USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES MARCH 2011 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared William Emmet, Leon Kessou, Jill Gay and Iain McLellan through the Global Health Technical Assistance Project. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES DISCLAIMER The authors‘ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 11-01-450-1) is available in printed and online versions. Online documents can be located in the GH Tech website library at http://resources.ghtechproject.net. Documents are also made available through the Development Experience Clearing House (http://dec.usaid.gov). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES i ACKNOWLEDGEMENTS The GH Tech Benin Family Health Activities Evaluation Team thanks the United States Agency for International Development in Benin for commissioning the evaluation of the PISAF and IMPACT/POUZN projects and for selecting us to undertake these challenging tasks. The time, patience, goodwill, and depth of knowledge of the 162 persons interviewed as part of this evaluation made it possible for the team to understand the significance of the many and diverse reports and technical interventions associated with the three projects‘ multiple technical initiatives. The willingness of project staff to provide us with a full array of project documentation and to share with us their thoughts on the projects‘ development and their perspectives on the projects‘ initiatives enabled us to understand the complex nature of issues associated with meeting their objectives. The fact that staff from the projects exhibited exemplary patience and good humor in fully responding to our repeated calls for clarification and information throughout the four weeks of this evaluation made it possible for the evaluation team to feel less guilty as we increased the intensity of our inquiries. In terms of framing the technical focus of this evaluation, the contributions of Ms. Michele Seibou, deputy team leader, and Dr. Milton Amayun, team leader, of USAID/Benin‘s Family Health Team and their entire professional and administrative team cannot be overstated. Finally, the evaluation team would like to acknowledge its appreciation for the time set aside by the representatives of the Ministry of Health (MOH) and of the departments and zonal health facilities in Zou/Collines and in the six communes to discuss the projects‘ progress toward strengthening Benin‘s family health services. The evaluation team is especially appreciative of the time spent with the projects‘ beneficiaries, discussing their perspectives on the effectiveness and sustainability of these complex and sometimes overlapping projects. Their insight was essential in helping us look not only to the past but also to the future and enabled the evaluation team to cut through superlatives and focus on the reality. The four members of this evaluation team acknowledge with appreciation the projects‘ assistance and that of its own Tatiana Amoussou for ensuring that the many logistical and administrative details associated with this evaluation were addressed. The evaluation team also expresses its appreciation to the staff of GH Tech, especially Ms. Allison Tombros Korman, for providing it with technical and administrative support throughout this interesting and challenging assignment. ii USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES iii CONTENTS ACRONYMS .................................................................................................................... vii EXECUTIVE SUMMARY............................................................................................... xiii I. INTRODUCTION .................................................................................................... 1 Purpose .......................................................................................................................................................1 Audience ....................................................................................................................................................1 II. BACKGROUND ....................................................................................................... 3 Overview of MNCH, Reproductive Health, and HIV/AIDS in Benin ..........................................3 USAID Strategic Framework .................................................................................................................3 PISAF Project Overview .........................................................................................................................4 IMPACT Project Overview ...................................................................................................................5 III. METHODOLOGY ................................................................................................... 7 Description of Process ...........................................................................................................................7 Constraints and Gaps..............................................................................................................................9 IV. FINDINGS BY PROJECT ...................................................................................... 11 Introduction ........................................................................................................................................... 11 Integrated Family Health Project (PISAF) ........................................................................................ 13 IMPACT/POUZN ................................................................................................................................. 26 V. INTEGRATION AMONG PROJECTS ................................................................. 39 VI. CROSSCUTTING ISSUES .................................................................................... 41 VII. LESSONS LEARNED ............................................................................................. 43 Impact of the Design and Delivery of Integrated Programs on the Achievement of Overall Health Goals............................................................................................................................ 43 Intended and Unintended Consequences of Integrated Programming .................................... 44 PISAF and IMPACT Lessons Learned from Application of Integrated Programming ........... 44 General PISAF and IMPACT Lessons Learned .............................................................................. 45 General PISAF Lessons Learned ........................................................................................................ 46 Specific Lessons Learned for PISAF .................................................................................................. 47 General IMPACT Lessons Learned .................................................................................................. 47 Specific Lessons Learned for IMPACT ............................................................................................. 48 VIII. RECOMMENDATIONS ......................................................................................... 49 IX. FUTURE DIRECTIONS ......................................................................................... 51 Introduction ........................................................................................................................................... 51 General Considerations ...................................................................................................................... 51 Health Systems Strengthening ............................................................................................................ 52 Improved Health Behaviors ................................................................................................................ 55 Integrated Health: Priority Interventions ........................................................................................ 58 iv USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES ANNEXES ANNEX A. SCOPE OF WORK—EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES ................................................................................................ 61 ANNEX B. USAID/BENIN FAMILY HEALTH ACTIVITIES EVALUATION JANUARY– FEBRUARY 2011: RESPONDENT LIST............................ 77 ANNEX C. USAID/BENIN FAMILY HEALTH BENIN EVALUATION BACKGROUND DOCUMENTS .............................................................. 85 ANNEX D. EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES METHODOLOGY ...................................................................................... 93 ANNEX E. USAID/BENIN FAMILY HEALTH EVALUATION SCHEDULE .......... 95 ANNEX F. EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES ................ 99 ANNEX G. USAID/ BENIN FAMILY HEALTH EVALUATION: INTERVIEW SUMMARY TEMPLATE .......................................................................... 101 ANNEX H. PISAF PROGRAM DESCRIPTION ........................................................ 105 ANNEX I. USAID/BENIN IMPACT PROGRAM DESCRIPTION ......................... 153 ANNEX J. PISAF PERFORMANCE MONITORING PLAN (INDICATORS LEVEL FY'07) ................................................................ 213 ANNEX K. USAID BENIN—IMPACT PROJECT PERFORMANCE MONITORING PLAN ............................................................................. 225 ANNEX L. POUZN PERFORMANCE MONITORING PLAN: REVISED JULY 2008 ................................................................................................. 237 TABLES Table 1. USAID/Benin Family Health Activities: Results-Oriented Framework ............................ xiii Table 2. Evaluation of USAID/Benin Family Health Activities—Future Directions ..................... xviii Table 3. Evaluation of USAID/Benin Family Health Activities—Future Directions ...................... xix Table 4. Evaluation of USAID/Benin Family Health Activities—Future Directions ....................... xx Table 5. USAID/Benin‘s SO and IR Framework ........................................................................................ 3 Table 6. USAID/Benin Evaluation of Family Health Activities—Summary of Number of Informants Interviewed and Observations and Discussions/Meetings Held on PISAF and IMPACT/POUZN (January 24–February 7, 2010) ................................................ 8 Table 7. PISAF/Benin: Focus on Technical Assistance by Geographic Area .................................... 14 Table 8. Preliminary Health System Strengthening Results, Zou-Collines, 2006-2010 (EQGSS—2–2010) ........................................................................................................................ 14 Table 9. Selected PISAF 2010 Annual Report Performance Indicators in Zou/Collines .............. 15 Table 10. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 1 ............................................ 17 Table 11. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 2 ............................................ 17 Table 12. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 3 ............................................ 18 Table 13. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 4 ............................................ 19 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES v Table 14. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 5 ............................................ 19 Table 15. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 6 ........................................... 20 Table 16. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 7 ............................................ 21 Table 17. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 8 ............................................ 22 Table 18. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 9 ............................................ 22 Table 19. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative I0 ........................................... 23 Table 20 USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 11 .......................................... 23 Table 21. Changes in Public Health Facility‘s Organization and Management in Zou/Collines, 2006–2010 (EQGSS-2)................................................................................................................. 24 Table 22. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 12 .......................................... 24 Table 23. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 13 .......................................... 25 Table 24. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 14 .......................................... 26 Table 25. IR 1: A Supportive Policy Environment Created.................................................................... 29 Table 26. IR 2: Access to Quality Services and Products Increased .................................................... 29 Table 27. IR 3: Demand for Health Services, Products, and Preventive Measures Increased ....... 30 Table 28. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 1 ...................... 30 Table 29. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 2 ...................... 31 Table 30. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 3 ...................... 32 Table 31. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 4 ...................... 32 Table 32. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 5 ...................... 33 Table 33. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 6 ...................... 34 Table 34. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 7 ...................... 34 Table 35. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 8 ...................... 35 Table 36. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 9 ...................... 35 Table 37. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 10 .................... 36 vi USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 38. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 11 .................... 37 Table 39. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 12 .................... 38 Table 40. Evaluation of USAID/Benin Family Health Activities—Future Directions ....................... 53 Table 41. Evaluation of USAID/Benin Family Health Activities—Future Directions ....................... 56 Table 42. Evaluation of USAID/Benin Family Health Activities—Future Directions ....................... 58 FIGURES Figure 1. PISAF‘s Activity Map....................................................................................................................... 12 Figure 2. IMPACT/POUZN Activities in Benin ......................................................................................... 27 Figure A-1. Carte Administrative du Bénin ................................................................................................ 67 Figure H-1. Diagram of Project Mission ...................................................................................................105 Figure H-2. Diagram of QA .........................................................................................................................114 Figure H-3. Continuum of Improvement in Health System ................................................................. 122 Figure H-4. HIV and Infant Feeding Question and Answer Guide .....................................................125 Figure H-5. Community Mobilization and BCC Training Process......................................................130 Figure H-5. Community Mobilization and BCC Training Process ......................................................132 Figure H: I-1. Pathway to Survival and EONC Best Practices .............................................................147 Figure H: I-2. Diagram of Demand, Access and Quality of Care in Local EOC Systems ...................................................................................................................................................149 Figure I-1. Community Based Intervention Strategy ..............................................................................175 Figure I-2. PERForM .......................................................................................................................................205 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES vii ACRONYMS AB Alibori/Borgou ABC Abstinence, be faithful, and correct and consistent condom use ABMS Benin Social Marketing and Communications Association (Association Béninoise pour le Marketing Social et la communication pour la santé) ABPF Benin Association for the Promotion of the Family/Association Béninoise pour la Promotion de la Famille Abt Abt Associates ACP Africa, Caribbean and Pacific ACT Artemisinin combination therapy AIDS Acquired immunodeficiency syndrome AIM AIDS impact model AIMI Africa Integrated Malaria Initiative AMSTL Active Management of Third Stage of Labor ANC Antenatal care consultation Aquatabs Brand name for household water treatment ARI Acute respiratory infection ART Antiretroviral therapy ARV Anti-retroviral medications (Antirétroviraux) AWARE Action for West Africa Region BASICS Basic Support for Institutionalizing Child Survival BCC Behavior change communication BHAPP Benin HIV/AIDS Prevention Program BIFHP Benin Integrated Family Health Program BSS Behavioral Survey System/Study CA Cooperative agreement CAME Central Medical Stores (Centrale d‘Achat des Médicaments Essentiels) CBO Community-based organization CBSA Community-based service agent CCS Communal health complexes/Complexe Communal de Santé/ CCM Country coordinating mechanism CFCA CFA franc CHD Departmental referral hospital/Centre Hospitalier Départemental CHW Community health worker C-IMCI Community Integrated Management of Childhood Illnesses CIPEC Centre d‘Informations, de Prospectives et de Conseils CIRTEF Conseil International des Radios-Télévisions d‘Expression Française CLUSA Cooperative League of the USA CNHU National referral and teaching hospital CNLS Comité National de Lutte contre VIH/SIDA COGEA Comité de Gestion des Arrondissements viii USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES COGEC Commune management committee/Comité de Gestion des Communes COGES Sub-prefecture management committee COP Chief of Party CPN Consultation Prénatale/Prenatal Care CPNR Consultation Prénatale Recentrée CPR Contraceptive prevalence rate CRS Catholic Relief Services CS Child survival CSA Arrondissement Health Center/Centre de Santé d‘Arrondissement CSC Communal health centers CSCU Urban Health Center/Centre de Santé de Circonscription Urbaine/ CSP Country strategic plan CSSP Centre de Santé de Sous-Préfecture/Sub-prefecture health center CSW Commercial sex worker CTL Local Technical Committee (Comité Technique Local) CTO Cognizant Technical Officer CYP Couple-years of protection DDS Ministry of Health Departmental Health Director (Directeur Départemental de la Santé) DDSP Departmental Directorate of Public Health DEXS Development experience system DH District hospital DHAB Directorate of Programming DHAPP Department of Defense HIV/AIDS Prevention Program DHS Demographic and Health Survey DNPS National Directorate of Health Protection DOD Department of Defense DPP Directorate of Planning and Programming DRC Democratic Republic of the Congo DSF Directorate of Family Health/ Direction De La Santé Familiale EEZS Health Zone Supervision Team (Équipe d‘Encadrement de Zone Sanitaire) EmOC Emergency obstetric care EONC Essential obstetric and neonatal care EPI Expanded program of immunization EQGSS Evaluation of the Quality of Health System Management (Evaluation de la Qualité de la Gestion du Système Sanitaire) ERPA Rapid evaluation of health worker performance EU European Union FERCAB Fédération des Radios Communautaires et Assimilées du Bénin FHT Family health team FP Family planning GAPTA Active management of the third stage of labor (AMTSL) USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES ix GFATM Global Fund for AIDS, Tuberculosis, and Malaria GHI Global Health Initiative GIS Geographic information systems GH Tech Global Health Technical Assistance Project GoB Government of Benin GTZ German Association for Technical Cooperation HEPS Health education in primary school HIPC Highly indebted poor countries HIV Human immunodeficiency virus HIV/AIDS Human immunodeficiency virus/ Acquired immunodeficiency syndrome HORECA Hotels, restaurants, cafes and bars network HRH Human Resources for Health HRM Human Resources Management HSS Health Systems Strengthening HZ Zonal hospital/Hôpital de zone HZMT Health zone management team ICRW International Center for Research on Women IEC Information, education, and communication IGA Income generating activities IMCI Integrated management of childhood illness IMPACT Integrated Assistance to Family Health and the Prevention of HIV/AIDS Project (Projet Intégré d'Appui à la Santé Familiale et à la Prévention du VIH/SIDA) IPC Interpersonal communication IPT Intermittent preventive treatment IR Intermediate results IRH Georgetown Institute for Reproductive Health IT Information technology ITN Insecticide-treated mosquito net IUD Intrauterine device KAP Knowledge, attitudes, and practices KfW Kredistanstalt für Wiederaufbau (German Development Bank) LLITN Long-lasting insecticide treated net LMIS Logistics Management Information Systems LQAS Lot quality assurance sampling MAP World Bank Multisectoral AIDS Project MARPs Most-at-risk populations MCZS Health Zone Head Physicians (Médecins Coordonateurs des Zones Sanitaires) MDEF Ministère du Développement, de l'Économie et des Finances MDG Millennium Development Goals MdM Association Maison des Médias M&E Monitoring and evaluation x USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES MFI Microfinance Institution MH Maternal health MNCH Maternal neonatal and child health MOH Ministry of Health MOPH Ministry of Public Health MOU Memorandum of understanding MVU Mobile video unit M/WRA Men and women of reproductive age NGO Non-governmental organization OC Oral contraceptives OGAC Office of the Global AIDS Coordinator ORS Oral rehydration salts ORS/Zinc Oral rehydration salts/zinc ORT Oral rehydration therapy ORTB Office de Radiodiffusion et Télécommunications du Bénin OSV-Jordan Organization pour le Service et la Vie-Jordan PATH Program for Appropriate Technology in Health PBA Benin German Project PBT Precedent birth technique PEP Post-exposure prophylaxis PEPFAR President‘s Emergency plan for AIDS Relief PHRplus Partners for Health Reform plus Project PISAF Integrated Family Health Care Program (Projet Intégré de Santé Familiale) PLWHA People living with HIV/AIDS PMI President‘s Malaria Initiative (Initiative Présidentielle de Lutte Contre le Paludisme) PMP Performance Monitoring Plan PMTCT Prevention of mother-to-child transmission PNC Postnatal consultation PNDS National Health Development Plan (Plan National de Developpement Sanitaire) PNLS National Program for the Fight Against AIDS (Programme National de Lutte contre le SIDA [MOH]) PNP Policies, norms and protocols PoNC Post-natal care POU Point of use water treatment POUZN Social Marketing Plus for Diarrheal Disease Control: Point-of-Use Water Disinfection and Zinc Treatment PPH Postpartum hemorrhage PPLS Projet Plurisectoriel de Lutte contre le SIDA – World Bank funded multisectoral AIDS project (formerly the MAP project) USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES xi PNLS Programme National de Lutte contre le SIDA/National AIDS Control Program PNLP Programme National de Lutte contre le Paludisme/National Malaria Control Program PPT Pre-packaged malaria treatment PRIMASIDA Integrated Social Marketing and HIV/AIDS Project PRIMS Integrated Social Marketing Program Project MAP Measuring access and performance studies PROLIPO Projet de lutte Intégrée contre le Paludisme dans l‘Ouémé/Plateau PROSAF Benin Integrated Family Health Program/Promotion Intégrée de Santé Familiale dans le Borgou et l‘Alibori PSAMAO Prévention du SIDA sur les Axes Migratoires de l'Afrique de l'Ouest PSI Population Services International PSP Private sector project PSS Swiss Health Project QA Quality assurance QI Quality improvement RBM Roll Back Malaria REsulTS Responsiveness, evidence, targeting, and support initiative RFA Request for assistance RH Reproductive health ROBS Benin Network of Health NGOs/Réseau des Organisations Béninoises de Santé SAP Structural Adjustment Program SBM-R Standards-based management and recognition SDM Standard days method SMFP Super Moustiquaire Famille Protégée bednet SNIGS Routine health information system SO Strategic Objective SONE Basic Obstetrical and Neonatal Health Care (Soins Obstétricaux et Néonatals Essentiels) SONU/GAPTA Emergency Obstetrical and Neonatal Health Care (Soins Obstétricaux et Néonatals d‘Urgence) SOW Scope of work STI Sexually transmitted infections SWS Safe water system TB Tuberculosis TBA Traditional birth attendant TFR Total fertility rate TPM Team planning meeting TRaC Tracking Results Continuously UNAIDS Joint United Nations Programme on HIV/AIDS UNDP United Nations Development Program xii USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES UNFPA United Nations Population Fund UNICEF United Nations International Children's Emergency Fund URC University Research Co, LLC U.S. United States of America USAID United States Agency for International Development USD U.S. Dollar USG U.S. Government UVS Village health units/Unité Villageoise de Santé VCT Voluntary counseling and testing VHC Village health committee WAEMU /UEMOA West African Economic and Monetary Union (Union économique et monétaire ouest-africaine) WARP West Africa Regional Program WB World Bank WHO World Health Organization ZC Zou/Collines ZS Health Zone (Zone Sanitaire) USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES xiii EXECUTIVE SUMMARY INTRODUCTION USAID/Benin charged the Global Health Technical Assistance Project (GH Tech) with the responsibility of conducting an evaluation of the PISAF and IMPACT projects as they came to a conclusion. The POUZN project, whose activities were absorbed by IMPACT following the 2010 completion of the POUZN implementation contract with Abt Associates, was evaluated under a single IMPACT/POUZN evaluative focus. The specific objectives for the four-person evaluation team were To assess for each project – Accomplishments – Progress against expected results – Extent of integration within activities of each project – Strengths, weaknesses, and constraints To recommend for each project ways to strengthen progress toward the completion of current contracts To assess ways in which integration was promoted within the three projects To quantify crosscutting issues associated with the implementation of the three projects To quantify lessons learned To recommend future directions BACKGROUND In their planning and execution of family health projects, USAID/Benin‘s implementing partners were governed by a results-oriented framework with a single strategic objective (SO) and three intermediate results (IRs), illustrated in Table 1. Table 1. USAID/Benin Family Health Activities: Results-Oriented Framework SO 5: Expanded Use of Family Health Services & Preventive Measures in a Supportive Policy Environment IR 1: Supportive Policy Created IR 2: Access to Quality Services and Products Increased IR 3: Demand for Health Services, Products and Prevention measures increased IR 1.1. Selected health products and approaches implemented IR 2.1. Selected products available at public service facilities IR 3.1. Knowledge of appropriate behaviors and preventive measures improved IR 1.2.Increased health system management capacity created IR 2.2. Selected products available at private service outlets IR 3.2. Appropriate research-based interventions and services introduced xiv USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 1. USAID/Benin Family Health Activities: Results-Oriented Framework SO 5: Expanded Use of Family Health Services & Preventive Measures in a Supportive Policy Environment IR 1.3. More effective civil society participation IR 2.3. Quality family health package of services available at targeted public health sector facilities IR 2.4. Financial access to health services increased Description of USAID/Benin’s Family Health Activities PISAF (Projet Intégré de Santé Familiale : Integrated Family Health Care Project) The design of the PISAF project was based on a framework of increasing access to health services through improving clinical skills and logistics, as well as reinforcing systems, such as supportive supervision, data management, financial management, and human resources. Project activities are scheduled for completion at the end of September 2011. To create a supportive policy environment, PISAF strengthened guidelines and procedures on integration. PISAF improved management of the decentralization process and expanded the minimum package of services offered for family health. Policy documents included protocols for family health and strategic plans to develop mutuelles (community-based health insurance plans). To improve access to quality services and products, PISAF trained health providers in quality care with reinforced clinical skills using curricula on provision of integrated care covering a number of key health needs: family planning (FP), prevention of maternal-to-child transmission (PMTCT), emergency obstetric care (EmOC), and malaria prevention. Community health workers (CHWS) were selected by communities and trained to promote insecticide-treated nets (ITNs) and other products, as well as health-seeking behaviors to prevent malaria. Logistical capacity was increased through Medistock a project-enhanced drug and medical supplies logistics computer program. Twenty health centers were renovated, and PISAF stimulated the creation of 59 mutuelles. To increase the demand for family health services, products, and prevention measures, PISAF created a package of integrated health services. The 151 CHWs trained to promote maternal and child health, especially those located in communities with limited access to other health workers, served a useful function in communicating health messages. PISAF also promoted community awareness of preventive measures in family health through radio and, on a smaller scale, theater and music groups. These efforts have the potential to reach at least half of the 1.5 million people of Zou/Collines. IMPACT (Projet Intégré d’Appui à la Santé Familiale et la Prévention de VIH/SIDA: Integrated Assistance to Family Health and the Prevention of HIV/AIDS Project) IMPACT built upon the strengths and lessons learned from past activities in Benin with social marketing, behavior change, service delivery, and institutional capacity-building.. The diverse interventions developed under the $14-million-dollar, five-year project were all evidence-based and came under the strategic objectives of the Government of Benin and USAID/Benin. Project activities are scheduled for completion in mid-October 2011. IMPACT provided diverse services and access to commodities through various locations combined with demand creation and behavior change through multiple mass media, USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES xv interpersonal, and community-based channels, as well as support for policy reform at the national level for HIV/AIDS. IMPACT partners, Population Services International (PSI), Abt Associates, and JHPIEGO, along with local non-governmental organization (NGO) partner Organization pour le Service et la Vie￾Jordan (OSV Jordan), provided technical expertise in social marketing, behavior change, service provision, and policy development. In 2008, additional responsibilities were given to IMPACT, including communications on malaria and social marketing of bed nets under the president‘s malaria initiative (PMI). In 2010, IMPACT absorbed diarrhea treatment and prevention activities, including point-of-use water treatment, implemented under the POUZN project and its socially marketed products Aquatab and Orasel/Zinc. Support for behavior change communication (BCC) and the national immunization program, as well as awareness and referral of women who suffer from obstetric fistula were also added to IMPACT‘s responsibilities for the fifth year of the project. EVALUATION METHODOLOGY Description of Process Scheduled between January 20 and February 18, 2011, the evaluation was conducted using participatory approaches and methods and employed a combination of the following qualitative techniques: Review of relevant documents Team planning and review meetings Briefing meetings: At the beginning of the evaluation, the team met with relevant MOH and USAID/Benin staff, as well as technical and management staff of PISAF and POUZN, to discuss and reach agreement on the evaluation process and expected deliverables. Thereafter, the team met twice with the same individuals, first to provide them with a preliminary briefing during the third week of the evaluation and again at the end of the evaluation to present findings incorporated in the first draft report. Interviews and site visits: The four-person team was organized into two teams that conducted simultaneous field trips to maximize exposure to field activities. Key informant interviews were conducted with senior department and zonal MOH staff, health facility staff, local NGOs, and current project staff. All interviews were conducted using a standardized interview template. A total of 162 persons were interviewed, and a combination of 19 observations, discussions, and technical meetings were held during the course of the evaluation. At the completion of each day‘s interviews, members of each of the teams met to summarize their findings, once again using a standardized interview summary template. Analysis and report writing: The evaluation team used interview and site-visit results, document reviews, and other relevant sources to obtain a comprehensive and in-depth understanding of PISAF and IMPACT projects to support sound analyses, arrive at inferences, and make recommendations for action. Constraints and Gaps The constraint associated with the evaluation team‘s late access to project-related documentation was significantly mitigated by the projects‘ careful and thorough preparation of initial project briefing sessions. While the time limitations assigned to the evaluation resulted in the team‘s being unable to visit the far corners of USAID‘s family health activities, those sites that were eventually chosen in consultation with USAID and project representatives resulted in a consensus by all parties that the sites were generally representative of the projects‘ initiatives. The fact that appointments with key resource persons and the schedule for site visits were xvi USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES delayed until the evaluation team‘s arrival was also a constraint. Finally, the strike by much of Benin‘s health staff during selected days of the evaluation significantly constrained our ability to observe service delivery at health center and hospital sites. FINDINGS In reporting on findings for PISAF and for IMPACT/POUZN, the following eight issues were addressed: Interventions: What is the focus of each project? Project progress (2006 - 2010): What have the projects achieved when measured against their workplans and their performance monitoring plans (PMPs)? Achievements: What have the projects achieved with reference to intermediate results (IRs)? – As a summary of findings on the projects‘ achievements, 26 dashboards associated with 14 PISAF and 12 IMPACT/POUZN indicators important to the management and implementation of the two projects provide a scaled assessment of the effectiveness and the promise of sustainability of each of the initiatives. For the purpose of the evaluation, effectiveness is defined as the extent to which, for each indicator, the two projects met technical and managerial expectations as expressed in the projects‘ design. Sustainability is defined as the extent to which, for each indicator, there were reasonable expectations that initiatives introduced by the two projects could continue and possibly be institutionalized following the projects‘ completion later in 2011. Integration between the two projects: To what extent did the two projects succeed in working together? Crosscutting issues: What common issues, both technical and managerial, positively or negatively impacted the ability of the two projects to meet expectations with reference to effectiveness and sustainability? Lessons learned: What lessons can be drawn from the projects‘ experience in implementing their initiatives? Recommendations: What opportunities or activities should each of the projects address during the remaining 8-12 months (March–October 2011) of their respective contracts? Future directions: As USAID/Benin is contemplating a follow-up project or projects to build on progress achieved under its existing family health activities, what concentration of activities might USAID/Benin consider for the future? PISAF: SUMMARY OF FINDINGS Project Progress Based on information collected in a 2010 survey (Preliminary health system strengthening results, Zou-Collines, 2006-2010 (Evaluation of the Quality of Health System Management (EQGSS) -2- 2010)) , the project‘s assistance resulted in significant achievement in the promotion of quality of care, knowledge and attitudes, selected household care practices, and health system management. Based on additional data presented in this report, the project achieved remarkable success in the reduction in postpartum hemorrhage among clients delivering at PISAF collaborative institutions but was decidedly less successful in community-level activities, most notably in the promotion and maintenance of new community membership in mutuelles. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES xvii Project Achievements Based on the results of respondent interviews, site visits, and a review of project documentation, 14 initiatives were reviewed and scored, as described above, in separate dashboards specific to each initiative. As a summary, PISAF was generally effective in most initiatives examined but less successful in ensuring that those initiatives would be sustained following the project‘s completion at the end of 2011. IMPACT/POUZN: SUMMARY OF FINDINGS Project Progress Based on information and documentation provided by IMPACT, the project‘s assistance resulted in an increase in effective civil society participation and in working with the private sector to promote quality in the integration of family health services. The project was less successful in promoting client access to method-mix, or family planning contraceptive alternatives. Project Achievements Based on the results of respondent interviews, site visits, and a review of project documentation, 12 initiatives were reviewed and scored, as described above, in separate dashboards specific to each initiative. As a summary, IMPACT/POUZN was significantly effective in most initiatives examined. IMPACT also achieved notable success in working toward ensuring that those initiatives introduced under the project would be sustained following the project‘s completion in 2011. In the assessment of effectiveness and sustainability, IMPACT/POUZN achieved significant results on the promotion of private sector capacity-building but less success in addressing gender issues. INTEGRATION AMONG PROJECTS PISAF and IMPACT project descriptions called for collaboration of the two projects since both focused on family health and, in some cases, were implementing the same or similar strategies. On the whole, there was little evidence of synergy between the two projects and significant evidence of many missed opportunities to collaborate. CROSSCUTTING ISSUES In reviewing progress achieved by PISAF and IMPACT/POUZN, the evaluation team identified nine issues that impacted the development and implementation of the two projects. These issues focused on a lack of consideration in the design of the projects and in their execution with reference to issues of sustainability. LESSONS LEARNED In examining lessons learned, as detailed in the scope of work, the evaluation focused on a number of issues, ranging from those specific to each individual project to those that were common to the two projects. The evaluation also looked at issues that could arise in the design and implementation of an integrated program of health initiatives. The lessons learned applied, to a large extent, to two projects whose activities were dependent upon a high degree of technical assistance and were concentrated in prescribed geographical areas. Accordingly, lessons learned from the projects‘ implementation, in terms of their impact on the achievement of overall national health goals, need to be regarded as precautionary pending a wider application of those activities throughout Benin. xviii USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES RECOMMENDATIONS As both of the projects are scheduled for completion prior to the end of 2011, the evaluation focused on ways in which each project might best apply its available resources prior to the completion of its contract in late 2011. Most of the recommendations focus on the importance of collaboration by the two projects on such issues as working with the national government to reach a consensus on an approach to quality assurance and ensuring that the technical innovations and products introduced by the two projects are widely disseminated among Benin‘s health care stakeholders. FUTURE DIRECTIONS Introduction In discussing the future of USAID/Benin‘s family health activities, it is assumed, based on information received as part of GH Tech‘s orientation and scope of work, that USAID/Benin‘s future family health procurement will center on one, five-year project, whose purpose will be to build upon progress achieved and lessons learned under PISAF, IMPACT, and POUZN. Recommendations for the future are summarized in the following four sections: General Considerations Build flexibility into the project‘s design Ensure a national as well as regional emphasis Increase country ownership Incorporate an emphasis on public/private sector partnerships Health Systems Strengthening The three dashboards below illustrate ways to build upon progress achieved: health system strengthening, improved health behaviors, and integrated health priority interventions. Each recommended area of concentration is ―scored‖ in terms of whether, under a new USAID project, the contractor will need to devote a minimal, low, medium, or high level of resources and effort to each specific technical area. It should be noted that the three tables do not suggest that any of the specific areas should receive a higher priority than another. Decisions on priorities should rightly be the responsibility of the project‘s design team, based on discussions with USAID/Benin and the Government of Benin and on knowledge associated with the availability of USAID/Benin resources. Considerations associated with each of the recommended areas of concentration are discussed in detail in the body of this report. Table 2. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Health Systems Strengthening Human Resources for Health Three Blue Quality Assurance Three Blue Capacity Development Four Green USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES xix Table 2. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Health Systems Strengthening Health Policy Reform and Development One Rose Health Financing Reform Two Gray Logistics and Commodity Management One Rose Operations Research One Rose Integrating Health Services Four Green Improved Health Behaviors Table 3. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Improved Health Behaviors Social Marketing Four Green BCC (Mass Media, Interpersonal Communication (IPC)) Three Blue Social Mobilization Three Blue Community Outreach Two Gray Research-Based Strategic Planning One Rose Gender Equity One Rose xx USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Integrated Health: Priority Interventions. Table 4. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Integrated Health: Priority Interventions Integrated Management of Childhood Illness (IMCI) (Diarrheal Disease, Acute Respiratory Infection (ARI), Immunizations) Four Green FP Four Green Maternal Neonatal and Child Health (MNCH) Four Green HIV/AIDS/STI Two Gray Nutrition One Rose Water and Sanitation Two Gray USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 1 I. INTRODUCTION PURPOSE USAID/Benin has charged the Global Health Technical Assistance Project (GH Tech) project with the responsibility of conducting an evaluation of the PISAF and IMPACT projects as they come to a conclusion. The POUZN project, which has been absorbed by IMPACT, is also being evaluated. As described in the scope of work (See Annex A), the specific terms of reference for the four-person evaluation team are the following: To assess for each project – Accomplishments – Progress against expected results – Extent of integration within activities of each project – Strengths, weaknesses, and constraints To recommend for each project ways to strengthen progress toward the completion of current contracts To assess ways in which integration was promoted within the three projects To quantify crosscutting issues associated with the implementation of the three projects To quantify lessons learned To recommend future directions AUDIENCE The primary audiences for this evaluation are USAID/Benin and its Family Health Activity implementing partners, including University Research Co., Abt Associates, Population Services International, JHPIEGO, and OSV Jordan. Other audiences include the USAID Health, Population and Nutrition Office in Washington, D.C., and the Ministry of Health (MSP) on behalf of the Government of Benin. 2 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 3 II. BACKGROUND OVERVIEW OF MNCH, REPRODUCTIVE HEALTH (RH), AND HIV/AIDS IN BENIN Benin has a population of 9 million, 45% of whom are below the age of 15. Benin ranked 134 out of 169 countries on the 2009 United Nations Human Development Index (UNDP, 2010) with a gross national income per capita of $1,250 USD. Six in 10 women and 4 in 10 men have never attended school. Only 32% of women report having control over their health care with or without their partners. Infant and child mortality rates in Benin are high due to preventable childhood illnesses. Malaria is a major cause of mortality in Benin. Benin has a high population growth rate with the average woman bearing five to six children. Unmet need for family planning (FP) is high. Use of modern contraception is less than 10%. Benin faces major challenges to improving family health due to poverty, limited knowledge about appropriate health-seeking behavior, and weaknesses in the quality, management, and delivery of health services. HIV prevalence is estimated at 2%. Among sex workers, HIV rates are estimated at 28%. Women constitute 40% of those living with HIV. Among youth, 13% have had sex by age 15 (Benin Demographic and Health Survey (BDHS), 2006). Only 5.3 million out of 8.8 million have access to safe water. Only 4% of households were found to have any disinfectant or cleaning products. For children under the age of 6 months, 13% are three deviations from the WHO standards for height for age, and 26% are two deviations from WHO standards for height for age (BDHS, 2006). In the past decade, the MOH has reorganized to facilitate decentralized planning and management, with each zone covering a population between 100,000 and 500,000 people. A wide variety of unregulated services exists in the private sector. NGOs provide an estimated 30% of health services. The Government of Benin (GoB) has prioritized ensuring the availability, accessibility, and utilization of quality family health services. Reinforcing human capital is among the GoB‘s top three priorities in its strategy for improving growth. The country has developed a 10-year National Health Development Plan (PNDS) to guide health interventions between 2009 and 2018 and lead to the achievement of the millennium development goals (MDGs) 1b, 3, 4, 5, and 6 by 2015. Benin expects to achieve the goals of the 10-year plan by providing universal access to quality health care. The GoB has announced several initiatives including free cesarean sections for eligible pregnant mothers; a waiver of user fees for children under five who consult public sector facilities; reinforcement of health financing schemes; and an increase in the number and capacity of CHWs. The health sector still faces important challenges, including the insufficient number of health workers at all levels, the quality of care provided by health workers, and the inadequate supply of drugs in health facilities. USAID STRATEGIC FRAMEWORK Table 5. USAID/Benin’s SO and IR Framework SO 5: Expanded Use of Family Health Services & Preventive Measures in a Supportive Policy Environment IR 1: Supportive Policy Created IR 2: Access to Quality Services and Products Increased IR 3: Demand for Health Services, Products and Prevention measures increased 4 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 5. USAID/Benin’s SO and IR Framework SO 5: Expanded Use of Family Health Services & Preventive Measures in a Supportive Policy Environment IR 1.1. Selected health products and approaches implemented IR 2.1. Selected products available at public service facilities IR 3.1. Knowledge of appropriate behaviors and preventive measures improved IR 1.2.Increased health system management capacity created IR 2.2. Selected products available at private service outlets IR 3.2. Appropriate research-based interventions and services introduced IR 1.3. More effective civil society participation IR 2.3. Quality family health package of services available at targeted public health sector facilities IR 2.4. Financial access to health services increased PISAF PROJECT OVERVIEW The design of the PISAF project was based on a framework of increasing access to health services through improving clinical skills and logistics as well as reinforcing systems, such as supportive supervision, data management, financial management, and human resources. Project activities are scheduled for completion at the end of September 2011. To create a supportive policy environment, PISAF strengthened guidelines and procedures on integration and improved management of the decentralization process. Through PISAF, the minimum package of services offered for family health was expanded. Policy documents included protocols for family health and strategic plans to develop mutuelles. Additional PISAF interventions included assistance on human resource management and on working with the departments and zones to develop an environment in which national legislation on decentralization could be integrated into planning below the national level. To improve access to quality services and products, PISAF trained health providers in quality care with reinforced clinical skills using curricula on provision of integrated care covering the following key health needs: FP, PMTCT, EmOC, and malaria prevention. CHWs were selected by communities and trained to treat malaria and diarrhea and to promote insecticide-treated nets (ITNs) and other products, as well as health- seeking behaviors. Based on the experiences in Alibori/Borgou, PISAF expanded the concept of collaboratives (zonal quality assurance teams focused on specific technical issues) to improve and monitor implementation of standards in response to findings associated with the 2006 EQGSS. Logistical capacity was increased through the use of Medistock, a project-enhanced drug and medical supplies logistics computer program. Twenty health centers were renovated, and PISAF stimulated the creation of 59 mutuelles. To increase the demand for family health services, products, and prevention measures, PISAF created a package of BCC activities to increase knowledge of, demand for, and utilization of basic health services. There were 151 CHWs trained to promote maternal and child health. PISAF also promoted community awareness of preventive measures in family health and products such as ITNs through radio, theater, and music groups, reportedly reaching the majority of the 1.5 million people in Zou/Collines. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 5 IMPACT PROJECT OVERVIEW The strengths and lessons learned from past activities in Benin with social marketing, behavior change, service delivery, and institutional capacity-building served as the basis for IMPACT‘s interventions. The diverse array of interventions developed under the $14-million-dollar, five￾year project were all evidence-based and came under the strategic objectives of the GoB and USAID/Benin. IMPACT planned to establish a balance of partnerships with both the public and private sectors. The intention was to create a lasting legacy for the project by working with the private sector, NGOs, and community-based partners. The provision of diverse services and the access to commodities through various locations, combined with demand creation and behavior change communication through multiple mass media, interpersonal, and community-based channels, were expected to create measurable changes. IMPACT partners PSI, Abt Associates, and JHPIEGO, along with local NGO partner Organization pour le Service et la Vie-Jordan (OSV Jordan), have diverse technical expertise that covers social marketing, behavior change, service provision, and policy development. In 2008, additional responsibilities were given to IMPACT, including communications on malaria and the social marketing of bed nets. IMPACT also absorbed the POUZN project and its socially marketed products, Aquatab and Orasel/Zinc, and took responsibility for the awareness and referral of women who suffer from obstetric fistula. POUZN Absorbed by IMPACT The POUZN project was designed to mobilize the private sector to introduce low-cost, point￾of-use water treatment products for diarrhea prevention and zinc bundled together with oral rehydration solution (ORS/Zinc) for the treatment of diarrhea among children under five. IMPACT subsumed the work of POUZN. The POUZN project used a combination of commercial marketing and community-based distribution by civil society partners to create availability and demand for the sale of Aquatab and Orasel/Zinc. The products were promoted though interpersonal communications and the mass media. The public sector channel and the central medical stores (CAME) were also used to increase access to POUZN products in all public health centers nationwide. Though POUZN has become part of IMPACT, which is entering its last year, its two socially marketed products stand a good chance of remaining available due to the demand for the products and their sustainable distribution networks. 6 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 7 III. METHODOLOGY DESCRIPTION OF PROCESS The evaluation consisted of a descriptive analysis of process, output, and outcome of USAID/Benin‘s three family health projects. As a quantitative assessment of achievements and a qualitative description of project activities, the evaluation methodology employed a combination of the following techniques: Review of Relevant Documents The evaluation team spent an initial period (January 14–20) reviewing existing key project data and reports (see Annex C). Project-generated documents, including annual and quarterly reports and qualitative assessments of activities, served as background to the evaluation. Additional documents and summary tables were requested from PISAF and IMPACT for information with reference to their achievement of outputs by IRs as specified by their PMPs and documented in their annual and semiannual reports. Team Planning Meeting Operationalization of the scope of work and methodological approach to the evaluation were developed by the team in a two-day team planning meeting (TPM) on January 21 and 22 and approved by USAID/Benin in a January 24 debriefing on the results of the TPM. (See Annex A). Initial Briefing Meetings Meetings with PISAF and IMPACT/POUZN technical and management staff (January 25 and 26) enabled the evaluation team to understand the project activities and prepare for field interviews and site visits. Interviews and Site Visits The four–person team was organized into two teams that conducted simultaneous interviews and field visits to maximize exposure to field activities. The teams spent a total of nine days (January 30–February 7) interviewing respondents and visiting sites in health zones in Zou/Collines and in the communes of Abomey, Littoral, Bohicon, Savalou, and Parakou. Key Informant Interviews Interviews were conducted with senior national, departmental, and zonal staff; health facility staff; local NGO staff; CHWs; women‘s groups; Local Technical Committee (CTL) representatives; representatives from the IMPACT-supported PROFAM Network (private health clinics); representatives of local radio stations; health promoters; and current PISAF and IMPACT/POUZN staff. (See Table 6 and Annex B). 8 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 6. USAID/Benin Evaluation of Family Health Activities—Summary of Number of Informants Interviewed and Observations and Discussions/Meetings Held on PISAF and IMPACT/POUZN (January 24–February 7, 2010) Key Informant Interviews Number Observations Number MOH officials 10 PROFAM Facilities 3 Local government (Ministry of Health Departmental Health Director (DDS)) Officials and Staff 13 Client–providers 8 Coordinating Mechanism 2 Discussions/Meetings Directors of Zonal Hospitals 2 Focus Groups or Group Discussions 2 Providers 18 Implementing Partners 2 CHWs and Focal Points 11 USAID/Family Health Team 4 Mutuelles 5 Total Observations & Discussions and Meetings 19 Representatives of Women's Groups 5 Community Radio Representatives and Staff 54 NGOs 41 Representatives from HIV+ Organization 1 TOTAL INTERVIEWS 162 Each team used a standardized interview and site-visit guideline (See Annex G) to ensure that the teams addressed the same issues. Each interview and site or facility visit took between 90 minutes and three hours to complete. Following each day of interviews, key points, issues, and observations were summarized using an interview summary form for each interview and site visited. At the completion of the data collection/interview stage, both teams met for two days (February 9 and10) in Cotonou in an intensive working session to summarize findings and to prepare the technical basis for the evaluation report. Preliminary Briefing Meetings (February 10) Meetings were organized with USAID and with representatives of PISAF and IMPACT/POUZN to present the preliminary findings and recommendations, with preliminary recommendations for the future being restricted to the USAID briefing. Analysis and Report Writing Following the preliminary briefing meetings, the evaluation team used the briefing participants‘ comments, interview and site-visit results, document reviews, and other relevant sources to support sound analyses, arrive at inferences, make recommendations for action during each project‘s final nine months of activities, and define recommendations for future technical assistance following the completion of the projects‘ contracts. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 9 Final Briefing Meetings (February 17) These meetings were organized with USAID and MOH representatives and, in separate meetings, with PISAF and of IMPACT/POUZN representatives to present final evaluation findings and recommendations, with recommendations for the future being restricted to the USAID briefing. Report Production The briefing, analysis, and final draft report writing process was completed during the last 10 days (February 8–18) of the evaluation. The final draft report was submitted on February 19, 2011, to USAID/Benin and to representatives of PISAF and IMPACT/POUZN for their comments. CONSTRAINTS AND GAPS In the execution of its scope of work, the evaluation team encountered the following constraints that impacted its ability to fully respond to USAID/Benin‘s and its own expectations: Limited USAID/Benin and Implementing Partner Pre-evaluation Preparation Although formal plans for the evaluation began in December, appointments with respondents and a field visit schedule were not finalized until well after the team‘s arrival on January 24. This gap in timing certainly represented a constraint to our ability to meet with individuals, in offices and at sites, whose work schedules did not permit last-minute scheduling of appointments. Late Access to Project-Related Documentation The team‘s access to project documentation was delayed until shortly before the team‘s arrival and consisted of project descriptions, quarterly and annual reports, and PMPs for the projects and USAID. Upon the team‘s arrival, important and timely access to relevant documents was an issue of significant concern for review of PISAF activities as access to additional PISAF documentation occurred only after the evaluation‘s first 10 days. After that time, information requested from PISAF continued to be provided in a less than timely manner. Indeed, information critical to the evaluation‘s findings was provided two days prior to the team‘s departure and even after the team‘s departure. The constraint associated with access to project data was somewhat mitigated by the projects‘ careful and thorough preparation of initial project briefing sessions in response to a request made by the team. Limited Time for a Dual-Project Evaluation Under normally-accepted guidelines, an evaluation of a single project calls for at least 10 to 12 days for field visits, 5 to 7 days for interviews, and an additional 5 to 7 for report drafting, initial briefings, and final briefings. However, in this instance, the team was charged with evaluating two projects within the time frame normally allocated to a single project evaluation. As a consequence, the team was unable to visit the far corners of USAID‘s family health activities, especially in Alibori, and did not have the opportunity, even in Zou/Collines, to visit those sites and to meet with those respondents who might have provided the team with a more comprehensive understanding of the beneficiaries‘ interaction with the projects. Nevertheless, this constraint was somewhat mitigated by the fact that those sites that were eventually chosen in consultation with USAID and project representatives resulted in a consensus by all parties that the evaluation‘s outreach, while not comprehensive, was generally representative of the projects‘ initiatives. 10 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Public Sector Health Care Staff Strikes The strike by Benin‘s public sector health staff during selected days of the evaluation significantly constrained our ability to observe service delivery at health center and hospital sites. Once again, this constraint was somewhat mitigated by the willingness of health service staff to meet with evaluation team members despite the sporadic curtailment of services. In noting this constraint, the evaluation team acknowledges that the same constraint existed for the implementing partners in the execution of their projects. Over time, both projects developed strategies to overcome this considerable constraint that has impacted not only the evaluation and the work of the implementing partners but also, and more importantly, the access to health care within Benin‘s public sector health services. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 11 IV. FINDINGS BY PROJECT INTRODUCTION In this section, the report addresses the following eight issues for PISAF and IMPACT/POUZN: Interventions: What is the focus of each project? Project progress (2006 - 2010): What have the projects achieved when measured against their workplans and their PMPs? Achievements: What have the projects achieved with reference to IRs? In assessing PISAF and IMPACT/POUZN achievements, the evaluation team drew on the perspectives of the evaluation‘s respondents (see Annex F), each of whom assessed the project‘s progress on USAID/Benin‘s SO 5 (expanded use of family health services and preventive measures in a supportive policy environment) based on their knowledge of and engagement with the project‘s many initiatives. At the same time, the team reviewed documentation provided by each project and by USAID/Benin to ensure that, to the degree possible, the evaluation‘s findings reflected a balanced and informed assessment of each project‘s achievements, strengths and weakness, and constraints. – As a summary of findings on the projects‘ achievements, 26 dashboards associated with 14 PISAF and 12 IMPACT/POUZN indicators provide a scaled assessment of the effectiveness and the promise of sustainability of each of the initiatives. For the purposes of the evaluation, effectiveness is defined as the extent to which, for each indicator, the two projects met technical and managerial expectations as expressed in the project‘s design and in subsequent planning documents. Sustainability is defined as the extent to which, for each indicator, there were reasonable expectations that initiatives introduced by the two projects could continue and possibly be institutionalized following the projects‘ completion later in 2011. The assessments of effectiveness and sustainability were drawn from comments received from the evaluation‘s respondents, from observations in the field, and from a review of available documentation. The evaluation team made every effort to ensure that personal opinion did not influence assessments of the 26 initiatives. – The assessment of the 26 selected key initiatives is summarized in each of the relevant dashboards in terms of whether, based on the above definitions of the two terms, a specific initiative‘s effectiveness and prospects for sustainability are limited, positive or very positive. In subsequent sections of the report, the findings focus on the following: Integration between the two projects: To what extent did the two projects succeed in working together? Crosscutting issues: What common issues, both technical and managerial, positively or negatively impacted the ability of the two projects ability to meet expectations with reference to effectiveness and sustainability? Lessons learned: What lessons can be drawn from the projects‘ experience in implementing their initiatives? Recommendations: What opportunities or activities should each of the projects address during the remaining 8-12 months (March – October 2011) of their respective contracts? Future directions: As USAID/Benin is contemplating a follow-up project or projects to build on progress achieved under its existing family health activities, what concentration of activities might USAID/Benin consider for the future? 12 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Figure 1. PISAF’s Activity Map USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 13 INTEGRATED FAMILY HEALTH PROJECT (PISAF) Interventions As defined by project documentation, PISAF‘s interventions were focused on three USAID/Benin IR-based strategies, all of which were expected to achieve the defined results summarized below: IR 1 Supportive Policy Environment Created Improve the policy environment through: Implementation of effective decentralization by strengthening effective planning and financial and human resource management at the health zone level Support at the national level for developing, updating, and finalizing policies on mutuelles, decentralization, community mobilization, PMTCT, and others that support access to quality healthcare Support for developing and implementing a national quality assurance (QA) strategy IR 2 Access to Quality Services and Products Increased Increase access to quality services and products through: Expansion of mutuelles and community-based service provision (CHWs and civil society groups) Integration of new technical areas into the family health minimum package, including PMTCT services, intermittent presumptive treatment for malaria, and distribution of ITNs Improvement of financial access to health services through the promotion of mutuelles Strengthening of capacity at various levels of the health system and among NGOs and civil society groups to provide high-quality health services and information Development of evidence-based clinical norms and the establishment of mechanisms to revise norms on a regular basis Strengthening of health worker competencies in critical clinical areas through training and monitoring performance through facilitative supervision and reviewing outcomes Implementation of improvement collaboratives to improve service quality through the facilitation of horizontal learning and rapid sharing of tested best practices Expansion and institutionalization of quality assurance activities at all levels of the health system IR 3 Demand for Health Services, Products, and Prevention Measures Increased Increase demand for health services and prevention measures through: Implementation of community mobilization and BCC efforts to stimulate demand for health services at the community level through peer education, local radio stations, participatory community theater, and materials development PISAF: Project Progress (2006–2010) In its implementation, PISAF activities centered on three principal administrative/geographic areas: national level, the health department of Zou/Collines (ZC), and the health department of Alibori/Borgou (AB) in the northeast of Benin (see Figure 1). Within these three areas of concentration, PISAF addressed the following technical priorities: 14 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES NATIONAL Health Department of ZOU / COLLINES Health Department ALIBORI / BORGOU (Former site of PROSAF 2001-2006) •Malaria case management and health systems strengthening activities under PMI (2008-11) • Systems strengthening at the department level and in six health zones. • Continue to strengthen the quality of health services. •Strengthen information and logistics systems • Strengthen service access in 147 public and 28 private health services • Upgrade the professional development of 24 mutuelles •Strengthen integrated guidelines and procedures •Improve financial access to services via 23 mutuelles •Strengthen management of the decentralization process • Moblize communities in four health zones. • Improve the guidelines on management of health service delivery Table 7. PISAF/Benin: Focus on Technical Assistance by Geographic Area As part of its monitoring process, PISAF has recently undertaken a second EQGSS (EQGSS -2- 2010) to assess progress since the project‘s launch in 2006. Table 8 represents preliminary results from the EQGSS – 2 on selected indicators of importance to the evaluation of PISAF‘s progress on the same indicators Table 8. Preliminary Health System Strengthening Results, Zou-Collines, 2006- 2010 (EQGSS—2–2010) Select EQGSS Indicators EQGSS I 2006 EQGSS II 2010 Quality of care (direct observations): proportion of cases observed that met at least 50% of the defined norms Prenatal consultations 40.9% (n=536) 79.1% (n=484) Deliveries 80.6% (n=32) 96.8% (n=100) Postnatal consultations 0% (n=14) 73.3% (n=36) Integrated management of child illness (IMCI) 35.4% (n=215) 85.7% (n=344) Knowledge, attitudes and practices by clients on danger signs (client exit interviews) Knows at least 2 danger post partum signs 65.0% (n=41) 91.5% (n=138) Knows at least 3 danger newborn signs 48.9% (n=41) 82.0% (n=137) Knows at least 3 sick child danger signs 25.5% (n=353) 44% (n=625) Household child care practices (randomized household survey of mothers with children under-five) USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 15 Table 8. Preliminary Health System Strengthening Results, Zou-Collines, 2006- 2010 (EQGSS—2–2010) Select EQGSS Indicators EQGSS I 2006 EQGSS II 2010 Exclusive breastfeeding of infants 0 to 6 months of age 43.1%* 56.2% (n=111) Children < 5 sleeping under an insecticide treated mosquito net 24.8%** 96.5% (n=458) Health system management (assessment of public health facilities) Use of data for planning and problem solving by health facilities 23.4% (n=134) 70.8% (n=50) Health facilities with no vaccine stock-outs over past 3 months 72.2% (n=134) 91.1% (n=50) Sound financial management systems operational in health facilities 38.2% (n=134) 97.8% (n=50) Health facilities receiving at least four formative supervision visits in past 12 months 67.9% (n=134) 80.5% (n=50) *2006 measure from DHS 2006 national ** 2006 measure from DHS 2006 for Zou/Collines Based on documentation received toward the end of this evaluation, there was no reason to question the EQGSS-2 methodology on which the findings in Table 2 are based. However, the previous table does present some results that raise questions. These questionable results include the findings that 97.8% of the health facilities have a sound financial management system and that 96.5% of children under five sleep under an ITN. Since the data in Table 8 were provided following the departure of the evaluation team and an evaluation of the data collection process was beyond the scope of this evaluation, the evaluation team suggests that the results of this survey be examined in more detail. In addition to the information provided above, the following table represents data, again on selected indicators, drawn from PISAF‘s 2010 annual report of activities Table 9. Selected PISAF 2010 Annual Report Performance Indicators in Zou/Collines Project Data available for 2007/2008 Project Data reported in 2010 IR 1.2 Health Systems Strengthening: % of health centers that have received at least four supervisory visits during a calendar year 37% (2007) 73% (2010) IR 1.2 Health Systems Management: Number of health zones that organized at least four sessions during a year to review and take action on collected information 5 out of 6 (2007) 3 out of 6 (2010) IR 1.2 Health Systems Management: % of health points of service that correctly estimated and submitted on time their need for essential family health products 16 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 9. Selected PISAF 2010 Annual Report Performance Indicators in Zou/Collines Project Data available for 2007/2008 Project Data reported in 2010 50% (2007) 79% (2010) IR 2.3: Service Delivery: Reduction of Maternal Mortality: Incidence of postpartum hemorrhage among all deliveries in the MNC/AMSTL collaborative sites 2.0% (May 2008) 0.39% (April 2009) IR 2.3: Service Delivery: Family Planning: Couple-Years of Protection (CYP) 5629 (2006) 12533 (2010) IR 2.4: Financial Access: Percentage of mutuelles that maintained at least two-thirds of subscribers from the preceding year 36% (2007) 6% (2010) With reference to data presented in Table 9, PISAF has achieved remarkable success in its collaborative sites in the reduction of postpartum hemorrhage and in the percentage of health centers that have received the necessary supervisory visits during a calendar year. At the same time, its achievement of 12,533 couple-years of protection (CYP), while certainly an increase from 2006, falls considerably behind its PMP target of 18,000 CYP. In terms of failing to maintain progress on indicators, the number of zones actively using collected information has decreased from 2007 as has the percentage of mutuelles who have maintained at least two-thirds3 of their level of subscribers from the preceding year. In their review of this report‘s first draft, PISAF representatives commented that the data on mutuelles refers only to new members. As one assumes that the same is true for 2007, where 37% of new members remained with the mutuelles, the fact that only 6% remained in 2010 is, as acknowledged by PISAF, a reason for concern. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 17 PISAF: Achievements PISAF Initiative 1: Policy Reform (National) Since its implementation in 2006, PISAF has made impressive progress in assisting the MOH in moving forward to develop (under USAID/Benin‘s SO 5–IR 1: A Supportive Policy Environment) an environment in which the GOB can strengthen the health care system. Collaboration with the MOH in preparing documents, such as the National Strategy for the Reduction of Maternal and Neonatal Mortality (March 2006), the handbook on the reform of Benin‘s health system, Réforme du Système de Santé au Bénin (2008), (2008), and the communications plan for health sector decentralized planning,Plan de Communication des Politique de Décentralization dans le Secteur Santé (July 2010) is to be commended. At the same time, national-level respondents indicated that the project could have been more effective in promoting national policies—such as the policy on mutuelles—had the project‘s design included the placement of a policy adviser within the central level of the MOH. Similarly, the work accomplished on national policy reform could have achieved greater prospects for sustainability had there been someone in place to ensure that the project-supported planning and policy documents were fully institutionalized. PISAF Initiative 2: Policy Implementation Policy implementation examines the effective application and anticipated sustainability of national guidelines at the departmental level and below. In this regard, PISAF directed significant attention to the introduction of best practices such as Soins Obstétricaux et Néonatals Essentiels (SONE)—Basic Obstetric and Neonatal Care (BONC); Soins Obstétricaux et Néonatals d‘Urgence—Emergency Obstetrical and Neonatal Health Care (SONU/GAPTA) ; and active management of the third stage of labor (AMTSL). PISAF‘s intensive work in introducing the policy of Équipe d‘Encadrement de Zone Sanitaire (EEZS)—Health Zone Supervision Team to form an effective zonal-level supervision team supported by a department and zonal-level budgets is a positive sign of PISAF progress in implementing national guidelines. The dashboard for Initiative 2 illustrates a positive assessment of the quality and effectiveness of the work accomplished in ZC under the project‘s tenure. At Table 10. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 1 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 1: Policy Reform (National) 1.1 Effectiveness Four Yellow Sustainability Four Yellow Table 11. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 2 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 2: Policy Implementation 1.1 Effectiveness Five Green Sustainability Three Yellow 18 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES the same time, as indicated in the dashboard for this initiative, many of our respondents raised significant concern about their ability to carry on with PISAF‘s application of the policies once PISAF has withdrawn its hands-on support. In most cases, the issue comes down to the respondents‘ reported inability to spend the time required to maintain the quality and intensity of interventions supported with funding from PISAF and to dedicate the funding required to support the training and transportation costs for staff assigned to continuous quality improvement teams. It is important to note that, while this is not a direct criticism of PISAF, it is a commentary on the culture of dependency promoted by projects that, in their professional zeal to deliver quality products, often forget the importance of promoting an environment of sustainability. PISAF Initiative 3: Integration of Services Under this initiative, PISAF built on the work accomplished under PROSAF, its predecessor project (PROSAF) in AB. More services of better quality are being offered and with better results. As a result of the findings of the extensively developed and well￾instituted EQGSS 2006, PISAF has taken impressive steps to introduce, in ZC, an expanded package of services including those focused on MNCH (including IMCI, Voluntary Counseling and Testing (VCT), and PMTCT. Moreover, PISAF‘s draft (December 2010) entitled Amélioration de la qualité des soins et services de santé familiale (Upgrading the quality of family health care and services) calls for the true integration of the above services during at least prenatal visits. In EQGSS 2 – 2010, whose preliminary results have recently been released, there appears to be marked improvement on the application of integrated service delivery norms in public health facilities in ZC. However, in visiting health services proposed by PISAF and in discussing the issue of integration with health care staff, the evaluation team saw limited evidence to suggest that health care services are currently appropriately integrated into the care of each client, mother or child, at each visit to a health facility. In balancing the evaluation team‘s observations and discussions in the field with recent findings from the research study, the team‘s assessment is that PISAF has achieved some progress in the integration of services but that, as indicated by PISAF‘s draft document cited above, more attention needs to be addressed to the issue of effective integration of essential services for it to reach a significant level of sustainability. Table 12. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 3 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 3: Integration of Services 2.3 Effectiveness Three Yellow Sustainability Three Yellow USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 19 PISAF Initiative 4: Quality Assurance PISAF‘s emphasis on quality assurance was one of the project‘s most effective initiatives. Following a frank and comprehensive report on EQGSS I findings, PISAF took steps to address many of the system‘s identified weaknesses through the establishment of the collaborative model (cohesive teams with sites working together to address priority care issues) envisioned in University Research Co.‘s (URC‘s) initial proposal, through the training of facility-based ―coaches‖ to implement needed changes and through a comprehensive approach to supervision and assurance of quality of care. PISAF‘s promotion of quality of care resulted in a remarkable achievement: From May 2008 to March 2009, project sites saw a 30% increase in providers‘ compliance (close to 100%) with AMSTL‘s three key tasks and a concomitant decrease of 40%, during the same period, in postpartum hemorrhage in PISAF-assisted health services. However, based on the human resource costs associated with maintaining the intensive supervision required under this intervention and on the current crisis in Benin‘s public sector human resources for health (HRH), it will be virtually impossible for the health services to support PISAF‘s laudable focus on quality assurance following the end of the project. PISAF Initiative 5: Relais (Community Health Workers) PISAF supported CHWs to prevent and treat childhood illnesses, , and diarrhea; to promote early referral for maternal complications; and to encourage the use of family planning methods. CHWs also promote health products, such as water purification tablets and ITNsto prevent malaria. In 2008–2009, 151 CHWs were selected and trained; of these, 143 CHWs (95%) are still functioning. In PISAF‘s 2010 work plan, 310 CHWs were scheduled to be trained, with initial trainings lasting five days. In one Zone Sanitaire (ZS), CHWs received profits from selling ACT treatment for malaria with other ZSs planning to replicate this. In the survey done by PISAF of 20 CHWs (17 men, 3 women), out of 145 CHWs in 4 ZS, 17 reported receiving a supervision Table 14. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 5 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 5: Relais (Community Health Workers) 1.3 Effectiveness Three Yellow Sustainability One Red Table 13. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 4 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 4: Quality Assurance 2.3 Effectiveness Four Yellow Sustainability Two Red 20 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES visit from the health worker in the past month.1 A workshop of 39 CHWs reported that they found supervision helpful. Supervision costs are covered by MOH, and PISAF-trained CHWs are viewed as MOH employees. Only half of the 20 CHWs interviewed reported having enough materials and products to follow norms. As only literate people can qualify to be a CHW, more men than women serve as CHWs. In a PISAF review in December 2010, five communities reported a loss of only 10% of CHWs who could no longer afford to volunteer. While the concept of the CHW is socially attractive and was reasonably effective under the project, its focus on volunteerism in a basic economy (e.g. extremely poor) and its reliance on the willingness of villagers to support CHWs suggests that, without external support, the CHW program has limited prospects for sustainability at the end of the project because CHWs cannot afford to volunteer. Needs assessments conducted with CHWs as part of project design would have clarified this lack of sustainability, decreasing the possibilities of missed opportunities. PISAF‘s interventionist, rather than participatory, methodology made the project unlikely to identify missed opportunities. While UNICEF, the Global Fund, and USAID are being asked to pay stipends for CHWs for the next period of time, it is unclear whether the GoB will assume these costs into the future and include CHWs as part of a paid workforce. If the CHWs are to be sustained, they will need more of a base to support their activities. PISAF Initiative 6: Mutuelles de Santé As articulated in a number of PISAF-supported studies, a community member‘s subscription to a mutuelle (health insurance scheme) is envisioned as a means to overcome financial barriers to access to health care, to reduce the financial risk associated with unforeseen health care needs, and to serve as a catalyst for increased knowledge and behavior associated with health care status, especially with reference to prevention of common illnesses and to the importance and benefits of family planning. PISAF has supported the development of 59 mutuelles and 9 mutuelle networks. PISAF has expended significant effort to strengthen the mutuelles so that they are in a position to fulfill their potential. According to PISAF‘s 2009 annual report, however, only 33 out of 45 mutuelles are still operational. Moreover, PISAF, in its annual report for 2010, noted that the percentage of new mutuelles that had succeeded in maintaining at least two-thirds of their membership from the preceding year had dropped from a level of 36% in 2007 to 6% in 2010. During 2010, PISAF strengthened mutuelles through, among other initiatives, developing a microfinancing plan, distributing ITNs through mutuelles, supporting the monitoring of mutuelles through zonal health collaboratives, providing training for mutuelles‘ union officials in managing exposure to financial risks, and supporting a national forum for the exchange of experiences associated with mutuelles. Of equal importance, PISAF has underwritten numerous studies on mutuelles in Benin, ranging from a study of the contribution 1 More recent data was provided to the evaluation team after the evaluation team had departed Benin so that questions of clarification could not be asked. This data did not provide numbers of CHWs (N) nor how CHWs were scored. Table 15. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 6 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 6: Mutuelles (Community Health Ins.) 1.3 Effectiveness Three Yellow Sustainability One Red USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 21 of mutuelles to community health (July 2010) to an analysis of factors influencing individuals‘ maintenance of membership in mutuelles. While the evaluation team did not have an opportunity to meet and talk with older PISAF-supported mutuelles in AB, many of which are still operational, USAID and PISAF have acknowledged that ―…those responsible for managing [these older] mutuelles admit that it took many years of support to be able to reach where they are today…‖. In discussions with mutuelles and ―mutuellistes,‖ considerable doubt was expressed with respect to the viability of mutuelles as currently implemented. The core issue appeared to be that without either external support or a national policy making enlistment in a mutuelle obligatory, PISAF-funded studies have all questioned the long-term sustainability of the concept. While Rwanda has been cited during the evaluation as a source of confidence on the concept of mutuelles, in Rwanda subscription to a mutuelle is obligatory for all citizens and the financial integrity and transparency of the mutuelles is an operational cornerstones of that country‘s mutuelle policy. Neither of these conditions currently exists in Benin. As currently implemented, Benin‘s policy on mutuelles and the current implementation of this policy under PISAF is deficient both in effectiveness and in expectations for sustainability. PISAF Initiative 7: Interpersonal Communication (Including Behavioral Change Communication) PISAF‘s introduction of EONC at the community level assisted communities in the early recognition and response to danger signs associated with women in labor and newborns. Extensive development of community liaisons in community-level IMCI and the production of numerous support materials for use in communities and in health services, all point to the seriousness that PISAF attached to this initiative. As earlier noted, exit interviews associated with the EQGSS – 2 survey also indicate that, in such areas as exclusive breastfeeding and the use of ITNs in the prevention of malaria among children under five, PISAF‘s focus on BCC appears to have had a positive effect. While the cost of maintaining a BCC program raises serious doubts about its sustainability, PISAF‘s effectiveness in promoting changes in behavior amongst its target audience was significant. Table 16. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 7 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 7: Interpersonal Communication (BCC) 2.4 Effectiveness Four Green Sustainability Three Yellow 22 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES PISAF Initiative 8: Mass Media Communication (Including Behavioral Change Communication) PISAF‘s approach to directly working with radio staff, local theater groups, and communities in the production of material centered on health topics, such as EONC, HIV/AIDS, STIs, malaria prevention, and mutuelles, was relatively effective given PISAF‘s lack of technical experience in the actual production and airing of mass media material. Although skills in media radio production were not increased, there were reports that radio producers appreciated training to increase knowledge on family health issues and applied this knowledge to programming content. However, both PISAF and IMPACT missed an opportunity to collaborate on the production of mass media material, since both projects were simultaneously working with several of the same radio stations. Finally, in the interest of sustainability, PISAF could have enlisted local support from businesses or NGOs to encourage them to buy into supporting future expenses associated with sustaining the communications program. Without such support, it is doubtful that the initiative is sustainable. One indication of the limited sustainability is the sudden end of family health programming and spot ads by the stations that did not have their contracts renewed by PISAF. PISAF Initiative 9: Civil Society Collaboration In December 2006, the project reported that ―…seven NGOs… were identified in the two health zones of Savalou-Bantè and Covè-Zangnanado￾Ouinhi alone.‖ The report further stated that ―…all the organizations and officials met hoped for harmonious collaboration with PISAF to strengthen community involvement in health promotion and prevention.‖ However, PISAF indicated that it had been difficult, until recently, to find competent NGOs with which to develop a professional relationship. The evaluation team recognizes and appreciates the pressure placed upon the project by USAID in 2007 not to spend additional time in attempting to develop an NGO partnership. However, persons outside of PISAF stated that the project should have insisted on working with NGOs that, over time, could have increased capacity leading to greater expectations for sustainability on selected initiatives, especially those that could have maintained effective mobilization of the Table 17. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 8 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 8: Mass Media Communication (BCC) 3.1 Effectiveness Four Yellow Sustainability Two Red Table 18. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 9 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 9: Civil Society Collaboration 3.1 Effectiveness Three Yellow Sustainability Two Red USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 23 community following the project‘s completion. At the same time, efforts to engage the civil society outside the NGO sector were undertaken by PISAF, including the development of the network of relais, support to women‘s groups, and the engagement of opinion leaders and traditional leaders. Community radio, traditional theater, and local musicians were instrumental in attempts to raise awareness for community engagement. PISAF Initiative 10: Capacity Development: Service Providers PISAF‘s work with service providers within their target service areas was highly professional, effective, and focused on priority health concerns. PISAF‘s innovative development of quality improvement collaboratives (or health teams) at a multitude of sites and within sites was a high point of PISAF technical assistance. The collaboratives‘ focus on working together to learn and serve as coaches on issues such as EONC, FP, malaria, and PMTCT could well serve as a model for the effective use of limited human resources. However, in the absence of a committed team such as PISAF with resources to support that commitment, it is questionable whether National Health Service staff will have the time or the energy to continue the support and development of this concept. Nevertheless, based on MOH support of this concept, it is conceivable that under the right circumstances (e.g. development of a MOH ―champion‖ during the project‘s final year) this concept could have some real hope of sustainability. PISAF Initiative 11: Capacity Development: Management During the last year of the project, PISAF worked with the MOH, the Ministry of Civil Service and Labor, the DDS of ZC, and with selected staff in developing the parameters for LOGIGRH. As a management tool for human resources management (HRM), LOGIGRH was designed to provide management personnel with an ability to identify gaps in HRM and to assist all levels in responding to these gaps. As part of the tool‘s development, PISAF trained 22 workers from the DDS in its application. When applied, LOGIGRH assists institutions in identifying gaps in human resources. When matched with training provided to health center staff in financial management, LOGIGRH‘s potential as a planning and Table 20. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 11 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 11: Capacity Development: Management 1.2 Effectiveness Four Yellow Sustainability Two Red Table 19. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative I0 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 10: Capacity Development – Service Providers 2.3 Effectiveness Five Green Sustainability Three Yellow 24 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES management tool is of significant importance. In information provided to the evaluation team, the recently-completed EQGSS-2 indicates (see Table 21) that there have been significant gains on two important management issues that, if supported in the future, especially through the institutionalization of LOGIGRH, could hold great promise for improved service management and management of human resources. However, in discussions and observations related to the practical application of LOGIGRH, respondents indicated neither working familiarity with LOGIGRH nor active day-to-day use of the tool and its outputs. While the concept associated with this tool is sound and the tool‘s outputs are focused and potentially useful, PISAF‘s annual report for 2010 has identified the need for follow-up if the tool is to be institutionalized within the two departments and on a national basis. PISAF‘s own assessment for the need for follow-up indicates that, without continued technical assistance, the sustainability of advances noted in Table 21 should be questioned. Table 21. Changes in Public Health Facility’s Organization and Management in Zou/Collines, 2006–2010 (EQGSS-2) Performance of facilities by activity 2006 2010 Evidence of data for decision-making 24,3% (n=37) 92,1% (n=38) Essential financial management procedures in place 41,2% (n=97) 70,0% (n=40) PISAF Initiative 12: Commodities and Logistics Management In 2008, PISAF introduced MediStock, a drug management inventory software, in ZC and reportedly rolled out the software throughout Benin. While not initially successful in terms of its application throughout the two initial departments, increased familiarity with the tool‘s application, through focused training of logistics personnel, has resulted in significant decreases in stock outages and in a true collaboration within the two department‘s zones in assisting each other in identifying and filling gaps in available drugs from one zone to another. In May 2010, PISAF introduced a web-based exchange program designed to facilitate and accelerate drug exchanges. As shown in the dashboard for Initiative 12, respondents indicated that the program is highly effective and responsive to their needs. However, while not advocating that Benin return to a paper-based logistics management system, respondents have indicated that, in an environment of scarce resources, the program‘s dependence on computer technology, including ready access to the Internet and to qualified technical (IT) assistance is an issue of significant concern with reference to the program‘s long-term sustainability. Table 22. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 12 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 12: Commodities and Logistics Management 1.2 Effectiveness Five Green Sustainability Four Yellow USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 25 PISAF Initiative 13: Gender PISAF staff defined the parameters of the project‘s focus on gender largely in terms of the number of women trained or providing health services for women. While the numbers of women trained have increased and training has been provided to improve services for women, a wider understanding and discussion of gender issues would have been warranted. For example, while the numbers of women who had posts in mutuelles as treasurers increased, the president, vice president, and treasurer (all male) of a mutuelle interviewed by the evaluation team believed that only men can join mutuelles, demonstrating a need to discuss gender within mutuelles. As another example, theater groups discussed the need to prioritize distribution of no-cost ITNs for pregnant women and children, as compared to men. This approach focuses on the biological vulnerabilities of pregnant women and children without adequate attempts to discuss power inequities between men and women. It is not enough to present gender conflicts to discuss without guidance on what could be a gender transformative approach. While URC‘s initial proposal and technical assistance contract called for gender-related initiatives, including the involvement of men in antenatal care (ANC) sessions, PISAF staff, determining that few men come to ANC, decided not to pursue this initiative. Studies on this issue (Ottolenghi, 2004; Varkey, 2006) have found that those services that succeeded in bringing men into ANC have also used this opportunity to provide services for men. As another example, respondents to this evaluation reported that, despite their obvious interest in microfinance and in selling socially marketed products, women were not engaged in health promotion activities beyond distribution of ITNs, Aquatab, and Orasel/Zinc. Needs assessments as well as assessments on women‘s microfinancing potential, based on a participatory process with women‘s groups, might have contributed to a more inclusive approach to gender issues by PISAF. Table 23. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 13 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 13: Gender Effectiveness 1.1 Two Red Sustainability One Red 26 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES PISAF Initiative 14: Collaboration within Benin’s Health System Although not a significant part of the project‘s design nor entirely accepted by USAID/Benin, the development of collaborative links with the national level received appropriate and targeted attention from the project. As a result, PISAF enjoyed a significant level of collaboration with counterparts in the MOH, both in terms of promoting policy essential to the strengthening of health services and in terms of ensuring that central level officials were aware of and in support of the project‘s department-level initiatives. Moreover, the MOH General Secretary‘s comments on PISAF‘s contribution to policy reform and development represented positive reinforcement of the project‘s central-level collaboration. However, based on respondents‘ assessments below central level, PISAF‘s focus on a relatively rigid contract-driven program of activities allowed for limited flexibility in responding to the concerns of departmental and zonal counterparts. While the current DDS in ZC was appreciative of PISAF‘s efforts to inform him of and, to some degree, involve him in, the planning of PISAF activities, comments from below department level indicate that a degree of resentment exists concerning the perception that PISAF imposed its activities without consideration for zonal priority concerns. IMPACT/POUZN Interventions IMPACT interventions respond to the following USAID/Benin IRs: IR 1 Supportive Policy Environment Created Improve the policy environment through strengthening the capacity of the MOH and GoB to manage and implement the national HIV/AIDS program Establishment of multi-sector approaches and collaboration with partners to strengthen local capacity for BCC Support at the national level for developing, updating, and finalizing policies that create a supportive environment for social marketing and the provision of HIV/AIDS services. Table 24. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: PISAF Initiative 14 PISAF IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 14: Collaboration 1.1 Effectiveness Three Yellow Sustainability NA USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 27 Figure 2. IMPACT/POUZN Activities in Benin 28 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES IR 2 Access to Quality Services and Products Increased Increase access to quality HIV/AIDS, reproductive health/family planning (RH/FP), and child survival services and products through Increased availability of health services and products in the private sector through expansion of a private network of clinics, PROFAM Adaptation and implementation of a national distribution system for socially marketed family health products Improve quality of services through: Promotion and support for implementation of best practices in integrated RH and HIV/AIDS services Strengthening of capacity at various levels of the health system and among private sector, NGOs, and civil society groups to provide high-quality health services and information Strengthening of health worker competencies in critical clinical areas through training and monitoring performance through facilitative supervision Expansion and institutionalization of quality assurance activities IR 3 Demand for Health Services, Products and Prevention Measures Increased Increase demand for health services and prevention measures through: Implementation of community mobilization and BCC efforts to stimulate demand for health services at the community level through mass media, peer education, local radio stations and other media venues, and materials development POUZN Absorbed by IMPACT The POUZN project was designed to mobilize the private sector to introduce low-cost, point￾of-use water treatment products for diarrhea prevention and zinc bundled together with oral rehydration solution for the treatment of diarrhea among children under five. IMPACT subsumed the work of POUZN in October 2010. The POUZN project promoted sales of Aquatab and Orasel/Zinc through a combination of commercial market and community-based distribution by civil society partners. The products were promoted though interpersonal communications and in the mass media. As indicated in Figure 2, IMPACT/POUZN concentrated on both the national level and select communes in AB in the northeast, Atacora/Donga in the northwest, ZC in the center, and Oueme, Atlantique, and Littoral in the south. IMPACT/POUZN Project Progress (2006–2010) As indicated in Tables 25, 26, and 27, IMPACT appears to have met or exceeded most USAID/Benin expectations. However, in indicators associated with FP, IMPACT has clearly failed to meet method-specific targets or to convince a sufficient number of new clients to adopt a modern FP method. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 29 Table 25. IR 1: A Supportive Policy Environment Created Select Indicators 2006 Base line Cumulative Target (2006- 2010) Actual (2006- 2010) 2010 % of Baseline IR 1.1.Slected health policies and approaches implemented: Number of HIV/AIDS national policies/protocols revised or disseminated to National Stakeholders 5 18 29 161% IR 1.2 Increased health system management capacity created: Number of quarterly review presentations made by the technical committees of coordination of National Program for the Fight Against AIDS (PNLS) and Comité National de Lutte contre VIH/SIDA (CNLS) 2 44 38 86% IR 1.3 More effective civil society participation: Number of additional private sector companies that commit to action against HIV/AIDS in their workplace by joining a Business Coalition against HIV/AIDS. 9 12 49 408% Table 26. IR 2: Access to Quality Services and Products Increased Select Indicators 2006 Base line 2010 target 2010 actual 2010 % of target IR 2.2.1 % of geographic areas with at least one sales point (Geographic area is defined as 1 village in rural areas, a neighborhood (quartier) in urban areas (for condom and Orasel/Zinc) an arrondissement (for ITN, Harmonie, Equilibre and Collier du Cycle) Prudence Plus condoms 77% 90% 90% 100 % Harmonie oral contraceptive 48% 75% 61% 87% Orasel/Zinc bundles 55% 58% 105% IR 2.3 Quality family health package available at targeted public health sector facilities: Number of persons trained in VCT, PMTCT, FP, or STIs 60 240 500 208% 30 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 27. IR 3: Demand for Health Services, Products, and Preventive Measures Increased Select Indicators 2006 Base line 2010 target 2010 actual 2010 % IR 3.1.3.3 Average minimum age first sexual relations boys 15.9 girls 16.4 boys 16.6 girls 17.4 IR 3.1.3.6: % of apprentices aged 15 - 24 who are sexually active and report having had only one partner during the last 12 months, disaggregated by gender boys 37.8 girls 62.7 boys 34.6 girls 60% IR 3.1.3.16 Knowledge of appropriate behaviors and preventive measures improved: % of Sex Workers 15-29 years who report systematic using condoms with clients during the last seven days. 91% 93% IR 3.14.3 Total numbers of new clients at PROFAM or integrated services who adopt a modern methods of FP ( oral, injectables, Cycle Beads, IUD and Implants) 12550 7078 56% IMPACT/POUZN Achievements Initiative 1: Policy Reform (National) IMPACT has achieved measureable results in policy reform at the national level. While the target was to devise or disseminate 18 national policies developed from 2006 to 2010, IMPACT assisted with 29 policies by 2010, while also holding meetings with key stakeholders and disseminating MOH policies concerning HIV/AIDS and other key reproductive health issues (See Annex F). Overall, policies were based on international best practices, such as youth interventions, anti-retroviral medications (ARVs), and strategic communication. IMPACT had a multilevel participatory approach that included advocacy. According to Dr. Akinocho, coordinator of PNLS, IMPACT was instrumental to the development of the key 2010 Ministry of Health document on the policies, norms, and procedures for people living with HIV. Dr. Akinocho stated that IMPACT also gave PNLS support to improve planning and management of programs. IMPACT assisted with the dissemination of policies and is credited with keeping policymakers on task and delivering outputs on time. While the target was for costing and planning exercises to be used by the CCM, 11 were done by IMPACT. Table 28. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 1 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 1: Policy Reform (National) 1.1 Effectiveness Five Green Sustainability Five Green USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 31 Engagement of the civil society makes IMPACT‘s work more sustainable. Although challenges exist in implementing policies outside the capital, much was accomplished on the policy front with modest resources. IMPACT POUZN Initiative 2: HIV/AIDS Outreach for Most-at-Risk Populations (MARPS) IMPACT conducted key studies to profile MARPS and systematic mapping to identify sites where they were concentrated, including transport hubs, red light districts, technical training locations, and schools among others. Based on this research (for example, Population Services International‘s (PSI‘s) 2007 study on sex workers (CSWs) and trucker use of condoms), CSWs, CSW clients, and truck drivers were targeted with interpersonal participatory communications conducted by NGOs. Peer education was used with particular success among truck drivers and CSWs. Regular supervision of NGOs by OSV-Jordan, an NGO partner, fosters sustainability. IMPACT regularly monitored MARPS behavior through its Tracking Results Continuously (TRaC) surveys. Measurable behavior change was achieved. Condom use at the last sex among sexually active youth apprentices was basically the same for males (46%) in 2006 and 2010. It rose, however, among females from 42.9% to 61.1%. In the case of sex workers, peer educators linked the women with user-friendly integrated clinics, where they went to treat STIs, receive regular HIV VCT, and access other health services. IMPACT provided free condoms for samples and demonstrations on how to use condoms. The NGOs conducting the outreach saw the free condoms as an incentive for participation and wanted more to give out. The fact that these outreach workers are employed by NGOs increases the chances for sustaining interventions. It should be noted that the numbers reached by these effective interventions are limited because they are located in selected sites. Impact on overall HIV/AIDS incidence in Benin will be limited. Table 29. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 2 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 2: HIV/AIDS Outreach for MARPS 3.1 Effectiveness Four Yellow Sustainability Three Yellow 32 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Initiative 3: IMPACT/POUZN Interpersonal Communication IMPACT NGO partners, public and private integrated service providers, peer educators working with MARPs, and community-based partners, such as CHWs and women‘s groups, were all trained in participatory interpersonal communication (IPC) techniques and provided with support materials in the form of flip charts, pamphlets, and posters. The contents covering FP, MNCH, malaria and diarrheal disease, and HIV/AIDS were developed based on behavioral research and designed to promote socially marketed products and inspire positive changes in behavior. Links were made between most of the same groups and the 10 community radio partners to allow then to amplify the same messages through the mass media. Youths in particular were targeted with IPC in schools and training centers using support materials and the popular magazine Amour et Vie. The same HIV and RH contents were also used in community radio station programming. Though the locations where the IPC occurred were chosen strategically, the limited number of interveners and the wide array of interventions resulted in limited coverage relative to the size of target populations and need. Limited resources for transport for IPC outreach and some management difficulties motivating those conducting IPC were encountered. The engagement of these community level partners and their IPC skills increases the chances interventions will be sustained. IMPACT/POUZN Initiative 4: Mass Media Communication Ten Community Radio Stations that covered the selected IMPACT intervention sites were provided with Training, and they signed contracts to produce and broadcast spot advertisements and programs as well as provide news coverage of community-level interventions. A typical community radio station reaches a potential audience of hundreds of thousands of people with a broadcast coverage of a radius of 75 kilometers. Spots were produced with partner stations in local languages based on a model provided by IMPACT promoting socially marketed products and specific behaviors. Different spots were broadcast several times a day. The impact of the spots and programs could be measured in increased sales of products and, in the case of Table 30. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 3 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 3: Interpersonal communication (BCC) 3.2 Effectiveness Four Yellow Sustainability Three Yellow Table 31. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 4 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 4: Mass media communication 3.1 Effectiveness Five Green Sustainability Four Yellow USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 33 PROFAM clinics, increased demand for integrated services. In the most dramatic example of the influence of mass media, Aquatab sales grew rapidly after it was advertised on television. Sales had been well below target after the water purification tablet was first introduced. All the socially marketed products promoted in the mass media, including FP commodities, condoms, Orasel/Zinc, and Aquatab had a steady growth in sales. The only exception was Super Moustiquaire, where sales declined when free impregnated bed nets were distributed. Initiative 5: Social Marketing (Family Planning, Condoms, ITNs) The strategy for the social marketing of FP products, including condoms, involved a combination of private sector, community level, and public level promotion and sales. Commercial distribution networks, including one operated by the government and one by commercial pharmacies, were also used. Sales outlets ranged from small dry goods retailers to CHWs supplementing their incomes. Annual condom sales have risen slowly but steadily from 9 million to 10.5 million between 2007 and 2010. Oral contraceptive sales have risen from 157,500 in 2006 to 200,000 in 2010. Prudence Plus condoms were found to be conveniently available to 86% of IMPACT target populations in 2010 as compared to 77% in 2006. Coverage and availability for the Harmonie oral contraceptive grew from 48% at the baseline to 77% in 2009–10. The commercially marketed impregnated bed net brand Super Moustiquaire had a more erratic sales record due to the widespread free distribution of nets. USAID indicated that the resources to purchase the Super Moustiquaire nets were limited as a consequence. Increased awareness of the value of nets in preventing malaria during the distribution may have inadvertently increased demand for the socially marketed nets. Most IMPACT socially marketed products were partially subsidized, though some were sold at cost- recovery prices, which increases their sustainability. PSI‘s global policy to develop real autonomy of its local affiliates, such as Benin Social Marketing and Communications Association (ABMS) in Benin, is a big step toward sustaining social marketing efforts in Benin. Continued clarity and transparency are needed as this evolution transpires and ABMS becomes a self￾sufficient entity with access to PSI technical assistance and quality control. IMPACT/POUZN Initiative 6: Social Marketing POUZN (Aquatab and Orasel/Zinc) The POUZN project started to promote oral rehydration solution and water purification products using interpersonal communication by engaging community-based partners, including women‘s groups, CHWs, and a microcredit association. These groups currently account for 25% of sales. However, it was not until a greater portion of the promotion budget was spent on mass media advertising that sales around the country took off, especially for the new water purification product Aquatab. During a three-year period, 811,379 packages of Orasel/Zinc were sold; during a two-year period, 870,090 packages of Aquatab were sold. There was also good public sector collaboration when the two products were added to the list of products Table 32. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 5 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 5: Social Marketing (FP, HIV and Malaria products) 2.2 Effectiveness Five Green Sustainability Five Green 34 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES distributed by the Ministry of Health. Both Orasel/Zinc and Aquatab are sold at a cost recovery price that is still reasonably affordable and makes sustained sales highly likely. For six months the Orasel/Zinc sales plummeted when UNICEF distributed ORS packages for free. There is evidence that the zinc regime is not fully followed. Continued promotion of the benefits of zinc for reducing the duration of diarrhea bouts as well as future cases is needed. The POUZN project collaborated with UNICEF to promote hand washing in schools and arranged for Orasel/Zinc and Aquatab to be sold in the neighborhoods around the schools. Abt Associates provided technical assistance to PSI for project implementation and, at the end of POUZN, the project activities were seamlessly absorbed into IMPACT for the final year of the IMPACT project. Initiative 7: Civil Society Collaboration NGOs are weak in Benin compared to many other West African countries. IMPACT and its collaborating partner OSV-Jordan, a Beninese NGO, conducted intensive capacity￾training for NGOs which resulted in improved management, accounting, and ability to conduct outreach. OSV-Jordan conducts regular supervision of NGOs and their outreach workers. Contracts and specific action plans guided the NGO interventions. IMPACT also provided support materials and encouraged NGO links with community radio stations. An example of NGO collaboration was the training of members of a group of people living with HIV and AIDS by OSV-Jordan to work as VCT counselors. Another IMPACT civil society initiative, the creation of municipal-level civil society advisory committees (CTLs) met with less success. Unrealistic expectations of the committees regarding financial support from IMPACT and municipalities and the absence of clear expectations and regular communication from IMPACT reduced the sustainability of the CTLs. Despite the weaknesses of NGOs in Benin, IMPACT and OSV-Jordan have accomplished significant civil society collaboration. The NGO RABEJ benefited from institutional capacity building and shows promise for fulfilling a monitoring role similar to OSV-Jordan in the future. Teaching grantsmanship to NGOs could increase sustainability. Businesses, such as Benin Telecom, were involved in developing Table 34. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 7 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 7: Civil Society Collaboration (NGOs) 1.3 Effectiveness Four Yellow Sustainability Three Yellow Table 33. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 6 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 6: Social Marketing of POUZN products (Aquatab and Orasel/Zinc) 2.2 Effectiveness Four Yellow Sustainability Five Green USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 35 workplace policies and programs. Engagement of the civil society makes IMPACT‘s work more sustainable. Initiative 8: Promotion of Integrated Services IMPACT first integrated services in 2 pilot sites run by NGOs and then expanded to 10 new sites operated by the Ministry of Health. In all cases, service providers were trained in HIV services, including VCT, STI diagnosis and treatment, FP services, ANC consultation, and malaria and other health services. The integration improved the collaboration among health providers in offering the different services and sharing skills for delivering them. The integration of services made it easier for clients who came to the clinic for FP to undergo VCT without having to be referred and seek out another service point. Offering VCT together with other health services made it more likely that clients would agree to VCT, as users of separate VCT facilities fear being stigmatized. Clinic staff conducted outreach with communities, including promoting services on community radio. Strikes by MOH health providers in 2010 reduced the reliability of the integrated services. The turnover of MOH health providers, including those trained by IMPACT, provided a challenge for sustainability. IMPACT/POUZN Initiative 9: Quality Assurance IMPACT developed a quality assurance system—Standards-Based Management and Recognition (SBM-R)—based on merit and community acknowledgement of good service. In addition to emphasizing respectful care for patients, providers are monitored to ensure use of current protocols and norms. Health workers are motivated through public recognition. Local authorities offer certificates denoting good service in official ceremonies, and clinics are provided medical equipment as a reward. MOH clinics were motivated to improve service to win the honors and gain the recognition of the communities they serve. One indication of client satisfaction with the integrated services was the 20% increase in use of the clinics between 2006 and 2008. Providers in Glazoue also reported that the combination of physical renovations and integrated care, along with an emphasis on team work, quality of care, and media and community outreach, increased the number of clients coming for services. IMPACT‘s approach to Table 35. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 8 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 8: Promotion of Integrated Services: (PMTCT, VCT, STI, FP) 2.3 Effectiveness Four Yellow Sustainability Three Yellow Table 36. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 9 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 9: Community Outreach for RH/FP/CS/Malaria 2.3 Effectiveness Four Yellow Four Yellow Sustainability Three Yellow Three Yellow 36 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES quality assurance was originally developed as a methodology for quality accreditation to be adopted by the MOH, but this has not occurred. Once services are integrated, ongoing supervision and retraining is needed. One challenge with quality assurance is PISAF and IMPACT employ two different QA systems. The use of both the IMPACT and PISAF Quality Assurance systems in the same zones has led to some confusion on the part of the MOH partners. Both IMPACT and PISAF‘s approaches to quality assurance have been adopted by the MOH staff in their intervention zones (the commune, district, and/or department level) and have received recognition at the national level. Neither, however, has been officially adopted at the national level. Supervision and retraining of the sites will be handled by the public health system. IMPACT/POUZN Initiative 10: Private Sector Franchise for Integrated Health Services (PROFAM) The 50 PROFAM clinics were selected to be part of a network of franchises for their potential to offer integrated services in modest income community neighborhoods. The benefit for the clinic was training for its staff, the addition of equipment such as sterilizers and HIV lab testing equipment, and an increased client load. Promotion of the PROFAM brand attracted clients to the new integrated services. The socially marketed PROFAM clinics steadily increased maternity, RH, and FP services. Voluntary counseling and testing of equal numbers of men and women rose from 244 tests in 2007 to 3,050 in 2010. Over the last five years, PROFAM clinics have experiences a fourfold increase in maternal health services offered. There were 1,219 births that required management of the third stage of labor in 2006, and that number rose to 4,479 in 2010. Half of all pregnant women were tested, but the percentage of women getting treatment at the clinics or referred for treatment was well below target goals. The private sector, including the PROFAM clinics, has been particularly useful during the strikes by the public health system. There were many private clinics interested in the PROFAM franchise, but only a few were selected. Only four, for example, can be found in the city of Abomey. One PROFAM midwife said monthly births rose from 50 to 82 at her clinic as women were attracted to the clinic by outreach and community advertising. The same clients tended to return for other services once they knew they were available. They would come to get an STI treated and then get an HIV test or come for maternal health services and then seek contraception. The PROFAM clinics are very sustainable because client fees cover most of the costs. Compared to the public sector, their labor force is more stable, which allows for a greater benefit from the IMPACT-sponsored training. There is a need for continued supervision to reinforce and control standards, and additional resources would be needed to expand the number of clinics and promote them. Following the completion of IMPACT, evidence suggests that some of the PROFAM clinics have realized the value of promoting integrated services through staff outreach and believe the promotion costs are justified by the increase in paying clients. Table 37. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 10 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 10: Private Sector Franchise for integrated health services (PROFAM) 2.2 Effectiveness Five Green Sustainability Five Green USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 37 The two PSAMAO clinics located in Parakou and Malanville offered STI and HIV services, as well as other family health services. Both increased the number of people undergoing VCT from several hundred a year to as many as 5,000 a year for each site in 2010. IMPACT/POUZN Initiative 11: Gender IMPACT has done some transformative work in gender by building gender as a crosscutting issue into the inception of the project. It has shown sensitivity to the special needs of women in its interventions and to the influence of men on decisions regarding family health. The percentage of male partners who participate in FP counseling reached 24% among all new FP clients in 2010, although the target was only 2%. IMPACT created the option of VCT for couples, so that relationships between men and women could be discussed in the context of VCT, and half of all those tested at the 50 clinics are men. Consideration has also been given to the special challenges of reaching and engaging women despite their low incomes and levels of literacy. For example, IMPACT added collaboration with microfinance institutions working with women‘s groups to their activities in response to information that women were not able to benefit fully from microcredit due to child and family member illnesses that cut into their time and loan capital available for their income-generating activities. Attempts were made to increase the number of women CHWs by having communities choose both a woman and a man and allow them to sell socially marketed commodities for a modest profit. Through IMPACT‘s youth outreach vehicle Amour et Vie, gender transformative messages are emphasized, including the importance of completing education for both girls and boys; the importance of girls and women taking charge of their own reproductive health; the negative health impacts of female genital cutting; and the importance of increasing male understanding of the consequences to both sexes of forced sex. IMPACT ensured that sex workers received respectful and nonjudgmental services from project-supported clinics. Providers are trained by IMPACT to promote the rights of women to access FP even without their partner‘s permission, though there remains a large unmet need for FP. The standard days method (SDM) is under￾programmed. IMPACT and IMPACT‘s activities benefited from working with the NGO RABEJ, which has institutionalized a rights-based, gender transformative approach within its institutional guidelines and has gender expertise on staff. A gender transformative approach is cost-effective as staff specialists consider intervention design and implementation from a gender perspective without adding activities. Ensuring the presence of a senior gender expert on staff both at USAID and in project staff increases the likelihood that gender perspectives are considered in decision-making. Table 38. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 11 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yellow Green Initiative 11: Gender Effectiveness 1.1 Three Yellow Sustainability Three Yellow 38 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Initiative 12: Research-Based Strategic Planning All IMPACT strategies have been driven by quantitative and qualitative research developed on an ongoing basis, permitting good analysis of target population behavior changes, trends in consumer habits, and mapping of intervention locations. IMPACT‘s work with high-risk groups illustrates how research is used. Detailed behavioral studies provided the foundation, and the regular Tracking Results Continuously (TRaC) studies looked at changes in key behaviors and consumer habits over time. For example, sexual violence was an issue identified as problematic for youth. The information was turned into content used in the Amour et vie magazine and radio program and in interpersonal communications outreach for youth. IMPACT staff set up the research protocols and then supervised the research data collection, and assisted with analysis. Eleven consultancy firms were selected to work with IMPACT on research and trained in the TRaC method of analysis of research results in 2007. Since then, three firms have completed most of the research for IMPACT beyond the ongoing TRaC studies after selection following tenders. In fact, the same research firms have been contracted by the national AIDS program and Africare to use the TRaC methodology. The contracting of these firms under IMPACT has greatly increased the availability of reliable, quality research expertise with the potential to be sustained over time. These firms are able to develop protocols and provide quality analysis with assistance from IMPACT staff. Table 39. USAID/Benin Family Health Activities: Assessment of Effectiveness and Sustainability of Key Initiatives by Project: IMPACT/POUZN Initiative 12 IMPACT/POUZN IR Ref. Limited Positive Very Positive Red Yello w Green Initiative 12: Research￾based strategic planning 3.2 Effectiveness Five Green Sustainability Five Green USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 39 V. INTEGRATION AMONG PROJECTS PISAF and IMPACT project descriptions called for collaboration of the two projects since both focused on family health and, in some cases, implementing the same or similar strategies. On the whole, there was little evidence of synergy between the two projects and significant evidence of many missed opportunities to collaborate. While USAID appears to have encouraged collaboration among the two projects, responses from the evaluation‘s respondents point to the following factors or instances associated with the two projects‘ limited levels of collaboration as well as to those instances in which there was evidence of collaboration: Lack of enforcement by USAID on the need for collaboration between PISAF and IMPACT. Although USAID/Benin repeatedly encouraged both PISAF and IMPACT to collaborate on crosscutting initiatives, such as those associated with mass media communications and quality assurance initiatives, USAID/Benin respondents reported that there was limited sharing across projects. The evaluation team assessed that USAID/Benin should have insisted on the development and implementation of concrete plans for collaboration and integration among the two projects. No regular meetings between the two projects. No regular meetings were held between project management and/or staff of the two projects to keep each other informed of project activities. The lack of such meetings represents a missed opportunity for the two projects to discuss ways to learn from the experience of the other project and to collaborate on any joint issues of concern, such as outreach to reduce unmet need for FP. Differing approaches by PISAF and IMPACT to QA. PISAF focused on building the technical skills of providers, while IMPACT focused on incentivizing providers through public recognition. These different approaches by two different USAID-funded projects, in some cases within the same geographic zones, led to confusion on the part of GoB partners. Respondents have reported that lack of a coherent and unified approach to QA created a climate of confusion at the national level, where the two project‘s approaches were duplicative and competitive. Duplication by PISAF and IMPACT of work done with community radio stations. On this issue, the lack of integration led to the two projects using the same community radio stations but with two entirely different approaches to communicating often similar but uncoordinated messages. PISAF‘s approach was to train radio station personnel on a number of key health issues, such as maternal health, while IMPACT provided programs with participation from radio personnel on topics of interest to their audiences. Had both approaches been coordinated, it is quite possible that messages would have increased in effectiveness and very probably that funding requirements for this initiative would have been significantly reduced. Good collaboration on the distribution and social marketing. From a positive perspective, both PISAF and IMPACT created good mechanisms for collaboration on the distribution and promotion of socially marketed commodities for FP and of products as such as Aquatab and ITNs. IMPACT ensured that socially marketed commodities were made available to PISAF partners and sites. Provision of Orasel/Zinc by IMPACT to PISAF for its emergency cholera response. When PISAF was tasked to respond to an emergency outbreak of cholera, IMPACT supplied Orasel/Zinc in a timely fashion to meet this public health emergency. 40 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Attendance by Both PISAF and IMPACT at quarterly meetings with the Ministry of Health and USAID. PISAF and IMPACT attended quarterly meetings that were held with the GoB‘s MOH and USAID. All indications point to their participation at these meetings as having been cordial, if not necessarily coordinated in terms of a consensus on issues of common technical interest. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 41 VI. CROSSCUTTING ISSUES In reviewing progress achieved in the implementation of PISAF and IMPACT/POUZN, the evaluation team identified the following nine issues that impacted the development and implementation of the two projects: Limited engagement by USAID of the GoB in project design: USAID policies regarding the engagement of Government partners in the process of project planning have changed since the time both the PISAF and IMPACT/POUZN projects were designed. In retrospect, a greater involvement and engagement in the design of the projects by the GoB would have resulted in a greater degree of Government ownership of the interventions and a greater commitment to sustainability and replication. Ambitious project design: As defined (see Annex H and Annex I), both projects, in the name of integration, took on an overly-ambitious program of activities. The design of both projects called upon the prime contractors to take on activities that were beyond their levels of established corporate and technical expertise to effectively implement and manage. In PISAF‘s case, venturing into the area of mass media communication and, to some extent, civil society collaboration, forced URC to take on activities beyond their established level of expertise. In IMPACT‘s case, contractual requirements that they address issues of service delivery and service delivery QA similarly forced PSI to take on activities beyond their established level of expertise. USAID‘s portfolio of family health activities resulted in wasted effort, duplicated activities, and a reduction in effectiveness and concentration in those areas for which each of the contractors was best suited. Existence of an environment of donor dependence: Due to the limited availability of GoB resources for public health expenditures, bilateral and multilateral donors provide significant resources for health programming. The combination of abundant need and donor generosity creates an environment of entitlement on the part of the GoB and optimism that projects will be renewed. Unrealistic expectations of sustainability in project design: The assumption that projects will be implemented on a pilot or regional basis, leading to sustainability, and replicated by the government was faulty especially in light of limited government resources and absorptive capacity. The engagement of civil society, particularly by IMPACT, in project implementation is an important step toward sustainability but not sufficient enough to achieve significant sustainability. Lack of gender analysis in design and implementation: There is an important difference between incorporating a gender perspective and simply targeting women in interventions. Addressing all aspects of gender in a systematic, inclusive, and transparent manner from the earliest stages of a project‘s design and throughout its implementation ensures that appropriate gender-focused consideration is given to all project interventions. Limited coverage to have significant national impact: The use of resources to strengthen health systems in well-defined and limited geographic areas certainly holds promise, at least during the life of the projects, for improvements within zones of concentration. However, without a concomitant focus on developing understanding, support, and ownership at the national level, project interventions, no matter how effective and innovative, stand a limited chance of impacting national health indicators. Replication unlikely: Both PISAF and IMPACT have achieved success in promoting selected initiatives beyond their target areas. Introduction of Medistock beyond PISAF‘s assigned zones and the national promotion of Orasel/Zinc are two such examples. Intensive and costly technical assistance in concentrated areas makes replication difficult due to 42 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES limited GoB resources and the lack of a national government commitment from the project‘s onset to sustain and scale up those project-specific interventions that prove successful in strengthening health services. Limitations in absorptive capacity: The ability of the GoB to take over activities from the PISAF and IMPACT/POUZN projects at their completion is compromised by government and civil society‘s limited capacity in terms of the time, knowledge, human resources, and funding necessary to absorb the two projects‘ initiatives and assume responsibility for their continuation. USAID/BENIN management challenges: The addition of programming responsibilities for malaria added complexity to the already diverse and complex PISAF and IMPACT/POUZN projects. Concerns were also raised regarding slow response times and gaps in the delivery of committed funds, especially with reference to PISAF and its implementation. Considering the complexity of the two projects, staffing at USAID/Benin‘s family health office has not been sufficient to fully oversee the projects. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 43 VII. LESSONS LEARNED IMPACT OF THE DESIGN AND DELIVERY OF INTEGRATED PROGRAMS ON THE ACHIEVEMENT OF OVERALL HEALTH GOALS Integrated programs designed without consideration of ways in which to replicate interventions are simply demonstration projects with limited prospect of contributing to overall health goals. To be effective in achieving overall health goals, the design and delivery of integrated programs must include a strong and sustained investment in research to ensure that initial designs are based on solid research and that ongoing research throughout the program‘s implementation is applied to identify requirements for midcourse change in the program‘s design. In an environment and economy of limited resources, designing and delivering integrated programs that include a significant emphasis on social marketing and on mass media communications is essential to the achievement of overall health goals. Integrated programs that include provisions for a viable public/private partnership develop collaboration and understanding among sectors of their respective roles in the achievement of overall health goals. The design and delivery of integrated programs must include due consideration for ways in which equipment and renovations can be funded if the program is to be recognized as contributing to overall health goals. While funding for material needs does not have to come from the program itself, links with funding sources are essential to the sustainability of integrated programs. The design and delivery of integrated programs must include due consideration for ways in which to address human resource constraints. While funding for human resources need not come from the program itself, adequate provision within the program must be directed toward assisting the nation in developing policies, such as task shifting, staff motivation, and career development, to address the crisis in human resources. In theory, integrated programs should reduce demands on human resources. In fact, integrated programs typically call upon limited staff to deliver increased services with the same amount of human resources. To be successful in achieving overall health goals, the design and delivery of integrated programs must include a strong national policy component and a commitment by the government to the development, reform, and institutionalization of health policies and guidelines to progressively support integration initiatives as they are introduced. Linking a variety of diverse interventions together under one project or contractor does not make them integrated especially if the ―integration‖ is more for the sake of expediency and without a clear benefit to the interventions themselves. PISAF and IMPACT never managed to create a critical mass of interventions in family health in part because they were designed as separate projects and were not obliged to integrate. Designing a single project taking the most effective components from PISAF and IMPACT and integrating them from the start with a clear plan for replication, sustainability, and scaling-up will increase the chances of reaching overall health goals. At the same time, with one single project focusing on a number of technically different initiatives, the project designers should ensure that sufficient resources and strong leadership for all initiatives are built into the project and that all initiatives receive adequate attention with respect to requirements for monitoring and evaluation, both within the project and at the mission level. 44 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES INTENDED AND UNINTENDED CONSEQUENCES OF INTEGRATED PROGRAMMING Integration and Coordination USAID cannot assume that two different projects, IMPACT and PISAF, both focused on family health with similar interventions, will coordinate and integrate simply because they are told to do so. Specific written instructions are needed to ensure that it occurs. Missed Opportunities Though the two projects developed similar interventions, they benefitted little from each other‘s experiences. These missed opportunities caused inefficiencies and some duplication of effort. Social Marketing Well Integrated The integration of IMPACT‘s socially marketed products into PISAF interventions was a good example of coordination and integration and ensured that community-level interventions were well linked to commodities. Duplicated Effort There was no difference between PISAF and IMPACT in terms of interest in coordination and integration with each other. For the most part, both were largely unaware of the activities of the other, even in situations where they were intervening with the same partners and developing basically the same interventions. Integration Approaches Similar Both PISAF and IMPACT made progress with their technical support to the integration of family health services. Their approaches were basically similar, though PISAF worked with MOH partners and conducted its own training and supervision. IMPACT worked more with the private sector and used NGOs as intermediaries. Both approaches achieved their goals, though the latter approach has a greater chance of being sustainable. PISAF AND IMPACT LESSONS LEARNED FROM APPLICATION OF INTEGRATED PROGRAMMING Need More Insights on Integration Analysis of the methodologies used to integrate services, including the documentation of lessons learned, is needed to improve the future design and delivery of these integrated program interventions. Integrated Services Well Appreciated by Staff Both public and private sector service providers welcome integrated services and greatly appreciate the increased skills, the increase in clients, the provision of equipment and commodities, and the refurbishing of facilities. Clients Appreciate Integrated Services Integration of HIV-related services into service points offering family health services increases client satisfaction and access, as clients are less concerned about stigma. There is increased interest in HIV VCT when it is offered at sites with integrated services. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 45 Family Planning Unmet Needs The demand for FP services exceeds the provision of services in general. The integration of FP services with other family health services increases access. More needs to be done to ensure user-friendly access for sexually active youths and sex education for youths. Maternal Health Progress The support for integrated services proved to be effective in responding to specific family health challenges. Measurable gains in maternal health were gained following the focus on increasing skills for diagnosing problem pregnancies. Local Radio Cost-effective Community radio proved to be a cost-effective partner for both IMPACT and PISAF, especially when annual contracts were signed permitting regular programming on family health, coverage of project events, and daily spot advertisements. The radio broadcasts promoted commodities, created a supportive environment for civil society involvement in family health, and contributed to positive changes in behavior. Research-Based Planning Keeps Focus Both PISAF and IMPACT conducted research-based planning, allowing policy decisions to be guided by both regular monitoring and periodic behavioral research on specific issues. Progress with Policy The influence of policy on creating a supportive environment for the interventions of both PISAF and IMPACT was given significant importance and resulted in the creation of numerous protocols, guidelines, and other rules and regulations. The policy work done by PISAF at the zonal level and by IMPACT at the national level with HIV policymakers enhanced the relationship between the project planners and their local partners. Integration Supports Essential Training, protocols, supervision, and support materials all contributed to improving the quality of work done by those conducting interventions and the maintenance of that quality under both PISAF and IMPACT. GENERAL PISAF AND IMPACT LESSONS LEARNED Limitations of Voluntarism In a resource-poor environment, it is unrealistic to expect individuals, who struggle daily with providing the very basics for their existence, to donate or volunteer their time for the good of the community. This issue is especially relevant to the development of the relais program and to the management of the mutuelles, whose key individuals are currently expected to donate their time without appropriate compensation. While possibly sustainable in the short-term, long-term prospects for sustainability, whether in Benin or in the developed world, are highly suspect. Linking women‘s groups and CHWs to the selling of socially marketed products for a modest profit can serve as a motivation, but it may not be enough to sustain the volunteers. Importance of Project Exit Plan Again, as noted in the above dashboards, much of the project‘s weakness is associated with limited expectations for sustainability. While PISAF‘s focus on sustainability was largely introduced toward the end of the project, the lesson learned from this experience is that plans 46 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES for progressive assumption of government responsibility for PISAF initiatives should have been built into the project‘s design and closely monitored throughout the project‘s execution. Every project should have a closedown strategy as part of project design. Such a strategy would also assist in preparing IMPACT/POUZN civil society partners for sustaining interventions and leveraging other funding. Links with Community-based Educational Initiatives At the community level, much of the success depended upon the capacity of the communities to appreciate the value of project initiatives, such as the mutuelles and relais. However, even at the level of USAID/Benin, the benefit of linking educational initiatives with health initiatives is overlooked. For example while initiatives for violence against women require a response in the health sector, USAID‘s project under education on violence against women managed by CARE had no links to either PISAF or IMPACT. As a result, opportunities were lost to integrate the introduction of community-level health interventions with educational initiatives and thereby strengthen the impact of both development programs. Because both male and female illiteracy is high in Benin, health initiatives could be strengthened through linkages with educational initiatives. For example, while Amour et Vie reaches a significant audience through both print and radio, efforts are needed to initiate sex education both within and outside of schools. Identification of Key Indicators While PISAF and IMPACT have done an impressive job of tracking a huge amount of data, a few key indicators should be chosen as key indicators to mark the overall progress of the project. While recognizing that USAID reporting requirements both within Benin and in Washington call for the tracking of myriad indicators, effective, on-the-ground management of a project‘s implementation would benefit from a joint project and local USAID/Benin agreement on a limited set of key indicators and key data. Such an agreement would have assisted projects of PISAF‘s and IMPACT‘s complexity in focusing more consistently on issues of vital importance to monitoring the project‘s progress toward effectiveness and sustainability. In addition, while recognizing that USAID reporting requirements both within Benin and in Washington call for the tracking of myriad indicators, effective, on-the-ground management of a project‘s implementation would benefit from an joint project and local USAID/Benin agreement on a limited set of key indicators and key data. Such an agreement would have assisted a project of PISAF‘s complexity in focusing more consistently on issues of vital importance to monitoring the project‘s progress toward effectiveness and sustainability. GENERAL PISAF LESSONS LEARNED Quality Materials Support Behavior Change PISAF developed quality support materials and trained its partners, who, in turn, effectively promoted changes in behavior amongst their target populations. Balance between Public and Private Sector Initiatives As noted in the above dashboards, much of the project‘s weakness is associated with limited expectations for sustainability. However, if the project, both in its design and its execution, had succeeded in developing a better balance between public and private sector initiatives, especially with NGOs, prospects for sustainability would have been greatly enhanced. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 47 Rigidity in Project Design and Implementation Although every USAID project carries with it performance requirements and the need for implementing agencies to adhere to these requirements, application of the project‘s design in the field calls for a delicate balance between what is required of the implementing partner and what the beneficiaries will perceive as being unresponsive to their own priorities. Failing to achieve this balance results in limited prospects for midcourse participatory correction, sustainability, and replication. SPECIFIC LESSONS LEARNED FOR PISAF PISAF Lessons from Application of Integrated Programming Sustainability Challenge PISAF created a model that does not lend itself to replication or sustainability due to its high dependence on its own staff and its partnering with the MOH, which does not have the capacity to take over the intervention in ZC or replicate it elsewhere. Civil Society Restraints The two most promising programs employed by PISAF to enhance civil society engagement in public health (the mutuelles and the relais) had disappointing results largely due to circumstances beyond the control of PISAF, including poverty, labor unrest in the public health system, and lack of support for volunteerism. Staff Instability PISAF found out that no matter how well training and supervision is developed and conducted, if those trained are transferred outside areas of intervention, it will be costly to the project, slow progress, and decrease the chances of sustainability. Limited Government Resources In a resource-poor environment, technical assistance to strengthen the health system permitted the MOH to develop skills and efficiencies, particularly in the area of supervision, commodity logistics management, and QA. Regrettably, limited MOH resources make sustaining the strengthened systems beyond the life of PISAF a challenge. GENERAL IMPACT LESSONS LEARNED Progress in Evolution of Benin Social Marketing and Communications Association (ABMS) to Self-Sustaining Organization PSI is in the process of implementing a global policy to develop real autonomy of its local affiliates, such as ABMS in Benin. This is a big step towards sustaining social marketing efforts in Benin. Clarity and transparency are needed as this evolution transpires and ABMS develops its own entity with access to PSI technical assistance and quality control. For example, MOH officials, largely unaware of the existence of ABMS, need to be informed of the progress of the changes. Good Links between BCC, Commodities, and Service IMPACT BCC methodologies are based on thorough analysis of target populations and guided through each step of the process by behavioral research. The communications are also targeted to specific populations and linked to affordable and accessible commodities and services, increasing their effectiveness. 48 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Private Sector Self-Sustaining Partners Private sector involvement through the engagement of the business community and commercial marketing of pharmaceutical products by IMPACT have potential for sustainability since these partners have their own resources to spend on programming. SPECIFIC LESSONS LEARNED FOR IMPACT IMPACT Lessons from Application of Integrated Programming Social Marketing Supports Integration The social marketing of commodities and services with both private and public sector partners allows for the widespread distribution of those commodities among the diverse partners offering integrated services and provides a source of revenue for some civil society partners. Room for Expanding PROFAM Network The PROFAM model of integrated services has good potential for expansion considering the demand among private sector health service providers to be part of the network and their dependence on these services when public health services are not available. Lack of Critical Mass in HIV Outreach The methodologies used to reach MARPS for HIV prevention and link them to commodities and integrated services proved to be effective but needs to be on a scale that reaches a critical mass in order to have an impact on HIV incidence. Progress with NGO Strengthening Working with the NGO sector in Benin is admittedly challenging, but IMPACT showed that through technical assistance in management and content, progress can be made in building up the skills of its NGO partners, increasing the opportunities to continue its interventions after the end of the project. Engagement of Civil Society Extends Project Reach and Chances for Sustainability Through partnering with the private commercial sector, strengthening the management capacity of local NGOs, and subcontracting local research firms, IMPACT will leave a legacy of strengthened civil society partners, who are available to continue the work initiated by IMPACT. Continued Availability of POUZN Products Though POUZN has become part of IMPACT which is entering its last year, its two socially marketed products have a good chance of continued availability because of the demand for the products and the sustainable distribution networks. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 49 VIII. RECOMMENDATIONS The USAID/Benin Family Health Evaluation Team recommends that, over the remaining months of its technical assistance project, PISAF and IMPACT should focus on the following items: Prepare and execute end-of- project plan: While decidedly late in its development, an end-of-project plan should be developed and implemented in consultation and collaboration with GoB and USAID/Benin counterparts. PISAF should focus its plan on implementing its 2009 institutionalization strategy. IMPACT should focus on the immediate preparation of its civil society partners for the post-project period. Joint working group on QA: PISAF and IMPACT should work together with other donors and the MOH to reach consensus on key elements of a joint approach to QA. Work on this issue will be of considerable importance to the MOH as it seeks to develop national standards on QA. Disseminate interpersonal communications support materials: Both PISAF and IMPACT have developed an impressive array of such materials, many of which are not currently available to key stakeholders within Benin. Accordingly, PISAF and IMPACT should work together to develop and implement a joint dissemination plan for sharing interpersonal communication support materials with government and NGOs. Social marketing options for relais: As discussed in this report, continued maintenance and growth of relais within communities is in doubt principally because of the concept‘s reliance on voluntarism for sustainability. Accordingly, PISAF and IMPACT should work with the government, with active relais, with community groups, and with NGOs to develop a strategy to increase the access to social marketing of health products as a means of providing a reliable income stream that will encourage relais to remain with the program. Analysis of progress on integrating services: In its recent assistance to the government on the development of a national strategy for the integration of services, PISAF developed a thoughtful plan for progressive integration of family health activities. PISAF should take the lead in working with the government to facilitate a national symposium on the progress of integration of family health services with the objective of reaching consensus on ways to roll out the national strategy throughout Benin. Effectiveness of integration: Policy work by both IMPACT and PISAF contributed to a 2011 MOH policy promoting integration. Studies should be conducted to assess if these integrated services are effectively reaching the poorest of the poor. Analyze the concept of mutuelles: As noted in this report, the concept of mutuelles, while holding great promise for increasing community-level financial access to critical family health services, is currently flawed due, in large part, to the lack of a national policy that will support an approach to the development of mutuelles within the context of Benin‘s current poverty environment. PISAF should take the lead in working with the government to analyze the concept of mutuelles as applied in other similar international environments and as currently practiced in Benin. The focus of this assistance should be on the development of draft policies that incorporate best practices associated with the development of mutuelles with specific attention to a realistic application of these practices within Benin‘s sociopolitical environment. Strategy for sustaining community-level outreach activities: Based on its experience in introducing family health initiatives at the community level, PISAF should work with local governments, health authorities, and other local stakeholders to facilitate a symposium of women‘s groups and NGOs to reach a consensus on recommendations for sustaining these 50 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES activities. Strengthening the links between the social marketing of commodities, microfinancing, and other initiatives with these groups will underscore the economic incentives associated with community involvement in health-related activities. National workshop on capacity-building: In the implementation of its project, PISAF has introduced a number of innovative initiatives focused on capacity- building of health care providers to ensure improved quality of care on critical family health issues. PISAF should facilitate a national workshop for family health care stakeholders to ensure that lessons learned and best practices introduced in the implementation of its innovations are discussed with the objective of developing a consensus on ways in which to institutionalize these innovations. ABMS should continue to evolve towards self-sufficiency: The creation of an ABMS management structure and staff that functions independently of PSI will position ABMS well for future partnering with USAID and other bilaterals and international organizations. Increase sustainability of civil society partners through grantsmanship training: In order to increase the sustainability of NGOs and community-based partners that now have increased management strength following IMPACT technical support, skills in resource development should be developed. Contents could include grant opportunities, how to write proposals, microfinancing, and selling socially marketed products. Analyze impact of free commodities on social marketing: IMPACT should conduct a systematic analysis of how distribution of no-cost commodities has impacted social marketing of the same products. No-cost condoms and impregnated bed nets have had an impact on the distribution and sales of branded versions. Based on this analysis, steps can be taken to encourage the coordinated procurement and distribution of health-related products, whether they are socially marketed or provided at no cost. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 51 IX. FUTURE DIRECTIONS INTRODUCTION For the purpose of the following discussion on USAID/Benin‘s future directions, it is assumed, based on information received as part of GH Tech‘s orientation and scope of work, that USAID/Benin‘s future family health procurement will center on one, five-year project whose purpose will be to build upon progress achieved and on lessons learned under PISAF, IMPACT, and POUZN. In the interest of setting a foundation for those who will design the project, the following discussion does not attempt to establish priorities for the new project. It is the opinion of this evaluation team that setting priorities, without knowing the resources available and without having access to USAID/Benin and GoB policy for the future and without the time required to adequately discuss priorities with the principals involved would be professionally irresponsible and presumptuous. Rather, the purpose of the following discussion is to identify the concentration of investment required of a new project should the mission and the GoB decide that a specific initiative is a priority within the limitations of available resources. Accordingly, in recommending ways in which to build upon progress discussed earlier in this report, the following discussion on future directions is divided into four subsections: General considerations Future directions for health systems strengthening Future directions to address improved health behaviors Future directions focused on integrated health priority interventions GENERAL CONSIDERATIONS Build in Project Flexibility Benin‘s next Demographic Health Survey (DHS) is expected to be released, at the earliest, in 2012, hopefully well after the next USAID/Benin family health procurement is awarded. Accordingly, it is recommended that the first-year, start-up activities of the new project be designed to build upon activities and initiatives that have proven effective over the past five years. At the same time, it is recommended that considerable flexibility be built into the project‘s succeeding years to permit the contractors to adjust their project monitoring plan in light of the findings of the new DHS. Such flexibility would allow for changes in project emphases, geographical concentrations, and configurations of the contractor‘s technical assistance team. As part of this flexible approach, it is recommended to build in mechanisms to identify and respond to gaps in national coverage, while ensuring that all project activities are harmonized with activities of other USAID projects and those of other donors. Increased flexibility as part of the project‘s design will require USAID/Benin staff to be particularly attentive in assessing progress and tracking changing public health priorities, making certain that such priorities are identified in concert with the host government and are based on emerging data. Ensure a National as Well as Regional Emphasis If the next USAID/Benin project is to improve its expectations of sustainability, the project will need to address the importance of a strong national presence to ensure that those initiatives developed at the departmental level are supported with strong and thoughtful attention to policy development and reform. At the same time, much of USAID/Benin‘s focus has been concentrated on heavily-funded activities whose impact has been minimal in terms of 52 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES demonstrated expectations for replication. Accordingly, the new project will need to solidify its technical progress in those areas in ZC and A/B, where much of USAID/Benin‘s resources have been concentrated, while being prepared, within available financing, to scale-up activities in years 2–5 to underserved areas. Increase Country Ownership The U.S. Global Health Initiative (GHI) policy requires all new assistance by USAID to be structured in a way that increases recipient country ownership. The new structure for activities should include a mechanism for the GoB to assume costs for various activities in an escalating fashion between the start and the end of the project. This approach calls for the active engagement of the GoB in all aspects of project development, starting with the project‘s design and the award of the procurement and extending on to include monitoring, evaluation, and a commitment to a joint project exit strategy. Under this strategy, the GoB should be encouraged to agree to a progressive assumption of responsibility for project innovations, both in terms of a commitment of personnel and in terms of an inclusion of national budgetary line items for selected technical initiatives. While it would be unrealistic to assume that the GoB will be in a position to devote its scarce national funding resources to support the full array of activities introduced through a USAID/Benin procurement, it is not unrealistic to expect that the GoB should begin, from day one of the project, to plan for assuming responsibility for a number of these activities by taking advantage of other external funding opportunities, such as those offered by the Global Fund. Increase Private/Public Sector Partnerships To date, little attention has been paid to the potential role of the private sector in assisting the government to respond to its family health priorities aside from the social marketing of FP commodities and franchising of clinics under IMPACT. While Benin‘s past experiment with socialism is certainly a mitigating factor to the willingness or ability of the private sector to establish a partnership with the government, IMPACT‘s work in establishing the PROFAM network and in enlisting the MOH‘s participation in supervising the work of PROFAM clinics represents strong evidence of the potential for a public/private sector partnership. Accordingly, the challenge for the new project is to serve as a catalyst in helping to develop this relationship, to overcome the mistrust on both sides, and to identify ways in which both sides can work together toward identified common objectives and priorities. Select Key Indicators IMPACT and PISAF were encumbered by a large number of indicators, many of which USAID obliged them to include. In the next project, a consensus should be developed between USAID, the project contractors, and the government on the selection of a small number of key indicators that provide clear and succinct data for decision-making and midcourse changes. HEALTH SYSTEMS STRENGTHENING As illustrated in the table below, eight principal technical areas of concentration are recommended as ways to build upon progress achieved in health systems strengthening. Each of the eight areas is presented in terms of the degree to which USAID/Benin will need to dedicate resources given progress achieved to date for each area. Accordingly, each area is ―scored‖ in terms of whether, under a new USAID project, the contractor will need to devote a minimal, low, medium, or high level of resources and effort to that specific technical area. It should be noted that the table does not suggest that any of the specific areas should receive a higher priority than another. Considerations for each of the eight areas are discussed in detail in subsequent paragraphs. Minimal Low Medium High Human Resources for Health Quality Assurance Capacity Development Health Policy Reform and Development Health Financing Reform Logistics and Commodity Management Operations Research Integrating Health Services Table 40. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Health Systems Strengthening USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 53 Human Resources for Health (HRH) The next five years represent a critical juncture in Benin‘s ability to build upon the progress achieved under PISAF and IMPACT. According to WHO (2011), Benin is one of 57 countries globally with a health workforce crisis. While 23 doctors, nurses, and midwives are required per 10,000 people, Benin has less than one physician per 10,000 people2. Assisting the nation in being able to respond to its current crisis in human resources for health (HRH) is one of USAID‘s areas of comparative advantage given the progress achieved under the agency‘s Capacity Project. The importance of staff motivation, up-to-date data (such as that now possible under PISAF‘s LOGIGRH initiative), on-the-job training, career ladders, and task-shifting, among other strategies and mechanisms, is well-documented as a means of responding to the crisis in HRH. Particular attention is needed to the obstacle presented by the reluctance of some physicians to accept nurses and midwives to insert implants, for example, or some health providers to accept CHWs to socially market oral contraceptives. Working toward addressing HRH issues is difficult and requires persistence and the availability of good solid research to establish what will work in Benin. This is a challenge worthy of USAID/Benin‘s new Family Health Project. In responding to this challenge, it is recommended that the future project address the following issues: Policies to assess the following HRH needs and ways to meet these needs: determination of which categories of providers with which skills are needed in which geographic locations; determination of incentives needed for providers to live in certain geographic zones; and determination of need for and feasibility of task-shifting. Policies to certify public and private health providers Policies to encourage the private sector and the MOH to respond to identified priorities Pre-service training to include current best international practices In-service training, along with institutionalized continuing education 2 Van den Broek, A et al, 2010. Policies and Practices of Countries that are experiencing a Crisis in Human Resources for Health: Tracking Survey, World Health Organization. 2010. 54 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Quality Assurance Both PISAF and IMPACT have adopted effective approaches to address quality in the delivery of health services. USAID should build on the work that has been done under these two projects. In a participatory fashion, USAID should collaborate with the MOH to assess what is the best QA approach that could be sustained in the future in Benin. While the approaches to QA by PISAF and IMPACT have proven effective, this report has noted that the existence of two approaches to QA has created confusion within the central government. Accordingly, as noted earlier, PISAF and IMPACT should work together, prior to the end of their projects, to reach a consensus on a QA model to be promoted under the new project. This model should incorporate approaches to QA currently being implemented by other donor projects. Realistically, it may not be possible to develop a single national approach to QA prior to the end of the two projects. It that likely event, working on the national model by building on the consensus achieved between PISAF and IMPACT should constitute an important element of the new project. It should also be noted that the two projects have achieved remarkable progress in addressing QA issues associated with delivery of services in the area of maternal and neonatal health. One would expect that these two areas would be a priority for USAID/Benin‘s next project. Capacity Development of Health Service Providers As noted in this evaluation, although PISAF has been most closely associated with capacity development of service providers, IMPACT‘s contribution in Glazoue and through PROFAM has also been significant. The evaluation has also called for both contracting agencies to bring together stakeholders to reach a consensus on those best practices that constituted a part of the lessons from their experience and work in the field. However, as indicated in the above table,, the new project will need to devote considerable effort to continued work on capacity development so that best practices and lessons learned will be applied as capacity-building is institutionalized within the nation‘s public and private health institutions. PISAF‘s significant progress on addressing the issue of ways in which health care staff can work in teams toward the resolution of quality care issues and especially the reduction of maternal postpartum hemorrhage must not be a lesson learned but forgotten. While this may appear to be simply a QA issue, working to develop synergy among different grades of health professionals is also a key component to capacity development. Health Policy Reform and Development PISAF and IMPACT‘s work with the MOH on health policy formulation has helped USAID/Benin establish its comparative advantage as a donor agency committed to practical and applied policy reform. However, the fact that PISAF, in its design, did not include the placement of a policy adviser within the MOH has been cited by officials within the ministry as a significant project weakness. It is recommended that the new project place a policy adviser within the MOH to address, among other issues impacting family health, policies associated with the true integration of services, links between the community and service providers, remuneration of CHWs, effective action against HIV/AIDS stigma and discrimination, and development of a realistic model for mutuelles. Health Financing Reform While the need for health financing reform can cover a multitude of interventions, the evaluation team recommends that the new project focus on ways in which Benin‘s communities can effectively address the importance of reducing financial barriers to access to health care. The concept of mutuelles, while important in its potential, has not taken root in the fabric of Benin‘s socioeconomic culture. Similarly, the concept of relais, or CHWs, has yet to be adjusted so that it will work effectively within the context of Benin‘s subsistence economy. It is USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 55 recommended that the new project take on the multisectoral challenge of working toward a solution that will address ways in which to make both the mutuelles and the relais functional within Benin. Insights into the mutuelles experience in Benin and globally should inform the revision of the current approach and the establishment of government policy, taking into account the GoB‘s policy to align health financing schemes along the lines of UEMOA standards. Logistics and Commodities Reform By all accounts, PISAF‘s Medistock has received wide acceptance within the MOH as a result of its practical application in ZC. IMPACT has established links with both private and public sector distribution networks for its socially marketed products. The new project should ensure that the progress gained by IMPACT nationally and by PISAF in its zones of intervention continues and systems are applied throughout the country‘s health care delivery system. Sufficient resources will be needed to ensure that the intermediate level has both the skilled staff and information technology to manage an effective system. Operations Research As the new project unfolds, information coming from the new DHS will raise fresh concerns and will suggest areas of intervention not yet considered. The team is recommends that the new project include an operations research component that will permit the new contractor or contractors to explore the feasibility of these interventions without initially committing significant resources to their adoption. Once again, the new project should provide the new contractor with sufficient flexibility to test these interventions and, if found feasible, to introduce them as full-fledged project components. Integrating Health Services The key to effective integration of services is providing all Beninese with access to all appropriate care within a single visit to a health facility. In this regard, those issues of primary concern are addressed below under ―Integrated Health: Priority Interventions.‖ Both IMPACT and PISAF have made strides in providing integrated services, and future work should continue in this area. In PISAF‘s case, the project‘s work in establishing collaborative models where ―… multiple sites work on similar problems to develop innovative care models…‖ has the potential for expanding to include teams that work together toward the true integration of services. Similarly, lessons learned from IMPACT‘s work with PROFAM clinics to integrate maternal and neonatal health with FP could form a part of the new project in terms of their application to both the public and private sectors. IMPROVED HEALTH BEHAVIORS As illustrated in the table below, six principal technical areas of concentration are recommended as ways to build upon progress achieved in improving health behavior. Each of the six areas is presented in table in terms of the degree to which USAID/Benin will need to dedicate resources given progress achieved to date for each area. Accordingly, each area is ―scored‖ in terms of whether, under a new USAID project, the contractor will need to devote a minimal, low, medium, or high level of resources and effort. to that specific technical area. It should be noted that the table does not suggest that any of the specific areas should receive a higher priority than another. Considerations for the six areas are discussed in detail in subsequent paragraphs. 56 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Table 41. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Improved Health Behaviors Social Marketing Four Green BCC (Mass Media, IPC) Three Blue Social Mobilization Three Blue Community Outreach Two Gray Research-Based Strategic Planning One Rose Gender Equity One Rose Social Marketing IMPACT has successfully developed social marketing to make products widely available through multiple outlets and to create demand through strategic communications. The evolution of the Beninese NGO ABMS into a self-sustaining entity with access to PSI technical assistance in social marketing should continue. Expanding those groups and individuals who can increase income through social marketing, such as Tontins,3 women‘s groups, and CHWs, should be considered. Behavior Change Communication (Mass Media, IPC) The work done by both IMPACT and PISAF with community radio, television, and interpersonal communications should be continued and expanded. The progress made with training, materials development, peer education, outreach by both civil society partners and government health staff should be consolidated. Progress made in IMPACT and PISAF partnering with NGOs, CHWs, women‘s groups, community radio stations, and peer educators should be expanded as much as possible to increase opportunities for sustainability. Links must be made with other interventions to benefit from BCC initiatives covering the same content and target populations to avoid duplication and allow all to benefit from progress made by PISAF, IMPACT, and the new project. For example, BCC strategies and materials developed by the USAID-funded Women‘s Justice and Empowerment Project on combating sexual violence have potential for use in family health interventions. Social Mobilization Both IMPACT and PISAF developed strategies for engaging civil society. IMPACT engaged the private sector and partnered extensively with local NGOs. It was less successful with the creation of municipal level health committees. PISAF made progress supporting the network of CHWs and supported community health committees in addressing village-level SONU to identify and avoid risk for pregnant women and newborn children. At the same time, PISAF has faced significant obstacles with the viability of mutuelles. Both projects worked with women‘s 3 Tontins are associations of women formed to pool funds to buy raw materials, such as foodstuffs, which they then divide amongst themselves and sell in smaller quantities. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 57 groups and community radio stations. The new project should consolidate social mobilization strategies in consideration of reach, sustainability, ability to be replicated, and cost-effectiveness. Community Outreach While community outreach has a clear link with BCC, community outreach focuses on establishing the link between health services and the communities they serve. IMPACT successfully conducted community outreach to MARPS for HIV and linked them to user-friendly public health services. PISAF supported outreach to women‘s groups and others on a community-based response to emergency obstetrics and neonatal care. These efforts represent avenues for building links between communities and health services. Considering the limited scope of these successful community outreach interventions, impact nationally was not expected. Full technical support for community outreach in the new project will rectify limited coverage. Research-based Strategic Planning IMPACT‘s strong emphasis on solid research to support its social marketing and other initiatives allowed the project to make midcourse corrections. Research-based strategic planning was one of the project‘s most notable strengths. Progress was made in developing civil society research expertise through training, subcontracting, and monitoring of research firms by IMPACT. PISAF‘s use of its findings from the EQGSS to identify areas in which to focus capacity development of health care workers helped it to define its program of collaboratives and to make important progress in the reduction of maternal mortality. LOGIGRH and Medistock, computer programs introduced in Benin by PISAF to manage human resources and commodities respectively, have been adopted by the GoB. Support for the continued use of data for strategic planning should continue and be expanded upon in the new project. In a climate of limited resources, the importance to the new project of linking strategic planning to issue-oriented research cannot be overstated. Gender Equity While not costly, expertise in gender equity is vital to the success of future work on family health in Benin. Establishing a mechanism to review all family health interventions with a gender perspective is advised. Given that gender is a crosscutting issue with a potential impact on all initiatives, the designation of a position devoted to gender within the new project would be warranted. Gender issues to consider include the following: Women‘s unmet health needs including needs assessments Sexual violence, including links to the Women‘s Justice and Empowerment Project to combat violence against women Women‘s economic dependence, including how it impacts their ability to access health services and strategies for improving women‘s access to financial resources Illiteracy and education levels of women, including links to ongoing USAID projects in the education sector to expand education opportunities for girls Income-generating options for low-income women, including social marketing Sex education within and outside of schools, including working in collaboration with Beninese NGOs such as RABEJ Female genital-cutting reduction through lessons learned from the work within Benin and internationally4 4 Diop, Nafissatou J., Amadou Moreau, and Hélène Benga. 2008. ―Evaluation of the long term impact of the Tostan programme on the abandonment of FGM/C and early marriage: Results from a qualitative study,‖ FRONTIERS Final Report. Available on website at www.popcouncil.org/frontiers. See also Ndiaye, Salif, 58 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Involvement of men in reproductive health, and increased VCT for men as well as establishing VCT for couples INTEGRATED HEALTH: PRIORITY INTERVENTIONS As illustrated in the following table, six principal priority interventions are recommended as ways to build upon progress achieved in integrating family health interventions. Each of the six areas is presented in the table in terms of the degree to which USAID/Benin will need to dedicate resources given progress achieved to date for each area. Accordingly, each area is ―scored‖ in terms of whether, under a new USAID project , the contractor will need to devote a minimal, low, medium, or a high level of resources and effort to that specific technical area. It should be noted that the table does not suggest that any of the specific areas should receive a higher priority than another. Considerations for the six areas are discussed in detail in subsequent paragraphs. Table 42. Evaluation of USAID/Benin Family Health Activities—Future Directions Recommendation Proposed Level of Project Resource Allocation Minimal Low Medium High Rose Gray Blue Green Integrated Health Priority Interventions IMCI (Diarrheal Disease, ARI, Immunizations) Four Green F P Four Green MNCH Four Green HIV/AIDS/STI Two Gray Nutrition One Rose Water and Sanitation Two Gray Integrated Management for Childhood Illness (IMCI) This area of intervention, which includes diarrheal diseases, ARI, and immunizations, is key to family health. Both PISAF and IMPACT have made strides to address IMCI topics. Increased provider and community outreach work on IMCI is needed. The incidence of diarrheal diseases could be reduced by increased attention to water and sanitation in addition to reducing women‘s workloads by improving access to water and the availability of clean water (please refer to section below on water and sanitation). Close collaboration is warranted with the activities under the President‘s Malaria Initiative (PMI), as young children are particularly vulnerable to mortality from malaria, and malaria detection and treatment are part of the IMCI protocol. The new project should also continue support for the initiatives undertaken by IMPACT to increase the use of Orasel/Zinc to combat childhood diarrhea. The new project would also benefit from focused research on ways in which to increase client compliance with respect to the recommended zinc regimen. Nafissatou Diop, and P. Stanley Yoder. 2007. See also, ―Évaluation à long terme du programme de Tostan au Sénégal: Régions de Kolda, Thiès et Fatick." Calverton, MD: Macro International. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 59 Family Planning The new project should place a strong emphasis on increased access to method-mix, something that appears to be lacking within Benin‘s health services, as research has shown that the more methods available, the more likely FP will be adopted. Increased attention to messages, both within services and through IPC/mass media, that provide accurate information on the advantages and disadvantages (including side effects) of various contraceptive methods is needed. The project should increase counseling on the advantages of contraception and the importance of respect for a woman‘s choices. This, in turn, could decrease unmet need. Reducing medical barriers to contraceptive use, such as the need to menstruate before being prescribed hormonal contraceptives, should be included in the new project‘s technical approach. Messages on dual method use to prevent unintended pregnancies and HIV transmission and acquisition are needed. Continuing to reach out to men as well as women with messages on the economic advantages of having fewer than the national average of 5.7 children should be emphasized. Continued emphasis should be given both to service providers and community outreach messaging on the right of women to access contraception with or without their husband‘s permission. Providing access to contraception for young women, including emergency contraception, through youth-friendly services and outreach could reduce the high number of abortions, which are illegal in Benin. Maternal, Neonatal, and Child Health Community-based interventions for neonates, such as those introduced by PISAF in its village￾based SONU program, should continue to receive emphasis under the new project. To reduce the current high rates of maternal mortality, the creation of no-fee emergency transport options is needed. Mechanisms to reduce financial barriers to Emergency Obstetric Care (EmOC), particularly for the poorest of the poor, should be considered. Training to increase the number of skilled birth attendants would greatly reduce maternal mortality. PISAF progress supporting pregnant women in the postpartum period and reducing postpartum hemorrhage needs to be continued and replicated on a national scale to reduce maternal deaths. Increased community outreach and mass media messages on the danger signs in pregnancy, birth, and postpartum that require immediate access to EmOC are also needed. Improving respect and interpersonal communication techniques of providers working with pregnant women is needed. The provision of privacy for birthing may also increase the likelihood of women accessing services when needed. Ensuring that facilities that can manage obstetric emergencies have adequate equipment, supplies, and trained staff is also a key to reducing the high rates of maternal mortality in Benin. Close collaboration is warranted with the activities under the PMI) as pregnant women and young children are particularly vulnerable to mortality from malaria. For pregnant women who are HIV positive, increased attention is needed to ensure that they receive ARVs. HIV/AIDS Many donors currently work in Benin on HIV/AIDS, but USAID should continue the work that IMPACT has done with outreach to MARPS and reducing HIV/AIDS stigma through tools developed and tested by the International Center for Research on Women (ICRW). While currently post-exposure prophylaxis (PEP) is only offered to health providers who have suffered from accidental needle-sticks, it could also be offered to rape survivors to reduce the likelihood of HIV acquisition. USAID should continue in its role as a catalyst to ensure that the GoB follows through on commitments made when accepting Global Fund monies, such as instituting sex education within all schools. Involving the NGO RABEJ is critical to the success and sustainability of youth interventions. 60 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Nutrition As indicated in the above table, the evaluation team has assessed that the amount of investment accorded to nutrition considerations should be minimal. This assessment is not to imply that nutrition is not a consideration of primary importance. Rather, the evaluation team has assessed that, given the amount of donor resources available and needed to address nutrition concerns, USAID/Benin‘s new project would do best to establish a technical link with projects whose primary focus is on responding to the nutritional needs of Benin‘s population, specifically of mothers, newborns, and children under five years of age. The key here is to develop the capacity of the new project to coordinate with such projects and to leverage their resources as the new project seeks to develop integrated health initiatives. Toward that end, the new project would do well to have sufficient technical expertise to coordinate with agricultural projects and with community food distribution projects. Within the new project itself, technical expertise, either long-term or short-term, could address the issues of prevention of maternal anemia and tracking the nutritional needs of neonates and children under five. Water and Sanitation IMPACT/POUZN‘s work with the socially marketed water purification tablet Aquatab should be continued. USAID will need to be a catalyst with other donors to ensure increased access not only to clean water but also to water in general. Working in collaboration with other donors on tube, well, and cistern activities should be explored to increase water access as well as ensure that BCC interventions with messages on hand-washing and other essential hygiene interventions accompany such activities. Coordinated messages between USAID contractors and other donors on these topics will be needed. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 61 ANNEX A. SCOPE OF WORK—EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES Statement of Work Evaluation of Benin Family Health Activities Final 1/5/2011 I. BACKGROUND USAID/Benin Family Health Program USAID/Benin supports a family health program which addresses family planning, maternal and child health, infectious diseases, and HIV/AIDS through the delivery of integrated, quality family health services and ensuring that these are available at all service delivery points. The Family Health strategic objective for 2006-2011 is ―Expanded Use of Family Health Services and Preventive Measures in a Supportive Policy Environment.‖ USAID health activities focus on 1) creating a supportive policy environment; 2) increasing access to quality services and products; and 3) increasing demand for health services and products. Integrated Family Health activities under the Health Program focus on a variety of interventions aimed at expanding the use of a minimum package of family health services. These services include: (1) treatment and prevention of childhood illnesses with a focus on malaria, diarrhea, respiratory infections and immunization; (2) services to mothers and babies to ensure pre-natal care for a safe pregnancy, assisted delivery, post-partum care, and health of newborns, including essential and emergency obstetric care (safe delivery through active management of the third stage of labor [AMTSL] and treatment of postpartum hemorrhage), essential newborn care, prevention of mother to child transmission of HIV/AIDS and IPTp; (3) malaria prevention and treatment; (4) family planning services, including expanding community-based programs and private sector involvement; and (5) prevention, correct diagnosis, and treatment of HIV/AIDS and other sexually transmitted infections, including promotion of voluntary counseling and testing services. Development Hypothesis The development hypothesis guiding USAID Benin‘s health strategy is that improvements in access, quality, and demand for services will lead to the increased use of services and that this expanded use of a package of family health services will result in a healthier population and a strengthened human resource base, thereby contributing to reduced morbidity and mortality. To achieve this, an enabling policy environment that allows for the decentralized management of sustainable family health services and the coordination of donors and private sector providers is necessary; affordable services and products need to be made available in order for them to be used; these services must be delivered in a manner which is respectful of the client and which promotes confidence in the safety and effectiveness of the services; and the customer‘s awareness and knowledge of the services and products must increase in order for them to see the value of adopting new health behaviors. A supportive policy environment refers to the need to ensure that adequate policies, management systems, trained personnel and community participation and oversight are in place. It also refers to the importance of establishing behavior change communications aimed at creating a favorable socio-cultural context. 62 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES ENVIRONMENT Some of the anticipated results of the USAID Family Health Program are that: The Ministry of Public Health will increase its ability to provide adequate health care to the people of Benin through continued improvements in capacity to plan, budget, train and supervise staff; The quality of service delivery will be improved; More people will have access to a minimum package of family health care services to protect their health and the health of their families; More people will take appropriate actions to protect their own health as well as the health of their children; Communities will be more assured of having financial access to health services; Communities will effectively contribute to oversight of planning, implementation, monitoring and evaluation of health services; and Collaboration among representatives of local government, Ministry of Public Health, and civil society in the planning and management of local health services will be improved. SO 5: EXPANDED USE OF FAMILY HEALTH SERVICES & PREVENTIVE MEASURES IN A SUPPORTIVE POLICY IR 1: A Supportive Policy Environment Created IR 2: Access to Quality Services and Products Increased IR 3: Demand for Health Services, Products and Prevention measures Increased IR 1.1Selected health policies and approaches implemented IR 1.2: Increased health system management capacity created IR 1.3: More effective civil society participation IR 2.1: Selected products available at public sector facilities IR 2.2: Selected products available at private sector outlets IR 2.3: Quality family health package of services available at targeted public sector facilities IR 2.4: Financial access to health services increased IR 3.1: Knowledge of appropriate behaviors and preventive measures improved IR 3.2: Appropriate research-based interventions and services introduced USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 63 Country Context Benin has a population of approximately 9 million, 45% of whom are below the age of 15. Benin ranked 161 out of 182 countries on the 2009 United Nations Human Development Index and had a gross national income per capita of $1,250 USD. Educational levels are low – six in ten women and four in ten men have had no schooling. For the period from 2001–2006, the infant mortality rate was 67 per 1,000 live births, the under-five mortality rate was 125 per 1,000 live births, and the maternal mortality ratio was 397 per 100,000 live births (with estimates taking into account issues of undercounting running as high as 850). Infant and child mortality rates in Benin are among the highest in coastal West Africa due mainly to preventable childhood illnesses, especially malaria, acute respiratory infections (ARI), and diarrhea. Malaria, endemic nationwide, is a major cause of morbidity and mortality in Benin. Benin has a high population growth rate with the average Beninese woman having five to six children in her lifetime. Unmet need for family planning is high. Actual family size continues to exceed desired family size while use of modern contraception is less than 10%. With an annual growth rate of almost 3%, Benin‘s population will double in the next 24 years. Any gains in economic growth will be adversely affected by the increasing population, which will increase demands for social services, including health and water. These demands also increase the need for health personnel, infrastructure, medicines, etc. Benin faces challenges to improving family health due to the entrenched poverty, low levels of education and knowledge about appropriate health-seeking behaviors, and persistent weaknesses in the quality, management and delivery of health services. Over the past decade, the MOH has reorganized its structure through the creation of 34 health zones (often called districts in other African countries). These zones are designed to facilitate decentralized planning and management, as well as to facilitate the efficiency of resource allocation and the rehabilitation of referral units. Each zone covers a population of 100,000 to 500,000 inhabitants. Through this reorganization, the MOH intends to reinforce and reorient current services, promote interventions for high-prevalence diseases, and ultimately promote the effective decentralization of health services. A wide variety of health facilities, pharmacies, and other services also exist in the private sector, and most of these are situated in the urban centers. In addition, many religious institutions and some NGOs run hospitals or dispensaries, or provide training, health education, and other health services. It is estimated that the private/NGO/confessional sector actually provides at least 30% of health services in Benin. The Government of Benin has prioritized improving access and quality and is committed to a significant and lasting reduction in child and maternal mortality by ensuring the availability, accessibility and utilization of high quality family health services. Reinforcing human capital is among the Government of Benin‘s top three priorities as provided in the strategy for reducing poverty and improving growth. The country has developed a ten-year National Health Development Plan (PNDS) to guide health interventions during 2009-2018 and lead to the achievement of the Millennium Development Goals (1b, 3, 4, 5, and 6) by 2015. Benin expects to achieve the goals of the ten-year plan by providing universal access to quality health care to the population. To achieve these goals, the GoB has announced several initiatives including: 1) free cesarean sections for eligible pregnant mothers, 2) a waiver of user fees for children under-five who consult public sector facilities, 3) reinforcing health financing schemes and d) revitalizing primary health care through increasing the number and reinforcing the capacity of community health workers (CHWs) to become real partners in promoting primary health care at the village level. 64 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES However, the health sector is still facing important challenges. Among the challenges are: 1) the insufficient number of health workers at all levels, 2) the quality of care provided by health workers, and 3) the frequent stock-outs of drugs in health facilities. II. PROJECTS TO BE EVALUATED The USAID/Benin‘s Family Health program (2006-2011) includes two major activities: a five-year $19.7 million integrated family health project, Projet Intégré de Santé Familiale (PISAF), implemented in the central region of the country; and a five-year $17.6 million social marketing and HIV/AIDS prevention, Projet Intégré d’Appui à la Santé Familiale et la Prévention de VIH/SIDA (IMPACT), implemented in seven convergence zones with a national focus for social marketing activities. Complementary activities in MCH and social marketing are conducted by the Social Marketing Plus for Diarrheal Disease Control: Point-of-Use Water Disinfection and Zinc Treatment Project (POUZN). Additional activities focused on malaria under the President‘s Malaria Initiative (PMI) were added in 2008 and 2009 but are not the subjects of this evaluation. Program implementation occurs at the national, intermediate and peripheral levels. National level efforts focus on policy work, system support, and enhancing sustainability and capacity at the central level. Intermediate level efforts at the regional level focus on capacity building in management (planning, implementing, monitoring, evaluation, and coordination/supervision) and in technical areas (health workers skill strengthening, quality assurance). Peripheral level efforts focus on capacity building and service delivery within the health zone and the community. Projet Intégré de Santé Familiale (PISAF) Program Description In March 2006, USAID awarded the Integrated Family Health Project (Projet Intégré de Santé Familiale or PISAF) to URC and its subcontractor, Abt Associates, to work in partnership with Benin‘s Ministry of Health (MOH) to improve health services. This five-year cooperative agreement aims to improve family health, malaria, and HIV/AIDS services in Benin. The project focuses on strengthening the health system‘s ability to provide evidence-based family health services that meet the needs of communities and families and to help communities become more active participants in their own health and in the health system. The approach relies on a multi-pronged strategy to strengthen healthcare service systems by working in partnership with Benin‘s MOH and collaborating organizations to: Improve the policy environment through: Implementation of effective decentralization by strengthening effective planning, financial, and human resource management at the health zone level At national level, support for developing, updating, and finalizing policies on Mutuelles de Santé (community-based group insurance programs), decentralization, community mobilization, PMTCT and others that support access to quality healthcare Support development and implementation of a national QA strategy Increase access to family health services and products through: Expansion of community-based group insurance programs (Mutuelles de Sante) and community-based service provision (community health workers and civil society groups) Integrating new technical areas into the family health minimum package, including services to prevent mother-to-child transmission of HIV/AIDS, intermittent presumptive treatment for malaria, and distribution of insecticide-treated bed nets Leveraging Mutuelles as vehicles for health education and promotion and as an effective partner in the search for quality services USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 65 Improve quality of services through: Strengthening of capacity at various levels of the health system and among NGOs and civil society groups to provide high quality health services and information Development of evidence-based clinical norms and the establishment of mechanisms to revise norms on a regular basis Strengthening of health worker competencies in critical clinical areas through training and monitoring performance through facilitative supervision and reviewing outcomes Implementation of improvement collaboratives to improve service quality through the facilitation of horizontal learning and rapid sharing of tested best practices Expansion and institutionalization of quality assurance activities at all levels of the health system Increase demand for health services and prevention measures through: Implementation of community mobilization and behavior change communication efforts to stimulate demand for health services at the community level through peer education, local radio stations, participatory community theater, and materials development The main geographic focus of this project is the Zou and Collines departments in central Benin, with complementary work in Borgou and Alibori in northern Benin and the Ouémé and Plateau in the southeast to maintain select activities begun under previous USAID projects. Projet Intégré d’Appui à la Santé Familiale et la Prévention de VIH/SIDA (IMPACT) Program Description IMPACT is a five-year cooperative agreement (Oct 2006- Oct 2011) with Population Services International (PSI). PSI's main partners in implementation are JHPIEGO Corporation, Abt Associates and the local NGO OSV-JORDAN. This five-year project is designed to support the Government of Benin‘s efforts to control the spread of HIV and increase access to family health services and products through social marketing campaigns. The approach relies on a multi￾pronged strategy to strengthen healthcare service systems by working in partnership with Benin‘s MOH and collaborating organizations to: Improve the policy environment through: Strengthening the capacity of the Ministry of Health and the government of Benin to manage and implement the national HIV/AIDS program Establishment of multi-sectoral approaches and collaboration with partners to strengthen local capacity for behavioral change communication (BCC) At national level, support for developing, updating, and finalizing policies that create a supportive environment for social marketing and the provision of HIV/AIDS services. Increase access to HIV/AIDS, reproductive health/family planning (RH/FP) and Child Survival products through: Increased availability of health services and products in the private sector through expansion of a private network of clinics, PROFAM Adaptation and operationalization of a national distribution system for socially marketed family health products 66 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Improve quality of services through: Promotion and support for implementation of best practices in integrated RH and HIV/AIDS services Strengthening of capacity at various levels of the health systems and among private sector, NGOs and civil society groups to provide high quality health services and information Strengthening of health worker competencies in critical clinical areas through training and monitoring performance through facilitative supervision Expansion and institutionalization of quality assurance activities Increase knowledge about HIV/AIDS, RH/FP and Child Survival to increase demand for and use of products and services through: Implementation of community mobilization and behavior change communication efforts to stimulate demand for health services at the community level through mass media, peer education, local radio stations and other media venues, and materials development Social marketing activities are implemented nationwide and cover a wide range of health commodities including, but not limited to, condoms, oral and injectable contraceptives, and bed nets. Integrated Family Planning and HIV/AIDS services are offered in select pilot public clinics and the private PROFAM network (50 clinics). Behavior change communication activities target at risk groups in the convergence zones of Cotonou, Bohicon/Abomey, Malanville, Parakou, Ouake, Tanguiéta, and Savalou. Social Marketing Plus for Diarrheal Disease Control: Point-of-Use Water Disinfection and Zinc Treatment Project (POUZN) Launched in September 2005, the Point-of-Use Water Disinfection and Zinc Treatment (POUZN) Project is a five-year project implemented by Abt Associates, Inc. in partnership with Population Services International. POUZN supports point-of-use water treatment activities in 7 countries: Angola, Benin, Democratic Republic of the Congo, Haiti, Kenya, Malawi, and Rwanda and zinc treatment program in 3 countries: Nepal, Benin, and Madagascar. POUZN activities in Benin are implemented by PSI alongside IMPACT activities. Activities began in Benin in September 2007 and end in November 2010. Total USAID funding for POUZN activities in Benin is $1.8 million. The POUZN Project mobilizes the private sector to introduce low-cost point-of-use water treatment products for diarrhea prevention and zinc together with oral rehydration fluids for the treatment of diarrhea among children under five. The goal of the POUZN project is to significantly increase the use of POU water treatment and zinc while creating vibrant commercial markets that will allow for program sustainability. This will be achieved through the use of social marketing as a technique to increase access to and demand for POUZN products. Social marketing activities are complimented by Behavior Change Communication activities (BCC) implemented through mass media and interpersonal communication (IPC) in an effort to encourage healthy behaviors. Amongst the communication tools developed to increase knowledge of the correct treatment of diarrhea and to create demand for all of the products are the following: television and radio commercials, mobile video shows, song contests, community theater performances, and point of sales materials. Improved health practices are also promoted through home visits and edutainment sessions. Program communications also underscore the importance of hand washing, continued feeding practices, and other elements of proper diarrhea management and hygiene practices. POUZN activities will be continued by PSI under IMPACT through to the end date of IMPACT. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 67 Figure A-1. Carte Administrative du Bénin All project activities are in their last 12 months of implementation. In the third and fourth years of PISAF and IMPACT, President‘s Malaria Initiative (PMI) specific activities were added to each project‘s scope of work along with additional financial resources. In FY 2011, the PMI malaria activities will be continued under a separate project, Accelerating the Reduction of Malaria Morbidity and Mortality in Benin (ARM3). III. SCOPE OF WORK Purpose of the Evaluation The Family Health five-year strategy and main integrated health activities are coming to end. The purpose of this evaluation is to understand how the three integrated health projects (PISAF, POUZN, and IMPACT) have been implemented in terms of program intensity, quality, and coverage relative to program plans; identify intended and unintended consequences of the integrated program; document lessons that can be shared throughout the Agency to contribute Département du ZOU 1 Abomey 2 Djidja 3 Agbangnizoun 4 Covè 5 Za-Kpota 6 Zogbodomey 7 Bohicon 8 Ouinhi 9 Zagnonado 1 Aplahoué 2 Klouékanmey 3 Lalo 4 Toviklin 5 Djakotomey 6 Dogbo Département du COUFFO 1 Lokossa 2 Athiémè 3 Bopa 4 Comé 5 Grand-Popo 6 Houéyogbé Département du MONO Département de l’ATLANTIQUE 1 Abomey-Calavi 2 Allada 3 Zê 4 Kpomassè 5 Ouidah 6 So-ava 7 Toffo 8 Tori Bossito Département du LITTORAL 1 Cotonou Département de l’OUÉMÉ 6 Avrankou 7 Bonou 8 Dangbo 9 Sèmè-Podji 1 Porto Novo 2 Adjarra 3 Adjohoun 4 Aguégué 5 Akpro-Missérété Département du PLATEAU 1 Kétou 2 Ifangni 3 Pobè 4 Adja-Ouèrè 5 Sakété 68 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES to development learning and improve future programming; and identify gaps and make recommendations on strategies to address any identified gaps and improve the delivery of follow-on integrated health projects, recognizing that they will contribute to the country‘s overall health strategy. Information from the evaluation will be used by USAID and other stakeholders to develop or refine relevant policies/regulations and improve the delivery of integrated program interventions in order to achieve overall health program goals. This evaluation will provide an informed basis to assist the Family Health Team and MOH to consider options for designing follow-on integrated health activities in Benin. The evaluation‘s findings, conclusions, and recommendations will also provide essential information to the Family Health Team and wider Agency in designing quality integrated programming in MCH, Family Planning, and Nutrition under the Agency‘s Global Health Initiative. The information from the evaluation will assist the Family Health Team in Benin develop their five-year integrated action plan under Best Practices at Scale in the Home, Community and Facilities (BEST). Evaluation Questions In this descriptive evaluation/investigation, we would like to answer the following questions: 1. How closely did the projects follow plans in terms of activities planned and program intensity, quality, and coverage? 2. What were the strengths and weaknesses in implementation of the projects? 3. Which activities or aspects of the projects were most successful and which were least successful? And why? 4. What were the intended and unintended consequences of individual project activities and the integrated program overall—for beneficiaries and the health system? How successful was the integrated approach? 5. What were the lessons learned from individual project activities and about implementing integrated health programs? 6. What evidence is there that the activities implemented contributed to the intermediate results of the Family Health results framework (an improved policy environment, increased access to products and services, and increased demand for produces and services)? What evidence is there that the development hypothesis and assumptions underlying the results framework were valid—that increases in the intermediate results would lead to expanded use of services, products and preventive measures? 7. What gaps in programming were identified, in terms of specific activities and in implementing an integrated package of family health services? Were there any missed opportunities in increasing coverage, improving quality of services, integrating different interventions, and building local capacity within our programs and/or in our coordination and harmonization with other donors? 8. What specific recommendations are there for strategies to a) address any identified gaps and b) improve the delivery of program interventions for the next five-year strategy and follow-on integrated health projects? The evaluation should cover USAID family health intervention zones under the PISAF and IMPACT projects: six health zones in Zou/Collines and eight communes of Littoral, Abomey, Bohicon, Savalou, Parakou, Ouake, Tanguieta, and Malanville. Evaluation Methods This will be an external evaluation but should be conducted in consultation with USAID/Benin to ensure that the team has the fullest possible background. The key issues to be addressed by the USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 69 evaluation team should be developed in consultation with the Family Health Team. The evaluation team should begin its work with a paper review of the documents cited in the ―Sources of Information‖ section below. Following document review, a two-day team planning meeting should be held during the evaluation team‘s first two days in-country. This time will be used to clarify team roles and responsibilities, deliverables, development of tools and approach to the evaluation, and refinement of agenda. A dedicated meeting space will be provided. In the team planning meeting the team will: Share background, experience, and expectations for the assignment; Formulate a common understanding of the assignment, clarifying team members‘ roles and responsibilities; Agree on the objectives and desired outcomes of the assignment; Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; Develop data collection methods, instruments, tools and guidelines, and methodology and develop an assessment timeline and strategy for achieving deliverables; and Develop a draft report outline for Mission review and approval. In addition, the Mission is looking for new, creative suggestions regarding this evaluation, and it is anticipated that the implementer will provide a more detailed explanation of the proposed methodology for carrying out the work. The methodology will be comprised of a mix of tools appropriate to the evaluation‘s research questions. These tools may include any combination of the following: Key informant interviews Review of historical documents Facility observations Client–provider observations Client interviews Provider interviews Interviews with Ministry of Health staff Interviews with community members Focus groups or group discussions Review of data collected by implementing partners (surveys and other data collection activities) Field visits Involvement of program beneficiaries, providers and other stakeholders to review and interpret the findings and make recommendations for program refinement Sources of Information There are already a number of documents available for review by the evaluation team including project reports and studies. Electronic copies of most of these documents can be provided in advance to the evaluation team. Paper copies will also be available in-country at USAID‘s office. The following key documents will be provided: 1. PISAF Program Description 70 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 2. IMPACT Program Description 3. POUZN Program Description 4. Family Health Program PMP 5. PISAF PMP 6. IMPACT PMP 7. POUZN PMP 8. PISAF Annual work plans 9. IMPACT annual work plans 10. POUZN annual work plans 11. PISAF Quarterly and annual reports 12. IMPACT Quarterly and annual reports 13. POUZN Quarterly and annual reports 14. Evaluation de la Qualité de la Gestion du Système Sanitaire (EQGSS) conducted as baseline in 2006 15. Draft EQGSS II (conducted in Aug-Sept 2010) 16. TRaC surveys: – HIV/AIDS: Sex workers, truck drivers, students and apprentices (2007 and 2009) – Family Planning (2008) – Diarrheal disease (2009) – Malaria (planned for November 2010) Other relevant documents will be provided as needed during the evaluation. Deliverables and Level of Effort Deliverables: Workplan: The team will prepare a detailed workplan which will include the methodologies to be used in the evaluation. The work plan will be submitted to the Family Health Team at USAID for approval no later than the third day of work and prior to beginning key informant interviews or site visits. Debriefing with USAID: The team will present the major findings of the evaluation to USAID/Benin and USAID/Washington staff in-country through a PowerPoint presentation before the team's departure from country. The debriefing will include a discussion of achievements and issues as well as any recommendations the team has for possible modifications to project approaches, results, or activities. The team will consider USAID comments in completing the draft report. Draft Evaluation Report: A draft report of the findings and recommendations should be submitted to the USAID CTO prior to team departure from country. The draft report will incorporate feedback and comments from the debriefing. The written report should clearly describe findings, conclusions, and recommendations. USAID will provide comments on the draft report within ten working days of receiving the draft report. Final Report: The team will submit a revised report that incorporates the team responses to Mission comments and suggestions no later than five days after USAID/Benin provides one set of written comments on the team's draft evaluation report. The format will include an executive summary, table of contents, methodology, findings, and recommendations. USAID will complete a final review and the Evaluation Team leader will respond to any last comments from USAID USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 71 and submit the final draft. The report will be submitted in English, electronically. The report will be disseminated within USAID. If recommendations in the report include procurement sensitive information, that information will be pulled out and put into a separate, internal, unformatted memo. The full report (with the sensitive information taken out) can be made publically available. Level of Effort: Activity/Task LOE (TL/team members) Review background documents & offshore preparation work. 3 days Travel to Benin 2 days Team Planning Meeting 2 days Meet with Family Health Team at USAID/Benin 1 day Information and data collection, key informant interview, site visit travel (est. 8-10 outside Cotonou) 14 days Briefing with USAID Mission to share direction of findings and provide an opportunity to advise on issues not considered. 1 day Discussion, data analysis and draft PowerPoint presentation on findings and recommendations; draft report preparation 5 days Debrief meeting with USAID/Benin 1 day Revise report and submit prior to departure from country 2 days Depart Benin 2 days USAID provides comments on draft (within 10 working days of report submission) Team reviews comments & revises report (Team Leader 3 days; members 2 days) 3 days/2 days USAID completes final review and submits final comments to GH Tech for TL Final revision/Edit, if necessary, and submission by Team Leader 1 day GH Tech edits and formats final report (3-4 weeks) TOTAL # days (Team Leader/Members) 37 days / 35 days * A six day workweek is approved when team is working in country. See attached timeline for specific project dates. Evaluation Team Composition USAID/Benin anticipates that the evaluation team will consist of a number of positions, including an evaluation specialist that will act as the team leader and 3-4 international and/or local consultants. 72 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES The team should include specialists with the following areas of expertise: family planning/reproductive health, maternal/child health, HIV/AIDS prevention, behavior change communication/community mobilization, capacity building/health system strengthening (particularly areas such as commodities management, community–based financing, planning and data use). As team member, the team should include at least one local consultant who has an excellent understanding of the Beninese health system and is fluent in French. At least one member should also have strong expertise in monitoring and evaluation. Team Leader/Technical Specialist: Should be an independent consultant and have an MPH or related postgraduate degree in public health. S/he should have at least 10 years senior-level experience working in health systems programs in a developing country. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required. The team leader should also have experience in leading evaluation teams and preparing high-quality documents. This specialist should have wide experience in implementation of USAID-funded MCH/FP/RH/HIV/AIDS programs. S/he should also have a good understanding of project administration, financing, and management. The team leader will take specific responsibility for assessing and analyzing the project‘s performance, factors for such performance, and benefits/impact of the strategies. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Benin Family Health Team. Family Planning/Reproductive Health, Maternal/Child Health, and HIV/AIDS Specialists: The FP/RH, MCH and HIV/AIDS specialists will have at least 7-10 years of experience in management of, or consulting on, FP/RH, MCH and HIV/AIDS programs. S/he should have a proven background and experience in FP/RH, MCH and HIV/AIDS and a strong understanding of the challenges facing FP/RH, MCH and HIV/AIDS programs in Benin or West Africa. S/he should also have a good understanding of the relevant national programs in FP/RH, MCH and HIV/AIDS, including the public and private sector. The FP/RH, MCH and HIV/AIDS specialists will be responsible for assessing the performance of the project and strategies used and provide technical leadership in FP/RH, MCH and HIV/AIDS. They will document key lessons learned and provide recommendations for modifications in approach or strategies and gaps to be addressed thorough future programming. More than one specialty area can be covered by the same consultant if appropriately qualified. Behavior Change Communication/Community Mobilization Specialist: This specialist should have wide experience in implementation of behavior change communication and community mobilization programs in the areas of FP/RH, MCH and HIV/AIDS. S/he should have a postgraduate degree in health promotion sciences or a related field with a minimum of 5-10 years experience working with behavior change and community mobilization programs in developing countries. S/he will analyze behavior change interventions in the communities and assess the effectiveness and appropriateness of the approaches adopted by the project to improve FP/RH, MCH and HIV/AIDS knowledge, health-seeking behavior, and health outcomes. S/he will also assess the technical foci of BCC activities, and whether they are the appropriate mix and topics for the intervention communities. Capacity Building/Health System Strengthening Specialist: This specialist should have at least 5-10 years experience working in the areas of capacity building/health system strengthening. S/he should have a good understanding of health systems in West Africa, preferably in Benin. S/ will look at the sustainability of the approaches and activities as well as the ability to leverage and USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 73 influence MOH programming, including adoption and institutionalization of project activities and innovations. Logistics Logistical Support: The Evaluation Team shall be responsible for arranging travel to and from Benin and providing computers for team members. Implementing partners and USAID will provide assistance in arranging meetings with project staff, GOB, program beneficiaries, and other key stakeholders. The Mission will make available all relevant documents. USAID/Benin will provide overall direction to the evaluation team, identify key documents, and assist in facilitating a work plan. A local logistics coordinator will be hired to arrange in-country travel, set up meetings with key stakeholders identified by USAID prior to the initiation of field work, and other logistics as needed. During field work, USAID and Implementing Partners will provide assistance as needed to arrange other meetings as identified during the course of this evaluation. USAID/Benin can assist with hotel arrangements if necessary but the evaluation team will be responsible for providing its own work space and computers. USAID/Benin personnel will be made available to the team for consultations regarding sources and technical issues, before and during the evaluation process. Additional mission responsibilities/roles include the following: Before In-Country Work 17. Consultant Conflict of Interest. To avoid conflicts of interest or the appearance of a COI, review previous employers listed on the CV‘s for proposed consultants and provide additional information regarding potential COI with the project contractors or NGOs evaluated/assessed and information regarding their affiliates. 18. Documents. Identify and prioritize background materials for the consultants and provide them, preferably in electronic form. 19. Local Consultants. Assist with identification of potential local consultants and provide contact information. 20. Site Visit Preparations. Provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line items costs. The Ministry may be helpful in setting up initial stakeholder meetings. 21. Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in￾country travel (i.e., car rental companies and other means of transportation) and identify a person to assist with logistics (i.e., visa letters of invitation etc.) if appropriate. During In-Country Work 22. Mission Point of Contact. Throughout the in-country work, ensure constant availability of the Point of Contact person(s) and provide technical leadership and direction for the team‘s work. 23. Meeting Space. Provide guidance on the team‘s selection of a meeting space for interviews and/or focus group discussions (i.e. USAID space if available, or other known office/hotel meeting space). 24. Meeting Arrangements. While local consultants typically will arrange meetings for contacts outside the Health Office, support local consultant(s) in coordinating meetings with stakeholders. 74 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 25. Formal and Official Meetings. Arrange key appointments with national and local government officials and accompany the team on these introductory interviews (especially important in high-level meetings). 26. Other Meetings. If appropriate, assist in identifying and helping to set up meetings with local professionals relevant to the assignment. 27. Facilitate Contact with Partners. Introduce the Evaluation Team to implementing partners, local government officials, and other stakeholders, and where applicable and appropriate prepare and send out an introduction letter for team‘s arrival and/or anticipated meetings. After In-Country Work 28. Timely Reviews. Provide timely review of draft/final reports and approval of the deliverables Reporting and Dissemination The findings, conclusions, recommendations, and suggestions on future directions resulting from the evaluation will be presented in a presentation at a full briefing with USAID/Benin and Washington staff before departure of the evaluation team. The format for the evaluation report is as follows: 29. Executive Summary—concisely state the most salient findings and recommendations (2 pp); 30. Table of Contents—(1 pp); 31. Introduction— purpose, audience, and synopsis of task (1 pp); 32. Background—brief overview of projects, USAID program strategy and activities implemented in response to the problem, purpose of the evaluation (2-3 pp); 33. Methodology—describe evaluation methods, including constraints and gaps (2-3 pp); 34. Findings/Conclusions/Recommendations—for each objective area; and also include data quality and reporting system that should present verification of spot checks, issues, and outcome (20-25 pp); (Note: separate Recommendations from Findings/Conclusions and limit to 2 pp) 35. Issues—provide a list of key technical and/or administrative issues, if any (2 pp); 36. Lessons Learned—(2 pp) 37. Future Directions—(2 pp); 38. References—(including bibliographical documentation, meetings, interviews and focus group discussions); 39. Annexes—useful for covering evaluation methods, schedules, interview lists, tables and any other relevant materials including evaluation data collection instruments. The final version of the evaluation report will be submitted to USAID/Benin in hard copy as well as electronically. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins 1" top/bottom and left/right. The report should not exceed 40 pages, excluding references and annexes. After the debrief meeting, the evaluation team shall incorporate oral comments received from USAID into the first draft of the report. Prior to departure, the evaluation team will send electronically the 1st draft to the Mission for written comments. USAID/Benin will provide GH Tech and the team leader with written comments within ten business days of receiving the team‘s first draft. Within five days of receiving one consolidated set of comments from the Mission, the Evaluation Team, through the Evaluation Team Leader, will send the revised report. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 75 Once the mission signs off on the final unedited report, GH Tech will have the documents edited and formatted and will provide the final report to USAID/Benin for distribution (2 hard copies and CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. This will be a public document. The USAID Mission will arrange for translation of the executive summary into French. Budget Estimate—TBD Level of effort: Approximately 37 days for the team leader, and 35 for the team members. Field visits: Estimated number of days in the field (outside of Cotonou) is 8-10 76 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 77 ANNEX B. USAID/BENIN FAMILY HEALTH ACTIVITIES EVALUATION JANUARY– FEBRUARY 2011: RESPONDENT LIST Name Organization Position Location Date USAID Benin Milton AMAYUN USAID/Benin Team Leader, FHT Cotonou 1/24/2011 Kevin ARMSTONG USAID/Benin Mission Director Cotonou 2/17/2011 Emile BONGO USAID/Benin PMI Adviser Cotonou 1/24/2011 Simplice TAKUBO USAID/Benin Commodities and Logistics Specialist Cotonou 1/24/2011 Abdou Salam GUEYE USAID/Benin CDC Adviser Cotonou 1/24/2011 Michele SEIBOU USAID/Benin Deputy Team Leader Cotonou 1/24/2011 Scott STOFEL USAID/Benin Assistant Director Cotonou 2/17/2011 Ministry of Health Officials Dr. Valere GOYITO Ministry of Health Secretary General Cotonou 1/27/2011 Dr. Marcellin M.A. AYI Ministry of Health Chief of Community Health Services Cotonou 1/27/2011 Mr. Adrien I. ADE Ministry of Health Health Economist - Community Health Services - Public Health Directorate Cotonou 1/27/2011 Ancel ADJEIN Ministry of Health Administrator - Community Health Services - Public Health Directorate Cotonou 1/27/2011 Alimatou ZOHOUN Ministry of Health In charge - Contraceptive Logistics for the Family Planning Service Cotonou 1/27/2011 Dr. Imorou YACOUBOU Ministry of Health Coordinator of the National Programme for the Fight Against Malaria (PNLP) Cotonou 1/25/2011 Dr. Gaston D. AHOUNOU Ministry of Health Chief of Reproductive Health Services of the Direction of Mother and Child Health Cotonou 2/4/2011 Dr. Olga A. HOUINATO Ministry of Health Child Survival Program Cotonou 1/27/2011 Dr. Evelyne AKINOCHO Ministry of Health Coordinator of the National Programme for the Fight Against AIDS (PNLS) Cotonou 1/27/2011 Dr. Gonte SHOUNOZA Ministry of Health Chef de Service/DSME Cotonou 2/17/2011 Alimatou ZOHOUN Ministry of Health Chef, Division Logistique Contraceptive/DSME Cotonou 2/1/2011 Marguerite ZOLIKPO Ministry of Health Retired-Direction of Mother and Child Health Cotonou 1/27/2011 78 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Name Organization Position Location Date Stanislas WHANNOU Ministry of Health Permanent Secretary of Country Coordinating Cotonou 2/2/2011 Dr. Sossa E.GBEDO Ministry of Health Director of Zou-Collines Health Department Bohicon 1/31/2011 Dr. Justin A.S. AKOHA Ministry of Health Chief of the Family Health Service of Zou-Collines Health Department Bohicon 1/31/2011 Theodore ALOMA Ministry of Health Chief of the Studies, Planning and Monitoring Service of Zou-Collines Health Department Bohicon 1/31/2011 Marcien CADASSOU Mayor's Office Member, Local Technical Committee (CTL) Bohicon 2/2/2011 Leonie O. DOSSOU Mayor's Office Member, Local Technical Committee (CTL) Bohicon 2/2/2011 Maryse AGOUA Chief of Social and Health Affairs Service Parakou Donor Representatives Dr. Gilbert VISSOH UNICEF Specialist of Health PCIME Project Cotonou 2/7/2011 Dr. Alban QUENUM UNICEF Health Specialist Cotonou 2/7/2011 PISAF Aurélie AZOGAN PISAF Assistante Administrative/PISAF Bohicon 1/26/2011 Franck BADOU PISAF Spécialiste CIP/Plaidoyer/PISAF Bohicon 1/26/2011 Marthe AKOGBETO PISAF Director for Prise -C Bohicon 2.17.11 Cyrille ZOUNGAN PISAF Conseiller en Suivi￾Evaluation/ PISAF Bohicon 1/26/2011 Valentin TOSSE PISAF Conseiller en Systèmes d‘Information et de Logistique/ PISAF Bohicon 1/26/2011 Sara RIESE URC Progamme Officer Bohicon 126/11 - Célestin DJIVOH PISAF Conseiller en Décentralisation Renforcement et Gestion des Zones Sanitaires/PISAF 1/26/2011 Estelle CODO PISAF Conseiller en Santé Familiale/PISAF Bohicon 1/26/2011 Jean A. AFFO PISAF Conseiller en Communication pour le Changement de Comportement/ PISAF Bohicon 1/26/2011 Gaston KEKIN PISAF Conseiller en Assurance Qualité/PISAF Bohicon 1/26/2011 Michelle KOULETIO PISAF Directeur Adjoint/PISAF Bohicon 1/26/2011 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 79 Name Organization Position Location Date Frantz SIMEON PISAF Directeur/PISAF Bohicon 1/26/2011 Tisna Veldhuijzen van ZANTEN URC Senior Vice-President Bethesda, Maryland 2/10/2011 Barabara TURNER URC President Bethesda, Maryland 2/10/2011 Jacques Akpovi HOUNGNONVI PISAF Spécialiste en Financement de la Santé/PISAF Bohicon 1/26/2011 Pascal SOGLOHOUN PISAF Team Leader/Financement de la Santé Bohicon 1/26/2011 Sylvie T. AMADOU PISAF Conseiller Technique/PISAF Bohicon 1/26/2011 IMPACT Marguerite NDOUR ABMS-PSI Responsable VIH/SIDA/SR Cotonou 1/25/2011 Clarisse GOMEZ ABMS-PSI Coordinator of Community Activities Cotonou 1/25/2011 Prudence AYIVI ABMS-PSI SR Coordinator Cotonou 1/25/2011 Leonce A. DOSSOU ABMS-PSI Communication specialist Cotonou 1/25/2011 Koami GOUTON ABMS-PSI Communication Coordinator Cotonou 1/25/2011 Amour A. BALOGOUN ABMS-PSI Service Chief for Quantitative Research Cotonou 1/25/2011 Jules HOUNTONDJI ABMS-PSI Child Service Coordinator Cotonou 1/25/2011 Prince Renaud ETCHI ABMS-PSI Coordinator of Distribution and Decentralized Activities Cotonou 1/25/2011 Bienvenu WAKPO ABMS-PSI Malaria Coordinator Cotonou 1/25/2011 Paula AGBEMAVO ABMS-PSI Public Relations Specialist Cotonou 1/25/2011 Antoninette DOMINGO OSV-Jordan AAF/OSV-Jordan Cotonou 1/25/2011 Edmond KIFOULY ABMS-PSI Technical Adviser for Strategic Communications - Objective 4 Cotonou 1/25/2011 Josephat AVOCE OSV-Jordan Executive Director - OSV￾Jordan Cotonou 1/25/2011 Irenee KOTCHOFA Abt Technical Adviser for Institutional Capacity Development - Abt/Impact Cotonou 1/25/2011 Achille KOUNOU ABMS-PSI Deputy Director and Operations Coordinator for HIV/AIDS Cotonou 1/25/2011 Hilaire TOKPLO ABMS-PSI Responsible for Marketing and Communcations Cotonou 1/25/2011 Njara RAKOTONIRINA ABMS-PSI Counseling and Testing Cotonou 1/25/2011 80 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Name Organization Position Location Date Mathurin LOUGBEGNON JHPIEGO Coordinator of Integrated Services Cotonou 1/25/2011 Leger FOYET ABMS-PSI Chief of Party - IMPACT Cotonou 1/25/2011 Health Service Personnel Dr. Emmanuel AGBATCHOSSOU Ministry of Health Medecin Coordonnateur of COZO Zone Cozo 1/31/2011 Marius W. KOYA Ministry of Health Logistician Cozo 1/31/2011 Inoussa ASSOUMA Ministry of Health Director of Cove Zonal Hospital Cove 1/31/2011 Dr. Antoine AZON Ministry of Health Medical Director Cove Zonal Hospital Cove 1/31/2011 Paul SEGBEDJI Ministry of Health Director of the Zonal Hospital, Savalou/Bante Savalou 2/1/2011 Thomas DJAGOUN Ministry of Health Focal Point for Community Agents and for Mutuelles Savalou 2/1/2011 Marie NDEYE Ministry of Health Medical Coordinator for Zonal Hospital, Savalou/Bante Savalou 2/1/2011 Maxime KOUTANGNI Ministry of Health Pediatrician - Zonal Hospital, Savalou/Bante Savalou 2/1/2011 M'zeyi CRECEL Ministry of Health Midwife - Zonal Hospital, Savalou/Bante Savalou 2/1/2011 Edith DJENONTIN Ministry of Health Medical Director, Zonal Hospital GLAZOUE Glazoue Dr. Justin DJIDONOU "Zone Sanitaire DASSA-GLAZOUE" Doctor Coordinator DASSA - GLAZOUE 2/1/2011 Dr. Chakirou BELLO "Zone Sanitaire BOHICON - ZAKPOTA" Doctor Coordinator BOHICON - ZAKPOTA 2/2/2011 Raymond FANDOHAN Royal FM General Manager Bohicon 2/3/2011 Community Representatives Franck DAGBELOU Farmer Community Health Worker (Relais) Klongo Logozohe, Savalou 2/1/2011 Jacques ADEHA Farmer Community Health Worker (Relais) Sekame, Achakpa 2/1/2011 Jerome MIGAN Farmer Community Health Worker (Relais) Attah, Oke￾Owo 2/1/2011 Osseni ODJOUTCHOUI Farmer Community Health Worker (Relais) Segbeya, Lahotan 2/1/2011 Gilbert GBETANGNI Farmer Community Health Worker (Relais) Agbodranfo, Ouesse 2/1/2011 Pierre NANNANMOUWA Farmer Community Health Worker (Relais) Ayede, Fita 2/1/2011 Antoine TOGNI Farmer Community Health Worker (Relais) Lohoue Lohoue, Aklampa 2/1/2011 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 81 Name Organization Position Location Date Jonas TOSSA Savalou Commmune Vice President of the Union of Mutuelles of Savalou Savalou 2/1/2011 Gabriel TOBOSSOU UCMS Zagnanado President Zagnanado 2/3/2011 Célestin DOVONOU UCMS Zagnanado vice President Zagnanado 2/3/2011 Edouard TOKPO UCMS Zagnanado Trésorier-Adjoint Zagnanado 2/3/2011 Cathérine TOSSOU CBO/ IFEDOUN ASSIKO President Miscellaneous Anatole ADAHOU Radio Cite Programme Chief Savalou 2/1/2011 Christophe BATA Radio Cite Administrator Savalou 2/1/2011 Marcien CADASSOU Local Technical Committee Member Stanisla S. WHANNOU Country Coordinating Mechanism - Global Fund Permanent Secretary Cotonou 2/2/2011 Flore M. AZONDEKON PROFAM Clinic "Mon Etoile" Midwife Cotonou 2/3/2011 Christophe TOZO CEBAC Secretary General and Spokesperson Cotonou 2/3/2011 Judicäel BIAOU CEBAC Administrator Cotonou 2/3/2011 Basile ADJALLE CEBAC Administrative Adviser and Representative of the Informal Sector Cotonou 2/3/2011 Daniel DALOHOUN CEBAC Executive Director Cotonou 2/3/2011 Basile ADJALLE CEBAC Administrative Adviser and Representative of the Informal Sector Cotonou 2/3/2011 Charles SITHON ESF Health Promoter Jonquet Quartier, Cotonou 2/4/2011 Michel MEHA ALAVO Local Technical Committee 3rd Deputy Mayor Parakou 1/31/2011 Vincent KPAMBOURAGUI Departmental Group of PV/HIV AIDS (NGO) President Parakou 1/31/2011 Stanislas TOHA Local Technical Committee Youth Representative Parakou 1/31/2011 Rafiou BAGUIDI PROFAM Clinic "L'As de Cœur" Doctor Parakou 1/31/2011 Fortuné G. GANKPE Clinic OSV JORDAN Doctor Parakou 1/31/2011 Hermione ADIMI GRADE-ONG Accountant Parakou 1/31/2011 Audace ATTAKPENOU OSV-JORDAN Supervisor Parakou 1/31/2011 Hélène AWE GRADE-ONG Animating Parakou 1/31/2011 Félicité SABI GRADE-ONG Animating Parakou 1/31/2011 Awaou IDRISSOU GRADE-ONG Animating Parakou 1/31/2011 Damienne AÏMABOU SOVIM-ONG Animating Parakou 1/31/2011 82 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Name Organization Position Location Date Emma SOSSOU SOVIM-ONG Animating Parakou 1/31/2011 Mouhamed PARAPE USCRB " Pair Educateur " Parakou 1/31/2011 Issaka OUSMANE USCRB " Pair Educateur " Parakou 1/31/2011 Nicolas HOUETOHOSSOU Radio Fraternité Commercial Directior Parakou 1/31/2011 Ghislain HOUNVOU GoB Representative Parakou 1/31/2011 Guillaume ADEGBOLA UFABAP-ONG Representative Parakou 1/31/2011 Hyacinthe KOUGNAGLO Local Technical Committee Vice-President Representative Parakou 1/31/2011 Marise AGOUA Local Technical Committee Member Parakou Roumanou OSSENI Local Technical Committee Member Parakou 1/31/2011 Antoinette DOMINGO AAF/JORDAN (NGO) AAF/OSV-Jordan Parakou 2/1//2011 Fatioulath ISSA CIPEC Doctor Parakou 2/1/2011 Gado KORA PPS-DDS/BA Service Chief Parakou 2/1/2011 Zull-Kiff ASSOUMANOU GRADE-ONG Executive Director Parakou 2/1/2011 Léon CHABI SOVIM -ONG Executive Director Parakou 2/1/2011 Cyprien ADADJA Health Centre of GLAZOUE Nurse Glazoué 2/1/2011 Cyriaque KPEKOU Health Centre of GLAZOUE Laboratory Assistant Glazoué 2/1/2011 Chantal TOLOME Health Centre of GLAZOUE Mid Wife Glazoué 2/1/2011 Françoise MONTCHEOUNWA Health Centre of GLAZOUE Mid Wife Glazoué 2/1/2011 Edith DJENONTIN Health Centre of GLAZOUE Doctor Glazoué 2/1/2011 Germain GOUKPANIAN Health Centre of AKIZA Nurse (Chief of the Group) Akiza 2/1/2011 Charlotte DOHOU Health Centre of AKIZA Nurse Akiza 2/1/2011 Rémi DOKPEGAN Health Centre of AKIZA Secretry Akiza 2/1/2011 Désiré AGBOGAN Health Centre of AKIZA Member Akiza 2/1/2011 Gabin KINDJANHOUNDE Health Centre of AKIZA Member Akiza 2/1/2011 Josephine ATCHAOWE Health Centre of AKIZA Member Akiza 2/1/2011 Guy BEHANZIN Radio ROYAL FM Commercial Directior Bohicon 2/3/2011 Bill GANDOGO Radio ROYAL FM In Charge of "Suivi Contrat" Bohicon 2/3/2011 Cyriaque SEHOUETO Radio ROYAL FM Bohicon 2/3/2011 Fretas ATTIMAGBO Radio ROYAL FM Animating Bohicon 2/3/2011 Bruno SEKLOKA Radio ROYAL FM Animating Bohicon 2/3/2011 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 83 Name Organization Position Location Date Marc GANDE Radio ROYAL FM Production Chief Service Bohicon 2/3/2011 Raymond FANDOHAN Radio ROYAL FM General Manager Bohicon 2/3/2011 Moïse LOKO Radio/TV CARREFOUR In Charge of " Promoting and Partnership" Bohicon 2/3/2011 Evariste GUEDOU Radio/TV CARREFOUR Marketing Chief Service Bohicon 2/3/2011 Marguerite YEMADJE Clinic Profam St Uriel Mid Wife Bohicon 2/3/2011 Bernadin TOSSA ALDIPE - ONG Project Coordinator Bohicon 2/3/2011 Boras BEHANZIN ALDIPE - ONG Execitive Directeur Bohicon 2/3/2011 Placide GODOVO IMPACT Animating Bohicon 2/3/2011 Augustine ZINKPE IMPACT Animating Bohicon 2/3/2011 Gilbert GLELE IMPACT Supervisor Bohicon Magloire HOUNKPE RABEJ/SD National Coordinator Cotonou 2/4/2011 Flavien AIDJINOU RABEJ/SD In Charge of "Planification,Coordination et Suivi/Evaluation" Cotonou 2/4/2011 Total # of Respondents 126 84 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 85 ANNEX C. USAID/BENIN FAMILY HEALTH BENIN EVALUATION BACKGROUND DOCUMENTS Document Name Date Implementing Partner Contracts PISAF Cooperative Agreement 3/28/2006 POUZN Abt Contract 9/30/2005 IMPACT Cooperative Agreement 10/17/2006 Project Workplans - PMPs Plan opérationnel de suivi évaluation de PISAF 2006–2010 PISAF Program Decription 2006? PISAF Workplan 2006 Mar-06 PISAF Workplan 2007 Oct-06 PISAF Workplan 2008 Oct-07 PISAF Workplan 2009 Sep-08 PISAF Workplan 2010 Oct-09 PISAF Workplan 2011 Jan-11 POUZN Workplan 2002 Sept-02 IMPACT Program Description Jun-05 Family Health Team PMP 2005? Common Indicators for Benin - All 2007? POUZIN Revised PMP Jul-08 IMPACT Workplan—Year 3 Feb-08 IMPACT Workplan—Year 4 Aug-09 Annual Reports PISAF Annual Performance Report No. 1 03/28/06–09/30/06 PISAF Annual Performance Report No. 2 10/01/06–09/30/07 PISAF Annual Performance Report No. 3 10/01/07–09/30/08 PISAF Rapport de Performance Annuel No. 3 30/09/2008 PISAF Annual Performance Report No. 4 10/01/08–09/30/09 PISAF Rapport de Performance Annuel No. 4 30/09/2009 PISAF Annual Performance Report No. 5 10/01/09–09/30/10 PISAF Rapport de Performance Annuel No. 5 30/09/2010 IMPACT Annual Report 2007— French 17/10/06–16/10/07 IMPACT Annual Report 2007— English 10/17/06–10/16/07 IMPACT Annual Report 2008— French 17/10/07–16/10/08 IMPACT Annual Report 2008—English 10/17/07–10/16/08 IMPACT Annual Report 2009—French 17/10/08–16/10/09 IMPACT Annual Report 2009—English 10/17/08–10/16/09 86 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Document Name Date IMPACT Annual Report 2010—French 01/10/09–30/09/10 IMPACT Annual Report 2010—English 10/01/09–09/30/10 IMPACT - PMP Performance—Year 1 Produced in 2007 IMPACT - PMP Performance—Year 2 Produced in 2008 IMPACT - PMP Performance—Year 3 Produced in 2009 IMPACT - PMP Performance—Year 4 Produced in 2010 IMPACT - Indicator Results—Year 2 Produced in 2008 IMPACT - Indicator Results—Year 3 Produced in 2009 IMPACT - Indicator Results—Year 4 Produced in 2010 POUZN Annual Report—2007 10/01/06–09/30/07 POUZN Annual Report—2008 10/01/07–09/30/08 POUZN Annual Report—2009 10/01/08–09/30/09 POUZN Annual Report—2010 10/01/09–09/30/10 Quarterly Reports PISAF Q1 Report 2006— French 28/03/06–30/06/06 PISAF Q1 Report 2006— English 03/28/06–06/30/06 PISAF Q1 Report 2007— French 01/10/06–31/12/06 PISAF Q1 Report 2007— English 10/01/06–12/31/06 PISAF Q2 Report 2007— French 01/01/07–31/03/07 PISAF Q2 Report 2006— English 01/01/07–06/30/07 PISAF Q3 Report 2007— French 01/04/07–30/06/07 PISAF Q3 Report 2007— English 04/01/07–06/30/07 PISAF Q1 2007 Progress Table French Produced in 2007 PISAF Q1 Report 2008— French 01/10/07–31/12/07 PISAF Q1 Report 2008— English 10/01/07–12/31/08 PISAF Q2 Report 2008— French 01/01/08–31/03/08 PISAF Q2 Report 2008— English 01/01/08–03/31/08 PISAF Q3 Report 2008— French 01/04/08–30/06/08 PISAF Q3 Report 2008— English 04/01/08–06/30/08 PISAF Q1 Report 2009— French 01/10/08–31/12/08 PISAF Q1 Report 2009— English 10/01/08–12/31/08 PISAF Q2 Report 2009— French 01/01/09–31/03/09 PISAF Q2 Report 2009—English 01/01/09–03/31/09 PISAF Q3 Report 2009— French 01/04/09–30/06/09 PISAF Q3 Report 2009—English 04/01/09–06/30/09 PISAF Q1 Report 2010— French 01/10/09–31/12/09 PISAF Q1 Report 2010— English 10/01/09–12/31/09 PISAF Q2 Report 2010— French 01/01/10–31/03/10 PISAF Q2 Report 2010— English 01/01/10–03/31/10 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 87 Document Name Date PISAF Q3 Report 2010— French 01/04/10–30/06/10 PISAF Q3 Report 2010— English 04/01/10–06/30/10 POUZN Q1 Report 2008— English 2008 POUZN Q2 Report 2008—English 04/08–06/08 POUZN Q3 Report 2008—English 07/08–09/08 POUZN Q4 Report 2008—English 10/08–12/08 POUZN Q1 Report 2009—English 01/09–03/09 POUZN Q2 Report 2009—English 04/09–06/09 POUZN Q3 Report 2009—English 07/09–09/09 POUZN Q4 Report 2009—English 10/07–12/09 POUZN Q1 Report 2010—English 01/10–03/10 POUZN Q2 Report 2010—English 04/10–06/10 POUZN Q3 Report 2010—English 07/10–09/10 POUZN Q4 Report 2010—English Produced in 2010 POUZN Q4 Report 2010—French Produced in 2010 POUZN Semi-Annual Report 2008 Produced in 2008 POUZN Semi-Annual Report 2009 Produced in 2009 POUZN Semi-Annual Report 2010 Produced in 2010 IMPACT Q1 Report 2007—English 10/17/06–01/17/07 IMPACT Q1 Report 2007—French 17/10/06–17/01/07 IMPACT Q2 Report 2007—English 01/17/07–04/17/07 IMPACT Q2 Report 2007—French 17/01/07–17/04/07 IMPACT Q3 Report 2007—English 04/17/07–07/17/07 IMPACT Q3 Report 2007—French 17/04/07–17/07/07 IMPACT Q1 Report 2008—English 07/17/07–10/17/07 IMPACT Q1 Report 2008—French 17/07/07–17/10/07 IMPACT Q2 Report 2008—English 10/17/07–01/17/08 IMPACT Q2 Report 2008—French 17/10/07–17/01/08 IMPACT Q3 Report 2008—English 01/17/08–04/17/08 IMPACT Q3 Report 2008—French 17/01/08–17/04/08 IMPACT Q2 Report 2009—English 01/01/09–03/31/09 IMPACT Q2 Report 2009—French 01/01/09–31/03/09 IMPACT Q3 Report 2009—English 04/01/09–06/30/09 IMPACT Q1 Yr 4 Report 2010—French 01/10/09–31/12/09 IMPACT Q1 Yr 4 Report 2010—English 10/01/09–12/31/09 IMPACT Q2 Yr 4 Report 2010—French 01/01/10– 31/03/10 IMPACT Q2 Yr 4 Report 2010—English 01/01/10–03/31/10 IMPACT Q3 Yr 4 Report 2010—French 01/04/10–30/06/10 IMPACT Q3 Yr 4 Report 2010—English 04/01/10–06/30/10 88 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Document Name Date Evaluation Reports Evaluation de la Qualité de la Gestion du Système Sanitaire (EQGSS) dans les Départements du Zou et des Collines en Republique du Benin Mar-08 Evaluation de la Qualité de la Gestion du Système Sanitaire (EQGSS) dans les Départements du Zou et des Collines en Republique du Benin Mar-08 Evaluation de la Qualité de la Gestion du Système Sanitaire, 2e édition (EQGSS2) dans les Départements du Zou et des Collines en Republique du Benin Jan-11 Fourniture d‘une eau potable grâce au programme Aquatabsdu POUZN au Bénin : Résultats et leçons apprises Oct-10 Un partenariat public-privé pour l‘introduction du zinc pour letraitement de la diarrhée au Bénin : Résultats et leçons apprises October 2010 Oct-10 Evaluation de l‘audience des radios partenaires de URC/PISAF dans les départements du Zou et des Collines Feb-10 Evaluation à mi parcours de l'efficacité des actions de communication de comportement dans 10 collèges et lycées du Zou et des Collines Feb-10 VIH/SIDA: Enquête TRaC pour Evaluer les déterminants de l'utilisation systématique du condom chez les Jeunes Apprentis non mariés de 15-24 ans avec les partenaires occasionnels dans les zones d'intervention. 1er Passage Dec-07 VIH/SIDA: Enquête TRaC pour Evaluer les Déterminants de l'Utilisation Systématique du Condom chez les Jeunes Elèves/Etudiants non mariés de 15-24 ans avec les Partenaires Occasionnels dans les Zones d'Intervention. 1er Passage Dec-07 Evaluation de l'utilisation du Condom chez les travailleuses de Sexe de 15-24 ans dans les zones d'Intrevention des Projets IMPACT et KfW; 2ème Passage /2009 Benin 2009: HIV Etude TRaC pour Evaluation de l'Evaluation du Condom chez les Routiers dans les Zones d'Intervention des projets IMPACT et KfW. 2ème Passage Sep-09 Evaluation de l'utilisation sytématique du préservatif chez les Apprentis non mariés de 15-24 ans des zones d'intevention du projet IMPACT au Bénin. 2ème Passage 2010 Evaluation de l'utilisation systématique du Préservatif chez les Elèves non mariés de 15-24 ans des zones d'Intervention du Projet IMPACT au Bénin. 2ème Passage 2010 Evaluation de la portée de la campagne du condom Koool® auprès des jeunes élèves de 15-24 ans des zones d'intervention du projet IMPACT au Bénin 2010 Consultancy Reports Déterminants de l'Utilisation du Condom chez les travailleuses de Sex de 15-29 avec les Partenaires non payants au Bénin. 1er Passage. Jun-07 Déterminants de l'Utilisation Systématique du Condomchez les Routiers avec les Partenaires Occasionnels dans les zones d'Intervention des Projets IMPACT et KfW au Bénin. 1er Passage Jun-07 Rapport Pre-Test des emballages du kit d'Orasel-Zinc Jul-07 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 89 Document Name Date Etude sur les déterminants de l‘utilisation du condom chez les Travailleuses de sexe de 15-29 ans et les Routiers des zones d'intervention du projet IMPACT et de la KfW au Bénin Sep-07 BENIN 2007: Enquête TRAC pour Evaluer les Déterminants de l'Utilisation Systématique du Condom chez les Jeunes Apprentis non mariés de 15-24 ans avec les partenaires Occasionnels dans les Zones d'Intervention Dec-07 Repositionnement du préservatif masculin Prudence Plus au Bénin: Focus on Concept Development Jan-08 Déterminants de l'Utilisation des Contraceptifs Modernes chez les Femmes dans les Zones de Concentration du Projet IMPACT au Bénin. 1er Passage Nov-08 Mesure de la Couverture et de la qualité de couverture du kit de traitement de diarrhée ORASEL-ZINC et du produit de traitement à domicile de l'eau AQUATABS dans les Départements du Bénin. Jun-09 Mesure de l'Acces et de la Performance (MAP): Evaluation de la Couverture des Produits de PSI-BENIN. 2009 Mesure de la Couverture et de la qualité de couverture du kit de traitement de diarrhée ORASEL-ZINC et du produit de traitement à domicile de l'eau AQUATABS dans les Départements du Bénin. Round 2 2010 Bénin (2010): Déterminants de traitement de la diarrhée chez les enfants de moins de 5 ans dans les zones du projet POUZN au Bénin. 1er Passage 2010 Bénin (2010): Evaluation de l‘utilisation de produit de traitement de l'eau par les femmes ayant en charge les enfants de moins de 5 ans dans les zones du projet POUZN. 2ème Passage 2010 Pre Test du Spot Télévisuel sur AQUATABS, le produit de Traitement de l'Eau à Domicile. Mar-10 Pre Test du Spot Télévisuel sur ORASEL-ZINC Apr-10 Pre Test du Spot Télévisuel sur les Méthodes Contraceptives Modernes de longue Durée (DIU ET IMPLANTS) May-10 Rapport des Pres Tests des Spots TV & Radio sur PRUDENCE PLUS Jul-10 Diagnostic et approches de solutions au phénomène de grossesse en milieu scolaire dans le Zou/Collines Feb-08 Plan stratégique de développement des mutuelles de santé 2009-2013 May-08 Stratégie d'articulation entre les mutuelles de santé et les services à base communautaire May-09 Stratégie d'articulation entre les mutuelles de santé et les institutions de micro finance May-09 Stratégie d'extension des mutuelles de santé dans les départements de l'Alibori, Zou et Collines Dec-07 Etude du contexte d'implantation des mutuelles de santé dans la commune de Zagnanado Mar-08 Etude du contexte d'implantation des mutuelles de santé dans la commune de Savalou May-08 90 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Document Name Date Rapport de l'étude de faisabilité dans le cadre de l'implantation des mutuelles de santé dans la commune de Malanville Feb-08 Rapport de l'étude de faisabilité dans le cadre de l'implantation des mutuelles de santé dans la commune de Karimama Feb-08 Rapport de l'étude de faisabilité dans le cadre de l'implantation des mutuelles de santé dans la commune de Gogounou Mar-08 Rapport de l'étude de faisabilité dans le cadre de l'implantation des mutuelles de santé dans la commune de Ségbana Mar-08 Rapport de l'étude de faisabilité dans le cadre de l'implantation des mutuelles de santé dans la commune de Kandi Mar-08 Etude de faisabilité sur les services pouvant être offerts par les hôpitaux d zone des unions communales des mutuelles de santé dans le Zou/Collines Jul-09 Etude sur le fonds sanitaire des indigents et les mutuelles de santé Oct-09 Contribution des mutuelles de santé dans la promotion de la santé communautaire au Bénin Jul-09 Etude sur les facteurs associés à la fidélisation des membres des mutuelles de santé au Bénin Nov-10 Evaluation de l'approche collaborative d'amélioration de la qualité des soins et services de santé dans les mutuelles de santé de Banikoara et Sinendé Dec-10 Cadre Stratégique National de Lutte contre le VIH/SIDA/IST 2006-2010 Produced in 2010 Training modules, Marketing Stratégies, Workplan revision Mar/15–28/2008 Program reviews, develop workplan and budget for Year 3 Jun/15–19/2009 Program reviews, assess of research findings and prepare a country brief May/2–10/2010 Un partenariat public-privé pour l‘introduction du zinc pour le traitement de la diarrhée au Bénin :Résultats et leçons apprises: POUZN Produced in 2010 Assurance de Qualité de base : guide du formateur + manuel de référence Apr-08 Supervision : Manuel de référence + guide du formateur Jan-09 Plan de formation 2010 Oct-10 Plan de développement des capacités Oct-10 Guide du coaching intégré Dec-10 Paquet intégré Dec-10 Gestion Axée sur les Standards et la Récompense et Services Intégrés de SR et VIH/SIDA (PTME, CDV, PF et IST) Sep-09 Miscellaneous Documents Proposal (Soumission technique révisée) Dec-05 PISAF Evaluation Presentation Jan-11 IMPACT Evaluation Presentation Jan-11 Résultats des collaboratifs Jan-11 Plan d'institutionnalisation - PISAF Nov-10 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 91 Document Name Date Fourniture d'une eau potable grâce au programme AQUATABS du POUZN au Bénin: Résultats et Leçons apprises. Oct-10 Un partenariat Privé-Public pour l'Introduction du Zinc pour le traitement de la diarrhée au Bénin: Résultats et Leçons apprises. Oct-11 Synthèse de l'Analyse de la Performance dans les Servives Intégrés de VIH/SIDA (PTME,CDV,PF,IST) par site. Produced in 2009 Accord de Collaboration - ABMS/PSI Sep-10 Value added to the approach to quality assurance (SBM-R) - IMPACT Produced in 2011 Mesure de la couverture et de la qualité de couverture du kit de traitement de diarrhée Orasel-Zinc® et du produit de traitement à domicile de l'eau Aquatabs® dans les départements du Benin - IMPACT Jun-09 Mesure de L'acces et de la Performance (Map): Evaluation de la Couverture et de la Qualite De Couverture des Produits de PSI-Benin Produced in 2009 Document de Stratégie nationale d‘intégration bidirectionnelle de la Santé Sexuelle et de la Reproduction et du VIH/SIDA - MOH Dec-10 Etude Rapide de l‘Intégration de la Santé Sexuelle et de la Reproduction et du VIH/SIDA - MOH – Dec-09 Atelier de Revision des Tableaux d‘Acquisition des Contraceptifs (TACs) au Benin (MOH) Dec-10 92 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 93 ANNEX D. EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES METHODOLOGY January 14–March 18, 2011 METHODOLOGY 40. Document Review: Review key project documents prior to arrival in country (January 14–January 20); 41. Team Planning Meeting: Orientation and planning meeting to produce a work plan, timeline, interview instruments and draft outline of the report (January 21–22); 42. Initial Briefing Meetings : Briefings from USAID staff to discuss USAID perspective on three family health projects and to review and approve GH Tech Team schedule, methodology, draft report outline, and deliverables (January 24); 43. Interviews and Site Visits (January 25–February 7)* – Key informant interviews to include USAID Mission Health Office staff, representatives of the Ministry of Health and of donor agencies, and representatives of family health projects‘ (PISAF, IMPACT and POUZN) implementing partners. – Visits to representatives of departements and to project sites to observe the project in action and to collect evaluation data. Location of sites and scheduling of visits will be determined in collaboration with USAID and representatives of the three family health projects. 44. Prepare summary findings and recommendations ( February 8–9) 45. Provide preliminary briefing to USAID, URC, PSI, and Abt Staff (February 10)** 46. Prepare Final Draft Report (February 11–16) 47. Debrief with USAID, URC, PSI and Abt Staff (February 17) ** 48. Final Revision and Delivery of Draft Report (February 18-19) 49. USAID Review of Draft Report (February 21–March 4) with comments to Evaluation Team by March 4 50. Evaluation Team response to USAID Review (March 7–9) * During the interviews and site visits, the evaluation team will be divided into two teams—A and B. Each of the teams will be provided with standardized interview and site visit guidelines to ensure that both teams address the same issues. At the completion of each week of the interviews and site visits, both teams will meet in Cotonou to compare notes and summarize findings. Team review meetings are scheduled for Sunday, January 30th and Tuesday, February 8th. ** Due to procurement integrity issues, separate briefings will be held with USAID and with implementing partners 94 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 95 ANNEX E. USAID/BENIN FAMILY HEALTH EVALUATION SCHEDULE January 2011 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Background reading 15 16 17 Background reading 18 Background reading and departure for Benin 19 Travel to Benin Team arrives 20 Background Reading 21 TPM 22 TPM and prep for USAID 23 24 Orientation meeting and TPM Briefing with USAID with USAID Benin 09h00 25 Orientation Meeting with IMPACT and Meetings with MoH 26 Orientation Meeting with PISAF 27 Meetings with MoH 28 Joint Meeting with PISAF and IMPACT And Team Work on Indicators 29 Review of new documents 30 Team A and B Departure for Field Visits 31 Field Visits 96 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES February 2011 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Field Visits 2 Field Visits 3 Field Visits 4 Field Visits 5 Individual Team Meetings 6 7 (AM) Interview Meetings with USAID Staff (PM) Meeting with UNICEF ; Meet with MOH￾SIDA Meet with CCM 8 Team Recap and preparation for meeting with USAID (Meet at Villa – 9:00) 9 Team Recap and preparation for meeting with USAID (Meet at Villa – 9:00) 10 Briefing with USAID 11 Discussion/drafting day 1 12 Discussion/drafting day 2 13 14 Discussion/drafting day 3 15 Discussion/drafting day 4 16 Team Recap and preparation for briefing with USAID (Meet at Villa – 9:00) Discussion/drafting day 5 17 Debrief with USAID/Benin 18 Revise report 19 Prepare report for submission to USAID (send by email) 20 Depart Benin 21 Tatiana delivers hard copy of report to USAID Travel 22 23 24 25 26 27 28 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 97 March 2011 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 Deadline for comments from USAID 5 6 7 Report revision (full team) 8 Report revision (full team) 9 Report revision (TL only) 10 11 12 13 14 15 16 17 18 TL submits final report pending comments from USAID 19 20 21 22 23 24 25 26 27 28 29 30 31 98 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES—VOLUME 1I ANNEXES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 99 ANNEX F. EVALUATION OF BENIN FAMILY HEALTH ACTIVITIES DISCUSSION GUIDELINES FOR FIELD VISITS AND INTERVIEWS The USAID/Benin‘s Family Health program (2006-2011) includes two major activities: (1) Projet Intégré de Santé Familiale (PISAF), an integrated family health project implemented in the central region of the country and (2) Projet Intégré d’Appui à la Santé Familiale et la Prévention de VIH/SIDA (IMPACT), implemented in seven convergence zones with a national focus for social marketing activities. Complementary activities in MCH and social marketing are conducted by the (3) Social Marketing Plus for Diarrheal Disease Control: Point-of-Use Water Disinfection and Zinc Treatment Project (POUZN). 51. PISAF has focused on strengthening the health system‘s ability to provide evidence-based family health services that meet the needs of communities and families and to help communities become more active participants in their own health and in the health system. With an emphasis on strengthening health care service systems, the project‘s principal objectives were to: – Improve the policy environment; – Increase access to family health services and products; – Improve quality of services; and – Increase demand for health services and prevention measures. 52. IMPACT was designed to support the Government of Benin‘s efforts to control the spread of HIV and increase access to family health services and products through social marketing campaigns. The project‘s principal objectives were to: – Improve the policy environment; – Increase access to HIV/AIDS, reproductive health/family planning (RH/FP) and Child Survival products; and – Improve quality of services; and – Increase knowledge about HIV/AIDS, RH/FP and Child Survival to increase demand and use of products and services. 53. POUZN was designed to mobilize the private sector to introduce low-cost point-of-use water treatment products for diarrhea prevention and zinc together with oral rehydration fluids for the treatment of diarrhea among children under five. The project‘s goal is to – Significantly increase the use of POU water treatment and zinc while creating vibrant commercial markets that will allow for program sustainability. Under a contract with USAID/Benin, the GH Tech Project is undertaking an evaluation whose purpose is to: 54. Understand how the three integrated health projects (PISAF, POUZN, and IMPACT) have been implemented in terms of program intensity, quality, and coverage relative to program plans; 55. Identify intended and unintended consequences of the integrated program; 56. Document lessons that can be shared throughout the Agency to contribute to development learning and improve future programming; and 100 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 57. Identify gaps and make recommendations on strategies to address any identified gaps and improve the delivery of follow-on integrated health projects, within the context of contributing to the country‘s overall health strategy. Based on each respondent’s experience with and knowledge of the three separate projects, and with reference to the projects’ goals and objectives, the evaluation team’s interviews will focus on the following questions: 58. How closely did the projects follow plans in terms of activities planned and program intensity, quality, and coverage? 59. What were the strengths in implementation of the projects? 60. What were the weaknesses in implementation of the projects 61. Which activities or aspects of the projects were most successful? And why? 62. Which activities or aspects of the projects were least successful? And why? 63. What were the intended and unintended consequences of individual project activities and the integrated program overall—for beneficiaries and the health system? 64. How successful was the integrated approach? 65. What were the lessons learned from individual project activities and about implementing integrated health programs? 66. For USAID and Implementing Partners: What evidence is there that the activities implemented contributed to the intermediate results of the Family Health results framework: a. An improved policy environment; b. Increased access to products and services; c. Increased demand for products and services? 67. For USAID and Implementing Partners: What evidence is there that the development hypothesis and assumptions underlying the results framework were valid—that increases in the intermediate results would lead to expanded use of services, products and preventive measures? 68. What gaps in programming were identified, in terms of specific activities and in implementing an integrated package of family health services? 69. Were there any missed opportunities in: a. Increasing coverage; b. Improving quality of services; c. Integrating different interventions; d. Building local capacity within our programs; e. Coordination and harmonization with other donors? 70. What opportunities did the Family Health projects take advantage of? 71. What specific recommendations are there for strategies to: a. Address any identified gaps; b. Improve the delivery of program interventions for the next five-year strategy and follow￾on integrated health projects? USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 101 ANNEX G. USAID/ BENIN FAMILY HEALTH EVALUATION: INTERVIEW SUMMARY TEMPLATE Evaluation Team Interviewer: Respondent Name: Respondent Title and Affiliation: Interview Location: Date: Project Covered:  PISAF  IMPACT  POUZN Interview Summary: 1. Adherence to Project plans: intensity, quality, and coverage. 2. Strengths in implementation of the projects. 3. Weaknesses in implementation of the projects. 4. Most successful activities and aspects of the projects were most successful. And why? 5. Least successful activities or aspects of the projects were least successful. And why? 6. Intended and unintended consequences of individual project activities and the integrated program overall—for beneficiaries and the health system. 7. How successful was the integrated approach? 102 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 8. Lessons learned from individual project activities and about implementing integrated health programs. 9. Evidence that the activities implemented contributed to the intermediate results of the Family Health results framework: a. An improved policy environment. b. Increased access to products and services; c. Increased demand for produces and services. 10. Evidence that the development hypothesis and assumptions underlying the results framework were valid—that increases in the intermediate results would lead to expanded use of services, products and preventive measures? 11. Gaps in programming, in terms of specific activities and in implementing an integrated package of family health services? 12. Missed opportunities in: a. Increasing coverage; b. Improving quality of services; c. Integrating different interventions; d. Building local capacity within our programs; USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 103 e. Coordination and harmonization with other donors? 13. Specific recommendations for strategies to: a. Address any identified gaps; b. Improve the delivery of program interventions for the next five-year strategy and follow￾on integrated health projects? 14. Miscellaneous Comments and Observations 104 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 105 Mission of Project USAID Health Strategic Objective 2 Millennium Development Goals Intermediate Results National Health and Population Policies ANNEX H. PISAF PROGRAM DESCRIPTION ATTACHMENT B - PROGRAM DESCRIPTION The new USAID program, Integrated Family Health Program/Projet Intégré de Santé Familiale or PISAF, provides an opportunity to bring together achievements of previous programs and partners, integrate new evidence-based practices for maternal and child health (MCH) and HIV/AIDS, forge new partnerships with public and private sector institutions, and expand the institutionalization of these best practices to more regions and zones of the country. Figure H-1. Diagram of Project Mission The URC Consortium (URC, Abt Associates) proposes an approach that builds upon the work carried out by each of its partners and updates and integrates technical strategies, using the latest evidence from Benin, regionally and internationally. We will help institutionalize best practices for management, clinical care and community services, and assist the government to make rational resource allocation decisions by using up-to-date costing and cost-effectiveness data. The proposed program activities are designed to support the Government of Benin‘s National Health Policy and Strategy, its Population Policy, and USAID‘s Strategic Objective 2: ―Expanded use of health services, products and preventive measures within a supportive political environment,‖ focusing on Intermediate Results IR 1: Supportive implementation environment created, IR 2: Access to quality services and products increased; IR 3: Demand for health services, products and preventive measures increased. We bring a keen understanding of the opportunities and the challenges to improve health care in the country and what it takes to obtain success at national, regional and peripheral levels. URC and Abt Associates have established close relationships with government counterparts, local NGOs and CBOs, other donors and projects, and we are intimately familiar with previous and ongoing USAID programs both in health and education. In this section we will present our overall approach to meet the challenges in the Beninese health context and achieve results. Then for each IR, we will describe our strategy, focusing on the latest evidence and proven interventions, as well as emphasizing integration between the IRs and between the technical areas. Where we felt that additional ideas and activities might significantly contribute to the desired results, this has been indicated. This is followed by more detailed activity descriptions for each of the sub-Intermediate Results. Our management and staffing plan is presented in Section 2; the detailed workplan is presented in Section 3, and the monitoring and evaluation plan is in Section 4. Curricula vitae, past performance references and letters of commitment from our partners can be found in the Annexes. OVERALL APPROACH AND DRIVING STRATEGIES Our approach will focus on strengthening the capacity of all levels of the health system to ensure access to high quality services and on strengthening the role that communities and civil 106 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES society play in improving quality and increasing use of preventive health services. Many effective strategies and approaches have been tested in the Borgou/Alibori and Ouémé/Plateau regions and are ready to be scaled up to other regions. The strong foundation already created in service delivery, quality assurance and support functions will facilitate the integration of new technical areas into the family health minimum package (VCT/-PMTCT/ART, intermittent presumptive treatment (IPT) and insecticide treated nets (ITN)); and decentralization. We will continue to work on improving overall quality, while strengthening community-based health services and insurance schemes (mutuelles) and forging stronger links between them. In the Zou/Collines Regions, we will strengthen efforts to introduce quality-assurance approaches based on those used in the Borgou/Alibori, while introducing additional tools and approaches for work at clinic and community level successfully tested, and build on relationships and progress already present there through the work of Africare and ROBS. In Borgou/Alibori, we will support current achievements in health systems management and quality improvement, mutuelles, implementation of expanded elements of the minimum package, and testing of innovations. In Ouémé/Plateau we will continue to build upon Africare‘s success in introducing IMCI and malaria innovations by supporting monitoring and improvement of compliance with standards, updating protocols to include the latest treatment for malaria, and continuing to mobilize local community groups. We will discuss, upon award, about the level of support to be provided to Ouémé/Plateau in management strengthening and quality assurance activities, and to community activities, particularly promotion of ITNs through women‘s groups. Because there are strategies and approaches that are well tested and ready for national scale-up, and others that still require more work before replication, it is of critical importance to engage the national level into the process of innovation, both in terms of sustainability of efforts in the project areas, but also for scale-up beyond project areas. Thus, PISAF will support the national level to develop effective mechanisms to incorporate new medical evidence into national norms, institutionalize best practices (quality assurance, ascendant planning, mutuelles), develop appropriate policies and procedures to support these best practices, and support the empowerment of health zones and regions through effective decentralization. We will work with the national level to ensure that they can play their newer, leaner role in health sector effectively. The program proposed by the URC team incorporates the following cross-cutting strategies to ensure successful and sustainable project achievements: Fostering Beninese investment and ownership of improvement efforts at all levels of the health system: We will work to ensure that Beninese at national, regional, zonal, health centre and community level participate in the development of new approaches and tools and take them on as their own. We will engage stakeholders in developing and testing new approaches, and sharing the results. We will work to ensure that counterparts can envision the improvements by creating fora for discussion and sharing; we will help those who have carried out innovative approaches to teach and assist those who are replicating/scaling them up. For example, we will arrange for people from Zou/Collines to visit centers and zones in Borgou/Alibori and Ouémé/Plateau to see improved systems first-hand and learn from MOH and NGO peers, thus shortening the learning and adaptation cycle for improvements of systems organization and provider performance. We will continue to build upon our strong working relationships with national counterparts, donors, civil society, other Ministries, and the Ministry of Health. At Zonal and community levels, this will involve the search for local investment in health activities in both the public and private sector, through ROBS members, faith-based and private organizations. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 107 Putting decentralization into practice at zone-level: The MOH has developed the policies and decrees that should facilitate effective decentralized planning, financial and human resource management of health zones and foster allocation of health budget by local governments. Now effective implementation of this policy must be nurtured, by both building competencies at local levels to effectively carry out their new responsibilities, and continuing dialogue with central level actors on how to oversee and support these decentralized actors to carry out their jobs. Ensuring that norms are updated, communicated and integrated into service delivery: There have been significant breakthroughs in recent years in the treatment of malaria, in neonatal health and in prevention and treatment norms for HIV/AIDS, and breakthroughs will continue to appear over time. We will help develop mechanisms to review, revise/adapt new standards, and to communicate them to the health workforce and community, and ensure they are integrated into M&E and supervision guides. Emphasizing quality assurance and institutionalizing continuous improvement: Quality assurance is the process of ensuring that quality is defined, measured and improved so as to meet the needs of the population. Quality assurance will be promoted at all levels of the system, such that quality assurance activities, structures and values become an integral and sustainable part of the health system. Such changes have already begun in the Borgou/Alibori, leading to a strong interest from the Ministry of Health to scale up this approach to other regions. What is being scaled up is not so much a set of tools and plans, but changing the way people look at the work they do and building capacity and motivation to improve how they do things to get better results. The institutionalization of QA can benefit from further strengthening, as well as increasing the capacity of the central level of the Ministry of Health to provide technical support to other regions. This will include developing a specific strategy/policy to clarify the roles of all actors in the heath system on quality assurance. We will help the health system move towards a more widespread institutionalized QA system with more advanced characteristics, following the two main approaches used to date. Quality management is a system by which Health Zone Management Teams and the Regional Health Directorates clarify their standards, consistently measure progress (e.g., through the use of the scoreboard or other mechanism), and then use teamwork and other collaborative approaches to design and implement improvements so as to better meeting the needs of their target populations. Collaborative improvement, a mechanism for fostering rapid learning and improvements (see box for details) successfully introduced in Borgou/Alibori and recently used in 15 maternities in Mono/Couffo and Ouémé/Plateau, will be supported and scaled up to include all zones of Zou/Collines. The capacity developed in the Borgou/Alibori, Mono-Couffo and Ouémé-Plateau can be used to jump- start work in Zou/Collines. Together these efforts will create a critical mass of expertise at national level and in four regions to foster ownership and institutionalization of this approach. 108 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Strengthening linkages between communities and health facilities to stimulate demand and quality of care: The existence of community-based mutual health organizations (mutuelles) introduced in Borgou/Alibori by Abt Associates through PHRplus and strong community structures supported by PROSAF, and prior experience in collaborative improvement between health centers and communities provide an extraordinary opportunity to develop new ways to ensure affordability of health care and improve outcomes. Mutuelles, with the financial inputs they bring to health centers, can leverage their collective buying power to demand quality services and can also be used as a vehicle to encourage their members to use preventive services and pursue healthy lifestyles and in-home health practices. Linking mutuelles with micro-credit can increase the number of individuals who can join a mutuelle. Community-based efforts to raise funds for the health facility to improve the quality of services also offers many opportunities to decrease financial barriers through exemptions, cross-subsidization, and income-generating activities. Quality improvement can decrease the costs of providing care, through reductions in over-prescription, nosocomial infections, and wastage from poor management. The project will help strengthen the linkages between communities, heath facilities and health zones to make the most of these partnerships. Multi-sectoral approaches to strengthening local capacity for and civil society participation in decentralized management, community mobilization and BCC, capitalizing on the gains made by other USAID-supported projects in Health, Education and Democracy and Governance, working with the Ministries of Family, Social Protection and Solidarity, Education, and Agriculture, ROBS, the women‘s NGO network, PTAs, community financial institutions, local governments, and others. Using pilots and small applied research to test new or innovative approaches, with results informing policy. There are many tested approaches to scale up, but others would benefit from further experimentation. Examples of such innovations include the linking of mutuelles to quality improvement, community-based distribution and micro-finance initiatives; the exploration of alternative sources of drug supply; the use of incentives for quality improvement, as well as effective and sustainable mechanisms for community-based service provision and community involvement. Collaboratives Model The collaboratives model is one where multiple sites work on similar quality problems in order either to develop innovative care models or scale up successful interventions. Through a structured process of alternating group learning sessions and individual work on implementation and measurement, many sites are able to gain technical knowledge of standards, learn about quality improvement, share ideas, work on these in their home facility and then share and learn some more. This approach has shown itself value for rapid improvements over a wide area, and has been successfully applied to a variety of service delivery issues from PMTCT to infection control in Benin and other African countries. For example, the EONC collaborative in 15 sites in Benin has already produced dramatic improvements between February and August 2005 in such measures as correct partograph use (58% to 94%), correct practice of AMTSL (8% to 70%), and correct newborn care (<30% to almost 90%.) In addition, the EONC collaborative has raised national awareness of the urgent need to improve the quality of care, facilitated spread of best practices and improved service organization leading to significantly reduced client waiting times. The URC/QAP EOC collaboratives will provide an immediate venue for implementing the new PMTCT protocols in a highly structured setting. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 109 Strengthening the Management and Coordination of the Health Facilities Ascendant planning Computerization Use of performance data during the HZ quarterly meetings Collection and analysis of performance data Organization of the HZ/DDSP quarterly meetings (CODIR) Management training curricula Establishment of logistics management systems (including the department warehouse) Establishment of financial management systems Establishment of data quality control systems Strengthening human resources management These cross-cutting strategies will form the basis for the specific technical strategies presented for each of the Intermediate Results in the following sections. INTERMEDIATE RESULT 1: A SUPPORTIVE IMPLEMENTATION ENVIRONMENT CREATED This IR focuses on the policy and management environment which involves central, regional (regional), and health zone levels, and the broader civil society which should have a stake in how health services are organized to meet their needs. Strategic approach: The URC team views a favorable policy environment as critical to achieving access to quality health care for the citizens of Benin. Our strategy focuses on a top￾down and bottom-up approach to policy and advocacy, in which there is a healthy interaction and dialogue between levels, and in which each level plays their role effectively. The central level has a key role in the definition of policies and updating of norms using the latest international evidence for clinical guidelines and treatment protocols, such as for malaria treatment and PMTCI/ARVs. At the same time, proven processes and management procedures tested in the periphery will be used to inform the formulation of management guidelines and procedures adopted nationally. This will involve engagement of operational level staff in the process of adapting/revising norms and standards, and the involvement of national staff in collaboratives to improve implementation of standards. Key elements of our strategic approach to IR1 include: Engaging the central level in an ongoing dialogue and creating momentum for policy development: At the central level, our strategic focus will be on creating ongoing fora and momentum that empowers stakeholders to build consensual policy frameworks and to be comfortable with paradigm shifts that are sometimes needed, such as is the case with successful decentralization. Our staff will work with different administrative and technical units of the Ministry of Health to provide technical support and to inform central level of results achieved and critical issues surfacing at implementation level in the zones and communities, as was successfully done by PROSAF staff in recent years. Decentralization laws and regulations recently were passed that indicate a commitment to the new paradigm. Now the Ministry must act in the new fashion described in those laws and regulations. This step is often as challenging as the passing of the laws and regulations. In addition, there needs to be accountability of Regional and Health Zone management teams to the central level. Collaborating closely with partners: We are committed to proactive, strong alliances with critical partners (both USAID-funded and others) and donors at all levels to leverage support for needed policy changes, to disseminate new policies and to support their implementation, as done in moving forward the decentralization policy through adoption of ascendant planning. We see our role as leading in some efforts and supporting in others. 110 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Scaling up and reinforcing quality management capacity at regional and health zone level: At decentralized levels, our strategy builds on scaling-up the valuable experience and lessons learned in the Borgou and Alibori and other Regions with regard to improved management of the health sector and the collaborative improvement. The essential management components are now accepted at national, regional and peripheral levels, and we will introduce this package of approaches, ways of working and in some cases, tools, into Zou/Collines. We will also continue to foster leadership for change and quality among health care managers, using coaching, training in facilitation and negotiation to apply the three tenets of quality: standards, monitoring and improvement. These essential management systems are presented in the box. In Borgou/Alibori, much has been achieved with regard to improved management, but continued support will be provided so that all systems are completely institutionalized. Special focus will be on financial and human resources management policies. PISAF will use cross-learning, with those who have done it assisting those who will be doing it now – and letting the central level see with their own eyes what issues come up and how they should be dealt with. Expanding the role of civil society in health: Benin has a number of successful community￾mobilization programs currently in place, combining improved access to family health products, services and information with improved citizen participation in health management such as those developed under PROSAF, PROLIPO, BHAPP and CRS. In addition, work under democracy and governance and education have strengthened community structures to be more involved in running schools, managing local services, etc. We will work with health authorities to streamline community mobilization to ensure adequate supervision by health managers, but will also work to expand involvement of communities in broader terms with a more multi-sectoral approach. Particular emphasis in both regions will be on the expansion of community-based insurance schemes (mutuelles) and work with existing institutions. We will achieve this by strengthening NGO capacity to mobilize communities and support delivery of community-based services, by involving the private sector, and by creating local advocacy systems. National level activities will be coordinated by the Chief of Party, with technical staff dedicating varying percentages of time to national level activities, especially the Malaria advisor. National￾level activities will be supported by short-term assistance for key areas, including national level QA, commodity security, clinical guidelines especially VCT/PMTCT/ARV and ACTs, costing, mutuelles, and decentralization. Sub-IR 1.1 Selected Health Policies and Approaches Implemented Implementation of policies and approaches requires policy development (including building support for the policy), ensuring the existence of appropriate norms and standards for its implementation and communication of these policies, norms and standards to get the necessary ―buy-in‖ by those responsible for implement them. Key activities under this sub-IR are as follows: Strengthening the policy process: Support to developing, updating, finalizing health sector policies and strategies to support access to quality health care. Below is a list of current illustrative policy issues that PISAF will focus on. However, it should be noted that the URC team is not proposing to take the lead on all these issues, but these issues represent key national level activities to provide a supportive policy environment for the important work at departmental, health zone, health center and community levels, and thus are ones that PISAF would be engaged in, in collaboration with other partners. The breadth and depth of this policy work will depend the pace of urgency of these policy issues, which is always evolving: USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 111 Finalizing the strategic plan on mutuelles. We will continue the current work under PHRplus, working with the MOH and other partners to finalize the strategic plan for mutuelles: hold discussions and assisting with the adoption of the strategic plan as official policy; establish criteria for what constitutes an effectively functioning mutuelle and sketch a monitoring and evaluation plan; and assist the MOH with implementation of the strategic plan. Consolidating national policy on QA: At the request of the Ministry of Health, PROSAF helped the MOH begin the development of a National QA policy that sets out the roles of each level of the system and defines the mechanisms that will ensure defining, measuring and improving quality. PISAF will continue to support the national level in developing this national policy, based on the experiences of the Borgou/Alibori and the EONC collaboratives in two other regions PISAF will advocate for this policy to include definition of mechanisms for regular review, updating and communication of norms and standards. QAP work in Rwanda and Morocco show the advantage of policies that outline the organizational responsibilities for key quality assurance activities at all levels of the health system. Support updating of policies on community-level provision of services, such as the inclusion of re-supply of oral contraceptives and community-based treatment of pneumonia, which are being piloted, as well as inclusion of future pilot interventions such as point of use water purification, zinc supplementation for diarrhea, postpartum Vitamin A and prenatal iron in the community, etc. Develop consensus and policy on decentralized essential medicines warehousing (regional and zonal warehouses for drugs) and how to better ensure adequate response of the CAME to decentralized needs for drugs and other health products. The availability of such warehouses alone is not sufficient: it is estimated for Borgou/Alibori that only 70% of medicines and products ordered from CAME are actually delivered. This policy will be linked with commodity security issues. Support policy operationalization through more effective dissemination, advocacy and creation of public support for changes: We will support the MOH to develop a comprehensive communication plan to accompany advocacy efforts and reaching out to multiple sectors, by: Maintain dialogue with the central level of the Ministry of Health on decentralization (also with the Ministry of Finance and local government to build on the RTI decentralization project.) Create fora to share best practices and continue discussion about decentralization including decentralized human resources management, health financing and others. Implement ascendant planning at all levels as adopted by the Ministry Implement decentralized HR management including incentive pay for less-favored postings. Engage legislators and civil society through IEC and advocacy to support for necessary changes 112 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Support to the Development and Implementation of Mechanisms for Review, Revision, Updating and Communication of Clinical Norms Develop mechanism for reviewing norms and standards: We will support the Ministry to create a mechanism for assessing each of the main technical areas that are part of the minimum package of family health services in terms of the need to update policies and clinical norms. We will provide limited support (while working in collaboration with other partners) to the process of developing/updating policies/norms (based on evidence from pilot experiences or international norms). All clinical areas that are will be reviewed and norms and protocols updated as needed. As we support the Ministry to include HIV/AIDS and updated malaria guidelines into the FH minimum package, we will document the process as guidance for future updates and build mechanisms to institutionalize continuous updating of evidence base for policies (through networking, Internet access, scientific conference attendance, cross-regional study trips, etc.) HIV/AIDS: The most recent Norms and Procedures document from the National HIV/AIDS Program (PNLS) cover the management and coordination of numerous areas including STI prevention and case management, laboratory practices, and BCC. PISAF will support their integration into the FH minimum package of services. Although a national ARV policy has been developed, and some pilot implementation has been done by the MOH and MDM, the country is gearing up to expand ARVs greatly under 2006 GFATM. The URC team will help the PNLS ensure the quality of ARV program implementation, including drug management and patient compliance through development of clear protocols and guidelines supported with job aids, etc. Other HIV partners will be MDM, UNICEF, CRS and Plan International. The PISAF team will also help facilitate discussion at national and DDS levels about exploring other models besides CIPEC for expanding availability of HIV information and services at the peripheral level. Malaria: Our malaria advisor will assist with the ongoing work of introducing ACT and developing detailed new policy guidelines We can apply the cost estimation methods for a switch to ACTs that PHRplus developed (in collaboration with the World Bank and RBM) and applied in DR Congo and Tanzania. Additional policy decisions may need to be made about how ACT would be provided in the private sector and by community-based health volunteers, including associated changes in diagnostic protocols. In addition to supporting the integration of the new guidelines in the family health minimum package, we will work closely with the World Bank and the Global Fund to contribute to the five-year malaria strategic plan being developed. We will facilitate the adoption by the MOH of USAID-developed tools and methods related to malaria and provide analytical support and development of financial systems for ITN subsidies. Maternal and Neonatal Health: PISAF will support the finalization of the national strategy to reduce maternal and neonatal mortality, and will assist in refinement and implementation of the main strategies identified such as strengthening EOC training and supervision, better prevention and management of PPH and increased use of AMTSL, financing for obstetric emergencies, building on community-based emergency transport initiatives, and strengthening PAC capabilities. We will disseminate the results of the EOC collaborative and the survey on health worker competency and use these mechanisms to make improvements and take them to scale, e.g. the new PMTCT guidelines. Pilot activities to inform policy might also include testing kangaroo newborn care and management of sepsis in communities. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 113 A More Detailed Description of our Strategy for EONC and PPH is in Appendix at the End of this Section. Provide support for understanding financial implications of changes in treatment policies. Based on needs identified in collaboration with MOH counterparts, PISAF will support cost-analyses for new major policies, such as the implications of presumptive treatment of fever versus increasing access to lab or rapid diagnostic tests for use of the new anti-malarial treatment, subsidization of insecticide-treated bednets; applying the AIDS Treat Cost Model used in Uganda, Zambia and Nigeria to assist with scale-up of ART,; and expansion of the expanded minimum package of services. Funds for short-term technical assistance on these subjects have been included in the proposed program budget. Explore and evaluate cost-effective communication of norms strategies: Examples include 1) Refine and institutionalize the pre-service curricula already adopted for IMCI and EONC in health worker training institutions in Parakou and Cotonou, along with appropriate modules to reinforce supervisory improvements. This will include training the faculty at these pre-service training institutions to provide pre-service training to established standards of quality, updating curricular and doing TOT on new clinical norms included in the FH minimum package of service. Modules will also be developed on community-based insurance (mutuelles) and on health management; 2) Collaborate with professional associations such as the Public Health Association and the Medical, Nursing and Pharmacists Associations to keep them up to date on changing policies, guidelines, etc.; 3) Explore the use of collaboratives and other innovations as mechanisms for in-service training: The use of learning sessions in collaboratives provides time for discussion of new or revised norms in the discussions of how to best operationalize them. Disseminate norms and update regions so their regional training teams can provide in-service training. Collaborate closely with other donors: URC and Abt Associates have demonstrated through their work their ability to collaborate and leverage other donors‘ activities and resources. PISAF will continue PHRplus‘ collaboration with the Swiss, French, and Belgian Cooperations on mutuelle development in Borgou/Alibori and we are ready to collaborate with the ADB and UNFPA in Zou/Collines, where these latter plan to work on mutuelles too. The URC team will continue to actively participate in key intervention-specific coordination mechanisms, such as for those for HIV/AIDS, malaria, etc. We will also discuss assisting the MOH to take a more active role in coordinating their donors, for example in developing coordinated pricing policies related to ITNs. Provide assistance on commodity security to the Government of Benin, in the development and adoption of a National Commodity Security Strategic Plan. This will include technical assistance to contraceptive projections and planning, data collection and analysis, and donor coordination. Benin has begun development of a Commodity Security strategic plan with support from UNFPA, the Benin mission and technical assistance from AWARE-RH. We will continue to work closely with AWARE, facilitated through Abt Associates, and provide assistance to: reviewing existing workplans and identifying additional technical inputs needed to finalize contraceptive security plan; incorporating efforts being carried out at sub-region level; facilitating the adoption of contraceptive security plan by the MOH and donors. Other activities include reviewing the contraceptive procurement tables for 2005 and provide assistance to develop CPTs for 2006 and possibly following years. The provision of ART will be part of commodity security development. Scale up the successful model used under PROSAF by creating regional and then zonal warehouses in Zou/Collines; and explore alternative supply sources for products and drugs. 114 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Assist the MOH to acquire additional financial support: Provide technical support to the GOB for leveraging GFATM funds, implementation and proposal development, as requested by PNLP and the RBM country partnership. Such support could be provided as well to the PNLS. Such assistance can include cost analysis and budgeting, malaria ACT scale up and ART scale-up planning. Our partnership brings experience in supporting other countries such as Uganda, Ghana, Nigeria and Zambia develop successful proposals. We will work to help the CCM develop feasible health system strengthening interventions as part of their GFATM proposals. Support the government’s epidemic preparedness program with limited technical assistance to strengthen regional response capacity for meningitis and cholera epidemics and for the challenges posed by the Avian flu. Our partnership has experience with assisting with avian flu influenza preparedness planning in Ghana and Georgia. Support for HMIS, surveillance and notification systems, commodity management will maintain an eye towards ensuring that policies are clearly articulated and capacity is built for epidemic response, although this will primarily be supported by WHO. Provide support to the Benin USAID Mission by sponsoring training participants and organizing meetings as agreed upon with the Mission, including providing data in the TraiNet data base, as well as providing technical expertise and coordination as needed to enhance SO results. Illustrative Key results: Strategic Plan adopted and implemented for mutuelles Policy on decentralized warehousing Commodity Security Strategy in place and implemented Minimum package of Family Health Services expanded Mechanism created and used for continuous updating of clinical norms and policies Cost-effective methods for communicating norms tested Central MOH engaged in scale-up/institutionalization of effective strategies Updated policy on community-based service provision SubIR 1.2 Increased Health System Management Capacity Created Most of the approaches listed below have been shown effective in the Borgou/Alibori and will be strengthened there, while being introduced in Zou/Collines. Figure H-2. Diagram of QA QA Improving quality Defining quality Monitoring quality These approaches (described below) have helped focus departmental and health zone efforts on strengthening weak management areas, on monitoring progress, and on making continual improvements to improve performance. These methods have been effective in changing the way people view their work and their responsibilities, as norms become clear and people can see how they are doing. We would suggest including participants from Ouémé/Plateau and others the MOH would like to support in training events to facilitate expansion in other regions. For Borgou/Alibori, we will work closely with the USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 115 regional and zonal management teams to take their health system to the next level of performance, exploring and developing a longer-term plan for improving performance, linking clinical and management performance and epidemiological data with client input and cost analysis and resource allocation. We will provide opportunities for key leaders and decision-makers to visit and learn about high performing districts and health systems through study tours and support from appropriate short-term technical assistance. Key activities under this sub-IR are as follows to be implemented in Borgou/Alibori and Zou/Collines: Carry out management assessment for the Zou/Collines and Borgou/Alibori. This comprehensive evaluation tool focuses on the performance of each management system as well as compliance with clinical standards of the minimum package of family health services. The methodology provides in-depth data for health care managers at all levels to determine gaps in performance against established standards and thus allow them to set priorities for improvement. The management assessment is a capacity building activity in itself, as well: the assessment is done by and with health care managers themselves, who participate in every step of the process starting with the review of existing standards and corresponding assessment instruments, through data collection to review of data and planning accordingly. These management assessments will allow PISAF to focus its support on key priorities at health zone and departmental level, tailored to local priorities. Lessons learned from implementation of the management assessment in Borgou/Alibori allow us to carry them out much more quickly and easily. Strategic planning of clinical and management improvement for health zones and regional directorates. This process will use the management assessment data, but will simultaneously organize regional workshops to share best practices and develop regional strategies for Borgou/Alibori and Zou/Collines as well as Ouémé/Plateau. These plans help foster an emphasis on what is needed to achieve the desired outcomes of the heath system. We will assist the regions to develop regional QA institutionalization plans in accordance with national QA priorities and directives and support their implementation, and explore innovations and fostering ongoing improvements in quality through the use of ―recognition‖ systems such as the gold star program, balanced scorecard, the European model of excellence, or barometers. Fostering Monitoring of Performance: Use of the scoreboard (tableau de bord) or a simpler tool for monitoring performance: The scoreboard facilitates better use of data for decision-making to increase program impact, by providing a regular measure of performance. The scoreboard has become a well-established tool in Borgou/Alibori, providing regular information for priority health indicators. Data are collected through rapid assessments of quality and other methods (particularly the quarterly monitoring). With the help of the Swiss-funded PSS project, these scoreboards are now generated electronically in Borgou/Alibori. However, the scoreboard is still quite elaborate, and we will work with staff in the Borgou/Alibori (with participation of Zou/Collines) to review constraints to continued use of the scoreboard, how they can be mitigated or other alternatives for monitoring performance. For the resulting approach, we will work to include new areas in the family health minimum package, and in the Zou/Collines, we will support the installation of the scoreboard software and training of staff in the six zones of Zou/Collines as appropriate,, and will explore feasibility of doing so in Ouémé as well. We will also assess success of the piloting of the preceding birth technique (PBT) as a tool for evaluation and decision-making and expand it to the Ministry as requested 116 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Reviewing performance towards Annual action plans (i.e. sharing results among zones): PISAF will continue to support quarterly reviews of annual action plans, mid-term review of strategic plans. We will hold periodic meetings with representatives from both regions to share and discuss performance. During these reviews we will raise the issues of including budget lines for key activities and support, such as community mobilization, BCC, maintenance and repair for equipment, etc. Develop and implement a capacity development plan in each region, based on the findings of the management assessments, subsequent ERPAs, and strategic plans. The capacity development plan will cover both management and technical/clinical capacity building, include direct training, mentoring and coaching, learning sessions in collaboratives, and using shared data workshops to analyze and improve performance and facilitative supervision. We will assess the need for additional capacity building for the Borgou/Alibori regional training team and will work with the Regional Health Directorate for Zou/Collines to strengthen the pool of trainers there. We will use the best of the approaches developed under the various projects, such as the ―Facilitated Training Approach‖ and ―Learning for Mastery‖.. Organize study tour for key health officials to a country that has made significant inroads in the establishment of integrated health district with comprehensive performance assessment and improvement. Install internet capacity in regional health directorates and health zone offices for Zou/Collines: Like in Borgou/Alibori, we will create the infrastructure for internet, for communication and access to research and information. Recurrent costs (including repairs and maintenance) will be the responsibility of the region and health zones, and we will work with the Departments and health zones to provide budget lines for internet service, repair and maintenance. Several health zones have effectively used their government budgets to this effect in Borgou/Alibori. In the Borgou/Alibori, where PROSAF supplied computers and other equipment, the repair and maintenance costs for malfunctioning equipment were budgeted in the DDSP and the health zone budgets. An emphasis on the need for such budget lines was included in capacity building for financial management, budgeting and planning. In some health zones, where they were given older PROSAF machines, they used their budgets to purchase needed spare parts. In addition, PROSAF also provided training in small repairs. The use of government budgets or other funds for repairs, maintenance and replacement, however, depends on the leadership of the individual health zone coordinators or departmental officers. Assist health zones and health centers implement ascendant planning to develop linkages and mechanisms for community inputs up through the departmental level (including coordination of development partners active in the departments). Improve human resource and financial management: Improved human resource management within the framework of decentralization implies that some responsibility for hiring, supporting and assessing performance lies with the health zones and regions. PROSAF developed an integrated Human Resource-Quality Assurance training course which brings together essential features of each into a single framework with shared vision and measurable outcomes for health systems performance. This training will be implemented in the Zou/Collines, while in the Borgou/Alibori support will be provided with the implementation of the newly developed HR-QA management system. We will also strengthen financial management systems by providing up-to-date financial management training and will work with the DDSP of Zou/Collines to introduce computerized accounting as was done with the DDSP of Borgou/Alibori, simply using Excel for improved management of medications, equipment, etc. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 117 Illustrative Key Results Management assessments carried out in Zou/Collines and Borgou/Alibori Management improvement plans developed and implemented in Zou/Collines and Borgou/Alibori Capacity building plans developed Annual and triennial strategic plans developed and monitored Scoreboards functioning in Zou/Collines and Borgou/Alibori Ascendant planning implemented in all health zones Supervision system updated to include supervision and functionality of community￾based services Regional QA plans developed Human resource development plans developed and implemented in Zou/Collines and Borgou/Alibori Sub-IR 1.3 More Effective Civil Society Participation Created The work successfully carried out in Benin by URC and Abt Associates for community mobilization is one important aspect of improved participation of civil society. This support to expand and strengthen community mobilization will continue under PISAF. Successful strategies for engaging civil society include ascendant planning through the COGEA and COGECs, and innovations being developed include mutuelle support mechanisms at commune level. PISAF will build on or create synergy with other USAID-funded community mobilization and local governance initiatives such as EQUIPE, PRISM, RTI‘s decentralization training work, CRS‘s micro-finance, community health and nutrition activities, MCDI‘s deworming with PTAs, etc. Additionally, the project will reach out to civil society structures including the for-profit private sector to establish key alliances for improved health and to provide support and resources for priority activities. To support these and other efforts to engage civil society in health services management and delivery, the proposed structure of support is shown in the accompanying table. This structure builds on strategies already used in the Borgou/Alibori and Oueme/Plateau to assign responsibility for community mobilization (and BCC) at departmental and health zone level. PISAF proposes strengthening civil society participation component that will be tailored to the needs and resources available for each health zone of each department in which the project will be working. The greatest focus of support will be for activities in Zou/Collines. A certain level of support will be maintained for community mobilization activities in Borgou/Alibori to strengthen and scale-up the ―concentration zones‘‖ CBSA activities, and for consolidation and expansion of community activities in Ouémé-Plateau, especially for the ITN and other community mobilization activities for malaria carried out under the AIMI project. This support will be channeled through the MOH and small grants to ROBS, ROBS members and other NGOs/CBOs/FBOs as appropriate. Valuable lessons have been learned by the URC team through their previous work with communities in Benin about the importance of actively engaging political and religious leaders, non-health groups and the communities themselves in determining priorities and plans. The achievement of all three PISAF IRs depends on working with and through the communities themselves, at their pace, on their priorities. It should be noted that the mutuelle development 118 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES process used under PHRplus and to be followed under PISAF is a community mobilization activity in itself, both in organizing community members at various levels around the creation of a community mechanism for risk-pooling, but also the Commune MHO Support Committees which bring in various community actors and local government to support and sustain the mutuelles. Because each department has a range of actors and programs working in it, and the community structures differ, the prioritization of possible partners and geographic concentration of activities must be determined at department and zonal levels. However, the approaches and criteria that will be used for prioritizing partners, platforms and activities will be the same: The project will confer with MOH, local government, other line ministries, other NGOs and actors to do a ―mapping‖ at Departmental and then zone levels of who is doing what community mobilization activity where. This will allow identification of coverage, needs, potential synergies and potential overlaps to ensure that PISAF support takes into account and complements other support for community health mobilization. Zou/Collines Ouémé/Plateau Borgou/Alibori Community Supervision by MOH, NGOs or other groups of community actors. Examples: Community-based health agents; School clubs; Women‘s groups who, in addition to their normal group activities, promote ITNS; Growth monitoring volunteers; Volunteer mothers (Hearth nutrition activities); Emergency preparedness volunteers-Red Cross, mutuelles Commune/ Arrondissement Appointed liaison within Mayor‘s office Support from ROBS member NGOs through grant to commune-level activities Commune staff assigned to promote and facilitate mutuelle membership Liaison within Mayor‘s office, NGOs Limited support from ROBS member NGOs through grants to commune-level activities Liaison within Mayor‘s office Commune staff assigned to promote and facilitate mutuelle membership Support to mutuelles and from ABPF to community￾based services and activities Zone-level Community Mobilization focal person to be assigned from EEZS staff Support from PISAF staff to ROBS members to strengthen their capacity, and support from ROBS members through grant to Zone staff Support to development of zonal level unions of Mutuelles Community Mobilization focal person to be assigned from EEZS staff Possible support from ROBS through grants Community Mobilization focal person already assigned in EEZS in all zones. Technical support from PISAF to DDSP and to mutuelles development Support to development of zonal level unions of Mutuelles DDSP/ Region Counterpart at this level Community Mobilization focal person (to be designated among DDS staff) Support to DDSP from PISAF Community Mobilization staff Support to ROBS Regional Committee Support to development of Regional level Union of Mutuelles Community focal person (already designated to work on community activities during PROLIPO) Some support from PISAF and ROBS using people that were previously trained under Africare Community Mobilization focal person already assigned in DDS during PROSAF Some support from PISAF (Technical coordinator) and Mutuelles specialist. Support to ROBS Regional Committee. Support to development of Regional level Union of Mutuelles USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 119 Three types of coverage will be considered: health information, health services, and health mobilization activities. Some areas that are close to health facilities may have good access to the first two, but may not have had an effort made to engage the community in mobilizing either for self-help health activities (such as clean-up campaigns or care and support for PLWHA) or for increased responsiveness of the health facility to community preferences for the way health services are delivered. Some areas may be well-mobilized but lack easy access to community-accessible health information and services. Other areas may have health needs specific to their area (high transmission rates of HIV on a transport corridor, obstacles to emergency transport.) A different strategy will be needed for each case. In some cases, support for community-based services agents will make sense. In others, identification of women‘s or religious groups who can take a leadership role in mobilization for health activities may be the best strategy. These decisions of what to do at the community level must be made locally, within the overall strategy framework which provides a menu of community mobilization tools and approaches adaptable to the needs and resources locally available. Once mapping of coverage, needs and available resources is done, the project will work with the MOH and local government to identify their priority geographic and technical areas for the project to provide support for community mobilization activities through support to the MOH and/or grants to NGOS, community-based organizations and faith-based organizations (FBO). Priority areas will probably be those with low coverage of information and services, higher level of health needs, etc. The MOH as well as the NGOs/CBOs/FBOs will share their expertise (co-train each other) in successful community mobilization tools and approaches, including those used under PROSAF and PHRplus (e.g. COGEC capacity building, community participatory diagnosis and action planning, training and supervision of CBSAs, use of traditional media, training related to mutuelle development, creation of commune mutuelle support committees) and other projects (using women‘s groups to promote ITNs, working through schools, song contests) in order to use best practices from many Departments. Periodic review sessions for groups working in community mobilization will ensure sharing of lessons learned and innovations, and will incorporate feedback from the communities. Some community mobilization and education activities will be done department-wide, e.g. through use of rural radio or widely disseminated print materials. Some will be zone or arrondissement- specific for the priority areas. Once the priority geographic areas are identified, the project will support the MOH and NGO partners to work with local government and traditional authorities to ensure their buy-in and support for the proposed work with communities. Project community mobilization support for the MOH, NGOs and other partners will emphasize activities that: – Build on existing health activities and actors, that link health to other sectors‘ development activities, or that use existing civil society structures e.g. adding health activities onto an active literacy program, tying health education to micro-credit activities, or using traders‘ associations as a venue for discussing men‘s roles in reproductive health – Use a mix of face to face communication, traditional and mass media to raise the profile of health issues and create discussion and dialogue- the first steps towards changing social norms – Make healthy behaviors and participation in health mobilization easier and more rewarding by creating positive reinforcements through recognition, increasing social value, etc. 120 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES – First target communities that have the critical mass of population, need, and dynamism that will make early investments pay off and serve as a role model to other communities – Begin with perceived needs (helping communities identify their priorities and responding to them before promoting less-favored or more controversial topics- e.g. ensuring access to quality anti-malarial treatment and helping improve community response to emergency transport needs before pushing for community-based family planning services) – Ensure that communities contribute to increasing access either through contributing to outreach services or by creating community based services – Ensure that communities participate in shaping and managing health services through participation in COGEAs and COGECs, improving community leaders‘ organizational and communication skills, creating periodic ―town hall‖ feedback meetings between health facilities and the communities they serve, etc. – Advocate for increased participation of women in COGEA and COGEC, mutuelle management committees, and the involvement of women‘s associations directly in mutuelles, micro-credit, and community level health activities. – Above all, are flexible and responsive to community preferences and opportunities. One of the first activities in the entire process of strengthening and expanding community mobilization activities in Zou/Collines will be assigning of responsibility for oversight of community activities to existing MOH staff members as done in Borgou-Alibori and Oueme￾Plateau under previous projects (described in detail in the table above) and providing them with appropriate training (based on management assessment findings) in managing and supporting community mobilization activities. In addition, PISAF will work with the departments and health zones to include line items in their annual budgets for community mobilization activities (transportation, etc) as well as leveraging with NGOs working in their area for collaborative visits and activities. Departmental and Zone-level community mobilization working groups will be created that bring together all the actors working in community development in a given area and will allow better coordination of efforts and the vital sharing of successes and challenges that fosters progress. The mapping, training of NGOs, community priority setting and activities described above will follow. Periodic high-level meetings will be held between MOH and administration officials to ensure that lessons learned on community mobilization and shared and discussed. PISAF will establish a lean set of process, output and outcome indicators for community mobilization activities that will allow tracking of progress and results. Specific activities include: Carry out assessment of status of community mobilization and mapping of civil society actors in all zones Develop and implement consensus implementation mode for increasing civil society participation. This model will certainly include some of the following steps, adapted as needed to existing structures and activities: 1) Carry out initial participatory problem analysis and development of proposed solutions, involving other line ministry agents and community groups such as credit associations, women‘s‘ groups, FBOs, and NGOs 2) constitute or strengthen COGEAs and COGECs and advocate for increase in number of women members or create a separate ―women‘s interest group for health‖; 3) training of COGECs in management, oversight and collaboration with other actors and support of monthly meetings 4) develop/strengthen partnerships between health centers and communities through Memoranda of Understanding; 5) based on proposed solution to local issues, identifying priority activities and implementing activities USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 121 Illustrative Key Results: Increased number of joint activities between communities, mutuelles, and health centers to improve quality and access Increase number of women in leadership positions in COGEC/COGEA Increased number of functioning COGEC/COGEA Effective COGEC/COGEA participation in ascendant planning Increased involvement of local leaders in health issues Increased number of Commune MHO Support Committees Work with and through MOH focal persons and other zonal and commune-level community mobilization agents to ensure that all community-based activities are integrated from the outset into supervisory structures of the health zone and in performance monitoring activities of the zones, and to ensure that health mobilization activities are harmonized with other civil society strengthening initiatives such as work with PTAs, etc. Support expansion and strengthening of mutuelle support from communes and the role of mutuelles in advocacy for desired aspects of health services: PHRplus has promoted an innovation in Borgou/Alibori to strengthen the role of civil society in supporting mutuelles with the creation of Communal Mutuelle Support Committee. In addition, mutuelles have the potential to use their financial leverage to negotiate with health facilities and to work effectively with them to improve the quality of care. Models for this have been tested in Senegal and the QA environment already existing in Borgou/Alibori makes this fertile ground for expanding the voice of civil society. Advocate at all levels of government for increased role of civil society including elected mayors, NGOs and CBOs and the private sector in provision of family health services and information Link community mobilization activities for health with other activities: such as small grants for infrastructure, income generating activities, literacy/numeracy courses or other activities that provide immediate and visible results and increase self-confidence and commitment of communities to mobilize for social change. NGO Grants: Grants will be given to NGOs and CBOs as described in IR 3.1. More information on URC‘s approach to strengthening civil society participation in creating a supportive policy environment is described under IR 2.2 and IR 3.2 122 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Health Center District Hospital Regional Hospital  Favorable Environment  Clinical Content / Knowledge, Behaviors et Key Practices  QA Content / Introduction changes and socio-cultural adaptation  Coaching (Intensive follow-up)  Documentation and sharing of improvement experiences Continuum of Improvement in Health System  Definition/Adaptation of Norms and Content of Care  Baseline assessment  Establishing improvement objectives  Establishing monitoring indicators  Continuous improvement Health Center District Hospital Regional Hospital  Favorable Environment  Clinical Content / Knowledge, Behaviors et Key Practices  QA Content / Introduction changes and socio-cultural adaptation  Coaching (Intensive follow-up)  Documentation and sharing of improvement experiences Continuum of Improvement in Health System  Definition/Adaptation of Norms and Content of Care  Baseline assessment  Establishing improvement objectives  Establishing monitoring indicators  Continuous improvement CBS INTERMEDIATE RESULT 2: ACCESS TO QUALITY SERVICES AND PRODUCTS INCREASED Figure H-3. Continuum of Improvement in the Health System The government of Benin has made major progress to expand services both in the public and the private sector over the past years. Much of the improvements are linked to the operationalization of the health zones and implementation of the family health protocols and clinical norms. Successful strategies that have been tested in Borgou/Alibori and Ouémé include offering an integrated package of services, quality assurance, IMCI, and improved management strategies and approaches are ready to be scaled-up in new regions. For example, using these strategies, 92% of health centers in Borgou/Alibori are now offering integrated services, 86% of health centers receive a formative supervision visit every quarter, and 87% ordered their drugs and products on time and in the correct amount. Strategic Approach: Quality and access are two related concepts. In fact, many definitions of quality include access as a component of quality. For the purposes of this program, access will refer to geographical and financial barriers to health service utilization and the availability of products and staff to provide care, while quality will refer to the service provided in terms of compliance with norms, attention to client needs, continuity of care. Improving access includes bringing services closer to communities through the use of community-based agents with health products at their disposal, strengthening the role of NGOs, and ensuring that medicines are available in the clinics. Quality also includes ensuring that a patient will receive appropriate care at different levels of the system, will be referred in a timely manner and that the different levels clearly understand their role and responsibility with regard to the care being offered. Our strategies will include: Reinforcing the quality of health services delivered using a quality assurance approach. We will be continuing to use the three-pronged approach of Quality Assurance: developing, updating and communicating clinical and service delivery guidelines (see IR1.1), fostering ongoing monitoring of compliance and implementation, and institutionalization of continuous improvement activities. Clinical improvement will be coupled with facility-based improvements focusing on essential management systems such as logistics, integrated service delivery and collaborative improvement based in health centers. Under the direction of the health zone, health centers collaborate to achieve identified improvement objectives, such as IMCI, emergency obstetric care, stock-outs and counseling and testing for HIV/AIDS and STIs. As much as possible we will foster learning across regions by supporting field visits and capacity building by local health agents. Implementation of innovations in capacity building for delivery of quality care. Capacity building will include a continuum of assistance which will use as its starting point analysis of USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 123 compliance with standards during the management assessments, update training to familiarize health workers with new or updated norms, monitoring performance, mentoring, and facilitative supervision, and the use of ―tutorat‖ (coaching) to ensure post-training follow-up and facilitate the ability of heath workers to integrate what they have learned into their practice at their health facilities. Reinforcing logistical management capacity to ensure access to essential health products: To improve availability of essential drugs and other family health products in health centers we will work with the regional health directorate in Zou/Collines to replicate the creation of regional and zonal warehouses. Our support at the national level described under IR 1 to strengthen CAME and develop a contraceptive security strategy are essential components. We will work with each regional health directorate to consolidate existing experiences, strengthen the role and involvement of the health zones and ensure that support for the expansion of community-based distribution is consistent with policy discussion and consensus on the national community mobilization strategy. The project will also help explore alternate sources of supplies and products for the public sector, and mechanisms for improving availability and correct use of recommended medications in the private sector. Assisting communities to improve their financial access to care. For improving financial access, the strategy includes evaluation and expansion of mutuelles, pilot testing improved cost￾effectiveness and cross-subsidization, and income-generation activities to support health services, and other community mechanisms to ensure that financial barriers do not prevent people from using care (especially emergency care). We will search for practical methods to target subsidies to the disadvantaged by working with health and social welfare personnel to develop practical definitions and indicators of who should be classified as disadvantaged. This will allow exemptions, subsidies, or the payment of mutuelle premiums to be used to address the financial access issues of the disadvantaged. We also will bring to the design of these programs international experience on incentives and disincentives operating in the process of granting benefits to the disadvantaged. We will also continue to evaluate options at community and health center level to ensure that women and children in need receive timely and affordable transportation to medical care. Staff working on these activities will include the Chief of Party, and the technical advisors in all clusters-- health systems strengthening, family health, and in community mobilization and behavior change. Short term technical assistance will be used as needed, particularly in the areas that are relatively new for Benin such as scaling up of ACT, ARVs, and mutuelles. Sub IR 2.1 Selected Products Available at Public Sector Facilities Specific activities include: Facilitate the creation of regional and zonal warehouses in Zou/Collines. Following the development of a consensus on supply systems for essential drugs, contraceptives and family health products, PISAF will assist in the validation of the recently completed RPM+ study and recommendations, carried out with USAID and UNDP. Completed documentation of the experience of regional and Health Zone warehouses in Borgou/Alibori will serve as an important input in the discussion on how to improve the reliable flow of products to the regions and zones. Close collaboration will be maintained with other donors, such as PSS, UNICEF, Médicins du Monde. Then, based on the experience of the Health Zone warehouses in Borgou/Alibori, decentralized warehousing will be introduced in Zou/Collines. PISAF will provide technical assistance in the process and work closely with other donors to obtain necessary funding. Build capacity for improved logistical management system (supply and distribution) of family health products at the Health Zone level through training in logistics management, thereby effectively transferring responsibility for improved logistical management of family health 124 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES products to regional and zonal level. Based on the approaches and materials developed for the Borgou/Alibori, we will provide training of Zou/Collines health managers and providers in the principles of logistics management, as well as the introduction, support and institutionalization of continuous monitoring and improvement of stock-out indicators by HZMTs. Exemplar health zone staff from Borgou/Alibori will assist in this training. Explore alternative sources of products: The use of IPPF contraceptives in MOH clinics to cover temporary stock-outs and the experience of PSI/Benin with UNICEF funding selling branded ANC ITNs in public clinics in Zou/Collines are examples of how private sources can help ensure that public structures have available and affordable supplies. The project will assist in identifying contingency plans when the CAME is not able to supply products for public or private outlets such as NGOs. Strengthen the capacity of community-based service agents (CBSAs) to provide access to expanded range of key family health products for their communities through training and supervision. Implement expanded number of services and products to be provided through CBSAs: Distribution in Borgou/Alibori by community-based health agents currently includes chloroquine, paracetamol. aspirin, ITNs, condoms, spermicide; re-supply of pills, and ITN re￾treatment kits. UNICEF has been piloting community-based ARI treatment, PROSAF and HEPS did de-worming, and CRS uses community-based HEARTH nutritional rehabilitation. PISAF will build the community-based services menu on these experiences and other internationally-recognized strategies to reduce child morbidity and mortality such as pre￾natal iron folate, post-partum Vitamin A, and zinc with ORS. After working with the MOH to review and update services and products distributed by CSW, PISAF will assess and develop monitoring system for re-supply of CBSAs and develop improvement strategy, including use of mutuelles. Include monitoring indicators for health zones to measure continued performance/presence by CBSAs. This will serve both to assess the relative role they play in provision of services, relative to fixed facilities, and to ensure that health workers carry out their responsibility to supervise community workers. We will also work with the Ministry of Health to pilot different models of CBSA support (including contracting to NGOs, use of EEZS focal person, linkage to programs in other sectors) to ensure improved performance and involvement. Sub IR 2.2 Selected Products Available at Private Sector Outlets Although not directly the responsibility of PISAF, we will work with social marketing partners and others as appropriate, and explore how to exploit supply channels such as IPPF‘s contraceptive supply chain, private wholesalers, NGO or religious hospitals‘ purchasing mechanisms as an alternate to CAME as necessary. We will also expand use of women‘s groups to promote ITNs and we will explore the possibility of inclusion of family planning products (such as standard days beads,) services, and ITNs as items in mutuelles’ benefit packages. We will also work with Mutuelles to organize and promote the sale of ITNs to their members as a way of both saving money for the mutuelle (less cases of fever to be treated) and of possibly Illustrative Key Results: Creation of departmental and zonal warehouses in Zou/Collines Health workers in Zou/Collines trained in logistics management Expanded network of community-based distribution of family health products Pilot test on adding products to CBSA kit and results validated and implemented USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 125 generating revenue. We will pilot test an innovative approach to strengthening of private vendors and pharmacies, using a successful model from Kenya of working with pharmaceutical wholesalers to disseminate new treatment norms and promote correct dispensing by shopkeepers and pharmacies. Sub IR 2.3 Quality Family Health Package Available at Targeted Public Sector Facilities Specific activities for the expansion and update of minimum package of family health services in Borgou/Alibori, Zou/Collines, and Ouémé/Plateau include communication of additional norms and standards, discussions with department and health zone staff about operational issues of integrating them into the current package of services, updating clinical skills, creating mechanisms for monitoring performance of the ―expanded‖ package of services (through assessments, updated supervision guides), creating appropriate job aids for workers, ensuring through logistics management that the needed drugs and supplies are available (see IR 2.1), and working on continuous improvement of service delivery for the expanded package. Through the use of collaboratives and other QA approaches, implementation barriers to the expanded package will be specifically addressed by the health facilities and health zones, both in terms of individual facilities and in terms of the linkages between levels of the health system. Specific activities include: Update clinical skills in Borgou/Alibori, Zou/Collines and Ouémé/Plateau. Health workers will need to have an opportunity to update their clinical skills for revised norms and new services to be offered. The capacity building plan discussed in IR1.2 will include both revised guidelines such as malaria treatment including IPT with SP, expansion of family planning to include standard day method and VCT and PMTCT. Capacity building will focus on a range of methods, including using Learning sessions at Collaboratives, and training sessions will be followed by ―tutorat‖ coaching where trainers go to sites where participants work to help them integrate their new skills into their work environment. We will continue to search for innovations in training methods that do not deplete health facilities of their staff, but are effective in transferring skills. Monitor and improve performance. Using data from health zone monitoring mechanisms and facility-based self-monitoring, we will support health zones in Borgou/Alibori and Zou/Collines to review and update indicators and best practices using the latest norms, as well as strengthening collaboratives managed by health zones. A sample diagram depicting indicators for improved essential obstetric care shows the interrelationships between the zones for improved quality of care, where an interaction of health centers, zonal hospitals and the Regional referral hospital work together to ensure adequate care of obstetrical emergencies. Figure H-4. HIV and Infant Feeding Question and Answer Guide Develop health facility model sites for quality of care which will pilot new models of care. Some health zones have been able to do exemplar work in certain areas, and PISAF will assist them to take their work a step further, but using them as pilots for testing innovative models of care. Based on a positive experience in Eritrea and WHO guidelines on safe drinking water, the URC team will propose to health authorities and USAID to pilot a low-technology, low-cost source for chlorine used by regional hospitals and distributed to health centers in their districts. This can replace thousands of dollars a year spent on commercial chlorine bleach of variable quality, and can also 126 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES be used by communities for point of use drinking water purification at a fraction of the price of boiling or infrastructure. Update supervision guides to include new guidelines and expand facilitative supervision; Disseminate job aids and develop new job aids as appropriate to support improved performance. URC has developed well-received job-aids which will be reviewed, if necessary updated and reproduced and rapidly disseminated. URC also has available from its ground￾breaking work in PMTCT a set of job aids for caregivers, mothers and providers to inform and counsel on the latest norms to prevent transmission from mother to child. These can be quickly adapted to the reality of Benin and introduced as well. Develop improvement collaboratives around key issues of the expanded family health service package: continous improvement strategies, using collaboratives or health improvement teams as appropriate, will be introduced in the Zou/Collines and Borgou/Alibori, to address operational and community barriers to coverage with quality services. Illustrative examples of specific issues to be addressed at service delivery level: Malaria – integrating subsidized ITNs for pregnant women and children, use of ACT, and IPT with SP into the family health minimum package. IMCI – use of the 6-day training course. Borgou/Alibori and Ouémé/Plateau health staff have all been trained in IMCI, but high turnover in staff will require periodic training programs to be organized. For Zou/Collines, PISAF will train those not already trained or not programmed to receive training by the Swiss project, followed by a supervision/post-training follow-up visit 4 weeks after the end of the training session. This visit to each of the trainees enables the supervisors to assess in-depth the level of compliance with norms and guidelines communicated during the training and identify any possible service organization issues which may hinder effective application of newly learned skills. Then after this follow up visit, routine supervision conducted by health zone coordinators will incorporate this new area in its targeted supervision. Workers already trained but who have not received appropriate follow-up supervision in Zou/Collines or Oueme/Plateau will receive follow up visits. To bring Ouémé-Plateau in line with the integrated approach used in Borgou/Alibori, it will be important to work with the DDSP/Ouémé-Plateau and all the IMCI focal persons at zone level to reach a consensus that IMCI supervision be included in routine supervision, and no longer treated as a vertical program. Thus, the PISAF Malaria Advisor and the Health Zone Management Strengthening Advisor will go to Ouémé-Plateau and work with them to ensure that all the health workers trained on clinical IMCI benefit from an IMCI-specific follow-up visit by IMCI trainers, and that IMCI supervision then be fully integrated into routine family health supervision. New malaria treatment guidelines will be integrated into the IMCI curriculum. Essential Obstetrical and Neonatal Care – Establish links between existing URC/QAP-led EONC improvement collaboratives (Zou/Collines, Ouémé/Plateau and Atlantique) with those in the Borgou/Alibori to report on key indicators and develop new strategies for improvement in specific areas. Introduction of PAC, Promotion of ITN in maternities and use of IEC materials developed by the AIMI and PROSAF projects for EONC and malaria. Reproductive Health – The project will focus on integration of FP and HIV into all RH services, and will ensure inclusion of the standard day method into FP curriculum. PMTCT and VCT – Currently, access to PMTCT and VCT are quite limited, and although GFATM resources will help increase coverage somewhat, shortage of equipment and commodities remains a major challenge. PISAF will use a combination of leveraging project resources with MOH and MDM resources and those of other donors, and partnering with USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 127 the private sector, to increase the number of sites offering PMTCT and VCT in public and private facilities as possible. STI and HIV/AIDS: Strengthen STI diagnosis and treatment through competency-based training and job aids. Recognizing the high rates of dual infection (TB and HIV) in HIV+ individuals and the impact of rising TB rates on broader public health, PISAF will advocate and support greater integration of TB testing and treatment for PLWHA in all project areas. PISAF will work with TB services to ensure that strong HIV counseling skills and referrals processes are in place to identify and bring into care the dually infected. Sub IR 2.4 Financial Access to Health Services Increased Specific activities include: Prepare a report on the Borgou/Alibori mutuelles to identify best practices and lessons for the expansion of mutuelles in Borgou/Alibori and Zou/Collines. The report will build upon data on hand from PHRplus and includes items like: Descriptions of schemes: benefits packages, premium amounts and frequency, waiting periods, provider agreements, governance arrangements, group basis for mutuelle, other features; Performance of schemes: number of members, membership retention, financial situation, premium payment rates, rates of use of indicator services. We will seek to collaborate with other donors and projects (such as those of the Swiss, French, and Belgians) to include similar data from the mutuelles that they support. Create and reinforce mutuelles in Borgou/Alibori: Specific tasks include: expand membership in and improve functioning of mutuelles in the original two Health Zones (Sinendé and Banikoara) in Borgou/Alibori, through IEC, support from Communal Mutuelle support committees, sensibilization, etc.; expand mutuelles: to two of other five Health Zones of Borgou/Alibori Region (the Swiss, French and Belgians are covering three of the Health Zones) for a total of 26 new mutuelles. Steps include providing training to commune-level agents and to COGEC as appropriate on mutuelles, their development and providing on￾going support. Create mutuelles in about 8 arrondissements in Zou/Collines: We will be working closely with the African Development Bank and UNFPA in the Zou/Collines region, focusing our efforts on the remaining 10 arrondissements that are not currently in the plans of the ADB and UNFPA for mutuelle creation and support. Assist with the creation of regional federations (unions) of mutuelles in Borgou/Alibori and Zou/Collines, based on the models used for zonal unions. Create capacity in the unions to provide technical support to their members to increase sustainability. Use mutuelles to promote high impact services utilization A new manual on quality improvement for MHOs is being published by PHRplus by the end of this year. We will adapt it and use it to promote improved quality. In addition, we will work with mutuelles to encourage their members to use preventive services (such as in Mali, where membership Illustrative Key Results: Providers delivering integrated quality health care Number of functioning collaboratives Providers trained in IMCI and expanded family health package Formative supervision provided to each health center 4x/year 128 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Illustrative Key Results Increased number of functioning mutuelles; increased membership Reduced financial barriers for the poor. Federation of mutuelles created Sustainability mechanisms in place for mutuelle development and support was contingent on promising to use preventive services such mosquito nets, vaccinations, ANC, etc.). Develop sustainability plans for Mutuelles in the zones and regions: Working with COGEC/COGEA, the members of the commune mutuelle support committees (comités communaux d’appui aux mutuelles) and zonal and regional mutuelle unions, develop mechanisms to ensure mutuelle sustainability, such as PHRplus‘ success in arranging for communes assigning or hiring personnel to support mutuelles (especially in promoting population membership), financial support through income generation activities, linking mutuelles, with micro-finance schemes, and sharing information about best practices. Test out mechanisms for ensuring access for the poor by reducing their health care expenditures: Conduct an analysis to identify the barriers to use of services by the disadvantaged. Use the results of the analysis to examine with health facilities ways of reducing these barriers, including costs to patients. Explore with mutuelles, COGEAs, and communes ways to address the issues identified, such as to subsidize mutuelle premiums for the poor and to explore the possibility of income generating activities (such as those developed under PROSAF) to subsidize the poor. Linking mutuelles with micro-credit and income generating activities to enhance sustainability: Participation in micro-credit programs allows participants to receive small loans that they use to make investments to increase their income earning potential. PISAF‘s community based service agents (CBSAs) will jointly promote the use of preventive and curative health services and good health practices within the home and participation in micro-credit schemes. This will be done by arranging for joint training and supervision of CBSAs and micro-credit promoters. Further, experience in Rwanda and more recently in Senegal indicates that many micro-credit scheme members are interested in using the micro-credit loans to pay mutuelle premiums—as the guarantee of payment for health services is viewed as a good ―investment‖ that ensures that members will be healthy and productive and better able to use their other loans for directly productive investments. Hence, PISAF will work with CRS and other micro-credit assistance organizations to find ways to ―introduce‖ micro-credit scheme members to the possibility and benefits of joining mutuelles. INTERMEDIATE RESULT 3: DEMAND FOR HEALTH SERVICES, PRODUCTS AND PREVENTIVE MEASURES INCREASED Our approach to demand creation goes beyond messages or promoting knowledge of correct practices. It includes enhancing community mobilization to take more responsibility for health outcomes, helping health workers learn to communicate effectively, strengthening community members‘ understanding of and access to information; and working with MOH at departmental and national levels to promote integrated strategies for information provision and community based services. Activities under this IR will be closely integrated with those under IR1, i.e., closer involvement of civil society groups in health promotion and service delivery, and IR2, i.e. provision of accessible and acceptable client-oriented services. Although the main focus will be USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 129 on departmental and peripheral demand creation activities, a minimum level of coordination with national level BCC strategies and activities will ensure harmonization and complementarity of results, as well as leveraging resources available at each level. Strategic Approach: Key to a strong BCC program is having people to focus on this activity. Experiences under PROSAF and PROLIPO have helped to institutionalize oversight at the regional level of community activities and behavior change through the appointment of a community mobilization/BCC point person at the Regional and Zonal levels, including these tasks in job descriptions and supervision guidelines, and institutionalized budgeting by local government for health support activities. PISAF will use this and proven BCC approaches for demand creation and community participation in Zou/Collines, continuing a minimal support for activities in Borgou/Alibori and Ouémé/Plateau, particularly to achieve synergy with other UNICEF and CDC-funded activities there, and will introduce innovative ways to further engage the population in improving their own health. Numerous BCC materials and strategies have been developed by PROSAF, PROLIPO, as well as PSI, CRS, MDM and others working in FH services. We will review existing materials and approaches and use those that have proven to be especially effective in the zones and communities. These include IPC/C, peer counseling, participatory theatre with facilitated peer discussion groups, counseling cards in short story form, small-format flipcharts, targeted radio broadcasting using local radios, and child‘s age￾specific health messages. Components of our strategic approach to demand creation and community interventions are: Use BCC approaches that go beyond simply creating awareness and knowledge: Through community mobilization and other efforts, PISAF will work to empower and engage communities, mutuelles, COGEAs and other local organizations to identify needs and prioritize and seek solutions, informing intervention design Multi-sector approach to creating supportive environment and promoting desired behaviors-: We will work with schools, women‘s groups, credit/producer associations, and mutuelles to help disseminate ideas, create discussion, provide support or help eliminate obstacles to adoption of healthy practices. For example, in Mali, mutuelles asked members to use preventive services as part of their membership commitment (mosquito nets, vaccinations, ANC, etc), Use of formative research to identify barriers to adoption of behaviors and use positive deviance or do-er/non-doer analysis to identify existing practical solutions within communities or health centers to help non-doers adopt desired behaviors Taking the long-term view of behavior change: PISAF will start both community mobilization and behavior change interventions with small, targeted attractive and visible actions that increase confidence and willingness to try more difficult changes. The long-horizon phased approach combined with frequently updated activities (something new) will help maintain interest and move people step by step along the behavior change continuum from awareness to understanding to interest to adoption to maintenance to promotion without overwhelming them. We will encourage a multi-channel/multi-media approach using local radio, print materials, testimonies, peer education, support groups, schools, community theater, working with community groups in other sectors, e.g. cooperatives, djemijan associations. All strategies will be culturally and locally adapted, and targeted to well-defined audiences. Conduct operations research to test the effectiveness and costs of various interventions, service models and communication channels: Both in the area of BCC and in training, service delivery and access, there are many innovations to be tested and evaluated, discussed at operational and regional levels, and fed up to the central level for wider-scale implementation. We will conduct several OR studies each year, based on a set of priorities determined annually, with results fed back into programming and policy decisions. Sub IR 3.1: Knowledge of Appropriate Behaviors and Preventive Measures Improved Figure H-5. Community Mobilization and BCC Training Process Specific activities include: Identify key priorities for BCC with regional and zonal heath teams, based on gaps in knowledge and obstacles to behavior change: PISAF will work with regional and zonal level community mobilization/BCC focal people (see IR 1.3) to determine what BCC and community mobilization strategies have been effective and what concrete results exist in order to identify BCC targets and areas needing formative research. Groups, local partners, and civil society organizations that could prove effective will be identified and engaged through capacity-building, provision of technical content and materials, and material support as discussed below. Develop Annual BCC workplans at regional, zonal and local levels: PISAF will support the creation of these plans, focusing on use of successful media and messages, and identifying areas where behavior change will encounter more obstacles and may require formative research or local adaptation, e.g, work with association of PLWHA to develop effective strategies for care and support, prevention of HIV stigma, and prevention of infection in vulnerable groups, including innovative techniques to enable PLWHA to give testimonials in ―safe‖ ways. Identify ways to support behavior change through collaborative efforts with service delivery system, e.g. through change in the services offered and how they are offered, promotion of community organization and problem solving. PISAF will encourage technical specialists and MOH counterparts at zone and regional level to introduce a focus on BCC into service delivery improvement efforts at health facilities through QA efforts (e.g., promotion of ITN use among pregnant women through awareness-raising when IPT is given.) PISAF will work with regional and zonal community mobilization/BCC focal persons to ensure BCC messages are passed through interpersonal counseling, and through strengthening and organizing local and traditional media, community theater, and peer educators (as in BHAPP) PISAF will distribute small grants to NGOs and CBOs to work with communities to promote family health behaviors and encourage use of family health services, including VCT, STI treatment especially in high-risk groups, etc. Joint meetings and planning efforts will guide the development of BCC activities and service provision and, helping to ensure effective referrals, welcoming and youth friendly services, and follow-up by the referring 130 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Exhibit 2.1: Community Mobilization and BCC Training Process HZMT Community Mobilization/BCC Team HC/COGEA/ COGEC/Health Workers COMMUNITIES COMMUNES AZ/IZ (ABPF) CPS, CARDER/ Local NGOs Village health committees/ CBSA/ Women's groups LOCAL NGOs train train train train home visits/ group discussions train USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 131 partners as well as links to additional services as needed, such as PMTCT, TB treatment, etc. Support community-IMCI through continued participation in the national C-IMCI working group, support to Malaria Mobilization Days, etc. PISAF will provide orientation to region and district level partners on community IMCI in Zou/Collines and Borgou/Alibori as well as continued support in Ouémé/Plateau. Sub IR 3.2: Appropriate Research-based Interventions and Services Introduced Key activities include: Identify annually topics for appropriate formative and operations research. As part of annual review of program results and opportunities, the project will help identify formative research needs, and innovative strategies for pilot testing either in Borgou/Alibori or Zou/Collines. We will start by targeting activities in areas that are ―wired for success‖ to create momentum and interest. Examples of such topics include: – using mutuelles for health education and promotion of use of prevention health services, linking mutuelles to micro-finance initiatives and to schools and PTAs; mutuelles contracting with private providers in addition to public; and mutuelle links to community-based distribution schemes and sales, subsidization, and promotion of ITNs. – expanding improved nutrition of children using the Hearth model – increasing the range of services and products CBSA can provide (e.g. treatment of pneumonia with antibiotics, post-partum vitamin A, or use of malaria rapid diagnostic tests and ACT treatment) and other alternative service delivery mechanisms to increase access and coverage – measuring the effect of zinc administration on inappropriate antibiotic dispensing by CBSAs and drug sellers – Promotion of ITN through Women‘s Groups: including provision of nets, linking to micro-finance and credit systems, selling at subsidized rates, follow-up in the homes on effective use, etc. – Explore increasing use of community radio, building on Africare‘s positive experience in Zou – Pilot test different incentives schemes for improved health work performance Implement pilots and assess impact and the variables contributing to success or lessons learned. Validate results and scale up or modify as indicated. These pilots will be Illustrative Key results: Annual BCC workplans developed with multi-sectoral partners Multi-channel BCC activities implemented Community-IMCI activities implemented and supported at zone and regional levels Expand peer education activities Work with PLWHA to design and implement strategies for care and support, testimonials, prevention activities and work against stigma NGO grants lead to effective community mobilization and health promotion Pilot test to use of mutuelles for health education implemented and evaluated 132 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES incorporated into each year‘s annual workplan and results will be discussed at both operational and national levels, focusing on the programmatic implications of findings. SUSTAINABILITY An important factor for sustainability is the investment of sufficient time into developing relationships, transferring skills and institutionalizing approaches. As described above, the URC team will build upon a combined 25 years of work in Benin and successful approaches to sustainability implemented under PROSAF and PHRplus portfolios. PROSAF carried out a series of institutionalization exercises with counterparts focusing on different components of improved management to plan for focused activities that would contribute to sustainability (see graphic). These reflections were periodically repeated to identify progress and obstacles. The proposed strategies for ensuring sustainability of gains made under PISAF include: Broaching the subject of sustainability and institutionalization from the beginning: Many of the approaches and strategies being implemented under PISAF have already been tested and institutionalization/sustainability issues are already well known. Thus, we will begin conversations about sustainability in the regions and scale up nation-wide from the beginning of the project. In the Borgou/Alibori, PROSAF Transition Phase began implementation with the creation of a sustainability plan that was well-planned from the beginning with specific targets over the life of the project. This emphasis on planning for sustainability of interventions and achieving specific steps ensured a focus on a phased, deliberate handover of skills and responsibilities. At the beginning of our work in each region, we will work with the regional and district teams to develop sustainability plans for PISAF activities, following the procedures used under PROSAF in the Borgou/Alibori. These sustainability plans will include specific targets for regions and health zones on an annual basis, with a five year horizon, and will be reviewed and revised annually. A sustainability plan will also be developed at the national level, based on a consensus of key strategies to be scaled up and the role of the national level in supporting scale up. Bringing the national level on board along the way and helping them to place their steering role in a decentralized system: Through the use of site visits, one-on-one working Illustrative Key Results: Annual priority for formative and operations research topics 2-3 studies conducted each year Study results communicated with zonal, regional and central level health authorities. Figure H-5. Community Mobilization and BCC Training Process USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 133 relationships, documentation of improvements, meetings and other fora, we will ensure that central level authorities and other partners are well aware of what is happening in the field and to incorporate best practices into national management systems. We will cultivate champions at central level to advocate and facilitate the policy level changes needs to ensure access to quality health services. We will assist the central level to fully play their role in ensuring appropriate, up-to-date norms and standards. National level engagement in the interventions and strategies promoted by PISAF will be fostered by a variety of activities. Firstly, national level actors will be sensitized to what is happening at the regional and health zone level through site visits, one-on-one working relationships, documentation of improvements, meetings and other fora. Secondly, we will work with key national level actors to improve their ability to advocate for the necessary policy changes, through, for example, production of advocacy materials, creation of fora, leveraging other partners to support needed changes. We will continue to work on the implementation issues related to decentralization and follow up on work done by the Comité de Suivi of the Forum sur la Décentralisation. We will also work with the Human Resources Directorate to develop and implement realistic decentralization of human resources management based on Benin‘s realities. We will also work to ensure that the DDSP be held accountable for their region‘s performance, especially related to formative supervision and efficient management of commodities. PISAF will advocate that such important topics be presented and discussed during the MOH CODIR meetings to demonstrate the central level‘s leadership. Strengthening the capacity of the health zone management teams, and fostering horizontal learning and collaborative improvement. Capacity building and working with counterparts to help them do themselves and learn from each other. For example, during PROSAF, the technical team started by doing joint supervision for QA activities with the regional health staff. They then moved to providing support before for planning and meeting after to debrief as the regional team gained competency and self-confidence. We will make sure that technical staff each have clearly identified MOH counterparts (sometimes multiple ones at regional and national levels), but all will work across multiple regions or sections, as well as with zonal management. The use of collaboratives and fora for sharing across zones and across regions will also facilitate capacity building using local intellectual resources. Capacity building is a key component of sustainability. This is not just a question of training. In fact the approach would be a combination of skills transfer and accompagnement where the PISAF team would work with health zone management teams in a ―see one, do one, teach one‖ approach. Cross-visits to other health zones and collaborative sharing across health zones and regions will form the basis of on-going capacity building. Increasing MOH/NGO collaboration and strengthening performance of NGOs. NGOs and CBOs have started to play a more prominent role in Benin. Members of ROBS, women‘s groups, local literacy instructors, and many other groups have contributed to health activities. In order to strengthen NGOs and establish a basis for ongoing future involvement with improved health, PISAF will provide technical and financial support to increase NGO capacity through contracting more activities to NGOs and will work with the Ministry of Health to evaluate the potential of NGO contracting for the future. Proposed activities to be considered for NGO contracting in collaboration with the MOH or other Ministries include VCT, community mobilization, training and supervision of community volunteers, literacy, BCC, and theater. PISAF has set aside funds to contract services to NGOs and will work to strengthen their management and financial skills. PISAF will work through regional and zonal NGOs and community groups to promote healthy behaviors by families and high-risk individuals. By distributing small grants to enable activities that provide immediate and visible results -- such as income-generating activities, 134 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES literacy/numeracy courses, health education messages and materials – PISAF will also support development of expertise by the community organizations offering these activities and strengthen them for ongoing work of this nature. Multimedia campaigns are an important part of our BCC strategy, employing drama troops, traditional singers (griots), and local radio PISAF will also work with NGOs to improve documentation and reporting of what they have accomplished to the MOH, both to increase the visibility of their role, and to help the MOH‘s ability to capture all the activities (and resources) implemented in the health sector. Given the important an increasing role of local elected entities, NGOs need to network with communes and mayors and help them implement the health activities included in their plans and budgets. We will foster this relationship during the development of COGECs‘, Health zones‘ and DDSP‘s strategic and annual workplans. Creating educated consumers who participate in demand-driven quality initiatives, through participation in mutuelles and COGECs; and documenting and communicating success stories through multiple channels. Foster leadership in communities and communes, for greater participation, and in health zones, directorates and the national level, to advocate for continuation of project-initiated or supported activities. Ensure use of data to help leaders understand how health issues are impacting their constituency in various ways, including economically. Through PROSAF and PHRplus, multiple mechanisms have been created and strengthen to ensure more engaged consumers—through the community mobilization and reinforcement of the role of COGEA and COGEC in planning, and the creation of Communal committees to support mutuelles. These models will continue to be pursued and strengthened, as well as engaging these community mechanisms in engaging more directly in health promotion activities, resource mobilization, and dialogue with providers on quality of care and other issues. Strengthening partnerships between the public and private sectors. If new evidence-based guidelines are to become standard for all health care being provided, it is paramount that the private sector is involved in efforts to improve quality of care. The informal private sector also plays an important role and significant consumer expenditures are directed to these providers. Health worker capacity building activities will target both public and private sectors, especially in offer of integrated FH services as well as data reporting, as most private clinics do not report or under-report services. PISAF will assist the DDS and health zone coordinators with private provider participation in the SNIGS data collection by facilitating provision of tools and training in SNIGS reporting, and implementing an active recall system with the DDSP and Health zone coordinators in order to collect filled forms, as PROSAF experience have shown that many private clinics did not sent their forms due to transport issues. We will help the DDSPs and Health zone coordinators to explore the possibility of involving private clinics in preventive activities (immunization, etc) and the possibility at providing support (medical equipment, training and coaching their staff ) to those involved in such activities and have demonstrated improvement in their practices. Strengthening coordination with and involvement of other Ministries in health issues, e.g. Ministries of Education, Social Welfare, Rural Development, Youth and Sports, and Local Development to ensure that these Ministries, which often have a wider reach into the communities as well as more political weight than the MOH, both contribute to and own health issues as part of their mandate. Involvement of other ministries will take place through planning mechanisms, co-opting these ministries in specific activities, and leveraging resources. Relationship between the departmental directions of public health, the social welfare, rural development, youth and USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 135 sports and local development need to be reinforce at decentralized level where resources are limited but with the human capital not used appropriately: Ssocial welfare agents at departmental and zone level can help in health promotion/ education campaigns and this has been demonstrated with the implementation of community-IMCI in Borgou-Alibori. We will help build/strengthen this collaboration in all our BCC/IEC activities and build the capacity of social welfare agents in BCC/IEC as well as in community mobilization Rural development agents have experience in sensitizing populations in agriculture-related issues and have a great audience due to the huge farmer or peasant organizations already existing. Given that health issues are cross-sectoral, especially HIV/AIDS, PISAF will foster collaboration between decentralized health system and agriculture sector in order to achieve greatest impact with BCC activities. Rural development agents at departmental and zone levels will be involved in our BCC/EIC training sessions as well as in our education campaigns planning and implementation. PISAF will encourage the DDSP responsible for adolescent health to work hand in hand with Youth and sports local authorities. Given the importance of local development entities in the decentralization, an important effort will be made by PISAF to foster better collaboration between decentralized health authorities at departmental and zone levels in order to involve these entities in the planning, implementation and evaluation of health activities. The development of strategic and annual plans offer a great opportunity to implement this collaboration. Working with staff from these various institutions at the departmental and zone level will help develop the spirit of multi-sectorality of health and fill the human resource gaps for health education. Recognition and innovative efforts towards development of motivation systems for community-based and facility and zonal level health workers, e.g. through their retention of a percentage of sales price, recognition within the community and the health system. PISAF will develop a series of pilots to test out various mechanisms to increase motivation. This will start with working with these staff themselves in developing potential ideas to test. COLLABORATION Both consortium members have a solid track record in Benin with collaborating with a counterparts, other donors and partners, NGOs, and civil society. Effective collaboration is based on mutual respect of roles and competencies and a willingness to meet the other side part way. Collaboration promotes sustainability, efficiency, and the possibility of leverage. As mentioned in our approach to sustainability and throughout the program, our approach to our counterparts in the MOH, regions, zones, facilities and communities is designed for collaboration with the goal of working ourselves ―out of a job.‖ With respect to partners, the consortium members have worked with wide spectrum of partners, at the institutional level and through joint planning and implementation, as well as collaboration with various other USAID funded projects (such as the PROSAF/PHRplus collaboration). PROSAF for instance collaborated closely with German and Swiss-funded projects in Borgou/Alibori to co-fund and co-plan activities. Quarterly and annual program planning was carried out with the regional health authority and other donors. Technical sharing and support was done with international and local NGOs, such as OSV-Jordan and MCDI. PHRplus worked in close concert with the donors on the decentralization agenda, such as the EU, the Dutch. PHRplus has collaborated with other donors on mutuelle development in Borgou/Alibori and we will seek to deepen and broaden that collaboration to share lessons to the benefit of the other partners and to PISAF. This will allow 136 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES the PISAF-supported mutuelles to learn and apply sustainability lessons from the mutuelles supported by others. It also is expected that the communal, zone, and regional mutuelle unions will be made up of mutuelles supported by all donors. The broader the membership of the unions, the more self-help they will be able to provide one another. As has been mentioned earlier, we will carry out all project activities in close collaboration with government institutions at peripheral, regional and national levels. Our primary counterparts reside within the Ministry of Health, but we will seek to support multi-sectoral collaboration as well. We are building on our existing strong relationships by establishing a collaborative relationship for this project with ROBS including ABPF, and will reach out to include a wide variety of community-based organizations to participate in the effort to improve health care and health status for the population. We will include in our efforts professional organizations who can become powerful advocates for change. Multiple donors are working in the health sector and on-the-ground collaborative relations already exist in all regions as well as at the national level. Among those with whom we will collaborate very closely are in Borgou/Alibori: Belgian, Swiss, and French Cooperation; in Zou/Collines: ADB and MOH. Our great collaboration with PSS during PROSAF implementation in Borgou/Alibori will continue and be reinforced in Zou/Collines where PSS is covering two health zones. In these zones PSS will provide training in all aspects of Family health including IMCI, EONC (including PPH). PSS will support training in HRM and quality assurance with the technical support of PISAF. The collaboration will cover joint planning, joint advocacy and cost-sharing of training sessions implementation in their zones of coverage. In Borgou/Alibori, PSS will continue using PROSAF tools and strategies in its support to the health zones of Tchaourou, Parakou-N‘Dali and Nikki-Kalalé-Pèrèrè (where PSS is supporting fully the implementation of the short training course on clinical IMCI tested by PROSAF). From 2007 it is likely possible that Bemberèkè￾Sinendé health zone will be included in PSS coverage. PISAF and PSS will share the cost for follow up activities in the Borgou/Alibori. UNICEF is providing support to Djija health zone in Infant and Maternal mortality reduction purpose. We will network with this institution in order to see areas of complementarity as the health zones capacity building is very key in the success of health system decentralization. German Cooperation is very active in the health Zone of Savalou/Banté. PISAF will work with them in order to see what else could be done in order to strengthen health zone capacity in management and define areas of complementarity for an efficient use of resources. Nationally we will work with the WB-funded activities as well as with country coordinating mechanisms for the Global Funded activities. Collaboration with other USAID-activities (PSI as done for malaria, DD and HIV activities) as above We will continue our successful collaboration with CRS, CARE, PLAN, UNICEF, IRSP, PSI (whose letter of collaboration is included in the program and other USAID funded activities such as Equipe. The URC team does not envision undertaking major work related to the broader private sector issues. We will seek rather to work with private providers in the various areas the project is active in. The Ministry of Health is already working closely with private health care providers primarily to ensure reporting for the National Health Information System (SNIGS). As with the previous USAID-funded PROSAF project, we will continue to support training to private sector providers was offered to improve their health data reporting. We would seek to leverage the USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 137 use of private organizations in capacity building as under PROSAF when we worked closely with ABPF which provided training in NORPLANT to public health staff. We envision working with the professional associations as well as through direct contact with private health care providers to explore to what extent they can be involved in preventive health activities, such as immunization, and be integrated into strategies to extend the family health minimum package to all people of the regions. Other options might be participation in clinical update training, and strengthened referral systems. GENDER Health issues, services and behaviors are not gender neutral; thus, it is necessary to consider the role that gender plays in fostering or hindering the objectives of PISAF. Evidence indicates that programs that enhance gender equity/equality are more effective at sustaining positive health outcomes. It is widely recognized in the international health community that gender inequality and its related cultural norms have negative impacts on women‘s health outcomes. However, its impact on men, while given less attention, is significant, as well. For example, cultural expectations dictating that men are knowledgeable about and experienced with sex may preclude them from seeking information, which may lead to unsafe sexual practices and health risk. Thus, under PISAF, it will be important to address gender inequality from several angles. Efforts to include gender considerations in implementation will focus on Increasing women‘s role in decision-making within the health system by establishing a quota for women‘s participation in COGES/C and in the leadership and membership of mutuelles Increasing awareness among men as to the importance of women‘s involvement through sensitization training for men on the benefits of gender equality and inclusion of women; and Addressing the particular gender norms that increase male vulnerability to negative health outcomes by developing BCC messages encouraging men to seek health advice from health care practitioners and reassurance that they need not know it all and sensitization of health workers to the particular information needs of men. The URC consortium has experience integrating gender into health activities, such as URC‘s efforts to provide space for both men and women to express their views openly during discussion groups following theatrical performances about HIV/AIDS, birth spacing, and safe; and Abt‘s involvement of women in leadership roles in mutuelles. We will incorporate gender equity and sensitivity into program strategies. Attention will be given to the following variables: Differing needs, roles and interests of men and women: Examples include providing opportunities during the community mobilization process for men and women to discuss and present issues particular to their gender; developing BCC materials that reflect the perspectives of each gender; and/or creating innovative approaches to reaching women in their homes. Control of resources and variable access as a result of gender differences: Examples include recognition of the need for extra effort to reach women to inform them of initiatives, services and programs from which they may benefit; and awareness of the limited mobility and financial resources some women may have and, thus, providing services and programs nearby through community-based health agents (CBSAs). Specific health program benefits for men compared to women: Examples include sensitizing health workers to the particular needs of men and women and the importance of providing equal, yet gender-appropriate, service to each group and/or developing BCC materials that reflect the perspectives of each gender. 138 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Spousal communication to support changing norms: For example, building the capacity of women to negotiate for their interests in their spousal relationship (e.g., for condom use or the use of another family planning method). Decision-making processes; power dynamics between men and women: For example, developing a play to be performed in villages and communities that depicts the benefits of an equal power dynamic between a man and a woman and, in contrast, the effect of an unequal power dynamic. Positive and negative programmatic impacts on men and women: PISAF will document the involvement or lack of involvement of women in its programs to contribute to the body of knowledge stipulating the benefits of involving women at every level and the negative effects of no involving them. Examination of gender issues will enable the design of appropriate communication messages, behavior change interventions, training and education programs, and policy activities consistent with achieving the goal and objectives of the project. At the project level, measures will be taken to ensure that gender considerations are taken into account at all levels and at every stage in planning and implementation of PISAF activities. A PISAF staff person will be assigned as the gender point person to assess gender equity in programming and participation throughout the project‘s portfolio. This individual will draw on the expertise of a Gender Advisory Group, which will be made up of men and women from within and outside of the project who are sensitive to the value of gender equality. Gender Advisory Group members will be those with a particular interest and experience in ensuring gender equity. PISAF will seek the participation of individuals from the various levels of the health system, NGO representatives, and CA and donor counterparts. Members will elect to participate in the group. The Gender Advisory Group will be a project-specific group that will meet on quarterly basis to review programs, identify opportunities for inclusion of a gender perspective in project activities and make recommendations for improvement. To ensure longevity and sustainability of the Advisory Group, chairmanship and membership will rotate on an annual, which will alleviate the burden of additional responsibility and allow for inclusion of fresh and innovative ideas. The Advisory Group also will have overlapping membership with the mutuelles associations and COGES membership. Some additional ways in which the project anticipates addressing gender are as follows: Policy Ensure that policy formulation and operationalization address gender issues in specific ways. Collaboration Collaborate with organizations working in other sectors, such as education and democracy and governance, to link efforts to empower women. Partners in this regard may include EQUIPE, PRISM, CRS and MCDI. Behavior Change In recognition of high illiteracy among women, develop BCC materials and job aids for low literacy populations. Representation Ensure proportionate representation of women COGEA/Cs and in leadership of mutuelles. This also will involve efforts to raise awareness of the benefits of women‘s participation. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 139 Providing leadership training for women to permit them to feel comfortable playing an increasing role in civil society participation and leadership for mutuelles Focusing some small grants exclusively on women‘s groups Community Involvement Working through the COGECs, gender sensitive approaches to health care service delivery will be identified. This will included the engagement of both men and women in reproductive health decision-making for their families. Encourage women‘s participation in civil society, increase their contribution to health system improvement, improve knowledge among men of RH/MCH issues, and illustrate the positive impact of female involvement through examples a positive role modeling approach will be implemented through mentoring, the promotion of participation in civil society structures by women themselves, and role modeling of male involvement in RH/MCH. Foster community support for gender equity by introducing new knowledge, explanations, and solutions to gender-related problems identified by community members. Training of Health Workers A gender sensitivity module will be developed and included in the interpersonal communications training curriculum developed under PROSAF. This module will walk providers through a process of setting standards for institutional practices, provider practices and client satisfaction that reflect sensitivity to gender issues in the care process. Monitoring and Evaluation Identify specific and measurable indicators to monitor progress toward gender equity and, where feasible and appropriate, disaggregate indicators by sex. Building Capacity Strengthen the capacity of women‘s groups to increase the effectiveness of project support, the future viability of the groups, and the status of women and their role in the community. Build on the excellent track record of women‘s groups in terms of effective, transparent management and dedication to health issues. Raising Awareness Provide gender sensitivity training for men and women within the health system and at the community level to demonstrate advantages of involving women in decision-making about health and demonstrate advantages of greater equality. Seek opportunities to build mutuelles around women‘s groups and/or to collaborate with women‘s groups to recruit members, an approach that has met with great success in Senegal, where women‘s group-based mutuelles tend to cover more key maternal and child health services and to be better performers financially. MANAGEMENT AND STAFFING Staffing URC has assembled a team of highly qualified personnel with complementary technical and managerial skills to meet and exceed the challenges and expectations of this project. The combination of long-term and short-term technical staff represents diverse areas of expertise, including malaria, reproductive health, family planning, maternal health, HIV/AIDS, IMCI, health financing and mutuelles, BCC, community mobilization, systems improvement including logistics, 140 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES decentralization, quality assurance, and monitoring and evaluation. In addition to technical expertise for their specific position, each staff person brings complementary skills and experience which will contribute to an integrated implementation of program activities. Long-term Staff Long-term staff include Chief of Party, Malaria Coordinator, Reproductive Health Advisor, Quality Assurance Advisor, Health Zone Management Strengthening, HIS/Commodities Management Specialist, Decentralization Advisor, two Health Financing Specialists, Community Mobilization Specialist and Behavior Change and Communication Specialist. Short-term Technical Assistance and Home Office Technical Support Short-term technical assistance will be provided on an as needed basis in the areas of decentralization, contraceptive security, mutuelles and health financing, quality assurance, behavior change and communication as well as in clinical areas of family health and HIV/AIDS. This support will be provided local, regional, and international consultants, many from West Africa, as well as by staff from URC and Abt Associates. Management Plan Overview of the Partnership University Research Co., LLC (URC) has formed the PISAF Partnership with Abt Associates and will collaborate closely with ROBS, Population Services International, Catholic Relief Services and the Institute of Reproductive Health for program implementation. As the prime grantee, URC will provide overall technical direction and management of the project, and offer technical expertise in IMCI, Reproductive Health, HIV/AIDS, malaria, quality assurance, management strengthening, community mobilization, behavior change communications, and income-generating. Abt Associates brings to the consortium its extensive experience in decentralization and health financing, particularly mutuelles, and in costing. All partners will contribute to the overall strategy of the project through consultations and short-term assistance as indicated. Organizational Structure of Project The project will be led by the Project Director, who will provide overall direction and leadership to the project. The technical staff are organized in a matrix-management system consisting of three technical teams, each team organized around one of the Intermediate Results. The team structure (see table) ensures that the technical components are managed in an integrated and interrelated manner and taking advantage of all staff competencies. Most staff will be based in Bohicon, from where they will support activities in the Zou/Collines Region, but also provide national-level support and assistance in the Borgou/Alibori and Ouémé/Plateau. Activities in Borgou/Alibori will be overseen by a technical advisor (Sylvie Amadou), based at the regional health directorate, who will have primary responsibility for ensuring that project technical resources from the Bohicon-based team are provided efficiently and effectively. She will be assisted by one of the health financing advisors who will also be based in Parakou. We have identified a pool of short-term technical advisors available on demand which enables the project to staff up in a flexible manner from the beginning to offer tailored expertise depending on the identified needs. Management Roles and Responsibilities Over the last 25 years, URC has established management procedures that have consistently enabled our developing-country projects to operate efficiently, successfully, and within budget. These procedures include: 1) a decentralized project management structure to ensure accountability, effective decision-making, implementation and monitoring; 2) formal systems for accounting and fiscal management; 3) information systems that provide current, relevant and USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 141 accurate information and data for decision-making; 4) efficient subcontracting practices; and 5) a relationship-building process that encourages collegiality, cooperation and close collaboration among the partners. The technical team based in Benin will work closely together and develop integrated program implementation strategies, while ensuring high quality technical support for each technical area of the project. Project staff is supported by key technical advisors from their organizations through on-line communication and periodic in-country visits. The partnership will also establish regular communication at headquarters to coordinate and ensure that all necessary support for the field team is forthcoming. Lines of authority and responsibility: URC will provide overall technical direction and management of the project. The project will be implemented as a partnership among the core partners and bring collaborating organizations into strategy discussions at crucial points in the program. Relationships will be developed with other CAs and donors. The Chief of Party will provide overall management and technical oversight. He will work with technical and administrative staff and oversee their performance. He will call upon the assistance of the short￾term technical experts as needed. He will work closely with USAID and establish close working relationships with collaborating institutions. Because of the multifaceted nature of the project's work, teamwork will be essential. The URC team will provide experienced headquarters technical support of field activities through various methods, including (1) review of technical strategies before implementation; (2) formal review of overall technical activities after six months, including reporting of results to date; and (3) annual technical reports emphasizing outcome data. Management of subgrantees: URC has had many years of successful relationships with subcontractors in a variety of USAID contracts. Our success in forging strong partnerships with subcontractors is based upon the establishment of a common vision for project implementation and upon clearly defined responsibilities of each of the partners. At the beginning of the project, URC will organize a team meeting, bringing together our key partners and collaborators, together with USAID and representatives of the government. The purpose of this meeting is to discuss the scope of work, establish well-defined standard operating procedures and clear lines of responsibility between team members. We also will review common criteria for performance evaluation of the staff, including emphasis on effective team participation and focus on results. Subgrantee supervision and performance monitoring: URC has well-defined procedures for monitoring subgrantee performance relative to the scope of work, level of services and cost of services. In addition to the subgrants issued to our core partners, we expect to engage local non-governmental organizations and community-based organizations in the implementation of the program through subgrants. URC will administer the subgrant program, under the guidance of the Project Director. Subgrantees will be required to adhere to URC‘s monitoring procedures, including monthly/quarterly reporting of costs and levels of effort, trip reports and annual reports. The Project Director will be responsible for certifying satisfactory performance before subgrantee vouchers are paid. Quarterly project review meetings will afford another opportunity for monitoring performance and coordination. All subgrants for technical services developed under the project will elaborate detailed scopes of work, expected outcomes and deliverables, and corresponding line-item budgets to ensure a mutual and clear understanding of the task, respective roles and expected performance. Subgrant tracking: A system for tracking subgrants is in place and will be adapted to track and monitor financial expenditures and subgrant budgets. The project director will oversee the implementation and monitor work progress, which will allow for payment of vouchers. 142 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Administrative and Financial Management Home office support: To facilitate smooth operation of the project, an experienced Senior Program Officer will manage administrative and financial operations of the project at URC headquarters and will coordinate and provide technical support under the direction of the Vice President, who will provide overall technical and management oversight to the project, serving as Corporate Monitor for the project‘s technical achievements as well as financial performance. The Vice President will work closely with the Project Director and will be in close contact with the field team via Internet and phone and make field visits as needed. The Vice President is very familiar with Benin and the URC work carried out under PROSAF, including having led the PROSAF assessment for sustainability completed in 2005. The Vice President will work closely with the Project Director and the field-based team to ensure that they receive necessary support from the home office. Personnel policies and compensation: All staff working on the project will be subject to corporate regulations. Salaries are in compliance with local Mission and USAID regulations; new employee salaries are based on past history; and annual raises are determined by performance, feedback from clients, and personnel policies. Full-time host-country and international staff receive benefits in accordance with local labor laws and URC and USAID policy. Travel policies: URC‘s travel and per diem policies comply with the USAID regulations and local Mission guidelines. Long- and short-term international consultants are provided with medical evacuation insurance for medical and/or other emergencies. Property management: URC has a system to inventory and track capital property and non￾expendable equipment at its home and field offices. This system will be implemented to control assets in field offices. Procurement: All purchases will be done in strict adherence to USAID‘s source and origin requirements. Appropriate waivers will be requested. Efforts will be made to develop relationships with other donors and global institutions to establish cost-sharing mechanisms. URC follows all required FAR and AIDAR procurement regulations, including obtaining multiple competitive bids for all procurements and purchase orders. In collaboration with USAID, we will develop a comprehensive list of goods and services to be procured. We will submit procurement status reports to USAID annually. Risk reduction: URC will put systems in place to mitigate risks of fraud, non-performance and environmental threats by conducting pre-award audits; requiring appropriate authorizations for approval and disbursement of funds; limiting access to cash by staff/ subgrantees; keeping appropriate and auditable records; and conducting periodic performance and financial audits. Office Location The main project office will be in Bohicon. A technical advisor and one of the two health financing specialists for mutuelles will be based in Parakou. Discussion will beheld with Ouémé￾Plateau about office space for intermittent technical assistance. Mobilization We are prepared to begin project implementation immediately upon signing the agreement with USAID. A rapid start-up will be facilitated by our long experience in Benin, our experience in the country, and immediate availability of all technical staff. Reporting and Monitoring of Performance URC recognizes the importance of performance monitoring to ensure that the project achieves and documents results and provides timely input to USAID/Benin; and to provide relevant information and feedback to the program‘s stakeholders and counterparts. Our project USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 143 management information system will be used to track project activities. This system will facilitate the collection, analysis and reporting of financial and program information to support management, logistics and monitoring of each activity and subgrant. Annual work plans will be developed in close collaboration with donors and other partners. Progress reports, project results, performance indicators, expenditure analyses, and trip reports will be submitted on a regular basis. The final report will summarize all accomplishments and results and any problems encountered, document use of funds and provide comments and recommendations for future improvement. Cost-share The URC consortium has agreements of collaboration with ROBS, CRS, PSI and MDM. These partners represent an opportunity for significant match funding and effective collaboration to extend the impact of the USAID investment in PISAF. Additionally, we plan to use our small grants program for cost share and in-kind contributions. We will also work with the government to identify other opportunities for cost-share such as through combined implementation with other donors, such as PSS and UNICEF. Abt also will leverage resources through its Aware Project for commodity security activities at no cost to the Benin PISAF project. PROGRAM MANAGEMENT PLAN The importance placed on achieving sustainable development results is reflected through a tight integration of the project‘s performance monitoring system with the strengthening of the monitoring function of the Ministry of Health both at the central and at lower levels of the system. We will make monitoring of program results a cornerstone of our partnership with the Government, strengthening the health information system, helping to simplify the presentation of relevant data at area and district level, expanding the use of facility and quality of care assessments as tools for continuous performance improvement, and developing goals and planning resources. To the extent possible we will integrate monitoring as part of SNIGS. Program results to be achieved under PISAF combine USAID and project indicators. The system will be synchronized with the national and local monitoring systems to ensure that the program monitoring continues after the termination of the project. The monitoring will put greater emphasis on outcomes, though program inputs and processes will be also monitored to show which aspects of the program are being productive and which ones are not. URC will design and implement a simple performance monitoring and evaluation system, the Project Management Information System (PMIS), to track project input, output, and outcome indicators. Input indicators will provide information related to expenditures, equipment and supplies, and technical resource utilization. Output indicators will provide information such as the number of health centers participating in an improvement collaborative, number of management trainings held, dissemination workshops. Outcome and impact indicators will provide information related to improvements in the access to and quality of care (percent of the population that has access to a minimum package of integrated family health services), levels of compliance with clinical guidelines among patients and providers, as well as changes in health-related processes and outcomes e.g. correct monitoring of patients, morbidity, and mortality. The team will use a number of tools for collecting and compiling information on various indicators. Intermediate and Final Program Results PISAF‘s success will largely rely on our ability to promote a supportive environment to facilitate policy development and updating of clinical norms, and the delivery of quality services through health care providers. Ongoing monitoring and assessment will permit us to (1) identify project activities that are progressing as planned and should be continued, (2) introduce corrections to 144 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES activities that are not progressing as planned, and (3) identify those activities that achieve their objectives ahead of schedule and can be terminated early. In addition, data will assist in identifying which interventions can be scaled up readily and which are sustainable. Description of Indicators, Baseline Values, Expected Results, Sources, Methods of Data Collection As part of the PMP, URC will develop a detailed list of indicators with definitions, an overview of current (or baseline) values, expected results, sources, method of data collection and distribution, and staff responsible for ensuring the collection and processing of information. For each indicator, we will establish a baseline value by reviewing existing reports and surveys. New surveys will carried out to collect values for indicators for which data are not available. The URC team will establish target performance goals for various indicators by reviewing past trends as well as level of effort being put on achieving specific results. A consultative process will be used to finalize the performance targets for various indicators with stakeholders including the MOH and USAID. Quantifiable and Qualitative Indicators of Progress Ongoing monitoring of programs will enable us to provide data to USAID and other stakeholders. We will conduct rapid surveys to assess performance and identify strategies that do and do not work. We will also measure programmatic impact by tracking morbidity and mortality data. The intermediate indicators will include process and output indicators showing improvements in compliance and service use. Results will be presented in run chart formats that show trends over time for various access and quality indicators at community, facility and municipality levels during the testing stages and during the scale-up phases. These reports and stories will also be stored in our PMIS database to permit rapid retrieval for reports and presentations. URC will identify the appropriate methods to collect data so that verifiable information is compiled for various indicators. The progress surveys will measure the same program effectiveness. Reporting Format The following presents an outline of the format that will be used to present project results quarterly. Executive summary: (a) key activities, (b) major outputs and results/accomplishments; (c) next quarter highlights – major activities; and summary table: Linking activities to results and accomplishments Activities and results: general activities; results and accomplishments each component; consultancies Coordination with local and other stakeholders Constraints/challenges to implementation Project administration Recommendations/next steps Budget and expenditures Timeline and workplan for the next quarter Data Sources and Quality Assurance The URC team will identify the appropriate methods to collect data so that verifiable information is compiled for various indicators. Periodic chart reviews, observations, document reviews, and facility assessments will measure the same indicators as those used for the baseline USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 145 survey to yield intermediate program results to measure improvements from project interventions. The URC team will collect much of the monitoring data using MOH information systems. Additional studies and surveys will be commissioned annually to collect qualitative and other data that are not available through the routine information system. URC, following USAID/Romania, will develop standards for data quality and carefully review the protocols. In addition, we will establish a Research Committee that will check adherence of protocols to the required standards, give approval for the implementation of the surveys, and monitor the ongoing research activity. Validity Check of Self-reported Data The URC team will commission external audits from time to time to validate the quality of data collected from MOH and other information databases as well as other internal mechanisms. The external validation will use a combination of chart audits, provider-patient interaction observations, as well as interviews with clients and providers as part of data collection. 146 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 147 APPENDIX I. URC PISAF MATERNAL AND NEWBORN HEALTH STRATEGY WITH A SPECIAL EMPHASIS ON EONC AND PPH General Strategy URC and partners will provide support to the MOH to integrate active management of the third stage of labor (AMTSL), the management of post partum hemorrhage (PPH) and Emergency Obstetric and Newborn Care (EONC) into an expanded minimum package of family health services. We will build on our past experience working with the MOH in Borgou/Alibori to create integrated local EONC systems to improve links between the different levels of care and thereby create a care continuum from household to hospital level. Figure H: I-1. Pathway to Survival and EONC Best Practices Pathway to Survival and EONC Best Practices Time Range: 1hr – 3 days Life￾threatening Illness Recognition of Problem Decisions Regarding Care￾Seeking Accessibility to “Adequate” Care Quality Essential Obstetric and Newborn Care Survival  Community BCC activities  Empowering womens groups Community birth preparedness Working with TBAs and community volunteers for referral of complications Building bridges between traditional and modern care providers Skilled attendance at birth: Partograph Active mgmt 3rd stage Episiotomy not routine Family support/cultural preferences Management of complications Immediate newborn care Special care for LBW baby An integrated local EONC system consists of a network of health facilities (health posts, health centers and hospitals) with the capacity to provide all the essential service elements that are necessary to ensure survival along the ―Pathway to Survival.‖ These facilities are linked to the community level through active outreach, IEC/BCC, and referral and counter-referral processes. At the community level we will strengthen community responsiveness to obstetric and neonatal emergencies and raise awareness of the importance of early recognition of danger signs, the consequences of delays in seeking care and the importance of a prompt response to bleeding. We will reinforce links between different levels of the service to ensure that delays in reaching referral care are minimized. At referral facility level we will ensure that facilities are able to provide basic EONC. Basic EONC facilities should be able to provide care for normal deliveries (including use of the partograph and essential newborn care for example) and the initial medical treatment of 148 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES complications related to pregnancy, delivery or abortion (for example: hemorrhage, sepsis, abortion-related complications, eclampsia); manual procedures (placenta removal, repairing vaginal tears or episiotomy). Basic EONC facilities should be accessible to all women and must have a skilled attendant present. At the district hospital level we will work with the MOH to support the availability of Comprehensive EONC. Comprehensive EONC includes all services of a Basic EONC facility plus surgical procedures, anesthesia, and blood transfusions5 and higher level care for the sick and low birth weight newborn. URC has extensive previous experience in creating EOC systems in Benin as well as in Latin America and Caribbean Region where URC has established national EOC systems in multiple countries. All these EOC systems have also addressed PPH by promoting the AMTSL. PROSAF in Borgou/Alibori organized training of all the maternity department heads on EONC and AMTSL. The trainers were members of the departmental training team and were trained by PRIME project. This experience was very successful as follow-ups demonstrated a decrease in the number of cases of hemorrhage and evacuations for hemorrhage. Ensuring that facilities at each level are able to provide the care may require additional funding and reallocation of resources. PISAF will advocate with the DDSP and Zone coordinators to make sure that the required drugs (e.g. oxytocin, intravenous fluids) are constantly available in all the health facilities of Zou/Collines and that essential equipment and supplies for EONC and addressing PPH are provided in the early phase of the project. A major focus of our work will to support the MOH to improve the integration of services and build capacity, sustainability and linkages of between all levels of the health system, including improved referral and counter-referral, see below. We will ensure that facility providers have the competencies to perform their EONC functions as well as the appropriate equipment, supplies and drugs. Even when health facilities provide good quality care, however, they may be unable to respond effectively to local emergencies because women prefer to seek care elsewhere, perceiving the services in the formal sector as not addressing their needs. Such perceptions are common where women feel they are not treated with respect or because services do not take into account cultural traditions. Through our research and community mobilization mandate we will explore with communities how to provide the most appropriate, accessible and gender-sensitive EONC services so that this leads to increased services use. The relationship between basic EONC, comprehensive EONC and referral linkages, and adaptation of services to the needs of families and communities is depicted in the next graphic. 5 Ensuring the availability of blood transfusion services is a critical issue but beyond the immediate scope of this project. We will however work with partners to maximize the availability of blood where possible and at a minimum ensure the availability of intravenous fluids. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 149 Figure H: I-2. Diagram of Demand, Access and Quality of Care in Local EOC Systems INCREASING DEMAND, ACCESS AND QUALITY OF CARE IN A LOCAL EOC SYSTEM Adaptation of services to the needs of clients Activities in the community to increase demand and access to EOC Basic EONC •Prenatal Care: •Care during labor: •Delivery Care: • Post partum Care • Newborn Care Comprehensive EONC •Basics EONC •Anesthesia •C-sections •Blood Clinical management and procedures for obstetrical complications Referral and counter-referral EOC continuous quality improvement teams Strategies for Specific Results Access to Quality Services and Products Increased At the facility level our main strategy for improving the quality of care will be to apply the collaborative model that was used successfully to scale up AMTSL in Bourgou-Alibori and build on this experience to scale up the geographical scope and technical content of services. The collaborative model brings facility providers together to identify gaps services provision and to identify the most locally appropriate strategies to implementing best practices. The EONC Technical Working Group that guided the development of the Bourgou Alibori collaborative will be reconstituted with representation from all the key stakeholders and experts in PPH and EONC. The TAG will be responsible for providing technical support and guidance to the improving the quality of EONC. This includes reviewing the EONC guidelines to ensure that they are SOTA and defining the content of the expanded minimum package of care. The TAG will also provide support for developing tools for identifying services gaps, training needs, supervision and monitoring and evaluating progress. Training will take place with experienced midwives and physicians who have been trainers in EONC and PPH in Borgou/Alibori. In addition to ensuring that providers perform to established EONC and AMTSL standards for complications, we will also ensure that appropriate preventive practices are optimized throughout pregnancy delivery and the postnatal period. For example, the consequences of hemorrhage are exacerbated if a woman is anemic so we will ensure during ANC that women are screened for anemia and prescribed iron and IPT according to national policy. At delivery we will ensure that providers are not only are competent to carry out AMTSL but can also recognize PPH and take action accordingly (stabilization and referral at 150 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES lower levels of care, transfusion and definitive treatment to address the cause of bleeding at referral level). Provider performance will be assured in a variety of ways. Assuring a system for continuous medical and nursing education will be set up to update and maintain provider skills (technical updates thought the collaborative, supportive supervision, facility meetings and audits) Establishing a Continuous Quality Improvement (CQI) approach. In the facilities staff will form CQI teams. The CQI teams will attend the collaborative learning sessions where they will learn about quality assurance approaches and self evaluation. As part of this CQI approach, teams will perform self assessment, checking and ensuring the availability of a basic supplies and equipment necessary for performing EONC (e.g magnesium sulphate, oxytocin injections, antibiotics, IV fluids, newborn resuscitation equipment etc) Encouraging facilities to carry out self assessment and perinatal audit to monitor maternal and newborn deaths, near misses and selected complications facilitating staff to discussing the findings in a non-judgmental way in regular ―Morbidity and Mortality‖ meetings to which all levels of clinical staff are invited. We will support facility staff to identify the main quality management problems and potential solutions highlighted by the audit and to implement and monitor the impact of the management changes. . Providing job aids and tools such as were developed for EOC, PPH and newborn care during PROSAF. Some special mention should be made about newborn health. A survey carried out in 2003/4 showed that newborn health had been virtually ignored in most facilities. This survey has led to the recent drafting of new national EONC standards which specifically address the needs of newborn health at the various levels of care. (For example, the provision of a basic essential newborn care6 at all levels; the resuscitation, stabilization and referral of babies with sepsis and low birth weight, at lower levels of care and resuscitation and extra care for the sick and low birth weight (LBW) baby at referral levels of care (Sourou Gbangbade, personal communication). URC will work with the MOH to implement these newborn care standards and to pilot new approaches to improving newborn survival, if necessary, such as Kangaroo care. Demand for Health Services Products and Preventive Measures Increased (Antenatal Care, Skilled Attendance, EONC, Post Partum Care etc.) Since a substantial proportion of deliveries occur at home, and women return home very quickly after delivery, URC will work with the MOH to promote awareness of appropriate care and care seeking behaviors, at community level, building on the work that has been done so far at to promote birth planning and complication readiness and in creating an enabling environment to increase the availability of emergency transport. We will identify and strengthen culturally appropriate practices to improve home-based care of the mother and newborn (adequate nutrition, rest, iron and folic acid supplementation, tetanus toxoid immunization, hygiene, IPT, bed nets, a clean and safe delivery, essential newborn care, and the importance of post-partum follow-up. There is a growing body of evidence that these practices prevent complications and improve survival. We will review what is known about community practices and the factors that influence them, conducting some formative research as necessary. This is particularly important for understanding how communities recognize complications. (Very little is known for example about the recognition of PPH or newborn complications internationally). Findings will be used to develop a comprehensive BCC strategy—identifying which behaviors to address, designing 6 Essential newborn care includes a clean and safe delivery, temperature maintenance, early exclusive breast feeding, cord care, eye care, recognition of danger signs, immunization and early postnatal care). USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 151 messages, and appropriate channels (for example, using traditional midwives, volunteers, and mother-to-mother peer support groups or the use of facilitators to promote awareness and services use amongst mothers groups as has been demonstrated successfully in Nepal (Manandhar et al, Lancet 2005). Strengthening Links between Services URC will assist the MOH to define, communicate, and strengthen the role of health personnel in EONC at each level of care and ensure that messages are harmonized between levels so that behaviors promoted at community and facility levels are mutually reinforcing. Roles and responsibilities for all different actors at each level of the health services will be defined so it is clear how the community and facility and all the components of an EONC system work together in a coordinated fashion to provide a continuum of care to enhance the lives of women and babies. Opportunities will also be sought to strengthen working relationships between community actors and facilities, using existing fora or creating channels where none exist, for raising awareness of important messages and sharing of issues, plans, and data. Facility health teams will be supported to work in collaboration with community delivery providers, community health workers and services users as is happening already in IMCI. Strengthening links between the Coges, community representatives and CQI teams will be explored in order to improve accountability for services and to increase the availability of small funds improving facility quality of care. Linking facility finance committees and the supervision process was a factor contributing to the success of the Basics II supported MOH newborn health program in Kebemer, Senegal (Mandy Rose personal communication). Coordination and collaboration between levels of facilities will be strengthened through joint planning sessions and sharing of lessons learned during the collaborative learning sessions. A particular focus will be the strengthening of referral and counter-referral mechanisms and their monitoring to that effective links are created between the community, health units and district and regional hospitals. Illustrative Activities Review existing research/assessments on quality of care, staff competence, availability of essential supplies and equipment, community care, and care-seeking practices and perceptions of care. Conduct targeted formative research with special emphasis on adolescent needs. Convene EONC TAG to develop consensus on expanded minimum care package and review existing norms, protocols and standards for EONC Procure necessary equipment and supplies for EONC, including neonatal resuscitation and PPH Develop/redefine roles and responsibilities and job descriptions for staff within reorganized EONC system (if necessary) Update existing in-service training curricula; integrate competency-based refresher training into on-site training whenever possible for facility level medical and nursing staff and auxiliary staff; coordinate with pre-service training Develop and support collaborative teams, learning sessions and supportive supervision Support facility teams to carry out perinatal audits. Develop comprehensive EONC IEC/BCC strategy Strengthen referral and counter- referral between facility and community levels 152 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Review & update existing job aids and IEC/BCC materials and develop new ones as necessary to support scale-up of improved EONC care for facilities and communities Increase skills of volunteers, TBAs, and CHTs in promoting EONC and provide support for community level EONC activities. Pilot model Kangaroo Care unit at one hospital. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 153 ANNEX I. USAID/BENIN IMPACT PROGRAM DESCRIPTION ATTACHMENT B—PROGRAM DESCRIPTION EXECUTIVE SUMMARY Population Services International (PSI) and its partners, JHPIEGO Corporation (JHPIEGO) and Abt Associates (Abt) bring together PSI‘s expertise and demonstrated experience in implementing results-driven HIV/AIDS, Family Planning (FP) and Child Survival (CS) social marketing and behavior change communications programs in Benin; JHPIEGO‘s expertise in training clinicians and management of integrated service delivery sites; and Abt‘s demonstrated ability to strengthen the capacity of the public sector to manage national HIV/AIDS programs and advocate for policies that create a supportive environment for social marketing and the provision of health services. In addition, important local collaborating partners such as OSV￾Jordan will bring expertise in implementing effective behavior change activities among key high￾risk populations and managing relations with local NGOs. Building on over 15 years of experience implementing HIV/AIDS, reproductive health and child survival social marketing programs in Benin, PSI will be the lead implementing agency for IMPACTS Objectives 1 and 2. Given its success to date, established infrastructure, productive relationships with key stakeholders, and experienced staff, PSI is confident in its ability to continue to produce health impact through integrated, innovative and performance-based results beginning day one of the award. PSI proposes an aggressive yet balanced strategy to expand the use of health services, products and preventive measures in Benin. PSI presents strategies to increase sales of existing condom, contraceptive, and insecticide-treated net (ITN) product lines and brands as well as add new products to the USAID social marketing portfolio, including a pre-packaged malaria treatment, female condoms, and zinc supplements. These important products will help to address unmet needs in Benin through empowering women to use a female-controlled method of HIV prevention, facilitating successful home-treatment of malaria while reducing drug resistance, and reducing the need for intravenous fluids during diarrheal episodes. PSI will also expand the socially franchised ProFam network of family planning service providers into four additional departments outside of Cotonou and will enlarge the network‘s service portfolio to include VCT and PMTCT services in select clinics. PSI‘s program includes provisions to ensure the sustainability of commodity supply. Commodity agreements with KfW will provide male and female condoms, oral and injectable contraceptives through December 2007. In addition, PSI will continue to use its own revolving funds to cover commodity-related costs for Orasel and ITNs, ensuring the ongoing availability of these important health products. With USAID approval and support for marketing costs, PSI can also establish sustainable revolving funds to cover other commodities if willingness-to-pay studies show that target groups can afford to pay for products at cost recovery price. PSI recognizes, however, that increased product sales alone are not an adequate measure of success; sustained behavior change and product use are the true keys to sustained health impact. To accomplish this, PSI proposes a range of evidence-based integrated communications campaigns to increase the integration of messages across projects, health themes, and communication channels for maximum health impact. Working in collaboration with local partner organizations, PSI will increase the scope and frequency of activities for greater impact and will introduce new activities that ensure a better balance of outreach in urban and rural areas, including translation of all communication materials into local languages. 154 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES JHPIEGO, a recognized leader in the field of integrated service delivery, will lead the PSI Team‘s efforts under Objective 3 to establish and scale-up PMTCT, VCT, and HIV prevention and treatment services and improve quality service provision of the ProFam clinic network. JHPIEGO will begin with an assessment of potential sites, select those sites with the greatest potential for success, support implementation of integrated services through training, operational support, and ongoing monitoring and quality improvement using state-of-the art techniques. JHPIEGO brings a wealth of experience to this initiative, having worked with the Ministry of Health of Benin, the Programme Nationale de Lutte Contre le SIDA (PNLS) of Benin, USAID/Benin, and other collaborating partner agencies since 1998 to improve public-sector health facilities in Benin in the areas of HIV/AIDS/STI prevention, family planning, reproductive health, maternal health, and prevention of malaria in pregnancy. Abt Associates, a leader in the field of health systems policy strengthening and capacity building, will lead the PSI Team‘s activities under Objective 4. Building on its role of providing technical assistance to strengthen the capacity of the Benin health system to deliver high-quality services, MOH policies, and implementing key health-financing strategies under the PISAF Project, Abt will implement a multi-level and participatory approach to policy and advocacy through the CNLS/PNLS, the private sector, and communities. Abt will focus on expanding and sustaining the technical and managerial capacity of the CNLS and PNLS, develop and implement an improved policy framework, and expand the role of communities and the private sector in the response to HIV/AIDS. Abt will provide an important linkage to the PISAF project, ensuring that the two projects‘ strategies and activities are complementary. PROGRAM DESCRIPTION This section begins with a discussion of the overall intervention context in Benin, presents the PSI Team‘s Technical Approach to IMPACTS and then describes the proposed strategies to address each of the four objectives of the program. Country Context Benin is a small country with an estimated population in 2005 of 7.2 million people, an annual population growth rate of 3.2% and a fertility rate estimated at 5.9. The population is increasingly young and urban. Over 44% of the population lives in urban areas, a percentage that has nearly doubled in the last 20 years, and poverty is widespread. In 2005, Benin ranked 161 out of 177 countries on the Human Development Index and has a per capita income of only $530. There are large gender disparities, with women and girls falling behind in most all socio￾economic indicators. Adult illiteracy is 74.3% among women 15 – 49 years and 51.0% among men in the same age group. Gender disparities limit women‘s ability to access information, which can significantly influence their health behavior decisions. For example, only 32% of women reported having any say (alone or with their husbands) in their own health care. Benin‘s infant and child mortality rates are among the highest in the region. Maternal mortality rests at 474 deaths per 100,000 live births. Although contraceptive use has doubled during the last five years, it remains below 10% and is largely concentrated in urban areas among more highly-educated women. In general, the health situation of those living in secondary urban and rural areas is significantly worse than those living in the capital, Cotonou. This context clearly points to the necessity of addressing the health issues of people living in rural areas and in urban areas beyond Cotonou, with particular attention to illiterate people, and particularly women. The Government dedicates nearly 10% of its national budget to health, yet the needs in this sector remain significant and unmet. The public health sector provides approximately 65% of USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 155 health care in the country but faces obstacles in the provision of quality services due to medical supply shortages, insufficient planning and management capacity, and a high level of turnover among health care workers. The private sector faces similar obstacles yet plays an increasingly important role in health care delivery in Benin, providing approximately 35% of medical services. Unfortunately, the private sector is largely unregulated, contributing to generally low-quality services. HIV/AIDS in Benin—The HIV and syphilis prevalence report published by the National AIDS Control Program (PNLS) in 2004 estimated the national HIV prevalence rate to be 2%. However, it varies greatly across the country and is significantly higher among at-risk groups. Prevalence rates are highest in the South, which includes a major West African transport route, although departments in the northwestern part of the country are also high. In 2004, the prevalence rate among prostitutes was reportedly 28% and 4.1% among their clients. When broken down by age group, the prevalence rate is 5% for youth under 15. This data is consistent with the fact that almost 40% of the youth in Benin have had their first sexual encounter before the age of 15. A baseline study conducted by the World Bank Corridor project in February 2005 reported 30% prevalence among prostitutes, 8% among persons in uniform, 6.6% among youth and adolescents aged 13 to 35 years and 5% among truckers. Across sub-Saharan Africa, including Benin, women are the most affected by HIV/AIDS: they constitute 58% of persons living with HIV/AIDS and 75% of infected youth, and they are often the ones left to care for sick family members and the first to leave work to care for the sick. Although considered a low prevalence country, Benin‘s concentrated epidemic threatens to spill over to the general population. This calls for a response that focuses various activities among high risk groups in order to reach a ―tipping point,‖ to cause a shift in social norms and behaviors to alleviate such things as stigma and HIV/AIDS. Any single intervention in isolation will have less impact than a combination of strategies and messages using multiple channels to reach target groups. The Ministry of Health has attempted to actively address the spread of HIV/AIDS since its first official plan was developed in 1987. The PNLS just adopted its fifth national strategy for the period 2006 – 2010, which lays out a series of targeted intervention areas. International donors, notably USAID and KfW, have supported these efforts with capacity-building support to the PNLS, condom social marketing activities, communication activities targeting youth and high-risk groups, STI case management and limited VCT services. However, the need and the opportunities for these services and activities are much greater than the current supply. Reproductive Health and Family Planning in Benin—Benin‘s high infant, child and maternal mortality rates are directly related to an unmet need for family planning products and services. According to the 2001 DHS, the desire to space births is 53% but actual use of modern contraceptives is under 10%. The average desired number of children is 4.6 per woman, but the fertility rates remains at 5.9. The government has taken concrete steps to address the issue of maternal and infant mortality, establishing norms and protocols for ante-natal care, the management of the third stage of labor, and post-natal care. However, the GOB has paid far less attention to promoting the benefits of contraception and birth spacing as a means to protecting family health. Women in rural areas are at a particular disadvantage due to limited access to and knowledge of modern contraceptive methods. The lack of male involvement in reproductive health further limits couples‘ use of modern contraception. Programs directed exclusively toward women fail to address the role men play in contraceptive decision-making. Men must understand the issues, recognize the benefits and be encouraged to support positive health choices for their partners and their families. 156 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Child Survival in Benin—High levels of infant and child mortality in Benin are attributable most notably to the high rates of malaria and diarrhea, particularly in rural areas. According to the 2004 Annual of Health Statistics, malaria is the leading cause of childhood deaths and is responsible for 37% of all health care consultations. Of the roughly 700,000 malaria cases reported in 2002, approximately 17,000 resulted in death. The GOB and its development partners have made measured progress towards achieving the Abuja targets. Notable success in expanding the demand for and use of insecticide treated bednets (ITNs) among pregnant women and children under five have been achieved in certain focus areas such as Oueme/Plateau, Zou/Collines and Donga. Yet overall demand for subsidized ITNs continues to outpace supply, with the provision of highly subsidized ITNs persistently plagued by inadequate and unreliable supply via the public sector. Access and affordability of effective malaria treatment is another key element to reducing malaria-related maternal and infant mortality. Support from the World Bank and Global Fund is expected to provide the public health sector with limited stocks of artemisinin combination therapy (ACT), but to achieve adequate coverage rates the strategy must take into account the fact that roughly 60% of patients access malaria treatment via private-sector outlets. 2004 Annual Health statistics indicated that diarrhea was the third most common reason for medical consultations and the fourth most common reason for hospitalization for children under five. According to the 2001 DHS, 13% of children under five and over 20% of children between six and 24 months experienced diarrhea in the two weeks prior to the survey. In general, rates are higher in rural areas where the lack of clean water and sanitation, poor hygiene practices, along with early weaning are the principal causes of diarrheal disease among children under five. Among rural mothers of children under five, 41% do not know that oral rehydration salts (ORS) or oral rehydration therapy (ORT) should be given during episodes of diarrhea. Increasing the availability and use of ORS/ORT and introducing zinc to treat diarrhea are two important strategies for reducing infant and child mortality due to diarrheal disease. In addition, point of use water purification products can be promoted and distributed to help reduce the incidence of diarrhea. Provision of HIV/AIDS-related Services—Although a large number of Beninese wish to be tested for HIV, few actually are. Barriers to testing include inaccessibility, low risk perception, lack of availability, stigma and discrimination, and lack of knowledge that treatment exists. Those who test HIV-positive have few health facilities that can or are willing to serve their needs, and most patients must travel far distances to access testing and medical evaluation and support services. Beninese youth and women are particularly adversely affected by the lack of integrated services, because their access to health services and their health behaviors are often dependent on factors that are outside of their full control. A successful model for integrated HIV/AIDS/STI and other health services like family planning and PMTCT, in addition to well-trained professional providers, would greatly enhance efforts to meet the expanding need for HIV/AIDS-related services. MOH and GOB Capacity to Manage and Implement HIV/AIDS Program—Important steps have been taken by the GOB to comprehensively address the HIV/AIDS epidemic and to prevent increased incidence. Importantly, the CNLS has been established and is supported by the Head of State and is made up of representatives of the many sectors of Beninese society that are most affected by HIV/AIDS. Nonetheless, the GOB and MOH lack the technical capacity to address the myriad policy and advocacy issues that continue to face the country. It is therefore crucial that the technical capacity of the GOB, MOH, and the CNLS in particular be strengthened to allow for a supportive environment for the coordination and provision of health services, as well as for social marketing. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 157 Technical Approach Overview Building on the strengths and lessons learned from its experiences in Benin as well as activities conducted as part of PRIMS, BHAPP and other programs, PSI proposes to improve upon successful interventions in social marketing, behavior change, service delivery and institutional capacity building through: 1. A tighter segmentation of target groups that addresses the needs of women and youth – with a strong focus on gender - and a concentration of activities and messages in targeted locations with a proportional balance between urban and rural areas; 2. Strong partnerships with local NGOs, community-based radios and public sector actors for community-based interventions, as well as international technical assistance for service integration and policy-level institutional capacity reinforcement both in the private and the public sector; and 3. Marketing existing products, re-positioning some products, and introducing new products. This Technical Approach section first provides an overview of the PSI Team‘s approach to implementing IMPACTS and the overarching themes that guide that approach. The subsections that follow provide more detail on our approach to achieving each of the objectives of the program. Overarching Themes The PSI Team‘s approach to implementing IMPACTS is guided by three overarching themes: Evidence Based Programming—All strategies conducted under the IMPACTS project will be driven by quantitative or qualitative research, and/or drawn from our relevant past experience in Benin and the region. For example, data show that Beninese youth are at particularly high risk of HIV due to their high behavioral risk factors. Further, youth make up a large portion of high-risk groups for HIV/AIDS, which also include prostitutes and long distance drivers. The general youth population – both in and out-of school – and targeted groups such as truckers and prostitutes will be a key focus for the PSI Team‘s HIV/AIDS prevention strategy under the IMPACTS project. Integration, Coordination and Collaboration—PSI‘s targeted health interventions will achieve measurable health impact via an integrated approach that will involve collaboration with various actors to deliver a wide variety of health messages and activities across a range of communication channels. At the institutional level, the PSI Team will continue its strong collaboration with the GOB, in particular the MOH and its specific agencies and programs, such as the Direction de la Santé Familiale (DSF), the PNLS, and the Programme National de la Lutte contre le Paludisme (PNLP), to ensure that all interventions are coordinated and support national strategic objectives. The PSI Team‘s strategy will also be closely coordinated with USAID‘s PISAF project, as well as its projects in the education and governance sectors as appropriate. PSI‘s diverse donor base will further enhance the project‘s collaboration with the Beninese government by facilitating the cost-effective use of resources. PSI/Benin currently is supported with funding from USAID, UNICEF, the KfW, the US DOD (DHAPP) and the Georgetown Institute for Reproductive Health (IRH). PSI will ensure that IMPACTS activities complement rather than duplicate these partners‘ efforts. At the operational level, the PSI Team will establish partnerships with local NGOs in seven strategically-selected sites to implement community-based social marketing and behavior change communication activities. Involving community-based actors in developing and implementing program activities will ensure greater motivation and buy-in to the project‘s messages and 158 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES themes. The project will also work with 13 local radio stations that cover these seven sites to build their capacity and bring the project‘s messages closer to the target groups, thereby reinforcing the social marketing and interpersonal communication activities implemented by local NGO partners. At the programmatic level, the PSI Team will integrate key themes central to positive behavior change that cut across all health intervention areas, such as gender. PSI/Benin‘s commitment to addressing gender inequality is central to the overall strategy and will be incorporated into all components of the program. PSI will train project and partner staff on the impact of gender and equality issues, and how to incorporate this into program design and activities, thereby instilling a cultural base that is supportive of gender issues throughout the project. The project will conduct a thorough review of all communication materials to ensure that they adequately and accurately take gender issues into consideration, and will also help to ensure that service delivery is equitable and accessible to women through the availability of female service providers. The PSI Team will also develop a series of at least six briefs addressing various gender issues to be used by PSI‘s youth program. Maximizing Sustainability—PSI‘s approach to sustainability includes not only a long– term commitment to the people its programs serve, but also a long-term commitment to the development of local social marketing capacity in order to sustain health impact. Five key issues in sustainability are central to the proposed project: capitalizing on Beninese capacity, the transfer of skills and technology to local counterparts, setting up cost-recovery mechanisms, behavior change among the target populations and supporting decentralized decision-making. Capitalizing on Beninese Capacity—PSI will continue to rely on local expertise in all of its program areas. PSI‘s many years of experience collaborating with the private sector in Benin includes not only social marketing distribution activities but also the production of materials and audio/visual supports and the implementation of research and communication activities. In working with local distributors, market research firms, advertising companies, drug manufacturing companies and training specialists, PSI demonstrates that appealing to the profit motive can be used for socially beneficial ends and strengthens local private-sector capabilities to carry out education, communication and marketing activities related to health. Trainings conducted as part of the program will maximize use of Beninese experts, utilizing outside expertise only if local capacity is unavailable. Transfer of skills to local counterparts is central to the proposed program. Through local partnerships with NGOs, community radios, health facilities and members of the commercial sector, PSI‘s proposed interventions will transfer lasting and useful health communications and marketing skills to key stakeholders. Transferring competence to local organizations will be an important focus of all of our work under IMPACTS. In each program component, the PSI team will work closely with local NGO, community, and private sector actors with the overall goal of increasing their capacity to implement similar activities by the end of the project. To aid in achieving this important goal, the PSI team will adapt a number of useful assessment and training tools developed by our REsulTS Initiative, a three year project funded by the Royal Netherlands Government to build the capacity of local PSI affiliates and local partners in evidence-based social marketing. REsulTS is staffed with an experienced group of social marketers, researchers, and development managers based in Johannesburg, South Africa. The initiative takes a broad approach to capacity building, offering face-to-face and distance learning programs in 7 areas: Evidence-Based Social Marketing, Management and Leadership Development, Research, Financial Management, Warehouse Management, Contracts, and Procurement. REsulTS has developed a number of capacity assessment tools USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 159 to examine the capacity of local partners in such areas and design capacity building plans to meet specific goals in these areas. IMPACTS will adapt these tools for use with local implementing partners and building their capacity through both formal training and day-to￾day co-management of activities will be a primary focus of our efforts. PSI trainings will always include a gender component and hands-on applicability components, as well as supervision and follow-up, to ensure that participants are able to replicate what they have learned. Establishing cost-recovery mechanisms to assure the sustainability of social marketing programs is essential to the project‘s long-term impact. PSI strives for full product cost recovery whenever feasible. Successfully selling products at cost recovery levels requires that local product demand is sufficient to support a price that is sustainable, covering the cost of the commodity and securing continued supply without donor support. PSI has made significant strides toward cost recovery prices for several products. For example, PSI‘s SuperMoustiquaire LLIN, Orasel ORS and Equilibre injectable hormonal are sold at commodity cost recovery prices, thereby limiting the reliance on donor funding and ensuring greater sustainability. Full cost recovery, which includes promotional and operational costs related to product sales, is more difficult to achieve given the importance of distributing products at prices that vulnerable people can afford and are willing to pay. However, PSI works constantly to increase efficiency, leverage resources, and negotiate lower commodity prices with suppliers in order to move closer to the goal of full cost recovery. Creating sustainable behavior change, not just product sales is fundamental to providing Beninese with the opportunity, ability and motivation to adopt positive health behaviors. PSI achieves this by making high-quality health products available and accessible and by influencing the individual‘s decision-making process through targeted behavior change communications. All of the strategies and messages proposed as part of this project are developed, pre-tested, adapted and evaluated to encourage sustainable behavior change among target groups, particularly youth. The empowerment to make informed health decisions will stay with these individuals long past the program‘s end date. Supporting Anti-corruption and Decentralized Decision-making—PSI fully supports USAID‘s decentralization mandate and will take advantage of opportunities to support the decentralization of its interventions with the goal of increasing transparency and accountability among stakeholders in the targeted areas. Specific ways in which the program will promote decentralization include, first: the inclusion of MOH officials, local authorities, community health management members, and representatives of key target groups including PLWHA, women and youth in the Family Health Advisory Committees. Secondly, the program will empower community radio stations and local NGO‘s to increase the decentralization of knowledge, equipping more people in more areas to be involved in health issues that concern them. The subsections that follow describe how these three overarching themes – evidence-based programming; integration, coordination, and collaboration; and maximizing sustainability – will be incorporated into the PSI Team‘s strategies and activities for achieving each of the four IMPACTS objectives. Objective 1: Improving Availability and Access to HIV/AIDS, RH/FP and CS Products Successes to Date Since beginning activities in 1990, PSI/Benin has improved availability and access to HIV/AIDS, RH/FP and CS products. Strategies such as commercial marketing, behavior change communication through community and mass media, and interpersonal communication have helped to create the knowledge, skills, opportunity and social support to motivate healthy 160 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES behavior among low-income Beninese. With funding from USAID and KfW, PSI has socially￾marketed male condoms in Benin since 1990. Since that time, PSI has dramatically expanded its portfolio of socially-marketed products to include family planning contraceptives, Equilibre (injectable hormonal), Harmonie (oral hormonal) and La Méthode du Collier (standard days natural method), as well as child survival products including Orasel (an ORS), Super Moustiquaire, Super Moustiquaire Bonne Maman, Super Moustiquaire Famille Protegee, (treated mosquito nets) and ALAFIA (a re-treatment kit). In 2006, PSI introduced Benin‘s first commercial long-lasting insecticide treated net under the brand name Super Moustiquaire Longue Durée. PSI‘s strong relationships with commercial wholesalers in the general and pharmaceutical markets throughout Benin have enabled the expansion of its portfolio of socially-marketed products while maintaining a high level of cost efficiency. In addition, PSI has benefited from access to internal PSI funding mechanisms that have allowed it to procure and sell crucial products at cost recovery prices, thereby achieving sustainability in the local market at very low cost to donors or the GOB. The table below summarizes successfully socially-marketed products by PSI/Benin to date: Product Launch Date Quantity Sold through Mar ‘06 Prudence Plus male condoms 1990 61,870,791 Orasel oral rehydration salts* June 1995 16,051,611 Harmonie oral contraceptives April 1998 694,492 Super Moustiquaire ITNs September 1998 238,426 Alafia ITN home (re)treament kit August 1999 205,081 Equilibre injectables April 2000 93,491 Super Moustiquaire Bonne Maman ITNs November 2002 (Zou/Collines) March 2005 (Donga) 188,200 38,705 Methode du Collier (cycle beads) December 2004 3,704 Super Moustiquaire LLIN* May 2006 37,100 * Indicates products procured by PSI and sold at commodity cost recovery prices. PSI has also pioneered the social marketing of reproductive health services in Benin with the 2004 launch of its franchised ProFam network of clinics. ProFam‘s goal is to provide high-quality, standardized, affordable services through a private-sector clinic network. Through PSI‘s 21 ProFam centers, Beninese couples can access quality family health services. With USAID/Benin and regional support, PSI has also offered quality HIV/AIDS, VCT and STI case management services through the PSAMAO clinic managed by its partner NGO, OSV-Jordan, in Parakou since 2000. With KfW support, this successful model will be expanded to the city of Tanguiéta with the establishment of a second PSAMAO clinic and mobile VCT unit that will serve communities in all 13 communes of the Atacora and Donga departments as well as 12 additional secondary cities across the country. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 161 Continuing Challenges Despite these successes, much remains to improve availability and access to HIV/AIDS, RH/FP and CS products. In HIV/AIDS, there is room to foster increased condom use for STI/HIV/AIDS prevention by addressing Prudence Plus quality perceptions and modifying the masculine image of the product to make it more attractive to women, and reviewing the wholesaler/retailer price structure to assure availability in high-risk locations as well as secondary urban and rural areas to minimize price discrepancies. Modern RH/FP products, including oral and injectable contraceptives, are nominally available exclusively through public-sector health facilities, registered private-sector health care providers and registered pharmacies. However, contraceptive availability in the public health sector is unreliable, with regular stock-outs of contraceptives and other medical products. The black market also offers some modern contraceptives at prices lower than those found in the pharmaceutical and public health sectors. PSI has successfully marketed two products related to Child Survival, oral re-hydration treatment and ITNs. PSI introduced Orasel ORS in 1995 to support the GOB‘s recommendations to increase in liquids and feeding of children to fight the effects of dehydration from diarrhea. Overall, 72.5% of mothers reported knowing about ORS for the treatment of diarrhea, although only 59% of mothers in Atakora reported knowing about ORS, compared to 92% in Cotonou. A continuing challenge is that the diarrhea disease burden remains higher in rural areas where knowledge of correct treatment including ORS/ORT is lower. The majority of mosquito nets available in Benin are either untreated or require regular re￾treatment. Since re-treatment rates are consistently low, this significantly limits their effectiveness. Some treated nets imported and distributed by PSI and private companies are available via commercial channels. PSI, with the support of UNICEF and USAID, also distributes highly-subsidized ITNs to pregnant women and children under five in Zou, Collines and Donga via antenatal clinics. The MOH conducts similar activities in the other departments of Benin, but these interventions are often limited by distribution inefficiencies, poor targeting of vulnerable groups, frequent stock-outs and the lack of an accompanying communication campaign to create informed demand. The World Bank Malaria Booster Program, scheduled to begin in 2006, plans a dramatic increase in the supply of long-lasting insecticide treated nets (LLITNs) for distribution to the most at-risk target groups. However, it is not yet clear what distribution or communications strategies will be employed. Despite the high incidence of malaria, use of treated mosquito nets remains relatively low in Benin, particularly outside the capital. Only 37% of children under five and 35% of pregnant women used ITNs in 2004. Continuing low usage is attributable to MOH budgetary constraints affecting supplies, limited financial access by target populations, insufficient private-sector involvement and inadequate re-impregnation of nets. Proposed Social Marketing Strategies To address these challenges while building on the successes of the PRIMS project, PSI will maintain key existing product lines while maximizing public health impact through innovations in marketing, distribution and the addition of new products to the USAID social marketing portfolio. Taking into consideration consumer preferences, new products available and/or soon to be available in the commercial sector and opportunities to maximize health impact, PSI proposes to maintain and to add the following products as part of the IMPACTS program: Expansion of existing product lines: Prudence Plus, male condoms Harmonie, oral contraceptives 162 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Equilibre, injectable contraceptives Super Moustiquaire Longue Duree, long-lasting insecticide treated nets Super Moustiquaire Bonne Maman Longue Durée, long-lasting insecticide treated nets Addition of new product lines7: Méthode du Collier, Standard Days Method (Cycle Beads) Orasel Plus, new formulation oral rehydration salts bundled with zinc supplements (actual brand name to be determined) PPT, pre-packaged malaria treatment (actual brand name to be determined) Female condoms, piloted with KfW funding during Year 1 with the possibility of an expansion in Years 2–5 with IMPACTS support pending assessment PSI currently has an agreement with the MOH and KfW that assures the provision of male and female condoms, OCs and injectable contraceptives to PSI‘s social marketing program through the end of 2007. However, KfW has indicated that due to shifting priorities, its support to the health sector in Benin will be phased out. Accordingly, PSI successfully lobbied to have commodity costs for male condom social marketing added to the country‘s Global Fund Round Five proposal, which was approved. This commodity support from KfW, the Global Fund, and PSI‘s own revolving funds for Orasel Plus and SuperMoustiquaire Longue Duree will help to provide cost share and defray the burden on donor and GOB support. However, long-term commodity support for female condoms, OCs and injectable contraceptives after 2007, as well as Super Moustiquaire Bonne Maman Longue Durée (subsidized LLINs), Méthode du Collier, and PPT is yet to be determined. The PSI Team will also explore additional distribution channels to expand the availability of key IMPACTS social marketing products into underserved areas. For example, PSI will collaborate with organizations such as CARE and the networks of local microfinance institutions (MFIs) to develop strategies that capitalize on the extensive community-based networks of microfinance institutions while making socially-marketed products available to support the income-generation activities among vulnerable Beninese populations. 7 The PSI team is entirely open to refining the list of proposed new social marketing products in collaboration with USAID. We read the list of new products on page 27 of the RFA as a menu of potential choices that could be proposed, rather than a list of what must be proposed. As we developed our proposal, particularly our detailed budget, it became clear to us that, given the budget resources available, we would not be able to propose a comprehensive, impactful social marketing program for every product listed. In our opinion, it was preferable to propose more comprehensive social marketing of a few products than spread resources across all of those products listed. Accordingly, we proposed adding pre-packaged malaria treatment (PPT), female condoms, cycle beads and zinc supplements to our existing product lines. These products were chosen due to their perceived market and health impact potential in Benin, the costs required to socially market them, and their estimated health impact. For example, PPT was chosen over point of use water treatment (POU) because of malaria prevalence amongst children under five (around 50% - UNICEF 2001) is higher than diarrhea‘s (13% - EDS 2001) and the fact that its negative health impacts are more acute. Again, however, we do believe that there is good market and health impact potential for POU in Benin. We have revised our Cost Proposal to include a feasibility study to examine this potential. In Annex 5 of our revised technical proposal, we present a plan for a pilot POU social marketing program that can be executed if additional funds become available, either through eliminating one (e.g. PPT) or more of the new products we proposed or by adding funding from another source. We would welcome an opportunity to sit down with the Mission upon award to weigh these choices and revise our proposed interventions accordingly. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 163 PSI‘s proposed product lines also will directly complement PISAF activities in the public sector as well as other planned donor interventions through the Global Funds and the World Bank Booster program. PSI will work closely with these actors as well as the MOH to assure that opportunities for synergies are maximized while avoiding duplication of activities. As such, the PSI Team has and will continue to play an active role in the development of a national commodities security plan currently being developed by the MOH. HIV/AIDS Successes and Opportunities— PSI has made remarkable progress in making condoms widely available, broadly known and increasingly accepted. Prudence Plus brand recognition is extremely high, which has reinforced its distribution network with sales outlets nearly ubiquitous across the country. Using 25 commercial wholesalers to reach over 17,000 sales outlets, PSI has established a highly-efficient distribution network: a distribution survey in 2005 reported Prudence Plus availability throughout the country and stock-out rates of less than 10%. In consultation with USAID, KfW and the MOH, PSI successfully increased the price of Prudence Plus in 2005. This was a large step toward price recovery and ultimate sustainability of the commercial condom market. PSI‘s branded promotional activities target both members of the distribution network and consumers and are largely responsible for making Prudence Plus a widely-known and accepted part of the commercial market. Proposed Strategy—PSI will foster increased condom use for STI/HIV/AIDS prevention in Benin by addressing Prudence Plus quality perceptions and modifying the masculine image of the product, reviewing the wholesaler/retailer price structure to minimize price discrepancies, and developing strategies to assure availability in high-risk locations, secondary urban and rural areas. PSI proposes to sell 9,400,000 socially-marketed male condoms in Year 1 of the program, with an estimated 5% annual increase through Year 5, using a balanced marketing approach and focusing on high-risk and non-traditional outlets. In addition, with KfW support, PSI will conduct a pilot distribution of 10,000 female condoms in Year 1 among prostitutes in Atacora and Donga. Following the pilot, PSI will assess the feasibility of scaling up this distribution in Years 2–5. Product Image: PSI‘s 2005 KAP research identified the influence of social support and partner discussion as key to increasing condom use among sexually active Beninese. Therefore, PSI proposes to re-position Prudence Plus for Beninese men and women who respect themselves and their partner. This will entail eliminating the masculine, aggressive panther image of Prudence Plus with a new image that will facilitate the promotion of the condom brand as a family planning product that can provide dual protection when using other non-barrier contraceptive methods. The repositioning of the brand will be done at the beginning of FY 08 in collaboration with the CNLS to assure that it complements the PNLS national strategy for STI/HIV/AIDS prevention. Based on formative research with sexually active individuals across high risk groups, the repositioning campaign will involve new packaging, branded mass media communications, and branded business-to-business promotions to attract new wholesalers and business-to-consumer promotions to increase demand. Youth 15–24 years are identified as a particularly at-risk group given their low risk perception and tendency toward high-risk behaviors. This is true both among the general youth population as well as youth within higher-risk groups such as prostitutes, migrant laborers and apprentices. Strategies for reaching at-risk youth will involve condom promotion coupled with messages on abstinence, partner reduction and the importance to get HIV testing and counseling as methods for reducing their risk. 164 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Pricing structure: While maintaining the end consumer price of 100 FCFA for a pack of 4 condoms, PSI will adjust the wholesaler price structure to include an incentive system designed to increase product availability within the HORECA (hotels, restaurants, cafes and bars) network. Wholesalers that achieve pre-determined quotas with regard to sales volumes and product coverage within the HORECA network will receive incentives. Quarterly wholesaler meetings and a new quarterly newsletter with information about best-of sales practices will support this new strategy. This new incentive-based pricing structure will help minimize the risk of price under-cutting by wholesalers. Along with the introduction of a new positioning and package, PSI will study the feasibility of increasing the price to cost recovery in light of the loss of KfW commodity support at the end of 2007. Product availability: PSI will build upon its commercial-sector distribution network by reinforcing and expanding product availability in secondary urban cities, rural areas and among high-risk populations. Efforts will center on the PSAMAO northern transport routes, focusing on Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguieta, Ouake, and Malanville, which are generally higher-risk locations due to the presence of highly mobile populations including truckers and migrants, and the prostitutes who follow these groups. This will complement the efforts of the World Bank Corridor project that is focusing on the southern route between Nigeria and Togo. PSI will also work with local retailers in the immediate proximity of long￾distance parking lots and in the port area of Cotonou due to the presence of many high-risk target groups in these areas. PSI assures the creation of Prudence Plus sales sites, possibly through the creation of Prudence Plus Kiosks as an outlet for information and condoms. PSI will expand its distribution network by developing a new distribution channel to increase condom availability in the HORECA market mentioned above, composed primarily of bars, restaurants, night clubs and hotels, locations often used as meeting sites for persons who engage in risky sexual behaviors such as transactional sex. Product availability will be tracked through regular MAP distribution surveys using Lot Quality Assurance Sampling (LQAS) methodology that permits tracking of overall availability as well as availability in areas identified as high risk. Promotion: Although current brand recognition is high for Prudence Plus, promotional efforts will be important to reinforce the product‘s new image, as well as to motivate wholesalers to open additional sales points to expand condom availability. To do so, PSI will develop and distribute a special wholesaler promotion kit, providing promotional materials for the wholesaler to use in promoting Prudence Plus among new and existing retailer clients, particularly the HORECA network. These activities will be reinforced by new Prudence Plus branded TV and radio commercials produced and broadcast through national, private and community radio stations. Additional promotional materials targeting consumers will promote the new brand image to high-risk youth and to women and their partners as an effective method against HIV/AIDS and unintended pregnancies. All promotions targeting high-risk youth will be consistent with USAID‘s ABC guidelines, promoting condoms with both "A" and "B" messages as well as presenting condom use as an effective dual protection method with other family planning methods. Additional theme-based generic campaigns using multiple communications channels (IPC, billboards, MVU, community and mass medias) will address identified obstacles to condom use including the individual‘s capacity to use condoms (promoting correct and consistent use) and the perception that condom use reduces sexual pleasure. Reproductive Health and Family Planning Successes and Opportunities— Since the launch of its first family planning program in Benin in 1998, PSI has widened its portfolio to offer socially-marketed oral (Harmonie), injectable USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 165 (Equilibre) and natural (Methode du Collier) family planning options, in addition to the male condom, as modern contraceptive choices for Beninese women and their partners. In addition to training service providers and pharmacists on these products, PSI has offered trainings in IUD insertion and removal as well as in the Active Management of the Third Stage of Labor (AMTSL). Sales of Harmonie have expanded from 11,652 during the first year to 124,440 in fiscal year 2005. However, the product remains heavily subsidized and movement toward cost recovery may be necessary to assure product sustainability, depending on future donor support and the national contraceptive security strategy. This is also the case for Methode du Collier, which is not currently sustainable without significant price subsidies, but has shown to be particularly popular among consumers with religious objections to other forms of birth control. Sales of Equilibre have climbed from 2,713 in 2000 to 21,512 in fiscal year 2005.8 Equilibre‘s three￾month cycle provides an important option for women and couples who want a longer protection period than that provided by the two-month injectable product distributed through the public health sector but often unavailable due to stock-outs. Equilibre‘s price is nearing cost recovery and comparable to the public sector price of injectable contraceptives. PSI has worked closely with all private pharmaceutical wholesalers in the country, providing training for both pharmacists and health care providers on family planning products and reinforcing consistent supply. All three products are available to public health centers throughout the country, although public health workers do not regularly include them in product requests due to lack of information, as well as certain myths and misconceptions surrounding the 3-month injectable. The majority of authorized private-sector distributors of family planning products are concentrated in Cotonou. Of the 172 officially registered pharmacies, 110 are located in the capital city. With frequent stock problems plaguing the public sector, this leaves women in other urban and rural areas throughout the country with significantly lower access to high-quality family planning products. Furthermore, promotion of modern methods of family planning is quite limited since Beninese law allows only generic family planning messages to be shared through mass media, although an exception was made for the Méthode du Collier. Proposed Strategy—In order to increase contraceptive demand and use and improve women‘s reproductive health, PSI will tighten its segmentation strategy to better meet the needs of groups desiring access to modern family planning and will expand the distribution and promotion of all contraceptive products, including the promotion of Prudence Plus as an effective dual protection product. The PSI Team will reinforce and extend the ProFam social franchise network by adding clinics in and outside Cotonou, and will integrate La Méthode du Collier as a permanent product in its family planning portfolio, thereby offering an even wider range of products that meets specifics needs of target groups. With a targeted annual sales increase of 5%, total commodity needs for the five-year project are estimated as follows: 718,332 cycles of Harmonie, 124,327 doses of Equilibre, and 59,566 Methode du Collier. PSI anticipates Year 1 sales of Harmonie at 130,000, of Equilibre at 22,500 and of Methode du Collier at 10,000. Brand positioning: According to the 2001 DHS, more than 70% of unmet needs in family planning fall in the 15-29 year old age group. With oral contraceptives being the preferred method of hormonal contraception among this age group, PSI will reposition the Harmonie brand to increase its attractiveness among younger women of reproductive age, specifically sexually active married and unmarried women living in urban areas. At the same time, using the 8 Equilibre is a brand of Depo-Provera, an injectable contraceptive that works for three months. 166 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES new Prudence Plus brand image that positions the product for both men and women, PSI will promote the male condom as a family planning option that is effective in providing dual protection when using a hormonal contraceptive product. Equilibre will continue to be promoted as an effective and discrete birth spacing method, particularly well adapted for women in rural areas who do not have easy access to health centers on a regular basis. La Méthode du Collier will be targeted to reach the 18% of women living in union in search of an effective natural family planning method. Recognizing that use of family planning methods increases with male support, PSI will develop a series of communication strategies and activities targeting men to help them recognize the benefits of family planning for themselves and their families, and to encourage them to support women‘s positive health choices (described in Objective 2). With the addition of the Methode du Collier to its suite of family planning products and services offered through the IMPACTS program, PSI will include this product in all health worker trainings and will ensure that it is included in all interpersonal family planning communication activities, including those held in partnership with ProFam clinics. Pricing structure: PSI will conduct surveys to assess client willingness to pay for Harmonie and Methode du Collier so that price increases may be considered if appropriate, therefore moving the products closer to cost recovery and sustainability in the local market. Product availability: PSI will use distribution strategies that increase availability beyond Cotonou and within non-commercial sectors to increase contraceptive access for Beninese women and their partners. PSI will work in close collaboration with PISAF to increase the availability and visibility of socially-marketed family planning products within the public health sector and through the expansion of the ProFam network outside Cotonou to increase access and availability in areas yet underserved by the pharmaceutical market. Harmonization of operational plans will ensure avoiding duplications of efforts. PSI will also continue to use a team of medical detailers that conduct regular visits to pharmacists and wholesalers in the interior of the country, and will organize information sessions with community-level health management associations (COGEAs) to explain PSI‘s program of social marketing and the impact it can have on the reproductive health of women in their community. COGEA board members will be encouraged to consider including Equilibre, Harmonie, the Methode du Collier and other important contraceptive products in their product requests to the CAME. Promotion: PSI will continue to promote its family planning products portfolio through a combination of generic mass media advertising as well as interpersonal communication (described in more detail in objective 2). Mass media will be used to promote modern family planning methods, stressing the advantages and importance of birth spacing to couples. Men will be targeted specifically in order to increase their knowledge, involvement and support for their female partners and for family planning. Medical detailers will distribute specially-tailored product information brochures to pharmacists and service providers to reinforce their capacity to explain the product characteristics to potential clients and to respond to misinformation about the products. Promotion of FP methods will always be accompanied with dual protection messages regarding STI/HIV/AIDS. PSI will also work to increase the family planning knowledge of future health care providers by conducting one-day capacity building and product information sessions twice a year for the graduating classes of the two largest medical schools in the country, as well as for the National Association of Pharmacists and the National Association of Midwives. These activities will be further supported through the incorporation of family planning training and promotion activities scheduled during National Pharmacist‘s Day, National Midwives‘ Day and National Doctor‘s Day. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 167 Social Franchising Successes and Opportunities —In 2004, under the PRIMS project with support from USAID, PSI launched a social franchise network of private family planning centers under the brand name ProFam. This program aims to increase the use and improve the quality of private￾sector reproductive health services through training, standardization of tools, and regular supervision. Branded signs and communication supports increase visibility and promote the network as having standardized, high-quality services. Launched in close coordination with the DSF, the ProFam network currently offers family planning services to thousands of women and men of reproductive age in and around Cotonou. Services offered as part of the franchise include family planning counseling and the sales of family planning products, such as injectable hormonal contraceptives, oral hormonal contraceptives, cycle beads, condoms, and spermicides. At least two providers from each clinic have been trained in family planning, health protocols, contraceptive technology and service protocols called for by the Government of Benin and the World Health Organization. With support from USAID‘s AWARE program (Action for West Africa Region – Reproductive Health), PSI will expand the ProFam network to include 25 sites in Cotonou and five in Oueme-Plateau by the end of fiscal year 2006. AWARE support has also enabled PSI to address specific needs identified by ProFam members, resulting in specialized training in IUD insertion and removal as well as in the Active Management of the Third Stage of Labor (AMTSL), which have been added as ProFam services. To reach their surrounding communities, centers use health and behavior change communications techniques such as one-on-one counseling, group discussions facilitated by a trained service provider, and mobile information booths. ProFam health centers are recognized throughout Cotonou as health care facilities that offer quality family health care. The network‘s success is due in part to the work of the advisory committee that oversees the network and brings together civil society, the MOH, and private sector actors to assure that activities respond to local needs. Proposed Strategy—Building upon successes of the pilot project, PSI proposes to reinforce the services provided and to expand the number of clinics within the ProFam network. Network services will be reinforced through ongoing provider training, the addition of certain key services, and the adoption of a targeted strategy to reach youth. At the same time, the network will be expanded in terms of numbers of clinics, building upon economies of scale and increasing the numbers of clients receiving services via the network. All of these efforts will be supported by an extensive promotional campaign raising awareness of the network and highlighting the quality services available via ProFam member clinics. Reinforcement of Services: Survey data show that client satisfaction still needs to be improved among network providers. Therefore PSI will work ProFam managers and staff to reinforce capacity and establish systems that support improved performance. More specifically, PSI will conduct annual trainings for at least two staff members (the manager/owner and another staff member –doctor, midwife or nurse) from each ProFam clinic on customer service and its impact on client satisfaction and retention, as well as develop a ranking system with an incentive system to reward best performing centers. ProFam centers with the highest customer satisfaction scores earned during site evaluations (mystery client and exit surveys) will be rewarded with promotional product supplies as part of an incentive-based initiative to motivate service improvements. A quarterly newsletter will promote the top 10 clinics and present a profile of some of their health workers. In addition to adding the above-mentioned trainings, PSI will offer revised refresher trainings that were offered to clinics as they entered the network. Topics include contraceptive 168 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES technology, Beninese family health standard procedures and protocols, and the Active Management of the Third Stage of Labor. An important part of the Beninese family health standard procedures and protocols includes recently revised procedures for the prevention of mother-to-child transmission (PMTCT) of HIV, which were implemented to improve the prevention of HIV transmission from mother to child. The Ministry of Health of Benin recently modified antenatal consultation health protocols to integrate HIV counseling and testing, STI testing and treatment, and PMTCT for mothers testing HIV positive. These were developed under a redefined antenatal consultation approach (Consultation Prénatale Recentrée - CPNR). The CPNR is a prevention measure that enables providers to identify risk factors of a pregnancy, monitor the pregnancy‘s progress, administer preventive care and counseling, and manage any complications during pregnancy. Follow-up activities are geared toward various stages of pregnancy, including fetal development. This approach dictates that a pregnant woman infected with an STI must be treated systematically according to the current national protocol. Additionally, any HIV-positive pregnant women must receive PMTCT care, limiting the risks of HIV transmission to their infants. In this context, STI case management must be systematically integrated into training of ProFam providers to conform to modifications to prenatal consultation legislation. While most ProFam clinics already offer HIV testing services and HIV counseling, STI counseling and testing is not common. In conjunction with trainings for the integrated pilot sites in Abomey/Bohicon and Parakou (described in objective 3 below), PSI will train ProFam staff to use the CPNR framework to provide comprehensive VCT, STI and PTMTCT services. Oversight will include detailed site evaluations and counselor supervision to ensure high-quality service delivery, such as counseling and laboratory testing and referrals for post-test care and support. PSI will also position certain clinics as ―youth-friendly‖ and train health workers on youth￾specific family planning counseling and service delivery. This strategy will build upon best practices learned from a successful USAID-funded pilot project launched in 2001 to make pharmacies in Mexico and El Salvador more youth-friendly9, as well as experiences with PSI‘s youth-friendly Top Reseau in Madagascar and the Centre Dushishoze in Rwanda. Network Expansion: With support from the USAID/AWARE project, the ProFam network will be comprised of 30 centers by September 2006. Under the IMPACTS project, PSI proposes to expand the network to a total of 50 clinics by the end of the project. Expansion sites will be selected based upon the identification of locations characterized by low levels of contraceptive availability and proximity to other key IMPACTS interventions. The network expansion plan will be implemented in stages, with clinics added in Abomey/Bohicon, Parakou, and Oueme/Plateau, and more in Cotonou. Network Promotion: The ProFam network will be promoted using a combination of mass media and interpersonal communication techniques. Using materials produced as part of its AWARE project, PSI will continue to raise the visibility of the ProFam network and the quality of services offered through ProFam health centers. PSI will also distribute flyers with maps and addresses of the ProFam member clinics as well as the free hotline number that provides information on clinic locations and family planning. This free informational hotline has been shown in other countries, such as the DRC, to be a popular way for men in particular to ask for family planning information anonymously. PSI will also introduce an innovative, targeted voucher program that promotes ProFam services to targeted clients in its local neighborhood by using a motivation system that involves client recruitment, behavior change communications and incentives for first-time family planning 9 http://www.fhi.org/NR/rdonlyres/e4too4ldyzlizvih4z7fsjk6wdj3idl4hzoy77t36tjsv3c2bi5wd6uzx4xz44g3ysfl smarlcmrib/YL17f.pdf USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 169 clients. These efforts will be reinforced through regular promotional/informational ―Soiree Santes‖ (Health Evenings) in close proximity to and in partnership with each ProFam center. The evenings will provide a forum to discuss the advantages of family planning, describe the products and services available through the ProFam network, answer questions about family planning and also provide important family health information on issues such as malaria, diarrhea and STI/HIV/AIDS prevention with a strong focus on the need for testing as a means to reduce stigma. Child Survival—Diarrheal Disease Successes and Opportunities—PSI introduced Orasel as a socially-marketed oral rehydration solution (ORS) in June 1995. All commodity-related costs are managed by a revolving fund established by PSI, which ensures product supply without donor subsidies. It is priced at cost recovery but is still affordable to the majority of the vulnerable population, making it an effective and accessible tool in treating diarrhea among Beninese children. As it is not considered a health product, PSI has been able to distribute Orasel through a variety of commercial outlets, reaching nearly the same level of coverage as achieved with Prudence Plus. Even with very limited promotion, PSI has been able to sell 16,051,611 packets since its launch in 1995, with 1,783,487 sold in 2005 alone. Recent improvements in the composition of oral rehydration therapy publicized by UNICEF and WHO suggest that Orasel could be more effective if it were updated with the new ORS formulation. As Orasel use in Benin has seemingly plateaued in recent years, a revitalization of the product‘s composition and image is both timely and impactful. Proposed Strategy—In 2005, WHO and UNICEF issued a joint statement to update previous recommendations for fighting diarrheal disease worldwide. New recommendations assert that it is most effective to use the new-formula ORS combined with a supplement of 20 mg of zinc per day for 10 days. To increase the health impact of PSI‘s socially-marketed ORS product, PSI proposes to update Orasel with the new WHO and UNICEF-recommended oral rehydration formula and to bundle it with a 10-tablet pack of zinc to be sold as a new bundled product tentatively branded Orasel Plus, although final branding will be based on consumer research. Bundling ORS with a zinc supplement will not only shorten the length of the diarrhea episode, but also reduce the likelihood of a repeat episode in the following 2 – 3 months. Pending MOH approval of the zinc supplement, new formulation Orasel will continue to be sold in packs of three. PSI will also add a flavor to the new formulation, making it more attractive and appetizing for children and their caregivers. A local taste test will determine whether lemon or orange is more attractive to children. Once the addition of zinc is approved by the Ministry, the new product, tentatively named Orasel Plus, will be comprised of 2 sachets of ORS and 10 tablets of zinc. In order to sell Orasel Plus at cost recovery, PSI will need to increase the consumer price by 25% over the current price of Orasel. This will allow PSI to continue to cover 100% of commodity costs via its revolving fund. However, if consumer research determines that this increase in price will be a barrier to use among the target audience, PSI will engage USAID, the MOH and other donors such as UNICEF to determine if some commodity cost support may be available. With the inclusion of zinc, it is possible that the MOH may wish to classify Orasel Plus as a medical product, thereby restricting its distribution to the CAME and pharmaceutical wholesalers. Therefore, during the approval process for the new ORS-zinc formulation, PSI will lobby the MOH to ensure that Orasel Plus continue to be classified as a non-pharmaceutical product similar to the current Orasel product, thereby allowing for a much broader product distribution via the commercial sector. Once approved by the MOH, PSI‘s distribution strategy will focus heavily on wholesalers and sales outlets in rural areas with the highest level of 170 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES diarrhea incidence. Product availability will be tracked through MAP distribution surveys using the LQAS sampling methodology. Based on historical sales of Orasel, and with the introduction of flavoring and a new formulation, PSI estimates annual sales increases of 5%; therefore, total commodity needs for the five-year project are estimated at 9 669 860 Orasel Plus sachets and 4 159 930 zinc blisters. The new Orasel Plus will be supported by mass media as well as interpersonal communication activities to raise awareness of the new product and promote its use in treating diarrhea. Mass media efforts will include television and radio mass media spots and promotional materials distributed to commercial wholesalers and retailers, as well as public-sector health centers, including all PISAF target sites. PSI will also develop a hygiene flip chart in coordination with PISAF that incorporates diarrhea prevention messages and information on Orasel Plus for diffusion through all appropriate communication channels used by PSI and other public health agents for interpersonal communication activities, including mobile video projections, ProFam Soiree Sante, as well as other PSI behavior change communication activities described in detail in Objective 2 below. Child Survival—Malaria Successes and Opportunities—Since the launch of its first socially-marketed mosquito net in 1998, PSI has developed a highly segmented and targeted portfolio of mosquito nets that attempts to balance both consumer need and commodity availability under the umbrella brand SuperMoustiquaire. SuperMoustiquaire, a pretreated net launched in 1998 in the general commercial market, was complemented by the introduction of: 1) SuperMoustiquaire Bonne Maman sold bundled with the Alafia retreatment kit, targeting rural pregnant women and children under five years of age through a highly-subsidized distribution in public health centers in Zou-Collines and Donga; and 2) SuperMoustiquaire Famille Protégée, targeting the rural commercial market and sold bundled with the Alafia retreatment at cost recovery prices. PSI recently replaced its original SuperMoustiquaire pre-treated net with SuperMoustiquaire Longue Durée, a long-lasting treated net purchased through a PSI revolving fund and sold at cost recovery prices targeting urban and rural commercial markets. SuperMoustiquaire enjoys an exceedingly high level of brand recognition due to intense mass media promotional campaigns and interpersonal communication activities. In many areas, SuperMoustiquaire has become the de facto term used to refer to any type of mosquito net. SuperMoustiquaire products are distributed through both public and private sector outlets, including pharmacies and wholesalers. Distribution of SuperMoustiquaire Bonne Maman to pregnant women and children under five in the Zou/Collines departments, in conjunction with UNICEF and the MOH, has also been very successful. Nearly 90% of all pregnant Beninese women attend at least one prenatal consultation, making antenatal clinics an effective way to reach the target audience. The highly subsidized price of 500 FCFA and the buy-in of public health sectors, which receive a profit margin with each sale to the target group, contribute to the success of the intervention, which has raised coverage rates of target groups from less than 10% to 45% in less than three years. With UNICEF support, the Bonne Maman net was recently converted to an LLIN in early 2006 to further increase health impact. Unfortunately, commercial sales of SuperMoustiquaire Famille Protegee (SMFP), which was priced at cost recovery and destined for the population segment not eligible for a targeted subsidy, have been negatively affected by the provision of highly-subsidized ITNs from the MOH that have been available to target and non-target populations alike. The relatively higher price of SMFP also limits the number of outlets able to purchase a substantial stock of nets, sometimes USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 171 leading to stock outs or difficulty with stock renewal. Pharmacies are often the most able to stock SMFP, which significantly limits product availability in nearly all non-urban areas. Because nets requiring re-treatment generally have lower health impact due to low levels of re￾treatment (less than 50%), the introduction of the long-lasting insecticide-treated net SuperMoustiquaire LLIN is particularly important. It provides three years of quality protection from mosquitoes, assuring a health impact much greater than untreated nets or treated nets that require regular re-impregnation. PSI promotes this product in the urban commercial market using mass media and interpersonal communication activities. Effective malaria treatment, a key component in reducing the negative health impact of malaria, is not widely available in Benin. The public sector CAME system offers limited supplies of the artemisinin combination therapy (ACT) Coartem® but stock shortages and distribution problems significantly limit its availability. A number of malaria treatment regimes are available via the commercial sector in Benin, but many of these are mono-therapies, which put at risk the long-term efficacy of new combination therapies, while many others are of unregulated dosage and quality. There is no widely affordable pre-packaged (ACT) treatment currently available through the commercial sector in Benin. Benin also just recently signed an agreement with the World Bank which will provide $31M as part of the Bank‘s Malaria Booster program. Although the strategy is not yet clearly defined, the vast majority of these funds will go toward supplying LLINs and Coartem® via public-sector health facilities and community-based distribution. Such an infusion of LLINs in particular is bound to have significant impact on any commercial-sector net activities. Proposed Strategy—PSI proposes to streamline its malaria prevention program to provide maximum health impact among target groups in selected areas. The strategy will include a consolidation of its current malaria bednet social marketing program, a wider dissemination of PSI‘s array of promotional materials, and the introduction of a pre-packaged malaria treatment (ACT) via the commercial sector. All elements of this malaria strategy will be directly coordinated with the MOH, the PNLP and PISAF so as to be complementary to the significant levels of donor funding for malaria expected in the next several years. Consolidated Net Strategy: PSI will phase out SuperMoustiquaire Famille Protegee due to its small market share and the estimated drop in demand for this product with the impending arrival of long-lasting insecticide treated nets through the WB Booster program. PSI will also phase out the Alafia re-treatment kit due to the consistently low levels of re-impregnation and in favor of the higher health impact LLIN, which does not require re-treatment. PSI will continue its successful ante-natal clinic (ANC) model for the promotion and distribution of Bonne Maman Longue Durée in the Zou/Collines and the Donga, and proposes to expand this strategy to include the Atacora department. The Atacora has been chosen in consultation with the MOH based on malaria incidence, low levels of awareness and net use, and historically low levels of donor support for malaria prevention activities in this region. Bonne Maman nets will continue to be sold at the subsidized price of 500 FCFA to pregnant women and children under five through public health facilities in target departments. The strategy will be closely coordinated with the MOH and USAID to ensure adequate commodity supplies for these target departments, and with PISAF so as to complement its malaria prevention activities in the Zou/Collines. PSI will continue to distribute and promote SuperMoustiquaire LLIN via the commercial sector at cost recovery prices to assure market sustainability and permit use of PSI‘s revolving fund for commodity purchases. Distribution will be expanded through the development of a private wholesaler network able to buy large quantities of the product for distribution to the urban 172 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES commercial sector. This activity will be highly complementary to the activities of the MOH and the World Bank Booster program, which will focus primarily on distribution of nets to vulnerable populations and those in rural areas who cannot afford commercial sector prices. Sales of Super Moustiquaire Bonne Maman Longue Durée in Zou/Collines, Atacora/Donga are estimated to reach a total of 565,000 units for the life of the project while Super Moustiquaire Longue Durée are expected to reach sales of 135,000 units. Promotional Activities: Interpersonal communication activities for the four target regions (Zou, Collines, Atacora, Donga) will be expanded to include mobile video units that will support the mass media and clinic-based activities targeting pregnant women and children under five. Since the target groups of this ANC malaria prevention strategy are the same as those in the child survival strategy, IPC activities will integrate malaria prevention messages with domestic hygiene and sanitation messages for prevention of diarrheal disease. PSI will also make all of its Bonne Maman brand and related communications materials available to the MOH/PNLP for use in the WB Malaria Booster Program targeting pregnant women and children under five on a national scale. At the same time, PSI will continue mass media campaigns promoting SuperMoustiquire LLIN in urban areas via TV and radio. New Product Strategy—PPT: According to the 2001 DHS, only 19% of Beninese children who suffer from a presumed malaria-related fever are taken to a medical center for consultation within 48 hours; and over 60% of all malaria cases are treated at home, the majority of which is treated with Fansidar, Chloroquine or Flavoquine. With home-based management of malaria increasingly common and the need to avoid creating drug-resistant strains of malaria among the general population, PSI proposes to introduce a pre-packaged malaria treatment that is easily administered by child caregivers and effective in providing superior treatment without creating drug resistance among mosquitoes. Working in close collaboration with the MOH, PSI will introduce this pre-packaged malaria treatment and ensure its inclusion on the essential medicine list within Year 2. The product will include specific low-literacy dosage instructions to permit safe and accurate home treatment of malaria for adults and children. Consumer pricing of the product will depend on commodity supply. With donor support for commodities, the product would be subsidized to assure its accessibility to the most vulnerable populations. Without donor support for commodities, PSI would price the product at commodity cost recovery with donor funding only required for promotion and operation costs, as have been included in the proposed program budget. Efforts will be made to price PPT similarly to the public sector‘s Coartem® malaria treatment. The new PPT product will be distributed through commercial pharmacies, ProFam clinics and the public sector CAME. PSI will ensure consistent product availability through visits to all private￾sector vendors and close supervision of the supply chain, ensuring the product‘s proximity to people who normally treat malaria at home. Promotion will initially be conducted through activities targeting health care professionals, including the incorporation of product information into the family planning trainings conducted in medical schools, with pharmacists, with midwives and during ProFam trainings. Following its introduction into the private and public distribution systems, PSI will develop mass media campaigns to promote PPT to the general public. Interpersonal communication tools will also be developed to facilitate the inclusion of PPT information into behavior change communications conducted with all target groups. PSI estimates overall demand for this product based on the number of malaria cases per year and an estimate of the number of these cases potentially covered by home based treatment. Based on 2004 MOH statistics, 945,788 persons were treated for malaria. Assuming that all the health system manages to treat 20% of these cases with ACT, this would amount to 189,157 treatments. Assuming further that the private sector handles an estimate 20% of these cases, USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 173 this would result in 37,831 treatments. We estimate that PSI‘s social marketing program could capture 25% of these cases in the first year. PSI‘s product sales would begin in the middle of year two of IMPACTS due to estimated time needed to register the product in-country. Thus, the first year of sales would mean a total demand of roughly 10,000 PPT doses. With increased promotional and educational activities during the subsequent years, we estimate demand and coverage rates increasing with sales of PPT expected to reach 236,280 doses by the end of the project. Performance Indicators—Refer to section 4. Research, Monitoring and Evaluation Plan for a list of indicators related to objective one. Objective 2: Increase Knowledge of Beninese about HIV/AIDS, RH/FP and CS to Increase Demand for and Use of Products and Services Overview PSI recognizes that simply making affordable health products available is not enough – sustained behavior change and product use are the true keys to sustained health impact. Strong behavior change communications support all of PSI‘s socially-marketed products and services to help empower target populations with the knowledge and capacities necessary for the adoption of healthy behaviors. All communication tools, from interpersonal flip charts and brochures to billboards and radio shows, are adapted to the needs of each target group, and channels for the diffusion of messages are selected to bring the messages as close as possible to the intended beneficiaries. Using a mix of generic and branded communications, PSI selects the most appropriate and effective means of communication for each specific target group; for example, messages in rural areas are shared through community radio stations and community-based interpersonal communication activities; television spots reach women of reproductive age in urban areas; billboards along major transport routes inform high-risk truckers about HIV/AIDS prevention options while our "Soyons Patients–Restons cool" bracelet support youth in delaying their sexual debut. Regular reach and recall surveys help to ensure that selected channels are the most efficient means of communication, and provide opportunities to better refine message dissemination. By partnering directly with local NGOs, PSI‘s behavior change communications and distribution are decentralized across the country through community-based channels. PSI trains and follows up regularly with its partner NGOs, providing them with behavior change tools that are used with target groups and other local, traditional and religious organizations in their communities. This also provides an opportunity to help build the NGOs‘ capacity to develop and implement their own BCC activities. Successes to Date—Over the last 16 years, PSI has constantly improved its understanding of the barriers, as well as positive influences, that affect the use of socially-marketed products and services and the adoption of healthy behaviors. The use of regular local research to guide behavior change and marketing decisions helps to ensure that PSI‘s strategies are continually updated, evaluated and refined based on concrete evidence. PSI has successfully developed communications strategies to target hard-to-reach groups like prostitutes, long-distance truckers and youth. For example, PSI has developed peer-education strategies with older and experienced prostitutes to reach younger and clandestine prostitutes in order to provide them with important information about protecting their health, while a supportive bracelet and hip/hop/rap songs were developed to encourage youth to delay their sexual debut. All of these strategies were implemented in collaboration with the CNLS, PNLS and local NGOs. 174 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Proposed Strategies—To increase the demand for and use of products and services proposed in Objective 1, PSI proposes to reorganize its communication initiatives to increase the integration of messages across projects, health themes, and communication channels for maximum health impact. PSI will increase the scope and frequency of activities for greater impact and will introduce new activities that ensure a better balance of outreach in urban and rural areas, including the systematic translation of all communication materials into 6 local languages. PSI will focus its communication activities in the priority cities of Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouake, and Malanville, which have the greatest concentration of persons at high risk of HIV/AIDS infection as well as some of the lowest contraceptive prevalence rates in the country. In addition to this community-based communication strategy, PSI will continue to reach additional members of all target groups through mass media communications on national and private radio and television to assure maximum health impact among target populations. All of these forces will be combined to reach a "tipping point" with key messages for each health area covered by IMPACTS to encourage a shift in social norms among target groups. Three primary innovations in the delivery of behavior change communications are proposed as part of this program: 1. Partnerships with local communities via the establishment of a local family health advisory committee in the seven high-risk selected sites; 2. The training and use of local, community-based NGOs to conduct integrated health communication activities with various target groups, including high-risk groups. This strategy will be supported by the expansion of mobile video projections that integrate messages on HIV/AIDS, family planning and child survival for diffusion in urban and rural areas outside Cotonou; 3. The training and use of community radio stations as family health promoters for all health areas of IMPACTS in the seven selected sites. To ensure sustainability, PSI will develop a multi-level monitoring and supervision strategy that will be implemented both by PSI and by its local strategic partner OSV-Jordan. OSV-Jordan is a Benin-based non-governmental organization (NGO) with more than ten years of experience in the fields of reproductive health and HIV/AIDS prevention. OSV-Jordan has been working with PSI since 1998 as a partner in the PSAMAO HIV/AIDS prevention program working with key high-risk groups such as commercial sex workers and long-distance truckers. As the HIV/AIDS representative of the National Network of Local Health NGOs, OSV-Jordan will be responsible for supervising both the work of the local family health advisory committees at the institutional level and the local NGOs in each selected site at the operational level. PSI will dedicate a full￾time middle manager with a team of 8 IEC/IPC agents to oversee and reinforce all committees and NGOs activities. The map below presents the seven selected sites and the community-based levels of intervention, in each site, covering high-risk areas across the country. The strategy is developed to complement (and not replicate) CORRIDOR strategies and other partners, including PSI‘s, HIV/AIDS prevention activities supported by KfW and USAID‘s PISAF project. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 175 Figure I-1. Community Based Intervention Strategy Partnerships with Local Communities—Community family health advisory committees will be established that are composed of representatives from the public education sector, the mayor‘s office, the local MOH department (as the representative of the COGEA), PLWHA associations, Youth National AIDS Network, women‘s groups, local community radio stations and the local NGO contracted by PSI to conduct behavior change activities in that city. Each committee will be set up with the collaboration of OSV-Jordan, the HIV/AIDS representative of the National Network of Local Health NGOs. These committees will be instrumental in assuring that the planned activities respond to the local needs of the target groups in each city. Members will meet once every quarter. Partnerships with Local NGOs—Building on its active partnerships with NGOs in four Beninese cities, PSI will establish additional partnerships with local NGOs capable of implementing behavior change activities in the target cities. Local NGOs will be selected based on past performance criteria, their expertise related to the health areas addressed by IMPACTS and their level of involvement in the community. In collaboration with OSV-Jordan, which will be responsible for the recruitment of the local NGOs, PSI will train three members of each NGO. 176 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES An increased decentralization of community-based interventions to local NGOs throughout the country will reinforce local capacity in health social marketing activities while increasing the sustainability of IMPACTS‘s health impact. PSI will train three members of each NGO in HIV/AIDS behavior change communications, communication techniques appropriate for each target group, monitoring and supervision, and budget management. Trainings and training materials will always include a cross-cutting gender approach. As part of PSI‘s proposed integrated community-based strategy, local NGOs will be responsible for conducting activities with target groups for each health area in each of their assigned cities. They will also participate in family planning and child survival outreach and MVU projections held regularly in each city or surrounding areas. PSI and OSV-Jordan will work with each NGO to develop a budget and action plan which will be finalized in conjunction with the health advisory committees. The commitment of the local partners will be assured through their contribution of 10-15% of budget cost. To support the interventions of local NGOs in each city, PSI will expand its popular mobile video projections geographically and thematically to expose an increased number of persons to integrated health messages covering HIV/AIDS, family planning and child survival topics. PSI‘s communication staff will conduct one projection per month in each target city, with three projections per month in Cotonou, in partnership with the local NGO intervening in each city. Each event will be dedicated to a specific theme, which will be coordinated with the community radios stations and mass media campaigns. All projections will be broadcast in the local language to assure maximum exposure. To ensure the sustainability of its collaboration with local NGOs, PSI plans to work in close partnership with OSV-Jordan, the NGO responsible for the STI/HIV/AIDS sector within the Network of Beninese Health NGOs (ROBS). In the context of IMPACTS, OSV-Jordan will be actively involved in developing and coordinating the seven Technical Health Committees, as well as selecting and supervising the work of the local NGOs in the seven selected cities for the project. OSV-Jordan will be heavily involved in planning and implementing all BCC and social marketing activities, which will strengthen its administrative and operational capabilities in BCC and social marketing. Through this strategic role, OSV-Jordan will transfer skills to all other member NGOs of this network, directly contributing to the sustainability of IMPACTS results. Partnering with Local Community Radios—In collaboration with the Federation of Community Radios and Associations of Benin (FERCAB) and the House of Media, a local NGO bridging all actors in the radio and television industry in Benin together, PSI will expand its radio coverage countrywide by working directly with 13 community radio stations covering the seven high-risk cities targeted by the HIV/AIDS prevention program, the 2 pilot integration sites and the expanded ProFam network (Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouake and Malanville). PSI will train on-air presenters and managers of community radio stations on key family planning, HIV/AIDS prevention and child survival issues. PSI will also train the radio station animators on the use of communication briefs, which will be developed by PSI and will address key messages on the same issues. The briefs will be updated on a quarterly basis to address a variety of themes related to these topics and to provide new material to the station animators. To ensure sustainability of the community radio program, PSI/Benin will train at least three people from each radio station, including one administrator (or if possible the owner of the station). The PSI Team will provide administrators with capacity to train newly-hired journalists when there is staff turnover. Such training will not only serve IMPACTS but also PISAF and all organizations wanting to use radio to promote healthy behavior in Benin. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 177 The PSI Team will strengthen the capacity of at least eight local NGOs, including one in the seven selected sites plus one for the peer education program with custom agents, and 13 community radios. The experience these local organizations gain through IMPACTS programs will endure beyond the duration of the project, allowing them to build upon their experience to pursue similar activities and to contribute to the sustainability and effectiveness of project activities. Training Media—The PSI Team strategy for community radio includes collaboration with l‘Association de la Maison de Medias (MdM) to train journalists to use communication briefs on various health themes in order to conduct health-related radio shows. In Benin, the Association of the MdM has the following objectives: Promote camaraderie amongst media professionals; Promote freedom of the press, pluralism and independence of the media; Encourage communication between the public and the press, serving as support structure for training and reinforcement of media professionals; Strengthen the production capacity of the media; Promote information exchange and exchanges between journalists and other communication professionals; and Create free space for media professionals. The trainings will be conducted by the association of the MdM with technical support from the International council of French-speaking Radios and Televisions (CIRTEF – Conseil International des Radios-Televisions d‘Expression Française). Since the MdM is relatively young (June 2005), CIRTEF will provide technical assistance for the trainings, particularly in using audio radio equipment. In Cotonou, CIRTEF has a functional and well-equipped audio and video post production centre, as well as training facilities. Mr. Amégnihoué HOUNDJI, responsible for training at the MdM, and Mr. Alain Kampinaire, responsible for Audio Training at the CIRTEF, will collaborate with the PSI Communications Department to develop training plans for 13 community radio journalists targeted by the project. Trainings will be conducted in French, but trainers will also strengthen the intervention capacity of the journalists in local languages, if necessary. Building on a successful strategy and tools developed by PSI globally, IMPACTS staff will monitor the press and look for articles that present incorrect or misleading information on the technical areas addressed under IMPACTS. PSI has developed training packets, press clippings, and letter to the editor templates for use in educating the media on HIV, FP, and other subjects. PSI has employed these in a rapid reaction fashion to respond to erroneous stories in the press. For example, an article on the use of family planning products might propagate myths such as oral contraceptives make one sterile, cause tumors, etc. In such a case IMPACTS would rapidly contact the reporter in question and supplying him or her with correct information, asking them to produce a second article dispelling myths and/or printing a letter to the editor from IMPACTS. Any effort of this type would be closely coordinated with USAID and government partners. Monitoring and Supervision PSI will train OSV-Jordan in order to reinforce its capacity to monitor, supervise and report on activities of all sub-grantees. PSI and OSV-Jordan will agree on a budget to cover the regular supervision and monitoring of activities of the local NGOs as well as costs associated with routine reporting of performance indicators to PSI. OSV-Jordan will also be involved in all research activities conducted with target groups in these cities. 178 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Targeted Messages PSI tailors communication messages and activities to specific target groups based on health statistics, epidemiological information, and evidence provided by regular TRAC studies (explained in the M&E section). HIV/AIDS prevention messages target higher-risk bridging groups such as prostitutes, their partners, long-distance truck drivers and youth. Pregnant women and children under five are targeted for all child survival interventions and women and men of reproductive age, specifically those 15 – 39 years, are targeted for reproductive health and family planning messages. One example of a successful, targeted campaign is the ―Delayed Debut‖ campaign launched in 2005 to encourage youth 13– 8 years to delay their first sexual relation. This campaign combined mass media television and radio spots, billboards and posters with other innovative strategies such as the production of a campaign song and music video and the social marketing of a ―Plus Tard Plus Sur!‖ campaign bracelet to increase the overall impact of its behavior change communications and activities. The following section describes the various targeted communications strategies by health area, building upon PSI‘s experience in Benin and in other countries around the region. HIV/AIDS/STIs Given the prevalence rate in Benin of 2%, indicating a concentrated epidemic, PSI will continue to prioritize activities toward high-risk groups, including prostitutes, their clients, truckers and persons in uniform. Given the sexual behavior of many youth (21.9% of youth 15-19 years and 44.4% of youth 20 – 24 years reported having had sex in the four weeks prior to the 2001 DHS and project baseline data from 2004), PSI will pay special attention to the vulnerable population of youth ages 15 – 24 years, including those in and out-of-school. PSI will continue its balanced approach to HIV/AIDS prevention that encourages abstinence, mutual faithfulness between concordant-negative couples and condom use for all at-risk sexual relations, including couples where one or both partners‘ status is unknown (the ABC model). Our approach will also emphasize testing and gender components, as socioeconomic inequalities usually leave women most affected by the epidemic. Successes and Opportunities Youth: Since 2000, PSI has promoted the ABC approach to youth via its Amour et Vie youth magazine, and in 2004 complemented this activity with the launch of the Amour et Vie weekly call-in radio program to encourage in and out-of-school youth to make responsible decisions regarding their sexual health. Amour et Vie reaches tens of thousands of Beninese youth around the country with messages centered on themes directly relevant to their lives such as negotiating the delay of sexual relations with one‘s partner, the dangers of teen pregnancy and STIs, negotiating condom use, and discussing the future in a romantic relationship. In 2005, 174,755 copies of the Amour et Vie magazine were distributed and 42 live radio shows were aired over the national broadcast station ORTB. The peer education program entitled ―Projet Panther‖, conducted in collaboration with the US Peace Corps, provides similar messages directly to out-of-school youth in targeted communities around the country. PSI has produced many high-profile generic media campaigns to address key HIV/AIDS/STI prevention themes including the ―Delayed Debut‖ campaign encouraging youth 13 – 18 years to delay their first sexual relation, ―Finishing My Education‖ which addresses the cross-generational sex phenomena of older men having relationships with younger girls, and ―Trusted Partner‖ encouraging youth to protect themselves consistently and to know their HIV status. In addition to using television and radio spots, billboards and posters to expose target groups to these messages, PSI developed innovative campaign-specific communication tools such as the ―Plus Tard Plus Sur‖ bracelet, music videos, and theater skit competitions on specific health themes. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 179 High risk groups: PSI has established strong partnerships with four local NGOs which were trained on how to conduct interpersonal communication activities with prostitutes and truckers. Working with these NGOs, PSI-supported activities in 2005 alone reached 176,488 people, including 18,495 sex workers, 88,759 long-distance truck drivers and 69,234 inter-urban taxi and motorcycle drivers10, while simultaneously reinforcing the capacity of local NGOs to conduct targeted HIV/AIDS prevention programs. PSI also has developed innovative on-the-job peer￾education strategies to reach young and often clandestine prostitutes, and has adapted these strategies to target Anglophone prostitutes from Nigeria under the campaign "No Condom – No Sex". PSI, with additional funding from the US Department of Defense, also conducts HIV/AIDS prevention activities with the men and women of the armed services, including training of military health personnel on HIV rapid testing techniques, the development of HIV/AIDS/STI training curricula targeting military, and monitoring and evaluation of HIV/AIDS/STI activities in the military. Proposed Strategies PSI proposes to tailor its key messages to address a balance approach of "A", "B", "C" and "D" messages using USAID ABC guidance, to which we add a ‗D‘ component that stands for "do get tested‖, along with specific efforts to combat the stigma surrounding HIV and getting tested. Abstinence: Abstaining from sexual relations as long as possible or returning to abstinence is the key message that will be communicated during activities with Beninese youth 15–19 years. "A" messages will combine HIV/AIDS/STI and FP messages since abstinence remains the most efficient way to avoid STI/HIV/AIDS and unintended pregnancy. Be Faithful: Remaining faithful to your monogamous partner is a key message that will be addressed to sexually-active youth 15 – 24 years, long-distance truckers, military and the sexually active population 15 – 49 years. Since marriage or partnership can be a risk factor when one or both partners‘ HIV status is unknown, ‗B‘ messages will be coupled with messages promoting the importance of testing and of knowing one‘s sero-status. ―B‖ messages will also combine HIV/AIDS/STI and FP messages since mutual monogamy coupled with use of an FP method can be an effective form of dual protection for preventing HIV and unintended pregnancy. Correct and Consistent Condom Use: Messages promoting the male condom as a dual protection product, reducing stigmas associated with condom use, and promoting correct and consistent use, including reinforcing the individual‘s belief that he/she is capable of correctly using a condom. These messages will target all prostitutes, long-distance truckers, military, and the sexually-active population 15 – 49 years with a particular focus on youth 15 – 24 years. ProFam clinics (and particularly the future youth-friendly ProFam clinics) and all community-based FP activities that will be held in each target city will also offer important venues to promote the correct and consistent use of condoms. Do Get Tested: The importance of knowing one‘s STI and HIV status will be promoted among all target groups, along with information on public testing sites (particularly CIPECs and IMPACTS) and private sector clinics (particularly ProFam and PSAMAO). Interpersonal and mass media communications campaigns will seek to convince all men and women of reproductive age that being tested is critical for their own health and the health of their families and advertise the availability of these services at PSAMAO and ProFam clinics and the integrated pilot sites. Building on a successful PSI experience in East Africa, we will promote the idea that knowing one‘s HIV status can provide great relief for the overwhelming majority who will indeed test 10 These figures likely include individuals who were exposed to messages on multiple occasions 180 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES negative. Separate communication campaigns on HIV/AIDS prevention targeting high risk groups such as truckers, their assistants, moto taxi drivers, and customs agents, nearly all of whom are men, will also promote the benefits of getting tested. This initiative may also include mobile sites co-funded by KfW. PSI will also train local NGOs and community radio stations to integrate STI and HIV prevention and family planning messages in all activities. PSI will integrate STI/HIV/AIDS prevention into the promotion of family planning products, promoting condoms as both a method of FP and STI/HIV prevention. PSI will also incorporate STI prevention methods into HIV and family planning mass media campaigns. Combating Stigma: Incorporating strategies to reduce stigma and discrimination will be an integral component of the PSI team‘s efforts under IMPACTS. Fear of stigma and discrimination can prevent people from learning their HIV status and, if they are HIV positive, from disclosing their status to others and accessing health services. In addition, stigma can place a significant psychological burden on an infected individual, thereby contributing to a decline in their health status. Stigmatizing attitudes and discriminatory practices from family and community members can prevent those infected from receiving adequate treatment and support. The PSI Team will thus develop and implement an anti-stigma communication campaign targeted at the general public to address individual fears and dispel myths surrounding HIV/AIDS. The PSI team will also work closely with community groups of persons living with or affected by HIV/AIDS not only to develop these campaigns, but also to provide support options to women who test positive. Such community groups will also be represented on the community advisory groups for the pilot sites, and the ProFam advisory council will seek to add a member who represents people living with or affected by HIV/AIDS. These efforts will be designed to help communities surrounding the integrated sites and ProFam clinics understand HIV/AIDS and overcome stigma associated with HIV. Specific Strategies for Youth Amour et Vie: PSI will reinforce its highly popular Amour et Vie youth activities by increasing direct youth participation in the development of topics and the production of both the magazine and the radio program. This will be achieved through the involvement of journalism and graphic design students as project interns, providing them with valuable professional experience. PSI will also work with the public education sector and USAID‘s health and education projects, such as the Project ABE-LINK, to promote the Amour et Vie magazine as an HIV/AIDS/STI interpersonal communication tool available to schools. PSI will also introduce a new ―Family Health Page,‖ included in each Amour et Vie magazine that will provide information on family health issues including hygiene, diarrhea and malaria prevention and gender issues, thereby broadening the scope of Amour et Vie beyond its traditional focus on HIV/AIDS/STI issues. PSI will also franchise the radio show to be produced by the community radio stations in local languages using briefs produced for each show. The Amour et Vie youth clubs will schedule group listening and discussion sessions based on the weekly radio shows. Depending on the success of this model, PSI will explore the possibility of creating an Amour et Vie television show beginning in Year 3 of the project. Projet Panther: In collaboration with the U.S. Peace Corps, PSI will provide all incoming volunteers in Benin with an HIV/AIDS toolkit to reinforce the HIV/AIDS training that PSI has traditionally conducted for each new group of volunteers. These kits will equip volunteers with materials to address HIV/AIDS/STI prevention topics within their communities regardless of their formalized participation in Projet Panther. PSI has also agreed with Peace Corps leadership to increase the number of volunteers working with Projet Panther from the current 9 to 15 in fiscal year 2007. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 181 Specific Strategies for High Risk Groups Interpersonal communication activities targeting high-risk groups will be conducted by the local partner NGO in each of the seven identified priority cities, with the support of PSI behavior change agents and the supervision of OSV-Jordan. Prostitutes will be targeted through the expansion of a pilot activity that trains prostitutes to conduct behavior change communications with other prostitutes. Unlike a strict peer-to-peer program, the animator will be older than the women with whom she conducts the training, reducing the risk of jealousy and increasing the acceptance of the messages she communicates among the target group. Truckers will be reached through IPC activities conducted in the major parking lots located in each city. These will be complemented by billboard campaigns along major roads and secondary routes addressing stigma, risk perception, partner reduction and correct and consistent condom use. Uniformed military personnel will be targeted through FY07 by PSI‘s intervention supported by the U.S. Department of Defense. As part of IMPACTS, PSI will partner with the Ministry of Development, Economy, and Finance (MDEF) and its HIV/AIDS Focal Unit to establish an HIV/AIDS prevention program targeting customs officials based at border crossings between Benin and neighboring countries through peer-based interpersonal communication activities. Away from their families for long periods of time and posted in high-risk locations along national borders, these agents are at a higher risk for HIV/AIDS exposure. Reproductive Health and Family Planning Given the imbalanced situation in Benin where the desire to space births is relatively high but contraceptive use is much lower, PSI proposes generic campaigns targeted at women and their male partners to increase awareness of the benefits of contraception, with an increased focus on rural areas. In select urban areas, women will be directed to ProFam clinics through mass-media and IPC campaigns in an effort to increase uptake of RH/FP services. Successes and Opportunities For years, PSI has used generic mass media communications coupled with interpersonal communications around service centers to reach sexually-active women aged 15 – 49 years with family planning messages. As such, communications have not effectively reached secondary urban and rural areas due to language barriers and the limited outreach of national television and radio channels. Thus far only media campaigns promoting Methode du Collier have included men as a target group. Proposed Strategies Based on results from PSI‘s latest KAP study, which identified various behavioral determinants linked to the use of modern family planning methods, PSI behavior change communications will target women of reproductive age as well as their male partners with the following key messages: Advantages of birth spacing, specifically the health benefits to the child and mother Social acceptance and support for family planning Correct use of available contraceptives for birth spacing Importance of male partner involvement in family planning decision-making PSI will increase message exposure in rural areas through community radio stations (particularly those covering the 7 selected sites) based on the family planning briefs provided quarterly to the family health promoters. Discussion of gender issues, including the importance of male participation in family planning issues, violence toward women, early marriage, widow inheritance, and polygamy will be covered in briefs providing an opportunity to introduce these 182 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES issues in the public forum. Issues presented by the community radio stations will be reinforced in an annual theatrical radio series produced and broadcast during the five-year program. Translated into the six local languages (Fon, Adja, Dendi, Bariba, Yoruba, and Wama), the six episode series will be broadcast over community, national and private radio channels, twice a year. In urban areas, PSI will continue to use mass media and targeted outreach from the ProFam centers to efficiently share key behavior change communications with target groups. PSI will develop and produce mass media television and radio spots targeting younger, sexually active women 15–29 years and their male partners that explain birth spacing methods available in Benin. The generic messages will be reproduced in billboards placed in proximity of ProFam centers and high-traffic areas frequented by target groups. Posters linked to the media spots will be distributed in ProFam centers, the integrated service pilot sites, PASMAO clinics and through public health centers, specifically those supported by PISAF. Interpersonal communication activities will continue to radiate from the ProFam service centers as well as in neighborhoods not targeted by ProFam through interventions held in market areas and other locations with a high concentration of targeted women. ―Health Evenings‖ conducted in proximity to each ProFam clinic will reach out to women and men using mobile video presentations to attract participants, who will benefit from information on all family health topics and the opportunity to ask specific questions of qualified health personnel who intervene during the activities. The new toll-free hotline will also offer opportunities for women and men to ask questions about family planning and to receive correct information about the various methods available. Child Survival and Malaria Malaria and dehydration due to diarrheal disease are two of the leading causes of infant and child mortality. Yet the burden of both of these health issues can be dramatically reduced via rather simple and inexpensive behaviors such as regular hand washing and the use of a treated bednet. To address these health issues, encourage healthy behaviors, and increase knowledge and use of available child survival products, PSI will conduct generic campaigns on issues such as personal hygiene, the timely treatment of diarrhea and malaria, and the importance of using bednets. Successes and Opportunities PSI‘s mobile video units, which have been used primarily for HIV/AIDS/STI prevention activities, present an excellent opportunity to convey child survival messages on issues such as malaria and diarrheal disease prevention. Since these messages are directed at the same target groups (pregnant women and children under five), there is an opportunity to systematically integrate malaria prevention messages with diarrheal disease prevention messages, increasing the health impact of each show. PSI‘s branded communications for Orasel oral rehydration solution and SuperMoustiquaire treated bednets have been successful in raising awareness about the availability of these products and their importance in preventing illness; however, the campaigns have been conducted primarily in urban areas, (with the notable exception of SuperMoustiquaire Bonne Maman, which targets populations in Zou, Collines, and Donga), while the greatest burden of these diseases falls on the rural areas. Proposed Strategies PSI proposes to increase child survival through the diffusion of key messages on hand-washing, domestic hygiene, timely treatment of diarrhea, and the efficiency, ease of use and availability of LLINs. Pending the introduction of PPT, PSI will also diffuse messages on the timely treatment of malaria and fevers. Messages will be developed specifically to target child caretakers. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 183 In rural areas, key messages will be delivered primarily through community radio stations (particularly those covering the 7 selected sites) using targeted communication briefs as well as through interpersonal communication activities implemented by NGO partners with technical support from PSI communication agents. PSI proposes to continue targeted anti-malaria behavior change communications focused heavily on promoting net use and timely use of PPT among pregnant women and children under five reached through its health-center based distribution (ANC model) in Zou, Collines, Atacora and Donga. In urban areas, PSI will use national and private mass media channels including radio and television to broadcast key child survival messages. Activities in both urban and rural areas will be complemented by messages delivered through mobile video presentations, the Amour et Vie ―Family Health Page‖, and behavior change activities conducted by NGOs in the seven target cities. Service Communications To respond to a need for increased information about available HIV/AIDS-related health services in Benin, PSI will form a working group with members from the PNLS, CNLS, UNAIDS, UNICEF and PISAF program to develop informational briefs on key service messages related to VCT, PMTCT, ART, TB and immunizations. The working group will also identify primary target groups for the messages, the most effective communication channels, and estimate the quantity of materials needed for each. Based on these briefs, PSI will collaborate with a local advertising/graphic design agency to develop service communication materials for the identified communication channels and target groups. Materials will be as visual and/or audio as possible to permit comprehension by low￾literacy populations and will be translated into the six primary national languages. Diffusion of these materials will be conducted by the MOH, PISAF and PSI and will include both public and private health facilities as well as distribution through community associations and NGOs active in the domain of HIV/AIDS prevention. While the final service communication materials will be decided upon by the working group, PSI anticipates the following materials and distribution channels will be used: 1. VCT Service Communications—Community radio briefs produced for use by the network of community radio stations. Posters and brochures for ProFam clinics, PISAF￾supported centers, public health centers, integrated pilot sites, PSAMAO clinics, and Prudence Plus kiosks in high-risk parking lots. 2. PMTCT—Community radio briefs produced for use by the network of community radio stations. Poster and brochure for ProFam clinics, PISAF-supported centers, public health centers, integrated pilot sites, PSAMAO clinics, and health centers with targeted ITN distribution. 3. ART—Community radio briefs produced for use by the network of community radio stations. Poster and brochure addressing misconceptions for distribution in PISAF sites, public health centers, integrated pilot sites, PSAMAO clinics and Prudence Plus kiosks. A module on ART for use in interpersonal communication activities will be developed and made available to all organizations conducting HIV/AIDS prevention materials. 4. TB—Community radio briefs produced for use by the network of community radio stations. Poster and brochure for ProFam clinics, PISAF-supported centers, public health centers, integrated pilot sites, PSAMAO clinics, and health centers with targeted ITN distribution. 5. Immunization—Community radio briefs produced for use by the network of community radio stations. Poster and brochure for ProFam clinics, PISAF-supported centers, public 184 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES health centers, integrated pilot sites, PSAMAO clinics, and health centers with targeted ITN distribution. Messages encouraging child caregivers to get their children immunized will be added to all Orasel and Bonne Maman promo and IEC materials as well as interpersonal communication activities with women targeted by the ANC ITN distribution model. In addition to the above communications, PSI will propose to the working group the development of a brochure targeting PLWHA that would include valuable information on key services that may be of particular interest to them including VCT, PMTCT, ART, TB and hygiene. It will be distributed through all VCT centers, integrated pilot sites, PISAF centers, PSAMAO clinics and the mobile VCT center, public sector health facilities and Prudence Kiosks. PSI will also collaborate with the working group to distribute the brochure to all interested NGOs including all members of the Beninese Network of PLWHA Associations. Performance Indicators—Refer to section 4. Research, Monitoring and Evaluation Plan for a list of indicators related to objective two. Objective 3: Promote and Support Implementation of Best Practice Integrated Services Overview Benin has moved ahead of many other West African countries to introduce and scale up public￾sector PMTCT, VCT, STI treatment, and FP services. However, the availability of these remains inadequate. Services are offered at different sites, requiring that clients travel long distances to meet all of their health needs. Furthermore, few people know of the existence of many of these different services (such as VCT or PMTCT). According to the 2001 DHS, 63.5% of women, and 62.1% of men in Benin would like to be tested for HIV. However, only 5.1% of women and 6.8% of men have actually been tested, not only because of lack of services available or knowledge of their existence, but also because they fear the discrimination and the stigma that still surround HIV infection. These issues of fear, discrimination and lack of knowledge create significant barriers to accessing needed HIV, STI and family planning services. These issues particularly block service access to Beninese youth who represent a large proportion of the Benin population, are at increased risk for HIV infection, and who also demonstrate a significant unmet need for family planning services. The development of integrated service delivery points providing quality and confidential care can help overcome many of these obstacles. Evidence shows that a ―one-stop shop‖ approach (as opposed to multiple service delivery points) is preferable to clients because it provides all care under one roof. In this regard, a patient can go to one location for all his or her needs, and consequently providers can capture more people and orient them to other health care services that they perhaps were not intending to use. For example, a provider might persuade a VCT client to discuss family planning, or a pregnant woman seeking ante-natal care can easily be referred to comprehensive HIV testing and counseling, and PMTCT services if warranted. Providing family planning services to an HIV-positive woman who does not want to get pregnant is the best way to prevent MTCT and is an essential component to both VCT and FP service delivery. Youth looking for STI treatment can be readily introduced to HIV testing and prevention services, as well as family planning counseling. Offering a full range of services at one site also reduces the fear of stigma that some clients might feel if they were entering an HIV￾only service provider. In particular, it has been shown that integrated services are the most effective means for serving adolescents as they benefit the whole individual, taking the reality of all aspects of their lives into account and not just one set of needs in isolation. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 185 Proposed Strategy The PSI Team‘s strategy and activities under this objective will be led by JHPIEGO, a global leader in clinical training and performance support for integrated services. The PSI Team will draw on JHPIEGO‘s experience in the establishment and scale-up of PMTCT, VCT, and HIV prevention and treatment services, care and support programs under the President‘s Emergency Plan for AIDS Relief (PEPFAR), as well as PSI‘s experience with improving quality service provision via its ProFam network of clinics. The overall strategy is centered upon the Standards-Based Management and Recognition (SBM￾R) approach for the integration of HIV/STI/FP/PMTCT services. SBM-R is a practical management approach for improving the performance, efficiency, and quality of health services, and is ideally suited to improving integrated service delivery because it uses a holistic approach to improving performance by addressing all aspects of quality beyond those factors, such as provider competency, that are traditionally thought to be the key determinants of quality. It consists of the systematic utilization of performance standards as the basis for the organization and functioning of services and rewarding of compliance with standards through recognition mechanisms. In comparison to other quality approaches, SBM-R focuses on specific content area rather than quality concepts or methodologies. Thus, SBM-R uses standards with specific verification criteria that not only prescribe what to do but how to do it. The emphasis is not only on evaluation for quality, but rather quality is incorporated as a regular function in the day-to￾day management of services. Similar in design to the Cercle d’Or Initiative implemented in family planning clinics under the Family Health and AIDS Project, JHPIEGO has applied SBM-R for PMTCT and VCT in Cote d‘Ivoire, has used SBM-R to standardize and improve care at 88 PMTCT sites in Ethiopia, and is currently using this approach among private-sector family planning clinics in Pakistan and VCT centers in Jamaica. Working in collaboration with the MOH and the PISAF project, the PSI Team will conduct a needs assessment of potential sites in the departments of Borgou and Zou/Collines to select the two most appropriate public sector sites for interventions. As the demand for quality of care and demand for services lies beyond the walls of health facilities, in the communities that they serve, the PSI Team, in coordination with the clinic leadership, will create community advisory committees which will be comprised of representatives of the COGEA, PISAF, local NGOs and community-based organizations, youth and PLWHA representatives, as well as members of the private health care sector, PLWHA, and youth. By having representatives from civil society – especially members who represent the most at-risk groups - on these advisory committees, the PSI Team will ensure each group‘s involvement in decision making, access, and increased participation in and use of the sites. Once the sites are chosen, the PSI Team will work with clinic staff to guide integration of new services. Given the resource constraints faced by the public sector, the PSI Team anticipates that there will be an initial need to train providers and strengthen the site with equipment and minor renovations. This initial training is crucial because those providers involved in developing clinical performance standards will need knowledge concerning the latest, evidenced-based standards for HIV/AIDS/STI, PMTCT and family planning services and on national policies, norms, and protocols to develop standards. The following key themes will guide activities at the proposed project sites: Increasing the Number of Men Seeking VCT—HIV testing rates are still low in Benin, particularly amongst men, and our communications campaigns will specifically encourage men to seek testing. First, PSI will undertake research in an effort to better understand the specific psycho-social barriers that prevent men from seeking testing. Interpersonal and mass media communications campaigns will be designed based on this research to address the specific 186 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES barriers to testing among men and to convince them that getting tested is critical for their own health and the health of their families. These campaigns will advertise the availability of these services at PSAMAO and ProFam clinics and the integrated pilot sites. Building on a successful PSI experience in East Africa, we will promote the idea that knowing one‘s HIV status can provide great relief for the overwhelming majority who will indeed test negative. Separate communication campaigns on HIV/AIDS prevention targeting high risk groups such as truckers, their assistants, moto-taxi drivers, and customs agents, nearly all of whom are men, will also promote the benefits of getting tested. VCT for Pregnant Women—Benin‘s family health code of was revised in early 2006 to require that all pre-natal clients receive HIV counseling and testing. The PSI team will work with providers to ensure that the revised code is implemented, and that all pregnant clients at the integrated pilot sites and ProFam clinics receive HIV counseling and testing. Rapid tests, which are now approved for use in Benin, will be used, eliminating the need for women to return for their results. Therefore, all clients will receive their results. This will allow providers to offer them the best information available about how to manage their status and possible avoidance of transmitting the virus to their infants during their first visit. Integration of PMTCT Services—Since those at high risk for HIV/AIDS are also likely to need STI counseling and treatment, the PSI Team will fully integrate STI services within HIV VCT services. PSI will ensure that both STI and HIV testing and treatment protocols are fully integrated into the design of the service provision plan for the pilot sites and that they are developed according to the standards of the Ministry of Health of Benin. A training module will specifically be developed and this training module will be used for all trainings for ProFam and PSAMAO clinics providers. To minimize disruption of services, innovative strategies such as structured on-the-job training will be done rather than traditional group-based courses which take providers away from patients they serve. PMTCT is a critical component of the integrated program, and the PSI Team regrets that this was not clear in the original proposal. The PSI Team aims to make PMTCT services available for all Beninese women of reproductive age; whether they are contemplating pregnancy, are already pregnant, or require post-partum care. The purpose of the initiative is to help enable all women to bear healthy infants. The PSI team‘s approach is based on the World Health Organization‘s comprehensive strategic approach for HIV prevention among infants, which is the best framework for integrated PMTCT services. The PSI team model will expand to include a strengthened platform for refocused antenatal care that addresses birth planning for all women and cutting edge interventions, such as intermittent presumptive treatment during pregnancy. The most recent evidence for effective PMTCT will be considered as related program initiatives are developed. The PSI Team will implement the following PMTCT program components in public integrated sites and ProFam clinics: Primary Prevention of New HIV Infections among Women and Their Partners: Counseling will be the foundation for HIV prevention among pregnant and non-pregnant women and their partners. This will include individual, couples and family counseling through introduction of VCT. Sessions will focus on the development of risk-reduction strategies for all clients, including ―prevention for positives‖ counseling for HIV positive individuals. VCT services are only effective when individuals are aware of services and able to access them. In addition to traditional demand-generating activities, JHPIEGO will encourage provider￾initiated VCT to ensure that individuals seeking other services, such as family-planning or STI treatment, are made aware of VCT and are encouraged to seek related services. Concomitantly, those seeking VCT services may also be referred for family planning counseling, antenatal care services, or STI screening. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 187 Prevention of Unintended Pregnancy among HIV-infected Women: Prevention of unintended pregnancy among HIV positive women has been documented as important in preventing new infections (The Futures Group, 2001 and Family Health International, 2004). Conversely, there is strong evidence from Rakai studies in Uganda (Gray et al. Lancet, 2005) that the risk for new HIV infections rises during pregnancy, presumably due to hormonal changes and changes in genital tract mucosa. Such evidence indicates a clear need for family planning counseling, with an emphasis on barrier methods, to help prevent unintended pregnancy among HIV positive women and VCT to help prevent new infections among uninfected pregnant women. Integrated PMTCT/STI/VCT services will help to ensure that providers do not miss opportunities to provide women with FP and HIV prevention services. The PSI Team will also help couples to make informed decisions about bearing children. Interventions will strengthen provider knowledge and capacity to counsel both HIV-infected and non-infected women and their partners about appropriate family planning methods and birth spacing, with a special emphasis on dual protection. Since many HIV positive women want to become pregnant, providers training will include the risk of transmission and appropriate interventions to decrease associated risk. Prevention of Mother-to-Child Transmission During Pregnancy: JHPIEGO will train providers in safe delivery practices, including avoiding invasive procedures such as premature rupture of membranes and episiotomy, and on use of ART or single-dose nevirapine in order to reduce the risk of transmission during antenatal care, postnatal care, labor and delivery. Other interventions will address prevention and treatment of malaria and anemia during pregnancy. The PSI team will also train women on the advantages of starting ART treatment during pregnancy and on delivery with the assistance of a trained health provider. When HIV status is unknown, the PSI team will work with providers to follow new health regulations that require HIV and STI testing for women receiving pre-natal care. Care and Support for HIV-infected Women and Their Infants: Follow-up care and support for HIV positive women and their children is essential. However, it is well-documented that many clients, both HIV positive and negative, do not return for postpartum care. The PSI team will establish procedures to encourage all women to return for postpartum care; including services such as safe infant feeding, family planning and cotrimoxizole prophylaxis. The project will also establish referral systems to link clinically￾eligible HIV positive women, infants and their families to ART, treatment of opportunistic infections and palliative care. To address the challenge of continuity of care for women and their families in the postpartum period, the PSI Team will build linkages with community and faith-based organizations, as well as PLWHA support groups. These groups may assist women and their families in obtaining housing, food and transportation. Combating Stigma—Incorporating strategies to reduce stigma and discrimination will be an integral component of the PSI team‘s efforts under IMPACTS. Fear of stigma and discrimination can prevent people from learning their HIV status and, if they are HIV positive, from disclosing their status to others and accessing health services. In addition, stigma can place a significant psychological burden on an infected individual, thereby contributing to a decline in their health status. Stigmatizing attitudes and discriminatory practices from family and community members can prevent those infected from receiving adequate treatment and support. The PSI Team will work with the clients themselves, their families, and the communities surrounding the proposed pilot integrated service delivery and ProFam clinics to address HIV stigma and discrimination on multiple levels. First, the PSI team will educate each client seeking services on the benefits of knowing their HIV status. The message will be that a positive result 188 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES does not mean that all is lost, that treatment and PMTCT services are available to arrest the virus‘ progression and transmission to unborn children. These messages will be delivered to all clients who visit health centers for reasons such as pre-natal visits, STIs, FP, or VCT. Counselors will also encourage those who test HIV positive to bring in their partners and families for counseling on the realities of having a loved one with HIV/AIDS and the services available to combat it. Through such efforts, stigma surrounding HIV will be incrementally reduced as more and more women, their partners, and families are educated about HIV/AIDS. FBOs and religious leaders, because of their significant influence in shaping cultural norms, will also be an important focus of our efforts to reduce stigma and discrimination. PSI will identify and reach out to influential religious leaders in the communities surrounding the proposed pilot integrated service delivery and ProFam clinics and educate them on the common myths surrounding AIDS and ask that they promote compassion for people living with the disease through their sermons, counseling, and day to day interactions with the members of their congregations. We will also develop and implement an anti-stigma communication campaign targeted at the general public to address individual fears and dispel myths surrounding HIV/AIDS. The PSI team will also work closely with community groups of persons living with or affected by HIV/AIDS not only to develop these campaigns, but also to provide support options to women who test positive. Such community groups will also be represented on the community advisory groups for the pilot sites, and the ProFam advisory council will seek to add a member who represents people living with or affected by HIV/AIDS. These efforts will be designed to help communities surrounding the integrated sites and ProFam clinics understand HIV/AIDS and overcome stigma associated with HIV. Increasing Men's Involvement in PMTCT Activities—The influence that men have on reproductive decision-making should not be overlooked. The PSI team will implement interventions targeting male partners and providers to encourage positive changes in attitudes related to PMTCT services. Performance standards for integrated care will reflect men‘s involvement in services. Examples of some of these interventions are outlined below. Encouraging male partners to accompany women to at least one ANC appointment. As men are more likely to control a partner‘s financial resources and access to transportation, providers will be encouraged to address a birth plan with the couple that ensures consistent access to care during the ANC period, as well as delivery in a health facility with skilled attendants; Reorganizing services so that men feel comfortable seeking care. Since clinics are frequently viewed as a ―female domain,‖ PSI will work to make men feel more comfortable by encouraging facilities to offer ―men‘s clinics‖ and other male-focused initiatives; Offering new services such as ―couples counseling‖ for HIV. There is strong evidence that men have a positive influence on pregnant women‘s willingness to get tested (Baiden et al. 2005). There is also evidence that shows that ANC couples counseling may be a useful intervention in HIV prevention (Farquar et al., 2004). Using this evidence, the PSI team will advocate for training of VCT counselors and ANC staff to offer couples counseling as an alternative to individual counseling; Linking facility efforts to mass communication campaigns that encourage men to participate actively in ANC care and family planning, as well as VCT and STI treatment services; Working with men‘s groups to advocate for the active role of men in the healthcare of their families; and Using mass media and inter-personal communication messages that target couples as users of health services and encourage men‘s involvement in pre-natal care. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 189 Strengthening COGEA’s and COGEC’s Partnerships with Health Providers—As active members of the Community Advisory Committees, COGEA and COGEC representatives will be actively involved in developing performance standards for integrated care, reflecting their involvement in health facility quality assurance activities. For instance, an indicator reflecting the role of the COGEA and COGEC might be ―quarterly meetings between the facility and COGEA are conducted‖ or ―agendas and meeting minutes for COGEA meetings with facility staff are available‖. COGEA and COGEC representatives will also be encouraged to participate in measuring performance at sites, in brainstorming the root causes for performance gaps, and designing and implementing interventions to improve the quality of services. Finally, these committee representatives should play an important role in setting targets and evaluating methods of recognition when targets are met. Quality Assurance Using the SBM-R Approach Simultaneous to the integration of services, the community advisory committees will develop performance criteria for integrated service delivery by agreeing on the definition of quality and creating precise definitions in the form of performance standards. Performance standards not only address provider competency (for instance, correct use of STI protocols and effective counseling) but they may also measure the success of linkages between the health facility and the community, quality of care for target groups, and the degree of youth-friendliness. For instance, examples of performance standards may prescribe that ―youth-friendly patient education materials are available in the clinic waiting area‖ or ―The provider addresses disclosure of HIV￾positive status and offers strategies for this disclosure to client‘s partner‖. The PSI Team will work with teams to ensure that performance standards measuring the degree to which clinic staff-monitor stigma and discrimination within the clinic environment exist. Once all aspects of quality are determined, the advisory committees will shape standards into the form of a monitoring tool, including measurable verification criteria. This tool will subsequently be used for internal monitoring of quality as well as external supervision and evaluation for recognition. With completion of performance standards and initiation of integrated services, the PSI Team will assist sites to make baseline measurements and do a simple analysis of performance gaps. Assisting clinics at this time to resolve problems that are easy to fix builds motivation among staff and instills credibility in the process. Following resolution of non-complex performance gaps, the PSI Team and the advisory committee will assist each clinic to conduct a root-cause analysis of more complex problems using techniques such as force-field analysis and why-why diagrams. Based on this analysis, specific interventions will be identified and implemented to eliminate or lessen performance gaps. The SBM-R approach allows the project and sites to tailor interventions that will lead to the greatest impact per intervention. Given the high rate of reassignment of public-sector employees, refresher training is one intervention that will be required to maintain provider competency and quality of services. In addition to this regular training and follow-up, the SBM-R process leads site stakeholders to identify other interventions that also contribute to quality service provision. For instance, more complex problems may require interventions such as reorganization of client flow, or interventions to improve privacy and confidentiality and alleviate discrimination by providers toward HIV/STI patients. Following the initial round of baseline measurement and interventions, the PSI Team will support ongoing performance measurements of progress to be used as a mechanism to guide the process, inform managerial decisions, and reinforce the momentum for change management. Measurement can take place as self-assessment by providers, internal assessment by facility staff as a whole, and/or assessment by external bodies. Under BHAPP, JHPIEGO trained department 190 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES and zone supervisors in facilitative supervision using a quality-of-care tool that is now becoming the standard for the PNLS in supervising HIV/AIDS health services. The PSI Team will build on this foundation by using SBM-R to create a tool that clarifies expectations of staff, is objective and measurable in tracking progress over time, and leaves health facilities, the advisory committees and external supervisors with a means of examining all aspects of quality services. This process also improves quality by promoting a healthy competition among sites as sites compare and contrast their achievements in attaining a target level of quality. As this process matures, measurement can be linked to an accreditation system where the site is recognized and is promoted as a high-performing site for reaching a set target, such as meeting 85 or 90% of performance standards. At the end of the pilot phase, the PSI Team proposes to present this to national health authorities as a way of accrediting sites for quality, and for determining means of recognizing those sites who attain high-performing status. The PSI Team will also develop and deliver referral directories and guidelines to ensure that all clients who test HIV-positive have access to post-test care and support. A key component of the integrated services strategy is service communications and demand creation, which must raise awareness of the services available and also address the stigma and discrimination surrounding use of HIV/STI prevention and treatment services. At the end of the first year of the IMPACTS project, when the integrated sites are functioning, the community advisory committees and the PSI Team will implement a communications plan using both mass media and interpersonal communication messages to reach the general public as well as key target groups, especially youth, truckers and PLWHA. The goal of the communications strategy will be to diminish stigma surrounding HIV and STI-related services as well as inform the public of the quality and range of services available at the two sites. Special emphasis will be placed on promoting couples‘ VCT in order to identify discordant and concordant positive couples. Couples‘ VCT will also help empower women and provide a supportive environment for women to get tested with their partners. The PSI Team, with the input of the advisory committees, will explore various incentive schemes such as vouchers for products and/or free services available to couples accessing VCT services together. In addition to the integration of comprehensive STI/HIV prevention/treatment and FP services, the PSI Team will assist the MOH to disseminate national VCT policies, norms and protocols (PNP) in all departments of Benin. During the first step of this process, the PSI Team, with the advisory committees, will identify facilitators from both public, non-profit, and faith-based organizations to VCT PNP and will build their capacity to conduct dissemination workshops. The second step of this process is to support facilitators to conduct dissemination workshops in all six departments. Finally, the PSI Team will use the facilitators to support ongoing dissemination at the site level and will measure the impact of dissemination. Performance Indicators—SBM-R is not only an intervention to provide quality assurance, but it can also provide ongoing data regarding the progress of sites and the impact of the project. During the initial development of performance standards, it is possible to develop standards reflecting inputs, processes, outputs, and outcomes, and tailor interventions to capture service￾delivery data necessary for project indicators. Examples of performance standards include strengthening recordkeeping and use of log books for systematic data collection, observing providers while conducting measurements to assess use of correct techniques, or record-keeping to improve forecasting of commodities. In addition, the PSI Team will collect monthly service data in order to report on indicators required by USAID. This includes program-level data such as the number of health workers trained to provide services and the number of clients receiving services. In addition, this data will be compared over time to document the outcomes of the intervention by showing the changes in the number of clients provided with HIV services. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 191 B.2.5.4 Expansion of Clinique PSAMAO Overview—Since 2000, PSI with its implementing partner OSV-Jordan, has run a pilot clinic offering STI treatment and care services and VCT as pare of the ―Prévention du SIDA sur les Axes Migratoire de l‘Afrique de l‘Ouest‖ (PSAMAO) program. Over the past five years, the clinic has established itself as the leading center for VCT and STI services in Parakou, a strategic crossroads in the north of Benin. Aggressive outreach activities toward key target groups (truckers and prostitutes) have enabled the clinic to conduct over 2,276 counseling and testing sessions and to diagnose and counsel over 3,258 STI clients. The impact of these services is magnified by the fact that the majority of clients fall into the high-risk category. Built upon an existing NGO clinic run by OSV-Jordan, this initiative has proven highly cost effective at providing important HIV/AIDS/STI and VCT services. In 2005, the cost per client tested was less than $12 per client served, which is far below the industry average cost per client served. Based on these results to date, PSI feels that this initiative merits a geographic expansion to other important transportation hubs. With the World Bank Corridor project focusing on the east-west transportation in the south of the county, PSI proposes focusing PSAMAO clinic activities on the other main transportation routes and border crossing in the north of the country. With support from KfW, PSI is already in the process of expanding the PSAMAO clinic model to the city of Tanguiéta, which will also include a mobile VCT unit. This new clinic will benefit from the local expertise of OSV-Jordan, presently running the PSAMAO clinic in Parakou. This new clinic will offer mobile VCT that will offer quality VCT and STI case management services to communities in all 13 communes of Atacora and Donga, particularly border sites like Ouaké, one of the 7 high-risk sites selected under IMPACTS. This mobile VCT will also cover 12 additional secondary cities across the country, reinforcing the PSI Team‘s HIV/AIDS strategy in the 7 target sites under IMPACTS. Proposed Strategy—Under IMPACT, the PSI Team will open an additional fixed PSAMAO clinic in Malanville at the northern border of Benin and Niger, increasing the accessibility of quality VCT and STI case management in the country. This extension of PSAMAO clinics will improve the coverage of clinics along Benin‘s main transport corridors, reaching a large number of long-distance truckers, cross-border prostitutes and others in need of such services. All PSAMAO private clinics will adopt an integrated prevention approach, including STI and HIV counseling and testing (potentially including a mobile VCT strategy funded by the KfW), and, when appropriate, STI treatment. STI positive clients will be treated at the clinics and HIV positive clients will be counseled, referred to the nearest health center offering ART, and followed by the clinic providers for proper care. Beginning in FY06, an assessment of existing facilities in Malanville will be conducted to identify the most appropriate site capable of integrating PSAMAO clinic activities. This will be followed by the training of clinic personnel in VCT and STI case management, which will be organized in conjunction with the training of the two integrated pilot sites described above. Clinic activities will be complemented by interpersonal communication activities designed to promote the VCT and STI services. These IPC activities will be conducted by the NGO identified to conduct the integrated communications activities described in Objective 2. As with PSI‘s other PSAMAO clinic activities, quarterly operational plans will be designed with input from the local family health advisory committee. Considering that these clinics will be targeting cross￾border prostitutes and international truckers, PSI will assess the need to develop radio spots in regional languages from Niger and Burkina, such as Peul, a language spoken in all three countries. PSI will also use community radio stations to broadcast messages promoting the PSAMAO clinic services. 192 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Training, supervision, and monitoring for PSAMAO clinicians and technicians will follow the same SDM-R principles at the two integrated public clinic sites. The PSI Team proposes professional exchange visits amongst staff at the PSAMAO clinics, the public integrated clinics, and ProFam clinics in order to share ideas and lessons learned on the job at the different sites. Performance Indicators—Refer to section 4. Research, Monitoring and Evaluation Plan for a list of indicators related to objective three. Objective 4: Strengthen the Capacity of the MOH and the GOB (Including the PNLS and CNLS) to Manage and Implement the National HIV/AIDS Program and Advocate for Policies that Create a Supportive Environment for Social Marketing and the Provision of Health Services Overview The CNLS is charged with coordinating the multi-sectoral fight against AIDS in Benin, supported by the PNLS, which is the technical body within the MOH that addresses the health aspects of the fight against the epidemic. The CCM manages aspects of Global Fund activities under the coordination of the CNLS. A review of the CNLS‘ strengths and weaknesses as an institution, and its work under the previous HIV/AIDS Policy Framework, show the lack of an advocacy plan and insufficient dissemination of epidemiological data as key weaknesses. By strengthening bodies such as the CNLS, PNLS, and the CCM, and by adding important epidemiological and operations research, leveraging donor support, and supporting training efforts, Benin can address the HIV/AIDS epidemic more effectively. Abt Associates, a leader in the field of health systems policy strengthening and capacity building with extensive experience in West Africa, will lead the PSI Team‘s activities under Objective 4. Abt Associates currently provides technical assistance to the PISAF Project in Benin, working to strengthen the capacity of the Benin health system to deliver high-quality services, and strengthening and facilitating the MOH‘s policy process by implementing key health-financing strategies. Additionally, Abt is supporting the MOH to put into practice decentralization policies at the zone level, through assistance in planning, financial and human resource management of health zones, and improved allocation of health budgets by local governments. Through this work, Abt is developing important and productive relationships with MOH personnel that make Abt well-positioned to lead Objective 4 of the IMPACTS Project and coordinate related efforts with the PISAF project. In order for the CNLS to accomplish its mission, it must have the capability to mobilize a broad set of organizations and stakeholders, including civil society in Benin, the commercial and NGO sectors, faith-based organizations, international donors, state agencies and ministries, and representatives of traditional leadership groups. Currently, the CNLS lacks the financial, technical and managerial capacity to assume this role on an operational level. As such, the PSI Team will develop and implement a program to strengthen the CNLS‘ institutional capacity and to enable it to take on a larger leadership role in the fight against HIV/AIDS in Benin. Our program will work with the CNLS to enable it to become a fully functioning organization, and so it can carry out its mission through the following steps: CNLS budget and its Secretariat specifically itemized in the National Budgets; Role of the CNLS clarified as the sole coordinating institution for HIV/AIDS, with international organizations; Broad dissemination of the legal framework and by-laws of the CNLS, as well as the National HIV/AIDS Strategic Framework of 2006-2010; A functioning, fully equipped and fully-staffed CNLS headquarters; USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 193 reinforced role for CNLS and its Secretariat as the principal oversight agency within the GoB for HIV/AIDS programs and projects; Strengthened leadership capacity for CNLS at the national level, through establishment of coordinating mechanisms, resource management tools, and organizational accountability systems; A system of information dissemination and communications at the national level, including a web site, publications and public relations capability within the CNLS, to inform all stakeholders of its activities and achievements; Technical and financial resources at the operational level to implement the Framework, through a needs assessment of program financing needs, by inserting a specific budget line item in the National Budget for HIV/AIDS programs, and by improving the coordination of technical and management resources at national and decentralized levels of government; Institutional improvements in financial management of resources dedicated to HIV/AIDS, through the establishment of annual operational plans and budgets, and a national action program budget financed through all funding sources and donors, including the national government; and Integration of the national HIV/AIDS program through specific plans at the Communal, departmental and national levels. Our technical assistance will be measured against progress achieved in enabling the CNLS to attain these institutional objectives on an annual basis, with specific activities, indicators and milestones that will be defined as part of first Annual Work Plan. Proposed Strategies—The PSI Team‘s strategy includes a multi-level and participatory approach to policy and advocacy through the CNLS/PNLS, the private sector, and communities. This includes dialogue and consensus among stakeholders from all sectors affected by HIV/AIDS, such as those in the public and private sectors and civil society, especially those affected most directly by HIV/AIDS. Our approach will build capacity at the individual, organizational, and institutional levels to address HIV/AIDS. In particular, the CNLS and PNLS will have stronger institutional capacity in key disciplines to address the epidemic in Benin, such as policy and advocacy, communications, and dissemination of critical data and information, development and promulgation of new policies, and engaging and collaborating with other sectors. The key elements of our strategic approach to this component include the following three approaches: 1. Expanding and sustaining the technical and managerial capacity of the CNLS and PNLS: The PSI Team‘s technical assistance will enhance the institutional leadership and management capacities of the CNLS and PNLS, so that over time their roles and responsibilities are clarified and better understood among the various sectors involved in HIV/AIDS and among the wider population. We will enhance their ability to coordinate the GOB‘s response to HIV/AIDS and in managing the national HIV/AIDS program from a technical and service delivery standpoint. This will encompass ongoing technical support in areas of communications and dissemination, advocacy, promulgation of new policies, surveillance, and dissemination of epidemiological data, research and data collection, and training. 2. Working with the CNLS and PNLS to implement policy framework: For these two key GOB institutions, our focus will be on providing technical assistance to enable them to implement the nearly-finalized HIV/AIDS policy framework and to understand the challenges and opportunities brought on by the process of decentralization in the health sector. Our technical staff will work with administrative and technical units of the CNLS and PNLS to provide technical support in key areas, especially policy development and dissemination, advocacy planning, and dissemination of key data. In advocacy planning, 194 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES the PSI Team will help the CNLS and PNLS to forge partnerships and collaborative interactions with other organizations and sectors, including those among civil society and community-based organizations. We will coordinate closely with PISAF to complement and not duplicate their activities and interventions, where they are working directly with the MOH on decentralization efforts. 3. Working with the GOB to expand the role of communities and the private sector in the response to HIV/AIDS: We will support the CNLS to expand the role and involvement of communities and the private sector to the HIV/AIDS challenge through a multi-sectoral approach. In particular, the PSI Team will help the CNLS to work with existing private organizations involved in health products and service delivery, to increase their ability to fill gaps in the population‘s needs for quality products and services. We will achieve this by helping the PNLS and CNLS to build a more favorable environment for the delivery of community-based services by increasing work with the commercial private sector, and by creating advocacy systems at the community level to ensure involvement of groups most affected by HIV/AIDS. We will provide short-term assistance to address key policy issues affecting communities and the private sector, such as social marketing of health products, targeting subsidized products and services, the introduction of generic products, and tax-exemptions for ITNs. Key Activities and Interventions—The PSI Team proposes the following comprehensive set of technical assistance activities and interventions to enhance the institutional capacity of the CNLS and PNLS to manage the national HIV/AIDS program, and to develop policies that will create a supportive environment for the implementation of social marketing and health service provision in Benin. The Policy and Institutional Capacity Building Coordinator will primarily focus on developing a long-term program to provide technical assistance to CNLS. The Policy Coordinator will be supported by short-term technical specialists, who will provide training and the necessary tools to the CNLS staff to manage their programs and resources. To assist the CNLS overcome its organizational challenges, the Policy Coordinator will be responsible for the following: Helping to establish the Steering Committee from among the CNLS and PNLS management, and provide inputs for structuring the deliberations and decisions of the Committee; Conducting a rapid organizational assessment of the CNLS to ensure it is properly staffed, funded and equipped to implement its functions; Ensuring that adequate funding has been budgeted and allocated to the CNLS to carry out the programs and interventions; Identifying new policies and legislation to improve the environment for HIV/AIDS programs; Recruiting and inviting stakeholders from civil society, the private sector, NGO and faith￾based organizations to be included in the deliberations and decision-making by the CNLS; Developing and implementing a detailed program of capacity building from the PSI team to provide short-term training in financial management, planning, and use of data for decision￾making, to strengthen the CNLS management and staff‘s ability to manage programs and resources; Helping the CNLS collect, analyze and utilize the information on the course of the HIV/AIDS epidemic for purposes of setting policies and establishing priorities; Assisting the CNLS in determining which strategies and interventions should be scaled up or replicated, based on the research and cost-effectiveness; and USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 195 Working with the CNLS to define and implement national policies and priority programs, by encouraging participation from a broad coalition of sectors and stakeholders, and through a transparent and informed process. Institutional Capacity Strengthening Institutional capacity strengthening approaches are required to deal with the myriad policy, service delivery, advocacy and communication issues facing the CNLS and PNLS. Our engagement will achieve several goals: institutionalize in-country capacity to engage a full range of sectors, promote understanding of benefits of the multi-sectoral approach, and integrate actors from key sectors of the economy in the process. This approach aligns closely with our team‘s guiding principles: (1) building advocacy and policy capacity through participatory approaches in order to foster ownership of the processes and decisions rather than offering ―outside solutions‖; (2) strengthening government institutions‘ and civil society organizations‘ capacity to collaborate in all aspects of the policy process; (3) ensuring that policy activities fully reflect the changing country and global context; and (4) leveraging existing national and regional knowledge and expertise ; (5) ensuring results can be sustained over time. A much-needed step for increasing transparency and accountability will be the specification of an operational budget for the CNLS in the National Budget, and the development of annual operating budgets for CNLS and PNLS that include all funding sources for HIV/AIDS-related program and projects in Benin. To achieve this, it will be necessary to conduct a funding needs assessment, and to develop a financial management capability to track and monitor funding sources and uses, and to publish the commitments, draw-downs and uses of funds against specific programs and projects. In addition, the CNLS should report on an annual basis, how funds under its jurisdictions and budget have been spent and what achievements have been attained by program and project. This reporting should also reflect how the Government has chosen to allocate and spend its resources, based on inputs from stakeholders and local organizations. Last, we will encourage the CNLS to evaluate the costs and effectiveness of alternative interventions, and to establish priorities for which interventions should be scaled up. These decisions will then be shared with stakeholders as part of the proposed participatory process we will help to promulgate as part of our efforts. Sustainability of the CNLS—The specification of an operational budget for the CNLS in the National Budget and the development of annual operating budgets for CNLS and PNLS that includes all funding sources for HIV/AIDS-related program and projects in Benin will help to increase transparency and accountability. To achieve this, a funding needs assessment must be conducted, financial management capability to track and monitor funding sources and uses must be developed, and commitments, draw-downs and uses of funds against specific programs and projects must be published. In addition, the CNLS should report on expenditures of funds under its jurisdiction and program and project achievements on an annual basis. Reporting should illustrate government allocation of resources based on inputs from stakeholders and local organizations. The PSI Team will encourage the CNLS to evaluate the costs and effectiveness of alternative interventions, and to establish priority interventions for scale-up. These decisions will then be shared with stakeholders as part of the proposed participatory process. The PSI team will review the final report of the health system assessment as soon as it is available; in the meantime, we have based our response on the draft report (to be revised if there are significant changes in the final report). The draft USAID/W assessment highlighted a strong tradition of community participation and involvement in health care, and the presence of a free and open media, along with active provider associations in Benin, as elements to foster in creating a more transparent and accountable health care system. The assessment also underscored the lack of information flows back to lower, decentralized levels and to 196 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES stakeholders as system weaknesses, along with a fledgling government capacity for internal enforcement, and lax observance of laws and regulations related to health care provision. In light of these conditions, the PSI team will extend its proposed efforts to establish, fund and support the proposed Steering Committee(discussed below), by proposing that the CNLS embark on a course to ―open its house‖ to external stakeholders, the media, and Benin civil society. The first step will be to publish its operating budget, including major donor funding, with details on key programs and interventions to be carried out during the year. We will also urge that the CNLS invite inputs from the community, the private sector and other stakeholders (professional provider associations) regarding principal elements of its program budget, through participation with COGECS, zonal-level health committees, and private-sector forums. Its budget will be published and disseminated on the web site, and shared with the Benin media, in an effort to create an environment of dialogue and open exchanges of information. Through the activities and ―open meetings‖ to be arranged through the Steering Committee, the PSI team will invite community and stakeholder participation on a regular (quarterly) basis, through reviews and presentation of key programs, accomplishments and challenges, as a way of eliciting external advice and support from natural stakeholders. The PSI team will recommend and advocate that the CNLS publish its Annual Report each year as a means of ensuring financial accountability and program transparency. The regular contacts with external stakeholders as well as periodic studies and analyses of programs, which will be shared and disseminated with the media and general public, will also contribute to creating an environment of full and open transparency with regard to the Government‘s programs in the fight against HIV/AIDS. Engagement of the GOB—Policy Steering Committee The PSI Team recognizes that the success of the national strategy requires full engagement and collaboration of appropriate partners from different sectors and disciplines, including the private sector and community groups. Importantly, the GOB must feel ownership of activities and will be key participants in deciding on priorities, timetables and processes for reviewing and approving new activities through the course of the project. To achieve this crucial engagement and participation, we will work with members of the CNLS and PNLS to form a Policy Steering Committee, to consider policy options and priorities, decide specific interventions, identify technical assistance from the PSI Team, and chart a capacity-building process. The Steering Committee will be our counterpart within the GOB for delivering support to the CNLS and PNLS, and for structuring our interactions through regular meetings. The proposed Steering (advisory) Committee will be composed of key PNLS and CNLS managers. The Committee will function as the primary point of contact between the CNLS and external entities, Benin civil society, donors and key stakeholders, with regard to the programs, activities and accomplishments of the CNLS in managing programs in fight against HIV/AIDS in Benin. As such, the Committee will also be the PSI team‘s principal ―counterpart‖ with the CNLS, in terms of the technical assistance to be provided to the CNLS and the activities to be supported through the project‘s Policy Advisor and Advocacy Advisor. It is essential that the Committee‘s financial and organizational sustainability be assured through allocated and budgeted funds and through a focused and purposeful institutional mission. The PSI team proposes that the Committee be funded on an on-going basis through the CNLS‘ annual operating budget, which should include a specific line item to support the work and expenses of this group. The PSI team, led by the policy advisor, will use findings from previous management and technical capacity assessments to help quantify the resource needs (financial and other) for the Committee on an annual basis. The funding will cover meetings with various constituencies, conducting cost-benefit studies and analyses, and documenting the work of the CNLS, particularly the uses of funds to stakeholders. From a programmatic standpoint, the USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 197 Committee should convene at least monthly, and publish its agenda and decisions. It will also invite representatives from stakeholders, donors, and private-sector forum participants to its meetings on a quarterly basis to report on progress and key programs. This will ensure that the Committee addresses issues related to coordination, financial and program management. The PSI team will provide the Steering Committee with technical support to enable it to review, analyze and present information from new studies and surveys to external audiences, to manage national budget resources and external financing, and to review and approve new programs and projects being developed at the communal, zonal and national levels. These steps will ensure that the Committee‘s work is essential for managing contacts and coordination with external constituencies and stakeholders, and that its functions are supported internally through the CNLS‘ operating budget. Key steps that we will propose to the Steering Committee include ten key elements described below. 1. Mapping and Integrating Key Partners in HIV/AIDS Activities. The PSI Team will assemble a technical team to conduct a rapid mapping of partners involved in testing and counseling. We will explore the possibility of establishing multi-sector boards representing health, education, and transport sectors, uniformed personnel, and others involved or affected by HIV/AIDS. These boards will function on a participatory basis to identify and document key issues and concerns, and to bring these issues to a broader national forum. 2. Building a Supportive Environment for HIV/AIDS and Social Marketing among Key Stakeholders. The technical team will consult extensively with stakeholders including GOB, PISAF, NGOs, donors, PLWHA organizations, and others on what should be included in the existing and new policies to create a supportive environment for HIV/AIDS, FP/RH, CS and malaria control programs. We will design a step-by-step HIV/AIDS Policy Support Strategy to develop and refine policies that create a supportive environment for HIV/AIDS that correspond to the country‘s HIV/AIDS policy framework. The strategy will necessarily include the participation of CNLS leadership and be approved by the CNLS to ensure implementation. The challenge in creating a supportive environment is often complicated by divergent and even hostile or opposing postures of key stakeholders, due to resource allocations and preferential policies or programs. To overcome this, the PSI Team will apply two interrelated activities to achieve success under this step. First, we will conduct a participatory review of existing policies, protocols, and guidelines. A major part of the review will focus on understanding how different policies are impacting HIV/AIDS activities among stakeholders and to clarify whether some groups are not reaping benefits from existing policies and programs. Second, we will work with partners to identify current challenges and opportunities to existing policies. The technical team, with stakeholders, will develop a common agenda, clarify who is responsible for coordination, and disseminate information. Again, the process and results must be endorsed by the Steering Committee, and the CNLS/PNLS leadership. 3. Conducting Advocacy—Appropriate policies and approaches can lead to a more supportive environment if they are informed by a credible advocacy process that includes relevant information and dialogue. Strategies to advocate for change in the current environment will follow a multilevel process that involves the following types of activities: – Assessment of current practices, policies, and successes in past advocacy movements; – Objective and honest brokering of focused discussions with lead associations, NGOs, and government ministries, development partners and others; – Mass media and educational campaigns in multiple settings (e.g. workplaces, schools, communities) and designed for multiple audiences; 198 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES – Gender training and awareness materials for project staff and in-country partners; – Editorial guidelines for all project documentation with appropriate gender-sensitive language, disaggregated data presentations by sex where possible; – Tools to support a policy dialogue in support of more gender-equitable policies and programs; – Involvement of PLWHA in media and educational campaigns to demonstrate their capability of leading normal and productive lives; and – Personal and media appearances of government leaders supporting the existing and new policies. Utilizing this approach, the PSI Team will provide technical assistance to strengthen the CNLS‘ ability to implement HIV/AIDS advocacy activities. The cornerstone of this effort will be the development of an information and communication infrastructure that will have the following elements: 1) an internet web site with all current GOB policies, government activities, local and regional organizations, and information resources on HIV/AIDS to permit users in Benin to access all relevant data on HIV/AIDS issues; 2) establishment of documentation centers at regional and national levels, with pertinent reports, studies and information resources from other regional and international efforts, to permit more thorough investigations and analyses; 3) the development of a regular newsletter, which will summarize current issues on HIV/AIDS, including new policies and programs, donor-funded interventions, access to HIV/AIDS products and services, and inclusion of articles and opinion pieces on HIV/AIDS from PLWHA. An Advocacy Advisor will provide ongoing technical support to the CNLS to develop an implementation plan for this communications infrastructure. Working with CNLS management and technical staff, the Advocacy Advisor will help the CNLS to draft an Advocacy Implementation Strategy within six months of contract start-up that will detail how the CNLS will build its communications systems and materials, and how advocacy activities will be rolled out. In addition, the Strategy document will identify areas of technical support from the PSI Team and its local resources, such as building a web site, establishing resource centers, and producing advocacy materials, newsletters, and information pieces on HIV/AIDS activities and successes. In addition, Abt will assist the CNLS to develop the AIDS Impact Model (AIM) in Benin, and to develop a plan for disseminating impact results from AIM on a regular basis. We will work with CNLS to prepare a staff training plan for enhancing its capacity to build and utilize AIM. Our approach for this effort will include: Adaptation of a data collection instrument for Benin Conducting a joint training of data collectors by the PSI Team‘s personnel and CNLS team CNLS will be responsible for data collection and entry, with guidance from the PSI Team CNLS will participate in report writing and dissemination of results, with review from the PSI Team Assisting CNLS to identify different audiences and recipients for the results, as part of an ongoing advocacy and outreach effort that would publicize the GOB‘s efforts in HIV/AIDS prevention and care. A parallel effort will be to work with the PNLS to disseminate sentinel epidemiological data, which would feed into the AIM. PSI and Abt communications experts will assist PNLS technical personnel to develop summarized and less technical versions of the data to share with public audiences, as well as managers/planners involved in HIV/AIDS efforts. These versions would be shared in various forums and meetings with stakeholders, multi-sectoral boards, policy makers USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 199 in the GOB, and opinion leaders in civic organizations, to inform on the progress and trends of HIV/AIDS activities, and to solicit feedback. 4. Policy Development Assistance—Our team will assist the GOB to develop and promulgate new policies that support the environment for HIV/AIDS and social marketing activities. The critical policy efforts to be undertaken are: – Updating the national youth policy - The team will engage multi-disciplinary experts to update the national youth policy on HIV/AIDS/STIs. We will provide technical assistance to enable the GOB to draft, review, and disseminate the policy, with inputs from civic organizations, parents, NGOs, and medical professionals. It is envisioned that this participatory process in the drafting and dissemination will lead to (a) a shared ownership of the national youth policy, (b) consensus on monitoring indicators, and (c) well-defined institutional roles and responsibilities. Finally, the team will facilitate a consensus-building workshop on how to move forward, especially on building partnerships and strengthening transparency and accountability processes which are critical to the success of this activity. 5. Disseminating New PMTCT Protocols and ART Guidelines—Abt will assist the GOB to disseminate new PMTCT protocols and ART guidelines. Two principal pillars of effective dissemination are the availability of good products and the ability of policy makers and advocates to use it to inform dialogue, build consensus among stakeholders, and promote transparency and accountability. The cornerstones of our strategy to assist the GOB address: – comprehensive consideration and harmonization of all key RH/MH/FP and HIV/AIDS intervention areas, technical guidelines, performance indicators, and system constraints; – use of country evidence and stakeholder engagement to inform policy revision and implementation changes; – development and costing of policy and implementation options along with efficient use and coordination of GOB and donor resources; – phased implementation and operations research for new approaches before scale-up; – M&E linked in a feedback loop of communication across communities, policy makers, and NGOs to inform ongoing decision-making and implementation practice. 6. Addressing Social Marketing Policy Issues—The PSI Team will work with the GOB to address policy concerns that impact the social marketing of FP/RH, malaria, and CS products, including the targeted distribution of ITNs, the introduction of tax exemption for ITNs, the use of generics alongside Coartem, and additional support for marketing of products through private- sector distribution channels. The PSI Team will bring our experience in countries like Uganda, Zambia, Ghana, and Nigeria to help the GOB develop successful models for facilitating social marketing activities that affect HIV/AIDS programs. We will conduct specialized research studies, using regional staff with experience in social marketing and product distribution, to determine the technical and policy feasibility of these issues, and to recommend potential GOB action or legislation to facilitate their implementation, as appropriate. 7. Leveraging Other Donor Funds—The PSI Team will work with the CCM Secretariat to enhance its ability to manage, implement, monitor, coordinate, and evaluate the utilization of Global Fund (GFATM) resources. We will assess the CCM Secretariat‘s capabilities in this regard and develop a strategy to strengthen its technical, financial, and managerial capacities. We will explore ways for the CCM to leverage GFATM funds, implementation and proposal development through cost analysis and budgeting, planning for malaria ACT and ART scale￾up, and collaboration with private-sector health institutions. 200 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 8. Epidemiological and Operations Research—The PSI Team will work with the MOH/PNLS to support its research and data collection efforts with regard to accessing and analyzing epidemiological data and information on HIV/AIDS and monitoring attitudes and behaviors among the general population and at-risk groups, such as youth. Abt Associates has proven capacity to provide rapid and comprehensive technical assistance to epidemiological and operations research in many countries. As the prime implementing partner of global USAID flagship programs such as Partners for Health Reform plus Project (PHRplus) and the Private Sector Project (PSP), Abt has conducted qualitative and quantitative research as well as evaluation on public and private sector health issues in HIV/AIDS, reproductive health, malaria and immunization. Our team will work with the MOH/PNLS to support its efforts in the following ways: (1) Strengthen the ability of local researchers in Benin to undertake independent and quality operations research that can influence policy from the bottom up; (2) Transfer technical skills in the use of epidemiological data and information to inform decision-making process. Abt Associates will equip researchers at MOH/PNLS to access and analyze epidemiological data and information on HIV/AIDS and monitoring attitudes and behaviors among the general population and at-risk groups; (3) Provide targeted on-site support to local partners to translate research results on HIV/AIDS, Malaria, FP/RH, and CS into program responses; (4) Provide state-of-the art tools and techniques, drawing on best practices and lessons learned from the field to design and disseminate powerful planning strategies and policy￾relevant information. The PSI Team will support the BSS implementation through the four steps outlined below. 1) We will conduct joint review of the data collection and analysis process to identify areas that require strengthening and technical remediation to ensure that data are sound and reflect proper sampling, handling and manipulation. As necessary, we will recommend alternatives to current practices and methods. 2) We will provide technical support in the analysis and interpretation of results, and assist with the dissemination of results to different audiences, which may include a business audience or technical specialists, both which has a different context for understanding the BSS. 3) We will help to identify and suggest specific targets that are considered high risk; such as youth, prostitutes, truck drivers, military personnel, for study and analysis. We will also identify geographic areas that may warrant attention based on previous research findings. 4) We will support the training of analysts in effective use of data to make evidence-based decisions, to evaluate specific interventions and to set national policies. For example, we propose to use the power of geographic information systems (GIS) to help policymakers and planners map out where health services are most needed. Mapping creates a visual representation of health service needs in a format highly accessible to people with all levels of technical knowledge. 9. Training Support—As requested by the MOH or NGOs, and approved by USAID, we will provide logistical support to specialized training requests in areas such as data collection and analysis, communications and dissemination, and conference planning. The PSI Team has extensive experience organizing and managing training seminars, conferences and technical meetings, and has ample, on-the-ground capability in Benin to assist the MOH and local organizations, on an as-needed basis. 10. Leveraging Programs and Best Practices—The PSI Team will seek government guidance and leadership to facilitate private sector HIV/AIDS initiatives and to identify models achieving significant impact to be replicated or scaled-up. Specifically, the PSI Team will collaborate with the government on the following initiatives: USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 201 – Business Council: The PSI Team will seek government assistance in forming a council of business leaders to address issues and challenges related to HIV/AIDS in the workplace. Members will discuss how HIV/AIDS affects their companies, and how they are addressing these challenges with workers, unions and the health care community. The Business Council will facilitate a collaborative partnership with the private sector. Success stories and models will be disseminated and studied for replication. – HIV/AIDS Advocacy in the Business Community: The PSI Team will seek government support in encouraging the business sector to be advocates for HIV/AIDS prevention and treatment. With input from the government, PSI will identify business leaders who can mobilize resources in their industries and who can be champions for effective, new policies. – Supporting Private Sector Projects: Consider co-funding projects that are initiated by the private sector, by bringing technical expertise to measure and evaluate results, by providing quality assurance methods and support from the public sector, and by sharing access to new information, technologies, access to training and treatment options with the private sector. – Accessing Private Sector Resources: The PSI Team will collaborate with government officials to tap the private sector for ideas and contributions for how prevention and treatment programs can be facilitated through public policies, such as tax treatment for health insurance coverage, and legislation that encourages companies to establish or outsource to NGOs workplace HIV/AIDS prevention and treatment programs. The proposed program of technical assistance to the CNLS and the GOB will enable it to implement its HIV/AIDS Strategic Framework and to improve the environment for a broad￾based approach to the fight against HIV/AIDS in Benin. The PSI team is mindful that a multi￾sectoral approach is essential, given the multitude of stakeholders and institutions involved in this critical effort. Therefore, the technical assistance program we are proposing is ambitious, yet realistic, in terms of specific milestones we propose to accomplish working with the CNLS and other GOB institutions. The essential results we intend to achieve are the substantial strengthening of the CNLS and PNLS as the central organizations managing, coordinating and implementing HIV/AIDS-related efforts within the GOB, in key technical disciplines such as advocacy, communications and dissemination of critical data and information, development and promulgation of new policies, and broad-based collaboration with other key sectors and institutions in Benin. Performance Indicators—Refer to section 4. Research, Monitoring and Evaluation Plan for a list of indicators related to objective four. PROGRAM MANAGEMENT The PSI Team—The PSI Team brings together PSI‘s expertise and demonstrated experience in implementing results-driven HIV/AIDS, FP and CS social marketing and behavior change communications programs; JHPIEGO‘s expertise in training clinicians and management of integrated service delivery sites; and Abt‘s demonstrated ability to strengthen the capacity of the public sector to manage national HIV/AIDS programs and to advocate for policies that create a supportive environment for social marketing and the provision of health services. Local collaborating partners, such as OSV-Jordan, bring expertise in implementing effective behavior change activities among key high-risk populations and managing relations with local NGOs. Management and Subcontracting Plan—As the primary recipient, PSI will be the Mission‘s main point-of-contact and will have overall responsibility for program strategy, implementation, monitoring, and reporting. PSI‘s local affiliate, PSI/Benin, will lead 202 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES implementation of Objectives 1 and 2. PSI proposes to manage sub-contracts with two international implementing partners, JPHEIGO and Abt, which will be the lead implementing partners for Objectives 3 and 4 respectively. PSI will sub-contract with OSV-Jordan, a Beninese NGO, to supervise the activities of local NGOs under Objectives 1 and 2, as well as manage the PSAMAO clinic expansion under Objective 3. All contracts signed with sub-contractors will clearly describe the responsibilities of each partner in contributing to the realization of IMPACTS objectives, specific activities to be implemented, and the monitoring and evaluation plan to assure full reporting on activities and performance indicators. Day-to-day implementation and operational oversight of all activities will be managed directly by PSI‘s Cotonou-based COP, who will be USAID‘s main point of contact and who will be supported by a five-person senior management team. Partner staff engaged through sub-grants will sit in PSI/Benin‘s offices and will be fully integrated into PSI/Benin to ensure maximum synergy and health impact across program activities. In managing IMPACTS‘s strategies and day￾to-day activities, the COP and the senior management team will focus on: Fostering Information Flow—The PSI team will employ a formal reporting procedure in addition to regular meetings and open communication among staff to ensure the flow of information and the harmonization of project activities. Annual work plans, monthly and quarterly reports, and the information required to build them, will flow from the bottom up. That is, IMPACTS field personnel closest to project beneficiaries will supply information on project activities, progress toward results, and any issues arising during implementation in monthly reports, and ad hoc communication with their supervisors and senior managers. Information will also flow back down and across project personnel through the distribution of IMPACTS‘s quarterly reports to technical staff members. Building Staff Technical Capacity—IMPACTS staff at all levels will be encouraged to reinforce their technical and professional skills through formal training, both within and outside Benin. This may include courses in computers, English language, or technical training organized by donors or other NGOs. Exchange visits to consortium partners‘ successful projects in other parts of the sub-region may also be arranged. IMPACTS personnel will also be exposed to state￾of-the-art approaches while working closely with the various international experts and consultants employed by PSI, Abt, and JHPIEGO to provide technical assistance to the project. Staffing Plan—By virtue of its ongoing programs in Benin and West Africa, the PSI Team is able to propose a highly skilled and experienced team to implement the IMPACTS Project. It is important that IMPACTS is implemented as an integrated whole to ensure that all project interventions across objectives are consistent and mutually reinforcing. To help achieve this, PSI has designed a staffing structure for IMPACTS (please see the organigram in Annex 1) which includes a five-person senior management team responsible for oversight of project integration. Annex 1 includes resumes and letters of commitment for key and supporting personnel listed by name below. Senior managers and their responsibilities are as follows: Key Personnel Chief of Party/Social Marketing Specialist—Brian Dotson, PSI (90% LOE): The Chief of Party will provide overall leadership, management and vision to the IMPACTS Project, serving as USAID‘s primary contact, and coordinating the work of all staff and partners (including the day-to-day management of the four senior technical managers below). Mr. Dotson will have ultimate responsibility for ensuring the effective and efficient implementation of all project activities and achievement of proposed results. He will facilitate open communication and the free flow of information between project personnel, ensuring that strategies and activities are implemented in an integrated, mutually reinforcing manner. As the Social Marketing Manager, he USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 203 will oversee and coordinate all social marketing interventions for IMPACTS, enabling high-quality implementation of the proposed activities and the achievement of proposed targets. HIV/AIDS and Family Health Technical Advisor—Foyet Tchechoupie, PSI (100% LOE): The HIV/AIDS Manager will report directly to the COP and will oversee IMPACTS‘s Integrated Services, Community-based, STI/HIV/AIDS, Reproductive Health, and Child Survival strategies and activities. He will ensure that strategies and activities related to these key project components are designed and implemented in an integrated fashion to reinforce other project areas. Supporting Senior Technical Managers The above key personnel will be supported by a group of talented technical managers. While not presented as Key Personnel, the following managers will oversee major components of the IMPACTS project: Marketing Technical Advisor—Frederick Persoons, PSI (50% LOE). The Marketing Technical Advisor will report directly to the COP and will supervise Research, Communications, and Sales & Distribution personnel, coordinate development of work plans, and ensure smooth implementation of related activities. In year one of the project, this position will be held by Frederick Persoons, and at the start of the project, PSI will recruit a local social marketing advisor who will be mentored by Mr. Persoons to take over the position in Year 2. Policy and Institutional Capacity Building Coordinator—Edmond Kifouly, Abt Associates (86% LOE): The Policy and Institutional Capacity Building Coordinator will report to the COP and will serve as IMPACTS‘s primary liaison with the MOH, leading the project‘s work with the MOH under Objective 4 to develop and implement integrated strategies, norms, and procedures for the fight against STI/HIV/AIDS in Benin. It will be particularly important that Mr. Kifouly is informed of all project technical strategies, activities, and issues so that he may serve as an effective representative to the CNLS and PNLS, which are key IMPACTS stakeholders, and that all project activities support Benin‘s national health strategies. Administrative Services Director—Achille KOUNOU, PSI (100% LOE): Other Supporting Professionals The senior management team above will be supported by a talented group of development and health professionals, selected members of which are presented below: Community-based Activities Coordinator —Clarisse Gomez, PSI (100% LOE): The Community-based Activities Coordinator will report to the HIV/AIDS and Family Health Technical Advisor and manage the PSI Team‘s partnerships with local NGOs including the provision of regular technical support, as well as supervise all community-based communications activities. Reproductive Health Coordinator—Prudencia Ayivi, PSI (100% LOE): The Reproductive Health Coordinator will report to the HIV/AIDS and Family Health Advisor and will be responsible for supervising all reproductive health-related activities, including the expansion and maintenance of the ProFam network. Michigan Population Fellow—Heather Robinson, PSI (100% LOE - until October 2007): This Fellow has been seconded to PSI as part of the University of Michigan Population Fellowship program to provide technical assistance with PSI‘s overall family planning and reproductive health program. Integrated Services Coordinator—Dr. Mathurin Lougbegnon, JHPIEGO (100% LOE): The Integrated Service Delivery Manager will report to the HIV/AIDS and Family Health 204 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Technical Advisor and be responsible for the implementation of all integrated service sites. He will provide ongoing support to the project through frequent visits, training, supervision, and support to the SBM-R process. This technical advisor will liaise with other IMPACTS staff, providing an important link to social marketing activities, advocacy efforts, and coordination with the PISAF project. Integrated Services Technical Assistance— Dr. Kodjovi John Agbodjavou. ( 50% LOE in Y1, 30% in Y2, 10% in Y3, 5% in Y4 and Y5) To assure the success of the two integrated service sites, the Integrated Service Delivery Manager will be mentored by Dr. Agbodjavou, based in Togo, and his technical assistance will then be phased out over the life of the project. Health Services Advisor—Dr. Sessi Irénée Kotchofa, Abt Associates (86% LOE): The Health Services Advisor will report to the Policy and Institutional Capacity Building Coordinator and will support IMPACTS‘s capacity building activities with the MOH under Objective 4, including development of strategies, norms and procedures for health services. Regional Researcher for West Africa—Dr. Virgile Capo-Chichi, PSI (25 days LOE per year): Dr. Capo-Chichi will provide support in the development and implementation of all research activities and will lead recruitment and training of the proposed M&E staff. . Supporting Local Field Personnel The managers above will be supported by a talented, experienced field team composed of selected members of PSI‘s and partners‘ current staff. One hundred percent Beninois and currently implementing ongoing programs in Benin, this team as a whole is intimately familiar with the local context, current issues, challenges, and best practices related to IMPACTS‘s areas of intervention. Supporting Headquarters Personnel IMPACTS field staff will be supported by a group of experienced project managers, administrators, and specialized technical experts from PSI, Abt, and JHPIEGO headquarters to facilitate top-quality technical management and administration of the project. Headquarters support costs have been kept to a minimum, with these personnel budgeted part-time to support program management, technical assistance, and administration. RESEARCH, MONITORING AND EVALUATION PLAN OVERVIEW The PSI Team will implement a comprehensive monitoring and evaluation strategy that combines performance-based management with monitoring and evaluation of behavioral determinants and resulting behavior change; monitoring of the quality of product and service delivery; and formative, qualitative research that targets messages for specific groups and communication channels. Data gathered will respond to program indicators, as well as provide accurate and complete information for programmatic decision-making throughout the program and the evaluation of program results. The PSI Team‘s monitoring and evaluation strategy is guided by PERForM, a Performance Framework for Social Marketing, for designing studies and activities and using secondary data (such as the Demographic and Health Survey and Behavioral Surveillance Surveys) to produce needed, timely, and actionable evidence for social marketing decision making. Figure I-1 presents PERForM, which is based primarily on the Andersen behavioral model of health services use.11 11 Andersen, R. M. (1995). Revisiting the behavioral model and access to medical care: does it matter? Journal of Health and Social Behavior, 36(1), 1-10. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 205 Figure I-1. PERForM PERForM has four levels, A to D, with Level A consisting of the goal of social marketing for health promotion, namely improved health status (such as reducing unintended pregnancies) or quality of life (such as preventing dehydration). Level B consists of the objectives of social marketing programs, stated as product or service use on the left side (such as sleeping under a treated bednet) and/or other risk-reducing behaviors that do not involve the use of a product or service on the right side (such as delaying sexual debut). The adoption or maintenance of these behaviors in the presence of a given need or vulnerability is causally antecedent to improving or maintaining health and/or quality of life. Level C consists of the determinants of behavior summarized in terms of opportunity, ability and motivation, given a set of population characteristics, that respond to Level D, the social marketing intervention or other influences of health, including health education, regulation, or other trends and impulses present in society. 12 Opportunity is defined as institutional or structural factors that influence an individual’s chance to perform a promoted behavior and that are mutable by social marketing. These factors include availability, quality of services, brand appeal, and social norms. Ability is an individual’s skills or proficiencies needed to perform a promoted behavior which again are mutable by social marketing. These factors include knowledge, social support and self-efficacy. Motivation is an individual’s desire to perform a promoted behavior. Motivation is mutable by social marketing and is defined in terms of attitudes, beliefs, intentions, outcome expectations and threat or risk perception. These factors will be directly influenced by PSI Team activities through interventions under objective 1, 2 and 3. 12 Rothschild, M. (1999). Carrots, Sticks, and Promises: A Conceptual Framework for the Management of Public Health and Social Issue Behaviors. Journal of Marketing, 63, 24-37. NEED OPPORTUNITY ABILITY MOTIVATION POPULATION CHARACTERISTICS SOCIAL MARKETING INTERVENTION USE HEALTH STATUS QUALITY OF LIFE RISK-REDUCING BEHAVIOR PRODUCT PRICE PLACE PROMOTION OTHER INFLUENCERS A LEVEL B C D 206 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Monitoring and Evaluation Process The PERForM framework involves four primary applications of social marketing research that form the basis of the PSI Team‘s monitoring and evaluation plan: concept development and pre￾testing, segmentation, monitoring of the marketing mix (the four primary marketing instruments used for intervention, sometimes called the 4Ps of products/services, price, place and promotion) and evaluation. Concept development and pre-testing—In PERForM, this occurs at Level D and is typically the use of qualitative research for developing and testing new elements from the marketing mix. This will particularly be the case for all brand repositioning or development (i.e. Prudence Plus, Harmonie, Orasel Plus and the new PPT) Segmentation—In PERForM, this is an association between Level B behavior, given need or risk, and Level C opportunity, ability and motivation. Segmentation is a process of dividing otherwise heterogeneous populations into homogenous groups that are identifiable in terms of easily measured variables, substantial in size, accessible to the intervention, stable long enough for the 4Ps to be developed, delivered and evaluated, responsive to the marketing mix, and actionable in terms of giving guidance to 4P decision making. This information is then used to complete the logical framework at output level and to elaborate marketing plans, which allocate resources among the marketing instruments of product, price, place and promotion. Monitoring and Evaluation—In PERForM, monitoring tracks trends at Levels B, C and D. Monitoring information is used to adjust the marketing plan and evaluation produces nine primary measures for evaluating social marketing performance: effectiveness, cost-effectiveness, a substitution effect, a halo effect, equity, coverage, quality of coverage, access, and equity of access. Monitoring and Evaluation Activities All research, monitoring and evaluation activities conducted during the course of the program will be developed and overseen by the PSI Research Manager with international technical assistance from the PSI Research Department‘s West and Central Africa research expert and in collaboration with the relevant technical advisor. The PSI Team will use a combination of internal and external research instruments in order to develop and pre-test concepts, segment, and monitor and evaluate the proposed interventions. These will include the following: External: Demographic Health Surveys; Behavioral Surveillance Surveys; Mid-Term and End of Project Review Internal: Tracking Surveys; Target Groups Surveys; Service Delivery Surveys; Qualitative/Formative Research; Programmatic Reports Demographic Health Surveys (DHS)—DHS surveys produce information on Level A (health status and quality of life), Level B (behavior) and population characteristics, but provide incomplete information related to Level C (opportunity, ability and motivation) and Level D (exposure). DHS surveys are thus most helpful in monitoring behavior changes by need category. The PSI Team will conduct secondary analyses of the upcoming 2006 Benin DHS to define baseline value for behavioral indicators such as condom use among youth, abstinence and partner reduction, CPR and ORS use among children with diarrhea in target areas. PSI will also compare the findings of the 2006 with the 2011 DHS as an independent source of monitoring overall program goals and purposes. Behavioral Surveillance Surveys (BSS)—A BSS provides information on Level B (behavior), but provide incomplete information related to Levels C and D. The BSS is thus most helpful in setting baseline levels of behavior and, if conducted again, for monitoring changes over time. Two BSS were conducted in 2001 and 2006 among youths, truckers and sex workers. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 207 Once available, results of the 2006 BSS will be used to set baseline values for behavioral indicators such as condom use, abstinence and partner reduction among high risk groups. Planning for additional BSS surveys is not now known. However, if conducted, results of subsequent BSS will serve as independent source for monitoring program objectives for high risk groups. The PSI team will work with the CNLS to leverage funding from other sources to accomplish the national-level BSS every two years. PSI will advocate and provide technical assistance toward achieving this goal. Tracking Surveys—The tracking survey complements the DHS and BSS by providing detailed Level C and D data important to social marketing segmentation, monitoring and evaluation. The PSI Team will conduct bi-annual tracking surveys that monitor progress on B, C, and D level indicators for HIV/AIDS/STI, Family Planning, Malaria Prevention and Diarrheal Diseases prevention-related activities. The PSI Team will also conduct two tracking surveys within the ITN ANC intervention zones, and two or more tracking surveys among truckers and sex workers for purposes of segmentation, monitoring and evaluation according to PERForM. In the event that additional BSS are not done and given consultation with stakeholders a final tracking survey may be conducted among these high-risk groups as part of a final evaluation of program impact. TRaC-M Surveys—These surveys will be done annually to monitor changes in reach and recall of mass media interventions addressing knowledge, attitudes, and behavior. TRaC-M surveys will be particularly useful in the context of IMPACTS as the program will significantly increase behavior change communications through community radio stations across the country. Service Delivery Surveys—Service delivery surveys produce monitoring information on coverage, quality of care and service delivery and equity of access. These Level C opportunity measures are known to be correlated with behavior. This survey is used as a management tool to facilitate monitoring of sales and service operations. During the project period, PSI will develop a GIS-based software application to make such monitoring routine. PSI may use quality assurance surveys, including mystery client and exit interviews as required to monitor service delivery performance. These may be quantitative or qualitative, depending on decision-making requirements. Qualitative Research—Qualitative research will be used for concept development, pre￾testing communications materials, promotional items and, if necessary, exploring questions raised in segmentation. The strategy will depend on the topic and could result in the use of key informant, focus group, in-depth interview or peer ethnographic research. For example, the PSI Team anticipates the use of exploratory research techniques to better understand clients‘ perspectives on various issues such as perception of hormonal contraceptives, trusted partners issues on HIV prevention, etc. Programmatic Reports—Regular reporting on programmatic activities will be produced and disseminated to key partners. This will allow for increased harmonization of activities and knowledge-sharing. Program reporting will be the primary tool for monitoring and evaluation progress of activities outlined under Objective 4. In addition to these more punctual monitoring and evaluation activities, the PSI Team will also monitor all activities via systematic record-keeping at all program levels. All behavior change communications activities will keep records of the number of people reached per activity, the location, and topics covered. Sales visits will be monitored through fixed objectives agreed upon by the sales/promotion agent and his/her supervisor and based on information gathered through Service Delivery Surveys. In ProFam, PSAMAO, and the integrated service sites the PSI Team will implement a Standards-Based Management Recognition (SBM-R) strategy to guide the 208 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES integration of new services, efficient client flow, and monitoring the progress of services, described in detail under Objective 3. Performance standards form the basis for the organization and functioning of services, as well as for rewarding compliance with standards through recognition mechanisms. Mid-Term and End of Project Review—An external evaluator will conduct a mid-term and end of project review. The scope of the review will be agreed upon with USAID. Monitoring and evaluation data and reports will be provided by PSI. The following table includes a list of proposed key performance indicators to be used in monitoring and evaluation the project‘s progress. Final indicators will be agreed upon with USAID/Benin‘s Cognizant Technical Advisor in line with the Mission‘s Results Framework and Performance Management Plan. The PSI Team will disseminate the results of all relevant research activities, such as Tracking Surveys, TRaC-M Surveys, and Service Delivery Surveys, among its partners and other stakeholders. Furthermore, whenever possible, the PSI Team will collaborate on KAP and other quantitative and qualitative surveys undertaken by partner organizations to measure baseline or progress on key indicators as appropriate. In particular, the PSI Team will collaborate with PISAF on any quantitative and qualitative research activities to measure key indicators in the public sector. An example of this collaboration among partners has been, and will continue to be, PSI‘s collaborations on conducting Behavioral Surveillance Surveys (BSS) in Benin. PSI proposes to continue its role in leading the qualitative component of the BSS survey, and we will rely heavily on the results of the BSS‘s quantitative component to measure, evaluate and guide our targeted prevention activities. Additionally, under the guidance of the program‘s Research Coordinator, the social marketing program will carry out in-depth analyses of other existing research such as the Demographic and Health Study (DHS) to ensure that our staff has the most up-to-date information as they design and adjust their interventions. ILLUSTRATIVE INDICATORS AND TARGETS The following tables present the PSI Team‘s sales objectives and indicators over the life of the project. We look forward to finalizing these indicators and targets in collaboration with USAID upon award. Product Sales Objectives Table Products / Services Sales target YR1 YR2 YR3 YR4 YR5 Prudence Plus (unit = condom) 9,400,000 9,870,000 10,363,500 10,881,675 11,425,759 Kit ISTOP (unit = kit) 2,000 2,100 2,205 2,315 2,431 Harmonie (unit = plaquette) 130,000 136,500 143,325 150,491 158,016 Equilibre (unit = dose) 22,500 23,625 24,806 26,047 27,349 Orasel (unit = sachet) 1,350,000 Orasel (unit=sachet; bundeled with zinc) 400,000 1,837,500 1,929,376 2,025,846 2,127,138 Zinc blisters (unit = blister of 10) 200,000 918,750 964,688 1,012,923 1,063,569 Super Moustiquaire (unit = LLIN) 35,000 30,000 25,000 25,000 20,000 Bonne Maman Zou/Colline (unit = LLIN) 75,000 50,000 30,000 30,000 30,000 Bonne Maman Atacora/Donga (unit = LLIN) 110,000 80,000 55,000 55,000 50,000 ACT – PPT (unit = treatment) 0 10,000 66,000 99,000 118,800 Cyclebeads (unit = beads) 10,000 11,500 12,075 12,679 13,313 Amour & Vie (unit = magazine) 200,000 240,000 280,000 320,000 360,000 Monitoring and Evaluation Table RESULT PERFORMANCE INDICATOR BASELINE DATA FISCAL YEAR Source￾reporting d (FY) 2010 USAID Framework SO2 Indicatora Unit of measure Disaggregation Baseline year b Baseline year b Target c Actual Objective 1: Improve availability and access by key populations to products related to HIV￾AIDS, RH-FP, CS, including malaria IR 2 Coverage: Geographic zones with at least one sales point that sells socially-marketed products % Product 2006 Service Delivery Survey IR 2 Coverage: Hot zones (HORECA) where socially￾marketed condoms are available either on the premises or with five minutes walking distance % 2006 Service Delivery Survey IR 2 Distribution: Number of socially-marketed products sold (see details in table below) Nbr 2006 Program MIS IR 2 Coverage: Targeted health centers where socially￾marketed LLINs are available to pregnant women and children under 5 years Nbr 2006 Service Delivery Survey Objective 2: Increase knowledge of Beninese about HIV-AIDS, RH-FP, and CS to increase demand for and use of products and services IR 3 Knowledge: Youth who know at least three means of transmission of HIV % Sex 2006 Tracking Survey IR 3 Knowledge: Youth who know at least three ways of preventing HIV infection % Sex 2006 Tracking Survey IR 3 Exposure: Youth ever exposed to Amour & Vie HIV prevention activities in the past year % Sex 2006 TRaC-M Survey IR 3 Behavior: Youth aged 15 – 24 year who are willing to abstain from sex % Sex 2006 Tracking Survey IR 3 Behavior: Sexually active youth aged 15 – 24 with only one sexual partner in the past year % Sex 2006 Tracking Survey IR 3 Female youth able to discuss condom use with partners % 2006 Tracking Survey 210 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES RESULT PERFORMANCE INDICATOR BASELINE DATA FISCAL YEAR Source￾reporting d (FY) 2010 USAID Framework SO2 Indicatora Unit of measure Disaggregation Baseline year b Baseline year b Target c Actual IR 3 Exposure: Members of high￾risk groups ever exposed to PSI‘s BCC activities % Age, high-risk group 2006 TRaC-M Survey IR 3 Behavior: Members of high￾risk groups who are correctly and consistently using condoms for commercial sex % Age, high-risk group 2006 Tracking Survey IR 3 Intention: Members of high￾risk groups who indicate a desire to get VCT/STI services % Age, high-risk group 2006 Tracking Survey IR 3 Knowledge: Men and Women of Reproductive Age (M/WRA) who know at least three modern methods for family planning % Age 2006 Tracking Survey IR 3 Knowledge: M/WRA who can identify 3 benefits of birth spacing % Age Sex 2006 Tracking Survey IR 3 Exposure: M/WRA who can cite ProFam clinics as place to receive quality care % Sex & Age 2006 TRaC-M Survey IR 3 Behavior: Couples accessing services at ProFam clinics Nbr Age 2006 Tracking Survey IR 3 Social support: M/WRA who report having discussed FP with their partners over the past year % Age, Sex 2006 Tracking Survey IR 3 Behavior: People using condom for dual protection % Age & Sex, high risk group 2006 Tracking Survey IR 3 Behavior: WRA who are currently using a modern FP method % Age, method 2006 Tracking Survey IR 3 Knowledge: Mothers of children under five years who can cite ORS as a means for preventing diarrhea – related de hydration % 2006 Tracking Survey IR 3 Exposure: Caregivers exposed to integrated malaria and diarrheal disease prevention messages % 2006 TRaC-M Survey USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 211 RESULT PERFORMANCE INDICATOR BASELINE DATA FISCAL YEAR Source￾reporting d (FY) 2010 USAID Framework SO2 Indicatora Unit of measure Disaggregation Baseline year b Baseline year b Target c Actual IR 3 Behavior: Children under 5 with diarrhea treated with ORS and zinc % 2006 Tracking Survey IR 3 Knowledge: Household head who can cite ITNs as a means for preventing malaria % 2006 Tracking Survey IR 3 Behavior: Pregnant women and children who sleep under ITN % Pregnant women & children 2006 Tracking Survey Objective 3: Promote and support implementation of best practice integrated services IR 1 Persons trained in VCT, PMTCT, FP, and STI‘s Nbr 2006 Program reports IR 1 Clients receiving VCT, PMTCT, FP, and STI services Nbr 2006 MIS Reports IR1 Persons referred for care and treatment Nbr 2006 MIS Reports IR1 Performance standards attained at service delivery sites % 2006 Program reports￾annual IR 1 Sites receiving accreditation and recognition Nbr 2006 Program reports IR 1 & 3 Exposure: people able to identify IMPACTS center sites for integrated services % 2006 TRaC-M Survey IR 1 Individuals receiving services at pilot facilities Nbr 2006 MIS Reports IR 1 Persons receiving HIV test results at pilot facilities Nbr 2006 MIS Reports IR 1 Male partners receiving VCT and PMTCT services % 2006 MIS Reports Objective 4: Strengthen the capacity of the MOH and the GOB (including the PNLS and CNLS) IR 1 HIV-AIDS Policy support strategy adopted and implemented Doc 2006 Program Reports IR 1 Advocacy implementation strategy in place and implemented Doc 2006 Program Reports IR 1 COGEA‘s participating in management and inventory of public health facility products % 2006 Program Reports 212 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES RESULT PERFORMANCE INDICATOR BASELINE DATA FISCAL YEAR Source￾reporting d (FY) 2010 USAID Framework SO2 Indicatora Unit of measure Disaggregation Baseline year b Baseline year b Target c Actual IR 1 COGEA‘s with women on the executive bureau % 2006 Program Reports IR 1 PMTCT protocols and ART guidelines disseminated Doc 2006 Program Reports IR 1 AIM model implemented and results disseminated Doc 2006 Program Reports IR 1 Sentinel epidemiological surveillance data disseminated Doc 2006 Program Reports IR 1 GFATM resources leveraged Doc 2006 Program Reports IR 1 PNLS operations research capacity strengthened Doc 2006 Program Reports Notes: a) Upon award, indicators will be discussed with CTO to ensure they are aligned with the Mission‘s Results Framework and the indicators USAID-Benin is using to manage and document its performance b) All baseline data will be gathered at the beginning of project in order to measure IMPACTS project performance over time c) Target values will be proposed by PSI based on baseline value and discussions with CTO Surveys will be designed to provide comparison between target intervention sites and national data. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 213 ANNEX J. PISAF PERFORMANCE MONITORING PLAN (INDICATORS LEVEL FY'07) Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 1. Utilisation du paquet de services de santé Familiale % de la population ayant accès (recevant au moins un élément du ) paquet de services de santé familiale dans la zone ciblée 24% 1. la planification familiale=5% 2. les soins prénatals=170% 3. l‘accouchement=5 7% 4. la prise en charge intégrée des maladies de l‘enfant (PCIME)=7% 5. les soins postnatals=52% 6. les soins obstétricaux et néonatals d‘urgence (SONU)=5% 7. les soins curatifs=22% 8.1 le dépistage volontaire VIH=2% 8.2 le dépistage/la prise en charge des IST=0,2% 9.1 la vaccination enfants=203% 9.2 la vaccination femmes=123% 10. le suivi de la 1. la planification familiale=9% 2. les soins prénatals=194 % 3. l‘accouchement =60% 4. la prise en charge intégrée des maladies de l‘enfant (PCIME)=12% 5. les soins postnatals=59% 6. les soins obstétricaux et néonatals d‘urgence (SONU)=5% 7. les soins curatifs=16% 8.1 le dépistage volontaire VIH=0% 8.2 le dépistage/la prise en charge des IST=0% 9.1 la vaccination 1. la planification familiale=8% 2. les soins prénatals=217 % 3. l‘accouchement =70% 4. la prise en charge intégrée des maladies de l‘enfant (PCIME)=21% 5. les soins postnatals=62% 6. les soins obstétricaux et néonatals d‘urgence (SONU)=8% 7. les soins curatifs=25% 8.1 le dépistage volontaire VIH=2% 8.2 le dépistage/la prise en charge des IST=0% 9.1 la vaccination 60% 8% Rapports SNIGS des CS x 214 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment croissance=41% 11. l‘IEC/CCC=56% 12. la PTME=22% 13.1 le TPI (Nombre de femmes enceintes ayant reçu la 2ème dose de SP)=11% 13.2 MILD (Nombre de femmes enceintes ayant acquis une MIILD)=10% enfants=384% 9.2 la vaccination femmes=57% 10. le suivi de la croissance=59 % 11. l‘IEC/CCC=41 % 12. la PTME=41% 13.1 le TPI (Nombre de femmes enceintes ayant reçu la 2ème dose de SP)=28% 13.2 MILD (Nombre de femmes enceintes ayant acquis une MIILD)=40% enfants=460% 9.2 la vaccination femmes=88% 10. le suivi de la croissance=61 % 11. l‘IEC/CCC=50 % 12. la PTME=37% 13.1 le TPI (Nombre de femmes enceintes ayant reçu la 2ème dose de SP)=33% 13.2 MILD (Nombre de femmes enceintes ayant acquis une MIILD)=52% USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 215 Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 3. Couple Années Protection Nombre de couples (en âge de reproduire) protégés contre une grossesse et utilisant des méthodes de planning familiale durant une année á partir de la quantité totale de méthodes contraceptives cédées ou distribuées au cours de la même année. 7,371 16,182 13,889 12,533 18000 SNIGS x Les ruptures au niveau des points de ravitaillement (CAME et dépôts répartiteurs) expliquent en partie cette situation IR 1 Amélioration de l'environnement politique IR 1.1 Mise en oeuvre de politiques et approches de santé selectionnées 1.1.1 Révision et Dissémination des Politiques Nombre de nouvelles politiques crées disseminées et ventillées aux départements et aux zones sanitaires 2 0 0 1 1 Rapports de PISAF x x x x IR 1.2 Les capacités de gestion du système sanitaire sont accrues 216 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 1.2.1 Renforcement de la gestion aux niveaux Régional et Zonal % de gestionnaires du système sanitaire aux niveaux départemental et zonal formés aux éléments de base de gestion (gestion des ressources humaines, financières et matérielles et assurance qualité) 0% 99% 99% 100% 100% Base de données des formatio ns (PISAF) x x 1.2.2 Système de Supervision mis en oeuvre % de centres de santé ayant recus au moins quatre visites de supervision formative par an 37% 69% 81% 75% 85% Cahiers de supervisi ons des centres de santé x x 1.2.3 Inventaire des Produits de santé Familiale % de COGEC prenant part a l'inventaire de tous les produits de santé Familiale reçus 94% 92% 94% 92% 100% Cahiers des COGECS x 1.2.4 Processus de Planification Ascendante mis en oeuvre effectivement % de EEZS utilisant la planification ascendante pour élaborer leur plan d'action 100% 100% 100% 100% 100% Rapports d'activité s des Zones sanitaires x x USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 217 Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 1.2.5 Plans d'Action Annuels des EEZS mis en oeuvre % de EEZS exécutant au moins 75% de leur plan annuel 78% 92% 96% 96% 100% Rapports d'activité s des Zones sanitaires x 1.2.6 Utilisation des données pour la prise de décision mise en oeuvre par les EEZS % de EEZS organisant au moins 4 sessions de prise de décision par an basée sur les données 89% 92% 100% 88% 100% Rapports trimestri els des Zones Sanitaires x IR 1.3 La participation de la société civile est accrue 1.3.1 Participation des femmes aux Structures de la Societe Civile % de COGEC avec une femme dans le bureau exécutif 42% 37% 37% 37% 60%(re vu à 37%) Repertoi re des COGECS /zone saniatire x 1.3.2 Respect des réunions des COGEC % de centres de santé avec COGEC tenant au moins une réunion mensuelle assortie de rapport 47% 43% 50% 52% 60% Rapports des COGEC et des EEZS x 1.3.3 Intégration des activités de santé dans les efforts des OBC % d'OBC ayant mené au moins une activité de santé au cours de l'année 45% 100% 100% 100% 100% Rapports des OBC et Rapport d'activité du PISAF x x x 218 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 1.3.4 Activités à base communautaire sont integrées dans les services du système sanitaire publique % de ZS ayant inclu les activités à base communautaire dans les descriptions de postes d'agents de santé selectionnés 33% 50% 100% 100% 100% Rapports d'activité s des EEZS x IR 2 L'accès aux services et produits de Santé Familiale de qualité est accrue IR 2.1 Disponibilité des produits essentiels de santé familiale dans les formations sanitaires publiques 2.1.1 Nombre moyen de jours de rupture de stocks pour des médicaments sélectionnés survenus dans les Centres de santé au cours du trimestre Nombre moyen de jours de rupture de stock survenu trimestriellement pour des produits sélectionnés de Santé Familliale (CTA, Ocytocine, MILD, SRO, contraceptifs oraux et injectables ne sont pas disponibles pour les patients et les prestataires de services) 9 10 9 9 5 Fiche de stock et REMECA R x USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 219 Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 2.1.2 Disponibilité des Produits de Santé Familiale Index de la gestion de la commande des Produits de Santé Familiale 50% 78% 87% 79% 90% Fiche de stock et Bon de Comman de x 2.1.3 Disponibilité des Produits de Santé Familiale pour le niveau Communautair e % villages ou les produits du paquet de santé familliale pour niveau communautaire est disponible 17% ND 93% 100% 40%(re vu à 95%) Rapports de supervisi on des ASBC par les centres de santé x IR 2.2 Disponibilité de produits de SF sélectionnés au niveau de vendeurs privés 2.2.1 Disponibilité des MILD à travers les groupes de la Société Civile # MILD vendus à travers les OBC et les Mutuelles de santé 0 ND 9775 8000 10000 Rapports du PISAF x x x IR 2.3 Disponibilité du paquet de Santé Familiale dans des formations sanitaires publiques sélectionnées. 220 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 2.3.1 Score de l'offre du paquet de Service de santé Familale par des agents de santé formes Score Moyen de la performance des agents de santé formés dans l'offre d'au moins un ou plusieurs éléments du paquet de Services de Santé Familiale (il y a au total 11 elements du paquet de santé Familiale dont certains sont appropriés seulement à certains groupes démographiques comme les femmes ou les enfants) ND ND ND EQGSS x IR 2.4 L'Accés financier aux services de Santé est accru 2.4.1 Adhesion aux Mutuelles de santé Proportion de la population adhérant aux Mutuelles de santé dans la zone cible 14% 16% 12% 13% 25% Régistres d'adhésio n des Mutuelles x x le taux de pénétration des mutuelles dans les ménages est autour de 10% sur le plan national, L'objectif de 25% fixé au début de l'année était trop optimisste, USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 221 Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 2.4.2 Perennisation et Continuite de l'adhesion au Mutuelles de santé % de mutuelles conservant au moins les 2/3 des adhérents de l'année précedente 36% 57% 23% 6% 90% (revu à 40%) Rapports du PISAF x x Les mutuelles enregistrent beaucoup de nouveaux membres qui cotisent pendant quelques mois puis relâchent malgré leur bonne volonté. Les conditions socio￾économiques difficiles des ménages expliquent en partie cette situation. La période de disette est celle où on enregistre le plus de relâchement. Ce phénomème est beaucoup plus observé dans les nouvelles mutuelles où les populations adhèrent massivement au début de leur 222 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment création. 2.4.3 Distribution des Mutuelles de santé à travers la zone ciblee % de communes/arrondi ssements de la zone cible ayant des mutuelles fonctionnelles 21% 79% 90% 83% 100% Rapports du PISAF x x IR 3 La demande des Services, des Produits et mesures Préventives de Santé Familiale est accrue 3.1 Distribution des produits contraceptifs Nombre de produits contraceptifs distribués/cédés dans les structures sanitaires publiques 10,713 16,576 15,667 16,810 23,000 Rapports des centres de santé x USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 223 Les indicateurs surlignés en jaune sont ceux du PMP de l'USAID Définition de l'indicateur Niveaux Source de données Regions Evaluees Commentair es, suppositions Nom de l'indicateur 2007 2008 Actual 2009 Actual 2010 Target 2010 Target 2011 Nat'l B/ A Z/ C O/ P Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment 3.2 Accès au Paquet de Santé Familiale Nombre de personnes ayant accès au paquet de services de santé familiale dans la zone ciblée 317,851 460,275 516,498 598,061 560,000 Registres des Centres de santé x IR 3.1 Amélioration des connaissances, des comportements appropriés et des mesures préventives IR 3.2 Introduction d'interventions et de services appropriés basés sur la recherche 3.2.1 Etudes de Recherches menées # études de recherche formative, operationelle et évaluative realisées et les résultats partagés avec tous les acteurs 3 3 0 1 1 Rapports du PISAF x x x 224 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 225 ANNEX K. USAID BENIN—IMPACT PROJECT PERFORMANCE MONITORING PLAN Y4, Fourth Quarter Indicator Definition Targets* Target Actual % Data source Target Indicator Notes Number Primary Implementing Partner Indicator Name FY 2006 (Baseline) CUMUL 2010 CUMUL 2010 CUMUL 2010 Geographic Areas USAID Strategic Objective 2: Expanded use of family health services, products and preventive measures within a supportive environment IR 1 A Supportive Implementation Environment Created IMPACT Objective 4 - Strengthen the capacity of the MOH and GOB, including the PNLS and CNLS IR 1.1 Selected health policies and approaches implemented 1.1.1 Abt Associates Policy Development Assistance Indicator Number of HIV/AIDS national policies/protocols revised or disseminated to National Stakeholders 5 18 29 161% IMPACT reports, MOH and CNLS Minutes and reports National 1.1.2 Abt Associates Technical Knowledge Training Indicator in Epidemiological and Operation research Number of dissemination of workshops in area of epidemiological surveillance systems n/a 12 13 108% IMPACT Project and MOH reports National 1.1.3 Abt Associates Advocacy Efforts to Policy development Indicator Number of Advocacy efforts produced to support policy development (such as HIV website and information resources,media reports, educational campaigns public pronouncements of political leaders, global/regional conferences, and other advocacy tools) 0 12 22 183% MOH, GOB, IMPACT, Media National IR 1.2 Increased health system management capacity created 1.2.1 Abt Associates Technical Coordination Committee Capacity Building Indicator Number of quarterly review presentations made by the Technical commitees of coordination of PNLS and CNLS 2 44 38 86% IMPACT Reports National 1.2.2 Abt Associates Training indicator on Management and Planning Number of CLNS /PNLS members trained in management and planning 0 2 0 0% Project, CNLS/PNLS reports National Activités annulées 226 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 1.2.3 Abt Associates CNLS Capacity Building Indicator Numbers of administrative documents/ operating procedures developed by CNLS 1 6 8 133% Project, CNLS/PNLS reports National IR 1.3 More effective civil society participation created 1.3.1 Abt Associates Civil Society Engagement Indicator Number of technical consultations with stakeholders (including GOB, IMPACT, NGOs, donors, PLWHA organizations, and others) done by CNLS Technical Coor. Committee to create a supportive policy environment 0 14 40 286% CNLS Tech. Coor. Committees, IMPACT reports National Multiplication des partenaires du CNLS et PNLS 1.3.2 Abt Associates Private sector Commitment Indicator Number of additional private sector companies that commit to action against HIV/AIDS in their workplace by joining a Business Coalition against HIV/AIDS. 9 12 49 408% Business sector, IMPACT, CNLS, PNLS National Intérêts croisssants des entreprises à s'activer dans les réseaux 1.3.3 Abt Associates Donors Funds Leveraging Assistance Indicator Number of costing and planning exercises used by CCM to facilitate policy decision-making and leverage donors resources N/A 4 11 275% CCM reports, IMPACT report National Document de restructuration revisité, amendements intégrés document final de Restructuration disponible et et effort d'appui du 10ème ROUND. IR 2 Access to Quality Services and Products Increased IMPACT Objective 1 - Improve availability and access by key populations to products related to HIV/AIDS, RH-FP, CS (including malaria) IR 2.1 Selected products available at public health facilities IR 2.2 Selected products available at private sector outlets 2.2.1 % of geographic areas with at least one sales point for: Geographic area is defined as 1 village in rural areas, a neighborhood (quartier) in urban areas (for condom, and Orasel) an arrondissement (for ITN, Harmonie, Equilibre and Collier du Cycle) 2.2.1.1 PSI Prudence Plus Condoms 77% 90% 90% 100% LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 227 de la couverture en l'an 5. 2.2.1.2 PSI Harmonie Oral Contraceptives 48% 75% 61% 81% LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité de la couverture en l'an 5. 2.2.1.3 PSI Equilibre Injectable Contraceptives 45% 75% 66% 87% LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité de la couverture en l'an 5. 2.2.1.4 PSI Orasel Rehydration Salts 52% 65% N/A N/A LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité de la couverture en l'an 5. 2.2.1.5 PSI Orasel/Zinc bundles TBD 55% 58% 105% LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité de la couverture en l'an 5. 2.2.1.6 PSI SuperMoustiquaires LLITN 41% 49% 42% 87% LQAS Results National L'enquête MAP a été faite; le niveau de couverture est acceptable mais des efforts supplémentaires sont en cours pour améliorer la qualité 228 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES de la couverture en l'an 5. 2.2.1.7 PSI Bonne Maman LLITN Donga N/A N/A N/A Produit non disponible 2.2.1.8 PSI ACT PPT treatments TBD N/A N/A N/A LQAS Results National Produit non disponible 2.2.1.9 PSI Collier du Cycle Cycle Beads 42% 70% 52% 75% LQAS Results National Faible demande, manque de sensibilisation des prestataires IR 2.3 Quality family health package available at targeted public health sector facilities IMPACT Objective 3 - Promote and support implementation of best practice intergrated services 2.3.2 Integrated Services in the Public Sector 2.3.2.1 JHPIEGO Number of public sector facilities with improved integrated HIV/AIDS prevention services Total number of public sector facilities with improved integrated HIV/AIDS prevention services including PMTCT, VCT, FP and STI services) 0 10 12 120% Project Records Borgou, Zou/Collines Glazoué, Kalalé et 10 autres cliniques de l'état qui travaillent avec IMPACT 2.3.2.2 JHPIEGO Number of persons trained in VCT, PMTCT, FP, or STIs Total number of persons trained in VCT, PMTCT, FP, or STIs disaggregated by content area 60 240 500 208% Activity Reports Borgou, Zou/Collines, Atlantique/Littoral et Ouémé 2.3.2.3 JHPIEGO Number of individuals who received counseling and testing, and who received their test results Total number of clients who received pre-test counseling, test results, and post test counseling at integrated service delivery sites 0 na 8,931 na Clinic records Borgou, Zou/Collines, Atlantique/Littoral et Ouémé 2.3.2.3a JHPIEGO Number of pregnant women who received HIV counseling and testing for PMTCT and received their test results Number of pregnant women who received HIV counseling and testing for PMTCT and received their test results na 4,647 na 2.3.2.4b JHPIEGO Percent of pregnant women who received HIV counseling and testing for PMTCT and received their test results Divide total number of pregnant women who received HIV counseling and testing, test results and post-test counseling at targeted sites by total of number of pregnant women coming to PCN 0 100% 100% 100% Clinic Records Borgou, Zou/Collines et Atlantique/Littoral 2.3.2.5 JHPIEGO Total number of HIV positive pregnant women receiving ARV prophylaxis or being referred for ARV prophylaxis for PMTCT Total number of HIV positive pregnant women receiving ARV prophylaxis at target site or being referred to another site for ARV prophylaxis 0 50 38 76% Clinic Records Borgou, Zou/Collines et Atlantique/Littoral 2.3.2.6 JHPIEGO Total number of HIV positive women accepting a family planning method Total number of HIV positive women who want to become mothers and who accept a family planning method NA na 12 Clinic Records Borgou, Zou/Collines et Atlantique/Littoral USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 229 2.3.2.6b JHPIEGO Total number of HIV positive for PMTCT Total number all HIV positive women NA na 27 2.3.2.6c JHPIEGO Percent of HIV positive women accepting a family planning method Total number of HIV positive women who want to become mothers and who accept a family planning method divided by all HIV positive women NA na 44% Clinic Records Borgou, Zou/Collines et Atlantique/Littoral 2.3.2.7a JHPIEGO Total number of HIV￾infected individuals Total number of individuals tested HIV￾positives TBD 83 2.3.2.7b JHPIEGO Percent of HIV-infected individuals who were referred for care support and treatment or receving traitment Total number of individuals identified as being HIV infected who were referred to wellness or treatment services TBD 100% 100% 100% Clinic Records Borgou, Zou/Collines et Atlantique/Littoral 2.3.2.8 JHPIEGO Number of health facilities achieving 80% of performance standards Total number of health facilities achieving 80% of performance standards NA 10 2 20% performance standard talley sheets Borgou, Zou/Collines IR 2.4 Financial access to health services increased IR 3 Demand for Health Services, Products, and Preventive Measures Increased IMPACT Objective 2 - Increase knowledge of Beninese about HIV-AIDS, RH-FP, and CS to increase demand for and use of products and services IR 3.1 Knowledge of appropriate behaviors and preventive measures improved 3.1.2 PSI Social Marketing Product Sales Sales of social marketing products at the wholesaler level. 3.1.2.1 Prudence Plus Condoms 8,825,280 ######### 34,461,504 86% PSI MIS National 3.1.2.3 Harmonie Oral Contraceptives 136,860 588,332 752,180 128% PSI MIS National 3.1.2.4 Equilibre Injectable Contraceptives 19,860 86,203 136,016 158% PSI MIS National 3.1.2.5 Orasel Rehydration Salts 1,198,071 880,000 794,344 90% PSI MIS National 3.1.2.6 Orasel/Zinc bundles N/A 2,638,616 810,819 31% PSI MIS National Objectifs surestimés et révisés en cours de mise en oeuvre 3.1.2.7 SuperMoustiquaires LLITN (PMI° 24,425 152,500 201,436 132% PSI MIS National 3.1.2.8 LLITN (others) 9,325 61,799 48,600 79% PSI MIS National 3.1.2.9 ACT PPT treatments N/A - - 3.1.2.10 Collier du Cycle Cycle Beads 6,630 21,807 7,001 32% PSI MIS National Produit de plus en plus pas très accepté 3.1.2.11 Amour et Vie Magazines 162,310 627,000 612,247 98% PSI MIS National 3.1.3 PSI Increase knowledge of HIV-AIDS 230 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 3.1.3.1 PSI Abstinence Students (Youth) % of non-married students aged 15 - 24 who say they report their first sexual relations after 15 years, disaggregated by gender Boys: 66, 0%; Girls: 80, 5% Boys: 71, 9% Girls: 84, 5% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.2 PSI Abstinence apprentices (Youth) % of non-married apprentices aged 15 - 24 who say they report their first sexual relations after 15 years, disaggregated by gender Boys: 72, 3% Girls: 76, 1% Boys: 69, 8% Girls: 74, 9% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.3 PSI Students Median age at firt sexual relation (Youth) Average minimum age that students aged 15-24 report having first sex, dissagregated by gender. Boys: 15, 9 years Girls: 16, 4 years Boys: 16, 6y Girls: 17, 4y TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.4 PSI Apprentices Median age at firt sex apprentices (Youth) Average minimum age that apprentices aged 15-24 report having first sex, dissagregated by gender. Boys: 16, 7 years; Girls: 16, 5 years Boys: 16, 9y Girls: 16, 6y TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.5 PSI Being Faithful Students (Youth) % of students aged 15 - 24 who are sexually active and report having had only one partner during the last 12 months, disaggregated by gender Boys: 45, 8% Girls: 77, 7% Boys: 40, 2% Girls: 67, 0% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.6 Being Faithful apprentices (Youth) % of apprentices aged 15 - 24 who are sexually active and report having had only one partner during the last 12 months, disaggregated by gender Boys: 37, 8% Girls: 62, 7% Boys: 34, 6% Girls: 60, 0% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 231 3.1.3.7 PSI Consistent Condom Use Students (Youth) with regular partners % of sexually active students aged 15 - 24 who report using condoms consistently and correctly with regular partners durint the last 12 months, disaggregated by gender Boys: 27, 4% Girls: 29, 1% Boys: 49,6% Girls: 36,5% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.8 PSI Consistent Condom Use Appprentices (Youth) with regular partners % of sexually active students aged 15 - 24 who report using condoms consistently and correctly with regular partners during the last 12 months, disaggregated by gender. Boys: 33, 1% Girls: 21, 3% Boys: 43, 1% Girls: 38, 4% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.9 PSI Consistent Condom Use Students (Youth) with casual partners % of sexually active students aged 15 - 24 who report using condoms consistently and correctly with casual partners during the last 12 months, disaggregated by gender. Boys: 7, 2% Girls: 24, 1% Boys: 51, 6% Girls: 50, 1% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.10 PSI Consistent Condom Use Appprentices (Youth) with casual partners % of sexually active students aged 15 - 24 who report using condoms consistently and correctly with casual partners during the last 12 months, disaggregated by gender. Boys: 39, 3% Girls: 17, 3% Boys: 38, 7% Girls: 44, 0% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.11 PSI Condom use at the last sex Students (Youth) of sexually active students aged 15 - 24 unmarried who report using condoms at the last sex , disaggregated by gender Boys: 60, 0% Girls: 60, 5% Boys: 55, 4% Girsl: 58, 9% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.12 PSI Condom use at the last sex Apprentices (Youth) of sexually active apprentices aged 15 - 24 unmarried who report using condoms at the last sex, disaggregated by gender Boys: 46, 9% Girls: 42, 9% Boys: 45, 9% Girls: 1, 1% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.13 PSI Consistent Condom Use (Uniformed Services) % of men and women in uniform who report using condoms consistently and correctly with casual partners during the last 12 months., disaggregated by gender. NA NA NA NA TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.14 PSI Consistent Condom Use (Truckers) with regular non spouse partners % of truckers who report using condoms consistently and correctly with regular non spouse partners during the last 12 months.. 24 31 TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 232 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 3.1.3.15 Consistent Condom Use (Truckers) with casual partners % of truckers who report using condoms consistently and correctly with casual partners during the last 12 months. 43 48 TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.3.16 PSI Systematic Condom Use (Sex Workers) % of Sex Workers 15-29 ans who report systematic using condoms with clients during the last seven days. 91 93 TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.4 PSI Increase knowledge of RH-FP 3.1.4.1 PSI Modern methods % of women of child-bearing age who use a modern or natural method of contraception, disagregatted by oral contraceptives, injectable contraceptives, and Cycle Beads. Oral =7,9% Inject = 4,5% CB=2,3% Oral =7,6% Inject = 9,5% CB=0,8% IUD =16,3% Jad =65,6% TRAC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.4.2a PSI Male participation Numbers of new FP clients who participated in FP counselling sessions at ProFam or integrated services clinics. NA 3200 7463 Clinic Records Communities with Clinics Effet de la campagne sur l'implication des hommes à la PF 3.1.4.2b PSI Male participation Numbers of men- of all new FP clients￾who participated in FP counselling sessions at ProFam or integrated services clinics. NA 64 1794 Clinic Records Communities with Clinics Effet de la campagne sur l'implication des hommes à la PF 3.1.4.2.c PSI Male participation Percent of males - of all new FP clients with or without female partner - who participated in FP counselling sessions at ProFam or integrated services clinics. NA 2% 24% Clinic Records Communities with Clinics Effet de la campagne sur l'implication des hommes à la PF 3.1.4.3a PSI Adoption of modern methods of FP Number of new clients at ProFam or integrated services clinics who adopt oral contraceptives NA 400 335 84% Health center Records Communities with Clinics 3.1.4.3b PSI Adoption of modern methods of FP Number of new clients at ProFam or integrated services clinics who adopt injectables contraceptives NA 800 1129 141% Health center Records Communities with Clinics USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 233 3.1.4.3c PSI Adoption of modern methods of FP Number of new clients at ProFam or integrated services clinics who adopt Cycle Beads NA 100 112 112% Health center Records Communities with Clinics 3.1.4.3d PSI Adoption of modern methods of FP Number of new clients at ProFam or integrated services clinics who adopt IUD NA 3000 1054 35% Health center Records Communities with Clinics 3.1.4.3e PSI Adoption of modern methods of FP Number of new clients at ProFam or integrated services clinics who adopt implants NA 8250 4405 53% Health center Records Communities with Clinics 3.1.4.3f PSI Adoption of modern methods of FP Total numbers of new clients at Profam or integrated services who adopt a modern methods of FP ( oral, injectables, Cucle Beads, IUD and Implants) NA 12550 7078 56% Health center Records Communities with Clinics 3.1.5 Increase knowledge of CS 3.1.5.1 PSI Diahorrea Treatment with ORS and zinc Among children under 5 having diarrhea, % of those who received zinc during 10 to 14 days, in complement to the new ORS formulation NA 6% TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.5.2 PSI Diahorrea Treatment with ORS and zinc % of parents of of children under 5 y.o. who report giving their child oral rehydration salts and zinc when they had diahorrea in the last 3 months. NA 29% TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.5.3 PSI Awareness Diahorrea Treatment with Zinc % of care givers who report that they know to give zinc to children with diahorrea. NA 6.4% TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 234 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 3.1.5.4 PSI Awareness of Effectiveness Diahorrea Treatment with Zinc % of health care workers who repot that zinc is an effective diahorrea treatment for children under 5 . NA 96% TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.5.5 PSI Awareness - Integrated CS % of care givers who report exposure to integrated health messages on malaria and diahorrea. NA N/A N/A TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville 3.1.5.6 PSI Mosquito net usage - children % of children under age five who slept under a mosquito net the previous night. 32,0% (DHS_200 1) TRaC in Dec 2010 TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville USAID was supposed to developp the KAP itself 3.1.5.7 PSI Mosquito net usage - pregnant women % of pregnant women who slept under a mosquito net the previous night. 33,2% (DHS_200 1) TRaC in Dec 2010 TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville USAID was supposed to developp the KAP itself 3.1.5.8 PSI Mosquito net awareness % of head of households who consider mosquito net and effect method to prevent malaria. TBD TRaC in Dec 2010 TRaC Survey Results Cotonou, Abomey/Bohicon, Savalou, Parakou, Tanguiéta, Ouaké, Malanville USAID was supposed to developp the KAP itself IMPACT Objective 3 - Promote and support implementation of best practice intergrated services 3.1.6 Private Sector Integrated Services 3.1.6.1 PSI Total Number ProFam Clinics Total number of ProFam clinics offering high quality integrated services. 30 50 50 100% Project Records Abomey Bohicon et Parakou, Cotonou, Porto Novo et Calavi 3.1.6.2 OSV Total Number PSAMO Clinics Total number of integrated services clinics offering high quality services. 1 2 2 100% Project Records Parakou, Malanville 3.1.6.3 PSI/OSV Number of individuals who received counseling and testing, and who received their test results at ProFam and PSAMO clinics Total number of clients who received pre-test counseling, test results, and post test counseling at ProFam and PSAMAO clincs. 384 16817 40416 240% Clinic records Clinic sites USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 235 3.1.6.4 PSI/OSV Percentage of pregnant women who received HIV counseling and testing for PMTCT and received their test results at ProFam clinics Percentage of pregnant women who come in for CPN who received HIV counseling and testing, test results and post-test counseling at ProFam clinics. NA 30% 31.1% Clinic Records Clinic sites 3.1.6.5a PSI/OSV Total Number of HIV positive pregnant women receiving ARV prophylaxis or being referred for ARV prophylaxis for PMTCT from ProFam clinics Total Number of HIV positive pregnant women receiving ARV prophylaxis at target site or being referred to another site for ARV prophylaxis NA TBD 84 Clinic Records Clinic sites 3.1.6.5b PSI/OSV Total Number of HIV positive pregnant women from ProFam clinics Total Number of HIV positive pregnant women receiving conseiling from ProFam clinics NA TBD 391 Clinic Records Clinic sites 3.1.6.5c PSI/OSV Percentage of HIV positive pregnant women receiving ARV prophylaxis or being referred for ARV prophylaxis for PMTCT from ProFam clinics Percentage of HIV positive pregnant women receiving ARV prophylaxis at target site or being referred to another site for ARV prophylaxis NA TBD 21% Clinic Records Clinic sites 3.1.6.6 PSI/OSV Percent of HIV positive women accepting a family planning method at ProFam clinics Percentage of HIV positive women who accepted a family planning method. NA TBD NA 0 Clinic Records Clinic sites 236 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 237 ANNEX L. POUZN PERFORMANCE MONITORING PLAN: REVISED JULY 2008 Point of Use Source Strategic Objective Increased use of POU water treatment % of households with children under five ever treating drinking water using promoted methods % of household with children under five consistently treating drinking water using promoted methods % of household with children under five treating drinking water using promoted methods in last 24 hours.13 % of households with children under five correctly storing treated drinking water % of households obtaining method by source of supply Liters of drinking water disinfected with USG￾supported point-of-use products National, regional or local population based surveys National, regional or local population surveys Self-reported use based on question in HH survey National, regional or local population surveys Verified by HH residual testing during survey (chlorine specific products only) National, regional or local population surveys Verified through observation of storage vessel National, regional or local population surveys (Ever purchased and private vs. public sector source) Based on project reports and derived from sales (USAID indicator) PSI quarterly sales data IR 1 Improved Knowledge % of households that know diarrhea can be contracted from water % of households that know they need to treat their water % of households that know they need to appropriately store their water to prevent contamination % of households that can cite National, regional or local population-based surveys National, regional or local population based surveys National, regional or local population based surveys National, regional or local population based 13 Rather than self-reported treatment of drinking water, the POUZN project will work with the CDC to test the feasibility of including actual water testing for chlorine as a measure of correct use, through its household survey. 238 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Point of Use Source one or more appropriate project promoted water treatment methods/products % of households who know at least one location where they can purchase POU products surveys Question on HH survey National, regional or local population-based surveys Question on HH survey IR 2 Improved access % of consumers residing within a specified distance (may vary by country) of a water treatment product outlet Use of GIS or manual mapping using census data and facility location to map target population and outlet location or self￾reporting through population based surveys. IR 3 Enabling environment Appropriate government authority accepts registration of POU product Appropriate government authority establishes quality standards for product to be regulated Project reporting, government documents, registration of product with government Project reporting, government documents SO Indicators: The purpose of these indicators is to measure % of households, and specifically households with children under the age of five, who are correctly and consistently treating and storing their water using a promoted product. Use can be related to different water treatment alternatives that include heat, UV radiation, physical removal process and chemical treatment. Use includes ensuring that the target audience is correctly following the specified instructions for the given drinking water treatment method and water storage. Ideally this would be gathered though self reports, direct observation, and water testing. Since the main goal of the PSP IQC and the POUZN project is increased use of water treatment products distributed through the private sector, the PMP will also gather information as to where consumers are obtaining their water treatment products. While not an impact measure, surveys will, where possible, include a question that will determine the household‘s primary drinking water source. IR 1: Knowledge of POU is defined in terms of caregivers‘ awareness of the problem (that unsafe/contaminated water causes diarrhea), awareness that POU is a solution/treatment for contaminated water, knowledge of different POU options for water treatment and where to obtain them. IR 2: The purpose of this indicator is to measure physical access of the population to the promoted product. Ideally this would be measured by physical mapping. IR 3: The enabling environment IR would look to measure changes in policy, protocols or guidelines that facilitate access, knowledge or use of POU technologies and government willingness and ability to establish and monitor product quality standards. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 239 Zinc Source Strategic Objective Increased use of zinc treatment % children under five who had diarrhea in the preceding 2 weeks who were treated with zinc % children under five who had diarrhea in the preceding 2 weeks who were treated with zinc by source of zinc supply % children under five who had diarrhea in the preceding 2 weeks who were given the correct amount of zinc for age for a full course of treatment (10-14 days) % of children under five who had diarrhea in the preceding 2 weeks who were given ORS/ORT in conjunction with zinc % of providers, pharmacists, and other drug sellers (including community-based distributors) who recommend zinc treatment along with ORS/ORT as the first line treatment for diarrhea as opposed to recommending antibiotics or anti-diarrheals Number of cases of child diarrhea treated through USG-supported programs Number of people trained in child health and nutrition through USG-supported health programs National, regional or local population-based surveys Direct question on HH survey National, regional or local population-based surveys Direct question on HH survey to determine where purchased (public vs private sector source) National, regional or local population-based surveys Question on HH survey National, regional or local population-based surveys Question on HH survey Mystery Client/Provider Surveys USAID indicator—ORS and zinc sales data for private sector programs USAID Indicator— program data IR 1 Improved Knowledge % of caregivers of children under five who know that zinc needs to be administered along with ORS/ORT % of caregivers of children under five who are aware that National, regional or local population-based surveys Question on HH survey National, regional or local population-based surveys 240 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES Zinc Source zinc is an appropriate treatment for diarrhea. Question on HH survey IR 2 Improved access % caregivers of children under five who know where to obtain zinc (and by source to compare knowledge of private sector access with public sector access) % of consumers residing within a specified distance of a zinc treatment product. National, regional or local population-based surveys Question on HH survey Population-based mapping surveys IR 3 Enabling environment # of zinc products/brands registered at appropriate dosage and indication Zinc legally permitted to be sold over-the-counter Quality control standards established and regulated through appropriate government entity Incorporation of zinc into national protocols and programs for the treatment of diarrheal disease Government Reports on product registration; program reports and pictures of products with registration number located directly on product Policy documents with evidence of government approval; pictures of zinc being sold/stocked OTC; program reports Government documents; program reports Policy documents; program reports; national/sub-national protocols SO Indicators: The purpose of these indicators is to measure appropriate use of zinc for the treatment of diarrhea including use of the appropriate dosage, completion of the full 10-day regime, and use of zinc with ORT/ORS. This indicator would measure overall use as well as use from private sector sources of supply. IR 1: Knowledge is defined as awareness by both caregivers and providers that zinc is an appropriate and effective treatment for diarrhea, and how to correctly administer zinc. IR 2: The purpose of this indicator is to measure physical access of the population to the promoted product. Ideally this would be measured by physical mapping, supplemented by measurement of their knowledge of where to access product, included above with IR1. USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES 241 IR 3: Indicators for enabling environment include changes in policy, protocols or guidelines that facilitate access, knowledge and use zinc for diarrheal treatment. This indicator also measures the range of pediatric zinc products available to consumers with the assumption that competition will reduce prices and improve consumer choice and therefore use. By having zinc available over-the-counter consumers will have easy access, ideally beyond the pharmacy and wherever ORS is sold. 242 USAID/BENIN: EVALUATION OF FAMILY HEALTH ACTIVITIES For more information, please visit http://resources.ghtechproject.net Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com