STRATEGIC APPRAISAL OF THE ACQUIRE PROJECT A PARTICIPATORY REVIEW: LESSONS LEARNED WITH A LOOK TO THE FUTURE September 2007 This publication was produced for review by the United States Agency for International Development. It was prepared by Joyce Holfeld, through the Global Health Technical Assistance Project. September 2007 This publication was produced for review by the United States Agency for International Development. It was prepared by Joyce Holfeld through the Global Health Technical Assistance Project. This document (Report No. 07-001-022) is available in printed or online versions. Online documents can be located in the GH Tech website library at www.ghtechproject.com/resources/. Documents are also made available through the Development Experience Clearing House (http://dec.usaid.gov). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. STRATEGIC APPRAISAL OF THE ACQUIRE PROJECT Agreement No.: GPO-A-00-03-00006 A PARTICIPATORY REVIEW A Look to the Future SEPTEMBER 2007 Disclaimer: The authors’ views expressed in this publication do not necessarily reflect the views of the U.S. Agency for International Development or the United States Government. Cover photograph: © 2007 by M. Tuschman/EngenderHealth Strategic Appraisal of the ACQUIRE Project v The Participatory Appraisal Team Members Independent Consultant: Joyce Holfeld, Senior Advisor, Public Health Programming – Appraisal Team Leader USAID Participants Bureau of Global Health, Office of Population and Reproductive Health: Jeannie Friedmann, Global Leadership Priority Team Leader, Service Delivery Improvement (SDI) Division – Appraisal Activity Manager Mary Ann Abeyta-Behnke, Senior FP/HIV Integration Advisor, SDI Division – Access to RH/FP Services and FP/HIV Integration, Global Leadership Priorities Michal Avni, Gender Specialist, Policy, Evaluation, and Communication (PEC) Division – Fistula and Gender, Global Leadership Priorities Kovia Gratzon-Erskine, Health International Development Intern – Maximizing Access and Quality and Post-Abortion Care, Global Leadership Priorities Marguerite Farrell, Health Officer, SDI Division – Management Mai Hijazi, Senior Technical Advisor for Policy, PEC Division – Improved Policy Environment Virginia Lamprecht, Senior Technical Advisor for Monitoring and Evaluation, SDI Division – Monitoring an Evaluation and Research Kevin Pilz, Contraceptive Security Champion, Commodities Security and Logistics Division – Access to RH/F Services and Contraceptive Security, Global Leadership Priority Lois Schaefer, Senior Technical Advisor for Human Capacity Development and Training, SDI Division – Provider Performance Allison Zimmerman, Program Assistant, SDI Division – Partner and Mission Surveys and Annexes d P The appraisal took place February–May, 2007. vi Strategic Appraisal of the ACQUIRE Project ACKNOWLEDGMENTS The authors express appreciation to the members of the ACQUIRE (ACQ) management team, partner representatives, technical staff, and field staff, and especially project director Lynn Bakamjian; all provided the team with their insights and perspectives, and clearly articulated the objectives of the ACQ project and its workings. The team also is grateful for the contributions of Dana Vogel, Ellen Starbird, and Scott Radloff, USAID/Global senior managers, who generously offered their time, expertise, and vision. We would also like to thank USAID’s Cognizant Technical Officer, Carolyn Curtis and senior technical advisor, Patricia MacDonald, for their constant availability to answer hundreds of questions that helped the team completely understand the project from the USAID point of view. The team that went to Tanzania especially thanks USAID/Tanzania staff Pam White, Charles Llewellyn, and Mike Mushi for taking the time to offer the field perspective on ACQ’s work. A special word of appreciation is also due to Grace Lusiola and her dynamic staff, who gave a realistic orientation to the ACQ field operations in Dar es Salaam, Arusha, and Kigoma. We are also grateful to the field implementation teams, the many service providers, and the clients who showed how the ACQ programs actually translate theory into practice, and the 14 mission health officers who answered a questionnaire that elicited a field assessment of ACQ’s work. The team was fortunate to have access to written materials that provided superb background and contributed significantly to our understanding of the ACQ program. We would like to acknowledge the entire Global Health Technical Assistance Project team, especially Camille Hart, for the superb technical, administrative, and logistical support they extended to the external consultant and for use of their facilities for synthesis sessions and debriefings. The combined work of these many people, and the thoughtful interaction of the entire appraisal team, laid the foundation for this strategic appraisal and for the findings presented in this report. Strategic Appraisal of the ACQUIRE Project vii ACRONYMS ACQ/ Cooperative Agreement: ACQUIRE, Access for Quality and Use in Reproductive Health ACQUIRE ADRA Adventist Development and Relief Agency AIDS Acquired Immune Deficiency Syndrome ART Antiretroviral treatment CA Cooperating agency CBD Community-based distribution COPE Client-ordered provider efficient CS Contraceptive security CTO Cognizant technical officer DfID United Kingdom Department for International Development EH EngenderHealth FBO Faith-based organization FHI Family Health International FP Family planning GBV Gender-based violence GH Bureau for Global Health GLP Global leadership priorities GTZ Deutsche Gesellschaft für Technische Zusammenarbeit HIV Human immunodeficiency virus IR Intermediate result IUD Intrauterine device LAPM Long-acting and permanent methods M&E Monitoring and evaluation MAARD Modified Acquisition and Assistance Request Document MAP Men as partners MAQ Maximizing access to quality MI Male involvement MIS Management Information System MOH Ministry of Health NGO Nongovernmental organization NSV No-scalpel vasectomy PAC Post-abortion care PEPFAR President’s Emergency Plan for AIDS Relief PMI Presidential Malaria Initiative PMP Performance Monitoring Plan PMTCT Prevention of mother-to-child transmission of HIV PNA Performance needs assessment PVO Private voluntary organization PRH Office of Population and Reproductive Health PRINMAT Private Nurse Midwives Association/Tanzania RH Reproductive health SDI Service Delivery Improvement (Division) SOTA State of the art SPARHCS Strategic Pathway to Reproductive Health Commodity Security SWAA Society for Women and AIDS in Africa TA Technical advisor UNFPA United Nations Fund for Population Activities UNICEF United Nations Children’s Fund USAID U.S. Agency for International Development WDA Whole District Approach WHO World Health Organization viii Strategic Appraisal of the ACQUIRE Project Strategic Appraisal of the ACQUIRE Project ix CONTENTS ACKNOWLEDGMENTS ............................................................................................................................................... vi ACRONYMS..................................................................................................................................................................... vii EXECUTIVE SUMMARY .................................................................................................................................................. 1 STRATEGIC APPRAISAL OF ACQUIRE PROJECT – MAJOR RECOMMENDATIONS................................ 5 I. PARTICIPATORY APPRAISAL FOR THE ACQUIRE PROJECT ..................................................................... 7 II. ACQUIRE’S MANDATE, RESULTS FRAMEWORK, AND FUNDING .......................................................... 9 III. GENERAL ASSESSMENT OF ACQUIRE’S PERFORMANCE ......................................................................... 13 IV. STRATEGIC APPROACH--FOCUS AND COHERENCE .............................................................................. 15 V. ACQUIRE’S OPERATIONAL MODEL: INCREASED ACCESS, QUALITY, AND USE OF FP/RH SERVICES ........................................................................................................................................................................... 23 VI. GLOBAL LEADERSHIP PRIORITIES ................................................................................................................... 29 VII. TREATMENT AND PREVENTION OF OBSTETRIC FISTULA .................................................................. 31 VIII. EVALUATION AND RESEARCH....................................................................................................................... 35 IX. PERFORMANCE MONITORING ........................................................................................................................ 37 X. MANAGEMENT OF THE ACQUIRE PROGRAM ........................................................................................... 39 XI. PARTNER, MISSION, AND COUNTERPART PERSPECTIVES ON ACQUIRE ...................................... 43 XII. MAJOR CONCLUSIONS ...................................................................................................................................... 45 PART 2: ANNEXES......................................................................................................................................................... 47 Major Program Components: Accomplishments, Lessons Learned, and Recommendations ....................... 49 SUPPLY .......................................................................................................................................................................... 51 DEMAND...................................................................................................................................................................... 54 ADVOCACY................................................................................................................................................................ 56 MAXIMIZING ACCESS AND QUALITY ............................................................................................................. 58 FP/HIV INTEGRATION ............................................................................................................................................ 59 CONTRACEPTIVE SECURITY ............................................................................................................................... 60 POST-ABORTION CARE ........................................................................................................................................ 61 GENDER ....................................................................................................................................................................... 62 SUPPORTING PROGRAM DOCUMENTATION.................................................................................................. 63 B.1 ACQUIRE Tools, Models, and Approaches ................................................................................................. 65 B. 2 ACQUIRE Country Programs ......................................................................................................................... 71 B.3 ACQUIRE Advocacy Interventions ................................................................................................................. 75 B.4 ACQUIRE Research Studies and Evaluation.................................................................................................. 77 SUPPORTING DEMOGRAPHIC DATA ................................................................................................................... 81 C.1 FP/RH Data: Matrix of Tier One/Two Countries LAPM Analysis ........................................................... 83 C.2 Reason for Supporting Investments in LAPM ............................................................................................... 87 PARTICIPATORY REVIEW .......................................................................................................................................... 93 D.1 Abridged Scope of Work ................................................................................................................................. 95 D.2 Self-Assessment Paper .................................................................................................................................... 103 D.3 Persons Interviewed ........................................................................................................................................ 213 D.4 Bibliography ....................................................................................................................................................... 217 D.5 Summary of Mission Surveys ......................................................................................................................... 221 D.6 Summary of Partner Surveys ......................................................................................................................... 223 D.7 After-Event Review of Process ..................................................................................................................... 225 x Strategic Appraisal of the ACQUIRE Project STRATEGIC APPRAISAL OF THE ACQUIRE PROJECT A PARTICIPATORY REVIEW: A LOOK TO THE FUTURE EXECUTIVE SUMMARY The Office of Population and Reproductive Health (PRH) of the Bureau for Global Health (GH), U.S. Agency for International Development (USAID), organized an interactive and participatory review of the Cooperative Agreement: Access for Quality and Use in Reproductive Health, known as the ACQUIRE Project (ACQ). An external senior consultant who worked with internal USAID technical experts in family planning and reproductive health (FP/RH) service delivery led the appraisal team. The purpose of this appraisal was “to review the project’s accomplishments and progress toward results in order to develop practical recommendations for future strategic programming in FP service delivery.” The ACQ Project is a five-year (2003–2008) leader with associate cooperative agreement awarded to EngenderHealth (EH) and its partners: CARE, IntraHealth International, Meridian International, the Adventist Development and Relief Agency (ADRA) and the Society for Women and AIDS in Africa (SWAA). The purpose of the agreement is to advance the use of quality FP/RH care, with a focus on facility￾based services, with emphasis on long-acting and permanent FP methods (LAPM). Four contraceptive methods are categorized as LAPM: intrauterine devices (IUD), implants, female sterilization, and vasectomy. Since the award in October 2003, USAID has allocated to ACQ $80.3 million, consisting of $25.3 million in global core support and $55.0 million in field funding. ACQ has created a strong global leadership program that articulates and disseminates best practices, approaches, and tools and strongly reinforces and supports its field programs. ACQ has programs in 19 countries and activities with two regional programs. As documented in this appraisal report, ACQ uses its core funds to support the design of global approaches, methods, and tools that it then adapts to individual countries. A secondary analysis of DHS; baseline studies in Bangladesh, Bolivia, and Tanzania; performance needs assessments; and family planning repositioning studies have helped ACQUIRE to identify needs and gaps that exist for both supply and demand sides of facility based family planning programs. ACQUIRE has built on a number of evidence based practices. Use of IntraHealth’s performance needs assessment (PNA); EngenderHealth’s Client Oriented Provider Efficient (COPE) model and the Men as Partners (MAP) approach; Meridian’s Communications Strategies; ACQUIRE’s Fundamentals of Care (FOC) and CARE’s participatory learning and action (PLA) methodologies have amplified their dissemination and use in countries. ACQ provides the seed money to pilot-test approaches and tools, while its client missions generally provide the funding for their replication and scale-up. Thus, ACQ is able to address the needs of the field, while at the same time, advancing technical leadership through this collaborative and synergistic approach with field programs. ACQ adds value in three distinct ways: • By bringing to a country technical skills that field programs otherwise cannot easily access or replicate. Technical assistance to strengthen the training, supervision and/or logistics processes and systems was given in 10 countries. Curriculum developed by ACQUIRE has been used in 19 countries, and resulted in 1623 training events for 73,838 provider participants to improve their knowledge, skills and performance. • By serving as a means of cross-fertilization between countries. Approaches and models such as the Whole District Approach have been used to mutually reinforce lessons learned from countries such as Bangladesh, Tanzania and Uganda. Lessons from nine countries in revitalizing the IUD have reinforced and informed each other. Through the GLPs, community-based approaches for PAC and FP for HIV positives have been tested in one country and replicated in others. Strategic Appraisal of the ACQUIRE Project 1 • By synthesizing experience from a range of activities and countries. A synthesis of PNAs done in 11 countries identified common issues across most countries. Identified needs included lack of attention to fundamentals of care in routine RH/FP service delivery; lack of up-to-date, evidence￾based knowledge skills, guidelines, and practices, especially for counseling and LAPM; the need for strengthened provider support systems; the need to update and standardize a clinical training approach; the need to expand the method mix with a focus on underutilized LAPMs; the need to strengthen contraceptive security; and the need to eliminate barriers which limit access to services. A synthesis of five case studies for Repositioning FP revealed that even successful programs are fragile and that Missions, donors, and country programs need to stay the course and continue supporting FP programs. ACQ’s broad sphere of influence and scope of programs include: (1) a flagship facility-based program that supports a range of clinical services, mainly LAPM; (2) efforts to champion and mainstream several global leadership priorities including repositioning family planning; postabortion care; gender; population, health and environment; and FP/HIV integration and (3) a congressionally mandated program to support the prevention and treatment of obstetric fistula that has grown from 5 sites in 2 countries in 2004/2005 to 18 sites in 10 countries in 2006/2007, with more than 29 surgeons trained and 2,000 fistula repairs completed. ACQ uses an integrated Supply - Demand - Advocacy approach to increase access to services. Some tools and approaches developed to respond holistically to gaps identified by PNAs include: the Whole District Approach which is a model to simultaneously strengthen training, supervision, and logistics at the District level; standardization workshops for updating trainers and providers’ knowledge, skills, and practice in IUDs and counseling; communications campaigns that incorporate media, community participation and interpersonal approaches to increase awareness, provide accurate information, and increase demand for LAPMs, and Reality Check, which is a method mix forecasting tool that can help national, regional, district and facility levels realistically plan for resources to provide those services. Its programming strategy is participatory and results in local work plans that demonstrate commitment to strengthening LAPM services ACQ is a global leader in selecting and implementing best practices, building consensus, and promoting changes in policies and approaches, especially at the country level. After participating with WHO to update the Medical Eligibility Criteria and Selected Practice Recommendations, ACQUIRE facilitated the updating of national FP policies, standards and guidelines and provider practices in 11 countries. National service guidelines that focus on the fundamentals of care have been adopted in four countries. In the three ACQ focus countries (Bolivia, Tanzania and Bangladesh), the number of FP sites supported has increased dramatically: from 954 to 3,297 in Bangladesh, from 189 to 357 in Bolivia, and from 55 to 400 in Tanzania. This represents a three-to-eight fold increase in sites, resulting in a five-fold increase in the number of clinical LAPM procedures performed in ACQ-supported sites. With technical assistance from Measure/Evaluate, ACQ designed and conducted baseline surveys in the three focus countries and monitors quarterly service delivery statistics. Local strengthening of health information systems is needed to make service delivery statistics meaningful. Thus, while it is difficult to use service statistics to measure change, they are a quick indication that movement seems to be in the right direction. Endline surveys are planned for the three countries; ACQ will do careful analysis of data from these evaluations to document changes and demonstrate the effect that its work is having on FP/LAPM outcomes. ACQUIRE is also conducting programmatic research in key areas of importance to the use of LAPMs, such as communications and discontinuation. The inclusion of strong M&E and program research in ACQ has meant that it can immediately apply findings to improve program approaches, and support expansion within and across countries. This methodology should be incorporated into future service delivery projects. 2 Strategic Appraisal of the ACQUIRE Project The success of the project is facilitated in a number of ways: • ACQ has strong senior leaders both at headquarters and in the field, and keeps its vision and purpose consistent by retaining senior managers and technical staff who have been with the project since its inception. • It brings together partners that offer a broad range of technical expertise to meet ACQ objectives. • It assembles excellent technical staff with extensive field experience who are empowered to act quickly and decisively. • It has designed a technically sound program revolving around needed clinical services. • It encourages ownership in country counterparts, who view ACQ as “assisting and supporting local efforts, not running its own parallel program.” • It has earned high regard from host country partners, USAID headquarters and missions, and cooperating agencies for the innovation, quality, and relevance of its activities, the competence of its staff, and its success in meeting programmatic and strategic objectives. In general, the team found that the ACQ program is well managed and its accomplishments, which are highlighted in this report, are significant. There are, however, a few areas for improvement. ACQ needs to coordinate better with other cooperating agencies to facilitate action at the country level. They were not able to make the best use of their field level partners, in some cases because of the organizational structure of the partner, and in other cases because the mission did not want the partners. Some tensions exist with the missions regarding “whose agenda it is” and “who pays.” This was especially true for GLP activities and LAPM communications, M&E and research activities. ACQ has moved actively to promote gender-specific activities but has not completely mainstreamed gender into all aspects of its programs. It also needs to strengthen M&E for service delivery by setting targets for specific indicators, so that it can better measure progress toward meeting those targets and demonstrate overall impact on LAPM use. ACQ is aware of these concerns and is moving to address them. The team suggests that the most important activity for ACQ over the next year is to take aggressive action to fully document and disseminate its work, so that it can seize every opportunity to institutionalize its successful models. There are important lessons to be learned from the ACQ experience that could inform future programming. Key among these are: 1. A project focusing on LAPM is a critical need within the GH/PRH portfolio of centrally funded projects. The team reached this conclusion based on its review of compelling demographic and service data and by observing ACQ experience to date. In order to meet the great and growing need for birth spacers as well as limiters, LAPM are a vital component of any FP services program. As clinical methods, LAPM depend on clinical skills and expertise, medical safety and quality, informed choice, and management of side effects. Each of these needs specialty attention. The team urges GH/PRH to champion the importance of clinical family planning methods and take aggressive action to promote LAPM more widely and make them strategically available. 2. A project such as ACQUIRE that supports a substantial number of global technical experts provides extremely effective support to and interface with the field. The team saw great synergy between ACQ’s global leadership and field programs. Global–field relationships were dynamic, reinforcing, and mutually beneficial. New designs should be allocated sufficient core funds to support a critical mass of global experts to enable a high level of interaction with field programs. 3. A service delivery program that integrates supply, demand, and advocacy interventions in a holistic manner is a useful, sound, and effective way to increase access to services. ACQUIRE evolved from a focus mainly on supply, to a focus on supply and demand, to a model integrating supply, Strategic Appraisal of the ACQUIRE Project 3 4 Strategic Appraisal of the ACQUIRE Project demand, and advocacy for LAPMs. ACQ has learned that if any intervention is weak or missing, it negatively affects the whole program. Models and approaches being implemented within this holistic framework include: • Supply Side: ACQUIRE is simultaneously strengthening district-level systems for training, supervision, and logistics, such that training strengthens provider performance and reinforces logistics procedures, and supervision reinforces both provider performance and logistics. • Demand Side: ACQUIRE’s communications strategies for LAPMs include a combination of media and materials, community mobilization, and interpersonal approaches. Strategies have been method￾specific for IUD and vasectomy, with one multiple-method approach developed. • Advocacy: ACQUIRE uses the Reality Check advocacy package (developed with TA from Policy Project) to generate commitment from national, regional and district government to expanded method mix, and facilitate realistic planning to introduce and scale-up services. 4. Facility-based services can incorporate a range of interventions that further the FP agenda. The need for clinical services is determined by the country and the program requirements. Although the ACQ program focuses on clinic-based LAPM in many countries, it was also successful in managing clinical services in fistula treatment and repair; post-abortion care (PAC); and integration with MNCH and HIV interventions. In considering the future, the team suggests, there is a need for a GH/PRH mechanism to respond to the expanded need for a variety of clinical and facility-linked outreach services. The chapters that follow attempt to appraise the value of ACQ’s work to date, highlight accomplishments and weaknesses, determine lessons learned, and make recommendations for improving the program. For easy reference, key recommendations are incorporated into the matrix on the following pages. The team hopes that this report will contribute to the analytical thinking for PRH visioning and decision making about future service-delivery programs. STRATEGIC APPRAISAL OF ACQUIRE PROJECT – MAJOR RECOMMENDATIONS USAID ACQUIRE Strategic FP/RH Programming • USAID must take a strategic approach to programming for LAPM. It might direct core funds to those countries that have a demographic need and the potential for results (“push”) while concurrently encouraging missions that express an interest in or are willing to provide funds for scale-up to use a GH mechanism (“pull”). • USAID should use and promote the LAPM Advocacy Package to help missions understand the importance of using LAPM to address unmet FP needs. • In its portfolio of centrally funded projects, USAID should include a program that addresses the critical unmet need for LAPMs • In future programs, USAID should encourage a holistic Operational Model, which integrates supply, demand, and advocacy interventions to improve access to, use of, and quality of FP/RH services. • On the supply side, USAID should design all future clinical programs to focus on clinical skills and expertise, supervision, medical safety, medical quality, informed choice, and side effects management. • ACQ should continue to document, disseminate, and institutionalize best practices, lessons learned, and proven interventions to ensure the long-term sustainability of its efforts. • ACQ should use and promote to missions and the international family planning community the new LAPM Advocacy Package to encourage action on meeting the unmet need of couples wanting to limit family size or space birth intervals. • ACQ should incorporate or strengthen community involvement in its programs and increase emphasis on creating demand for services. Global Leadership Priorities (GLP) • USAID and ACQ should collaborate strategically with partner organizations and GLP Champions on issues that concern improved service delivery. • USAID should sustain and build on GLP integration into GH programs, making sure to institutionalize lessons learned from ACQ. • USAID should advocate and promote GLPs with missions to help them incorporate GLP frameworks into National Strategies. • ACQ should gather and document successes and processes, such as (a) the provider perspective of contraceptive security, (b) the methodology of the PNA process, (c) program implementation in Bolivia and Kenya to determine ability to scale up, and d) gender interventions. ACQ should expand efforts to promote and disseminate best practices, lessons learned, and technical updates at the global and country levels. Obstetric Fistula • USAID should set clear parameters and expectations for each component (detection, repair, prevention, and reintegration) to help the program move forward and achieve maximum impact. • USAID should continue to review and evaluate indicators with ACQ and more clearly define indicators for fistula prevention. • ACQ should work systematically with men as partners and agents of change to promote repair and prevention. • ACQ should expand the gender-based violence (GBV) component of its fistula program through consultation and collaboration with organizations that have GBV expertise. 6 Strategic Appraisal of the ACQUIRE Project Evaluation, Research and Performance Monitoring • USAID should ensure that global projects have sufficient core funds to provide targeted assistance to field projects, including those funded in the field, especially if the global program is fully or partly responsible for the results. • Global service delivery projects should set their own research agenda; when they collaborate with another organization, study design, roles and responsibilities, schedule, deliverables, publications, and acknowledgement should be thoroughly discussed and documented in writing. • Global service delivery projects should have in-house expertise in monitoring, evaluation, and research and be able to access technical assistance from experts and specialty projects as needed. • USAID should encourage future global projects to include in their performance monitoring plans (PMPs) indicators that (a) respond to PRH results and (b) measure outcome results, in particular where field activities are significant. • Global projects should, where applicable, direct their country programs to better align the indicators in country-level PMPs with the data they are capturing through baseline and final studies. If indicators are developed, they can be included in the country￾level PMP. The project would then be able to demonstrate change in the indicators over the life of the project. • Future global projects should consider the structure of ACQ’s PMP when designing their own. ACQ’s well- organized and concise PMP shows the relationship between global and field indicators that respond to the project’s activity objective. • ACQ should streamline its tools and instruments for data collection so that the data are more manageable and findings can be shared more readily with project implementers and stakeholders. • ACQ should develop indicators based upon the data collected in baseline studies in the three focus countries and measure the change at the end to demonstrate results. Management • USAID should evaluate the advantages and disadvantages of encouraging large consortia in new awards. • USAID should use Mission Strategic Plans and Operational Plans (part of the new reform process) to ensure that FP programs address fundamental functional and technical areas to increase access, quality, and use. • USAID should ensure that the cost-share percentages required in grants and cooperative agreements are not so low that they are easily raised early in the project or so high as to be unattainable. • ACQ should continue with its “can do” problem-solving approach to management. In summary, the program is quite effective. ACQ has made significant achievements that contribute to the support and advancement of clinical services in FP/RH around the world. Strategic Appraisal of the ACQUIRE Project 7 I. PARTICIPATORY APPRAISAL FOR THE ACQUIRE PROJECT A. Background For the past 30 years, USAID service delivery programs have helped countries and institutions create accessible, high-quality voluntary FP/RH services. USAID programs around the world have increased the availability and quality of FP/RH products and services in partnership with a broad array of public, nonprofit, and commercial institutions. USAID funds 35 to 40 percent of donor-provided contraceptives in the developing world and offers technical and logistical support for many FP programs globally. It has worked particularly to address the large unmet need for birth spacing and limiting. To achieve this objective, USAID GH/PRH exercises global leadership in policy, advocacy, and services; generates, organizes, and disseminates knowledge; and provides field support to implement effective programs around the world. In 2003, GH/PRH developed the ACQUIRE: Access, Quality, and Use in Reproductive Health (ACQ) Project, a five-year project that was awarded to EngenderHealth and partners. The purpose of the agreement was to advance the use of quality FP/RH care, with a focus on a broad range of facility-based services and clinical methods, including LAPM. B. Purpose of the Appraisal As ACQ nears its 2008 completion date, GH/PRH assembled a team “to review the project’s accomplishments and progress toward results to develop practical recommendations for future strategic programming in FP service delivery.” From February to May 2007, the team, composed of an external senior RH consultant and internal USAID technical experts looked at a number of questions in assessing ACQ program components: • How does ACQ operate at both the global and field levels? • What is ACQ’s strategic approach and specific program interventions? • Is it appropriate to focus on facility-based services? • What are ACQ’s major accomplishments for various program components? • What interventions, tools, approaches, and service delivery models has ACQ developed, and have they been effective? • What is ACQ’s management structure, and how effective has it been? • What has been the demand for ACQ program activities through the various funding mechanisms? • What have been the issues, challenges, and lessons learned through monitoring, evaluation, and research? The team analyzed the implications of ACQ’s experience in strategic and technically sound programming, resource efficiency, and use of evaluation and research data in program decision-making, in order to inform future programming. 8 Strategic Appraisal of the ACQUIRE Project C. Appraisal Methodology The appraisal began with ACQ and its partners completing a self-assessment questionnaire. The team’s technical experts conducted topic-focused video conferences with ACQ technical staff, visited the New York headquarters, and interviewed project staff to gain in-depth understanding of the various aspects of the project. The team leader also queried the project technical and network partners to ascertain their opinions on EH’s management of the project (see Chapter XI). Team members also collected, reviewed, and analyzed the literature on all aspects of the program in their area of technical expertise, including contracts, work plans, annual reports, survey documents, field studies, case studies, and technical documents. The team interviewed USAID staff and other stakeholders, such as ACQ staff and managers; host-country officials, managers, and providers; cooperating and donor agencies; community networks; faith-based organizations; private sector service sites; and beneficiaries. To obtain a field perspective, team members met with country representatives from the core ACQ countries—Bangladesh, Bolivia, Tanzania, and Uganda—during their annual field staff meeting in New York. Three members of the team did a field visit to Tanzania to: • Interview representatives of the national government’s Ministry of Health. • Meet with local district-level government authorities, community leaders, service providers, and ACQ managers in three regions (Dar es Salaam, Arusha, and Kigoma). • Observe service delivery and meet with clients and beneficiaries at public and private hospitals and other facilities. • Visit health centers, health posts, commodity stores, and other facilities in rural and urban areas. • Interview staff from USAID and USAID-funded projects: Family Health International, the Academy for Educational Development’s T-MARC Project, John Snow’s DELIVER Project, the Johns Hopkins ACCESS Project, and the PRINMAT/Tanzania, Policy Project. • Debrief with Mission staff. The team surveyed or interviewed 14 USAID missions and regional offices (out of 21 queried) that had received core funds or bought services from the ACQ project through field support, a MAARD (Modified Acquisition and Assistance Request Document), or an associate award (Annex D.5). While on other business, team members interviewed informants in Nepal, Cambodia, and Kenya. The team reviewed trip reports, baseline surveys, special studies, assessment reports, and case studies for countries where ACQ was operating. The team met periodically to raise questions and compare and discuss findings. At the end of data collection, the team held two facilitated synthesis meetings. The first was an internal meeting to consolidate findings, elicit lessons learned, and discuss strategic, technical, and programmatic questions. The second took place with the USAID review team and ACQ staff to achieve consensus on major issues and solicit input for final recommendations. Formal debriefing of findings were held for USAID senior staff in both Washington and Tanzania, the GH/PRH staff, and the ACQ management staff and representatives of its partners. Recommendations are listed in the matrix at the end of the Executive Summary. The full scope of work is laid out in Annex D.1; documents reviewed in Annex D.3; and persons interviewed in Annex D.4. ACQ documents that served as a reference for the team are available at the EH Extranet home page (www.extranet.acquireproject.org). The team review of the advantages and disadvantages of the participatory process is on file at the GH/PRH/SDI Division (see also Annex D.7). Strategic Appraisal of the ACQUIRE Project 9 II. ACQUIRE’S MANDATE, RESULTS FRAMEWORK, AND FUNDING A. Mandate and Evolution of the ACQUIRE Project USAID in 2003 awarded a “flagship” project to the ACQ partners to advance and support global leadership and field FP/RH facility-based service delivery. While the program has a mandate to support facility-based services, it emphasizes LAPMs. The ACQ program aims to increase access to quality FP/RH services, improve the performance of service delivery providers, and make the environment supportive of FP/RH service delivery. USAID requested that the ACQ project pay special attention to gender concerns and generally attempt to mainstream gender into its programs. Starting in 2004, USAID asked ACQ to plan, coordinate, and implement certain GH/PRH Global Leadership Priorities (GLP) and other service areas. Of the ten GLP areas, USAID provided core funding for maximizing access to quality (MAQ), particularly for revitalizing use of the IUD; repositioning FP; PAC; gender and men as partners (MAP); FP/HIV integration; population, health, and environment; female genital cutting; refugee reproductive health, and contraceptive security. GH/PRH also requested ACQ to administer core funds for the Flex Fund and for two PRH Country Partnerships. In 2005, USAID asked ACQ to become the main partner for obstetric fistula. B. Results Framework ACQ has three intermediate results that are the basis for its program activities and its performance monitoring (Figure 1). In 2005, ACQ revised its initial PMP to base performance monitoring and reporting on both global and country-level indicators (Chapter IX). Finding the results framework somewhat deficient operationally, ACQ designed a more holistic operational model for increasing access and quality and using integrated interventions: supply, demand, and advocacy (Chapter V). ACQ carries out its programs through: 10 Strategic Appraisal of the ACQUIRE Project • Global leadership to advance technical development and application of best practices. These activities, which are usually core-funded, aim to advance critical thinking and transfer knowledge into practice. ACQ collaborates with a variety of international organizations on global leadership topics: multilateral organizations like the World Health Organization (WHO) and the U.N. Fund for Population Activities (UNFPA for fistula activities), other bilateral donors like the United Kingdom Department for International Development (DfID), and private organizations like the Elizabeth Glaser, Packard, and Gates Foundations. Technical and program assistance and training for field programs to help country nationals plan, implement, and evaluate program models, and start up, scale up, and sustain FP/RH programs. The focus is on facility-based programs. ACQ has implemented field programs in 19 countries and with two regional programs (Figure 2). It provides technical assistance and training in three focus countries (Bangladesh, Bolivia, and Tanzania) on facility-based methods, emphasizing LAPM. It is now initiating larger-scale programs in Azerbaijan, Nigeria, and Uganda. The other 15 programs have service delivery, GLP, and/or obstetric fistula programs. Funding is a mix of core funding for introduction and pilot projects and field funding for replication and scaling-up projects. • Figure 2: Map of ACQUIRE’s Country and Regional Programs Honduras Azerbaijan Bolivia Bangladesh Tanzania Uganda Kenya Mali Ethiopia Nigeria Rwanda South Africa Benin Ghana Cambodia Nepal Guinea Senegal DRC Madagascar Azerbaijan Bolivia Bangladesh Tanzania Uganda Kenya Mali Ethiopia Nigeria Rwanda South Africa Benin Ghana Cambodia Nepal Guinea Senegal DRC Madagascar C. Funding ACQ has a funding ceiling of $150 million. Thus far, USAID has allocated $72.3 million in core and field support and missions have provided another $8.0 million directly for large associate awards, for a total of $80.3 million (Table 1). Of the total, 32 percent is core and 68 percent is field funding. • Core funding ($25.3 million) supports headquarters and personnel costs, and global technical leadership activities. Approximately 50 percent of all core funding can be attributed to direct support of field programs. Within the $25.3 million of core funding, USAID earmarked funds for country-specific activities, such as the Flex Fund ($748,000) for three grantees in Ethiopia and PRH Country Partnerships for Uganda ($950,000) and Kenya ($650,000). USAID also provided core GLP activities ($2,781,000) and fistula ($2,454,000). All of these activities were accommodated within the ACQ mandate. • Field funding ($47.0 million) is transmitted via field support or MAARDs for ACQ field personnel and field implementation costs. ACQ is one of the GH projects most in demand by the field. The basic core-to-field ratio is $1.00 core for every $1.85 in field funds, but taking into account core funds that go directly to the field, the actual ratio is $1.00 core for every $4.71 that goes directly to the field. In essence, the field gets the full backing of a strong technical agency. The funding breakdown by type of program is shown in Table 1. The ACQ financial management system reports 85 separate account codes, 27 of which relate to the global leadership program. The majority of field funds are attributed to Africa (59%), followed by Asia (26%), and Latin America (15%). • Associate Awards ($16 million) have been issued by three missions, Azerbaijan ($8 m), Bolivia ($6 m) and Uganda ($2 m). The amounts obligated to date are included in the table below. Table 1: Total Funding Provided to ACQUIRE by Type of Funding, FY 2003-2006 ($000) Strategic Appraisal of the ACQUIRE Project 11 Type of Funding FY 03 FY04 FY-05 FY 06 FY07 TOTAL Global Core Funding (includes Core-Core, GLP, & Fistula) $ 5,750 $ 8,949 $ 5,440 $ 5,168 $ 25,307 Core-Core $ 5,000 $ 4,824 $ 4,400 $ 3,500 $ 17,400 Flex Fund $ 750 $ 700 ($ 702) 0 $ 748 PRH Country Partnerships 0 $ 1,600 0 0 $ 1,600 Global Leadership Priorities 0 $ 1,525 $ 742 $ 514 $ 2,781 - Revitalization of IUD 0 0 188 0 188 - Maximizing Access/Quality (IUD) 0 900 0 0 900 - Repositioning FP 0 0 195 75 270 - Post-Abortion Care 0 200 (60) 170 310 - Gender/Men as Partners 0 0 0 50 50 - FP/HIV Integration 0 260 65 24 349 - Population, Health, and Environment 0 0 275 175 450 - Female Genital Cutting 0 90 0 0 90 - Contraceptive Security 0 0 29 20 49 - Maximizing Access/Quality (IBP) 0 0 50 0 50 - Reproductive Health for Refugees 0 75 0 0 75 Fistula (MCH & POP) 0 $ 300 $ 1,000 $ 1,154 $ 2,454 Field Funding $ 3,510 $ 17,138 $ 12,826 $ 12,639 $ 900 $ 47,013 Field Support $ 3,010 $ 11,038 $ 9,950 $ 10,512 0 $ 34,510 President’s Emergency Plan for AIDS Relief (PEPFAR) $ 500 $ 4,130 $ 700 $ 950 $ 900 (core) $ 7,180 MAARD 0 $ 1,970 $ 2,176 $ 1,177 0 $ 5,323 Subtotal Core & FS $ 9,260 $ 26,087 $ 18,266 $ 17,807 $ 900 $ 72,320 Associate Awards $ 3,475 $ 3,512 $ 1,000 $ 7,987 TOTAL $ 9,260 $ 29,562 $ 18,266 $ 21,319 $ 1,900 $ 80,307 The breakdown of core vs. field funding is detailed in the budget chart below (Figure 3). 12 Strategic Appraisal of the ACQUIRE Project Figure 3: ACQUIRE Funding ACQUIRE Funding September, 2003 – March, 2007 (in millions) Total Revenue = $80m Global (Associate Awards = $8m) Field Core-Core GLP Other Fistula Fistula - $2m (10%) Core-Core - $17m (68%) Other Core* - $3m (11%) * Includes Flex Fund, PRH Country Partnerships & Incremental Funding GLP – $3m (11%) Asia - $14m (26%) Africa - $32m (59%) Latin America - $8m (15%) Asia Africa Latin America Global (breakdown) Field (breakdown) Global - $25m (31%) Field - $55m (69%) Strategic Appraisal of the ACQUIRE Project 13 III. GENERAL ASSESSMENT OF ACQUIRE’S PERFORMANCE ACQ’s unique value lies in its extensive expertise in supporting quality FP/RH services, especially facility￾based clinical services and more specifically LAPM. The project, however, is authorized to provide a range of clinical services (e.g., voluntary counseling and testing for HIV, maternal and child health care services, PAC, etc.). Missions and PRH have asked ACQ for a range of field activities, and it has been extremely flexible in responding to USAID requests to take on the GLP and the congressionally mandated fistula program. Specific ACQ activities are detailed in the following annexes: • Tools, models and approaches, Annex B.1 • Country interventions, Annex B.2 • Advocacy interventions, Annex B.3 • Research studies and evaluations, Annex B.4 As documented in the chapters that follow, ACQ’s integrated approach to increasing access, quality, and use through advocacy, supply, and demand seems to have been effective, although there is a need to evaluate and determine the full potential of this approach. ACQ’s strategy and work plans have been solid. It has effectively provided global leadership in selecting best practices, building consensus, and promoting changes in policies and approaches, especially at the country level. It has built on a number of evidence￾based practices—IntraHealth’s performance needs assessment (PNA); EH’s Client Oriented Provider-Efficient (COPE) model; the EH model for male involvement, particularly the Men as Partners Approach; Meridian’s Communication Strategies, ACQ’s Fundamentals of Care; and CARE’s participatory learning and action methodologies—and it has amplified their dissemination and use. The adoption of proven interventions enabled ACQ to move quickly into field applications in most countries. EH and representatives of its technical partners have provided excellent leadership through a unified and seamless ACQ management team. Senior leaders have been effective both at headquarters and in the field. All senior leaders and most technical staff have been on the job since the project began, keeping vision and purpose consistent. The partners bring broad technical expertise to complement EH’s known capabilities. Host country partners, USAID headquarters and missions, and cooperating agencies respect ACQ for its quality and the relevance of its activities, the competence of its staff, and its innovation in meeting programmatic and strategic challenges. FACILITY BASED FP/RH SERVICES – SITES-- PROVIDERS SERVICES: Family planning services focusing on LAPMs, informed choice and quality counseling post-abortion care services, integrated FP/MCH services, FP/HIV integration, and fistula treatment and repair SITES: Hospitals, clinical facilities health centers, health and other outreach posts, community outreach outlets, and pharmacies PROVIDERS: Doctors, clinical officers, midwives, nurses, counselors, peer educators, outreach and community workers, and pharmacists The scope of the ACQ program is extremely wide, incorporating facility-based family planning activities, many GLP interventions, and the obstetric fistula program. The project also administered the Flex-Fund to three grantees. ACQ generates knowledge, identifies and validates best practices, and encourages the mainstreaming of new ideas in a host of technical and functional areas. Over the life of the project, ACQUIRE has build relations with other projects that are critically important to service delivery, mainly in the area of contraceptive logistics. ACQ has worked with Deliver to integrate LAPMs into SPAHRCS, coordinate commodity needs for training in LAPMs, helped strengthen the “last mile” inventory management by providers and supervisors, and pilot-tested IUD kits to ensure that facilities have the consumable supplies necessary for IUD insertions. While ACQ has accommodated USAID missions and country programs on a wide range of service requests, it has kept its focus on clinical services and LAPM. For greater impact, 14 Strategic Appraisal of the ACQUIRE Project however, ACQ should continue to strengthen coordination with other agencies working in areas that feed into service delivery, such as preservice education, research, communications, and logistics. This could be further facilitated by increased coordination within the PRH Office across divisions in Washington. As one senior manager said, “Service delivery is everybody’s business.” One factor that limits ACQ’s impact is that field funding drives implementation at the country level. This is not the strategic way to determine which countries should be targeted for ACQ approaches. For example, there is a tremendous unmet need for FP for both spacing and limiting births, which LAPM could fill. ACQ is a great source of technical assistance for LAPM, but its LAPM interventions are being driven by funding from the field rather than demographic analysis of where needs and the potential for implementation are highest. There could be more push in terms of country selection for programming for maximum impact. This would require joint action between ACQ and USAID to reach out and negotiate with missions. In sum, the program is going well, and the team concludes, based on preliminary results in the three focus countries (Chapter V, Tables 4 and 5), that ACQ’s significant accomplishments contribute to the support and advancement of clinical services in FP/RH in general, and LAPM in particular. The chapters that follow elaborate on ACQ’s work to date, highlight accomplishments and weaknesses, determine lessons learned, and make specific recommendations for improving the program. The major messages for each topic area are synthesized as lessons learned and recommendations. Within each area, there are program details and general recommendations in individual program component matrices, which are included in Annex A. The team hopes that this report will contribute to analytical thinking for PRH visioning and will be helpful as PRH makes decisions for future service delivery programs. Strategic Appraisal of the ACQUIRE Project 15 IV. STRATEGIC APPROACH--FOCUS AND COHERENCE Findings, Lessons Learned, and Recommendations A. Strategic Approach: Global–Country Interface ACQ has built a strong global leadership program that interfaces substantively with the field. It uses core funds to support the design of global approaches that it then adapts to the country level. Although ACQ does sometimes provide seed money to pilot-test approaches and tools in the field, its client missions generally fund replication and scale-up. In a number of instances, countries have initiated the process: field-generated efforts have then led to global recognition and ACQ has adapted them for broader use. Figure 4 depicts the global-country relationship. The team believes this is an excellent model of what is meant by global leadership. Figure 4: Schematic View of ACQUIRE’s Global Leadership/Country Interaction GLOBAL USAID/Washington plementing partners ooperating Agencies Multilaterals COUNTRY Missions Ministry of Health Regions Districts Im C Communication ~ Collaboration ~ Documentation Apply, Adapt, Refine: Modified Based on language, culture, health systems, and socio-political context Develop, Promote: Evidence-based tools, approaches, and methodologies related to research & best practice Design/Trial: Innovative approaches Relevant to context, country-specific Barriers included in results framework Adapt, Lead: Further develop Country practices into global Approach, tools, and documents The global-country interaction as illustrated forms a unique symbiotic relationship in which program interventions, innovations, and adaptations are mutually beneficial in advancing successful interventions. The foundation of this relationship is the communication, collaboration, and documentation of lessons learned by global- and country-level personnel, which allows for sharing and adjustment. 16 Strategic Appraisal of the ACQUIRE Project Global projects add value in three distinct ways: • by bringing in technical skills that field programs cannot easily access or replicate; • by serving as a means of cross-fertilization between countries (not just between a country and the global team); and • by synthesizing experience from a range of activities and countries. B. The Strategic Approach in Practical Terms ACQ has demonstrated its ability to bring technical skills effectively and document and share lessons learned with most of its country partners, as illustrated in Table 2 below. The technical intervention examples of the Tanzania Case Study that follows show how the global-field model works at both levels. Intervention Examples On a global level ACQ designed, adapted, and promoted several innovative initiatives. For each type of intervention, communication and collaboration with countries resulted in global learning and documentation of success. Table 2 offers three examples of how mutually supportive functions are implemented at the global and country levels. GLOBAL COUNTRY Whole Develop/Promote: As programs were Adapt/Lead: The positive results of District increasingly decentralized, ACQ created the ACQ’s extensive district-level experience Approach WDA to address gaps by simultaneously with scale-up in Bangladesh, Tanzania, and (WDA) strengthening training, supervision, and Uganda are being used to define and refine logistics systems. a more global approach for further testing and replication. IUD Develop/Promote: ACQ developed a Apply/Adapt: ACQ brought this initiative Revitalization standardized approach to revitalization of the to Uganda, Kenya, Ethiopia, Mali, Nigeria, IUD by: Ghana, Guinea, Senegal, and Honduras. • Using PNA to assess provider needs • Conducting workshops and training to Design/Trial: Uganda created a positive provide staff at all levels with the latest environment for national programs by scientific evidence and skills revising national guidelines, drawing up • Developing or adapting training curricula action plans, and replicating workshops. • Creating an IUD tool kit with the USAID working group Design/Trial: Kenya developed a communication strategy to dispel myths • Communications strategy and rumors about the IUD Adapt/Lead: The success of this approach illustrates how similar methods can be used for other clinical skills, such as mini-lap, counseling, and training Training Develop/Promote: ACQ has updated and Apply/Adapt/Refine: All 19 countries curricula pilot-tested a number of state of the art have used at least one ACQ curriculum. curricula, such as Fundamentals of Care, No- As an example, the FP counseling Scalpel Vasectomy (NSV), Facilitative curriculum adapted for Bolivia Supervision, FP Counseling, and incorporates self-directed learning, group Contraceptive Technology Updates. training, and on-the-job mentoring. Table 2: Examples of ACQUIRE’s Mutually Supportive Programming Strategic Appraisal of the ACQUIRE Project 17 The assessment team had the opportunity to investigate this relationship on a site visit to Tanzania. The Tanzania Case Study demonstrates how this global-county relationship has worked in one country. Tanzania – A Case Study The ACQ Project/Tanzania vision, funded by mission field support, is to ensure that couples have access to and are using high-quality, affordable, sustainable, and integrated facility-based RH services, in particular FP, LAPM, and comprehensive PAC. The project’s strategic objective is to increase demand for, access to, and use of facility-based FP/RH services. It is built on two positive features of Tanzania’s health system: (1) the availability and reach of its health facilities, and (2) an institutional process of health sector reform. The program is aligned with health sector reform and is moving away from a national hospital-level to a district￾level focus. The strategy also includes national advocacy, contraceptive security (CS) initiatives, and capacity building for sustainable services. In FY 06/07, the mission asked ACQ to draw up strategies to scale up from 10 priority regions to national coverage of all 21 regions of the mainland and 5 regions of Zanzibar. ACQ developed a strategy to scale up activities across the country while building district and regional capacity. With the Ministry of Health (MOH) it designed a plan to increase access to quality FP/RH services based on the following activities: (1) building the capacity of master and regional FP/RH trainers in collaboration with public, private, and nongovernmental organization (NGO) providers; (2) mobilizing communities to support FP/RH services; (3) conducting outreach; (4) creating national service promotion strategies; and (5) crafting policies to address medical barriers and make contraceptive supplies more available. Most of these elements were themes in the global leadership areas. Core monies funded technical assistance to lay the groundwork for field-funded replication and scale-up. • The Tanzania program builds district support for FP/LAPM services. Working within the sector￾wide approach and devolved health systems at the district level, ACQ supports district leaders to become FP champions and effectively advocate for resources to carry out and scale-up sustainable FP/LAPM services. As a result, ACQ is working closely with district health teams in the 10 priority districts to build their skills in planning, budgeting, and advocacy. Service access and contraceptive use improved significantly in all the districts. As plans are underway for nation-wide scale up, ACQ is providing sub￾agreements to 20 districts to complement their local budgets, strengthen LAPM training, supervision, outreach and services, and build stronger commitment for local FP funding. • Tanzania inspired the Reality Check tool. The initial meeting (2004) of ACQ with the MOH, faith￾based organizations (FBOs), and regional and district authorities stayed true to ACQ’s principle of basing decision-making on solid data. While drawing up the project implementation plan for 10 regions, it became clear that there was no easy-to-use tool for local planners to see how national goals could be achieved at the local level. In response, ACQ’s global team designed the Reality Check tool, a FP forecasting tool that can take planning to the district and facility level. Today, district authorities anchor their vision and plans for FP/LAPM programming on specific goals based on local data; this evidence￾based contextually driven planning results in more realistic and attainable goals. • Cross-fertilization is the key to success, with global and country programs contributing to the body of knowledge, experience, and lessons learned. ACQ’s extensive work in technical and programmatic areas (see Table 3) greatly benefited the Tanzania program. Global expertise gives content area depth that a country program cannot afford to have on the ground; the country program gives a context and the reality to meld global learning into programs. 18 Strategic Appraisal of the ACQUIRE Project Table 3: Tanzania County Program Interface with Specific Global Interventions Technical Topic Global Contribution Tanzania Country Program Attention to contraceptive Through global collaboration on GLP- Contraceptive and equipment stock-outs security (CS) funded contraceptive security activities, are a major obstacle for FP. ACQ global including Ready Lessons II and the and country staff participated in the Strategic Pathway to Reproductive national SPARCHS assessment, which Health Commodity Security (SPARHCS) resulted in clearer definition of particular Supplement, ACQ highlights the needs concerns of facilities regarding of lower-level facilities (the “last mile”) contraceptives and LAPM supplies. ACQ and how systems must meet global introduced the COPE for CS tool supply/equipment needs for FP/LAPM. to gather data and find solutions for Tanzania and Uganda are often used as facilities and the logistics system in examples to illustrate situations where Tanzania. the last-mile issues need special attention from logistics professionals. Evaluation of the integrated ACQ global promotes the importance Tanzania is field-testing the approach of demand strategy of closely revitalizing supply/demand closely linking marketing, community, inputs for greater synergy. A study will and male involvement to increase review the methodology for integrating demand for multiple LAPMs three areas in the demand equation: simultaneously in areas where clinical marketing, community participation, and services have been strengthened. Once male involvement. The global team the effectiveness of this approach is brings expertise not readily available in determined, it will be a model for the local office on marketing, gender, country and worldwide demand-side and evaluation of demand activities. programming. Decentralized PAC and Global strategic and technical support ACQ global, Frontiers, and country stronger FP in PAC to decentralizing PAC services was programs collaborated to field-test the piloted in Tanzania, with technical decentralized PAC model in Geita assistance to test and replicate district. The study found that strategies for integrating FP into PAC decentralizing PAC services to health using a framework designed at the centers is feasible, affordable, and global level acceptable to communities and service providers. Findings were incorporated into work plans of 20 districts (via sub￾agreements with ACQ). ACQ’s work to improve access to and use of quality FP/RH services in Tanzania began at the global leadership level, was refined at the country level, and was then put forward as a proven methodology. Lessons learned from supply-demand-advocacy interventions in Tanzania no doubt will guide best practices and future scale-up activities in other countries. This model, at both global and country levels, has been extremely powerful. Strategic Appraisal of the ACQUIRE Project 19 C. Strategic Focus, Relevance, and Coherence The team found the ACQ program to be strategically sound, with interventions linked to its strategic results framework. Its approach has forged its global leadership function with field implementation and has melded theory with practice. The program has a holistic supply, demand, and advocacy approach to increase access, use, and quality. ACQ’s experience and involvement in promoting clinical programs and the demand from the field for this expertise, validates the approach. • ACQ’s program is focused by its mandate “to advance and support FP/RH services, directed toward facility-based services.” Given the prior work and organizational expertise of the lead agency and partners and the need for LAPM, ACQ has concentrated on, but has not limited itself to, LAPMs. Some missions wanted “one-stop shopping” (one mechanism to do a range of RH programs that take a variety of clinical or facility-based approaches). Other missions did not want specialty services or saw ACQ’s focus as too narrow. ACQUIRE is also a multi-faceted project for PRH. A review of ACQ’s portfolio makes it clear that ACQ has concentrated on LAPM and facility-based FP services but its program has accommodated a wider range of other clinical services and GLP activities. ACQ has responded to USAID requests, both PRH and missions, to provide a number of other types of clinical services (e.g., FP/HIV integration, GBV services for women with AIDS, integration of FP into maternity services, PAC, post-partum care, and prevention of mother-to-child transmission of HIV (PMTCT) and fistula prevention and repair). While the ACQ program does not generally focus on skills training for resupply methods, it does include all methods in any counseling or quality improvement training with service providers whether they work in fixed facilities or do outreach work. • ACQ’s path to increased access is an integrated approach of supply, demand, and advocacy interventions. Initially the program emphasized supply-side interventions to strengthen training, supervision, performance, quality and logistics. Learning early on that demand is an essential element, ACQ has incorporated communication strategies that include media campaigns, community mobilization, interpersonal counseling, and community/partner support,to enhance acceptance and increase use of services. To answer questions from country managers about how to meet their goals for contraceptive prevalence, ACQ developed Reality Check, a forecasting tool to project the number of users and adapters for each method and help plan to meet resource needs. The team deems this paradigm extremely useful, providing a coherent overlay to supply side interventions to increase access to FP/RH services. Designing ACQ activities around this model is a strategic program approach. (The next chapter of this report discusses the approach of supply, demand, and advocacy in greater detail.) • Field funds drive ACQ rollout. Application of best models, approaches, and tools is mainly driven by Mission interest, demand and available funding rather than by where the need might be greatest. ACQ has been successful in marshalling field resources: Most of its funding comes from the field, indicating a high level of demand for ACQ services. However, the missions that provide the funding also dictate what ACQ does in the field. This is not necessarily the ideal or most strategic way to determine what countries would benefit from the program and how they should be involved. This constraint is common to many global projects. Its impact is something GH/PRH should carefully consider. 20 Strategic Appraisal of the ACQUIRE Project • Special considerations for programming facility-based RH/FP services. Demographic and program data1 and ACQ experience to date clearly identify LAPM as a critical and continuing need within the GH/PRH portfolio of centrally funded projects because: – LAPM are a vital component of any program to meet the great and growing FP/RH needs. Because LAPM depend for their delivery on clinical skills and expertise, issues of medical safety, medical quality, informed choice, and side effects management are paramount to delivery of quality LAPM services. A global project can be expert in all of these parameters. – Missions often cannot devote enough resources or time to LAPM due to competing priorities, such as PEPFAR or the Presidential Malaria Initiative (PMI); and to external factors, such as the human resource crisis in many countries and restrictions on collaborating with certain private clinical organizations. A global program can help by focusing on vital aspects of a FP program. LONG-ACTING AND PERMANENT METHODS • Intrauterine devices • Implants • Female sterilization • Vasectomy – A mission’s implementing agency usually has only a few staff with clinical expertise in LAPM and finds it difficult to provide the depth and breadth of clinical capability, supervision, and knowledge and experience needed to make LAPM service more available and accepted. A global project can bring that specialty expertise. As in the current ACQ project, any future LAPM-focused, centrally funded project will need a critical mass of staff with clinical expertise who can work across countries, provide global technical leadership, transfer best practices and lessons learned, and contribute to state-of-the-art LAPM programming. ACQ has created an outstanding Advocacy Kit for promoting use of LAPM. It should present this package to all USAID missions and to the international FP community to increase awareness of the importance of LAPM in meeting the need for FP. Over the life of the project, ACQ has placed emphasis on building the skills and capacity of providers and programs to provide all LAPMs. However, because the demand for IUD and vasectomy is not as high as for implants and tubectomy, ACQ has focused its communications efforts on IUD and vasectomy to promote the image of these methods, dispel myths, and increase demand for these methods. Female sterilization has been consistently popular with women who want to limit their family size. Implants, although highly popular, are frequently in short supply, or out-of-stock; for these reasons, ACQ has not invested in a communications campaign for implants, although providers are widely trained and offering the method. In addition, based on ACQ’s innovative and successful work with men, USAID may want to consider how to replicate and scale up programs for men as both clients and supportive partners in both future LAPM-focused projects and in general FP programs. The team did observe that ACQ has been so responsive to all of the demands from GLPs and the field and has taken on such a broad array of activities that the program has become very complex. The breadth of the program is wide. While this may in some ways be good, it can also overextend the system and dilute its total impact. ACQ needs to advance cautiously in its last year, concentrate on its legacy, and work out how to institutionalize its interventions for the long term. 1 Roy Jacobstein, “Reasons and Supporting Evidence for Substantial Investments for Long Acting and Permanent Contraception” (Working Draft), Acquire Project, April 2007. Strategic Appraisal of the ACQUIRE Project 21 Lessons Learned About Strategic Programming • Sound programming models facilitate purposeful programming for maximum results. • Given the unmet need worldwide of women wanting to limit or space childbearing who can benefit greatly from LAPM, there is a need for robust clinical services. • If field funding (or the lack of it) alone drives activities, this may lessen strategic programming and skew the worldwide impact, as is currently the situation with the ACQ program. Recommendations for Strategic Programming 1. USAID must take a strategic approach to programming for LAPM. It might direct core funds to those countries that have a demographic need and the potential for results (“push”) while concurrently encouraging missions that express an interest in or are willing to provide funds for scale-up, to use a GH mechanism (“pull”). 2. USAID should include in its portfolio of centrally funded projects, a program that addresses the critical unmet need for LAPM. 3. USAID and ACQ should use and promote the LAPM Advocacy Package to help missions understand the importance of using LAPM to address unmet FP needs. 4. ACQ should continue to document, disseminate, and institutionalize best practices, lessons learned, and proven interventions to ensure the long-term sustainability of its efforts. 22 Strategic Appraisal of the ACQUIRE Project Strategic Appraisal of the ACQUIRE Project 23 V. ACQUIRE’S OPERATIONAL MODEL: INCREASED ACCESS, QUALITY, AND USE OF FP/RH SERVICES Findings, Lessons Learned, and Recommendations A. ACQUIRE’S Operational Model Over the past four years, ACQ has built a model that is now serving as its foundation (Figure 5). The team believes that ACQ’s operational model for FP/RH service delivery reflects a holistic understanding of the dynamics of the health systems of the country partners to whom the project provides technical assistance. ACQ seeks to foster change that will increase access, quality, and use of services by taking an integrated approach to supply, demand, and advocacy interventions. The model applies to FP/RH services2 in general as well as specifically to clinical FP services, the program area in which ACQ primarily works. It can be applied in national, regional, and district programs. Because the ACQ operational model is at the core of this report, the way it is presented in the ACQ Self Assessment Paper3 is represented visually here before being discussed. How this model has been applied and the results in the three focus countries are covered later in this chapter. Subsequent chapters show how the model is applied to ACQ’s Programs for the Global Leadership Priorities and Obstetric Fistula. Figure 5: ACQUIRE’s Operational Model Quality client-provider interaction Quality client-provider interaction Supply Demand Advocacy • Supportive service policies promoted • Human and financial resource allocation fostered • Gender equity advanced • Supportive service policies promoted • Human and financial resource allocation fostered • Gender equity advanced • Service sites readied • Staff performance improved • Training, supervision and logistics systems strengthened • Service sites readied • Staff performance improved • Training, supervision and logistics systems strengthened • Accurate information shared • Image of services enhanced • Communities engaged • Accurate information shared • Image of services enhanced • Communities engaged Fundamentals of Care Stakeholder Participation “More More More More More Services People Places Services People Places” to in Data for Decision-Making Increased knowledge + acceptability Increased availability Improved policy + program environment Improved Access, Quality and Use 2The model applies across the range of possible clinical services, service sites, and providers. 3 Self Assessment Paper for the USAID Evaluation of the ACQUIRE Project, ACQUIRE Project, submitted to USAID, Washington, DC February 20, 2007, and presented at a briefing to the Appraisal Team on February 26, 2007. 24 Strategic Appraisal of the ACQUIRE Project The ACQ model situates at its center the service encounter between client and provider. ACQ sees the ideal client-provider interaction as taking place between a knowledgeable, empowered client and a skilled, motivated service provider at an appropriately staffed, managed, and functioning site. It contends that quality client-provider interaction is enabled by well-functioning supply-side and demand-side elements that operate within a supportive policy and activity environment. The client-provider interaction contributes to improved access to, and use of, quality services. In the aggregate these interactions result in “more services to more people in more places,” the program goal of both ACQ and those it assists. • On the supply side, solid service delivery components or subsystems—training, supervision, and commodity supply logistics—ensure service site readiness and result in increased availability of skilled, motivated, well-equipped, and well-supported FP/RH service providers offering a range of contraceptive methods. The service providers are attuned to gender issues and are skilled in addressing the FP/RH needs of men and women, separately and as couples. Supervisors work to enhance performance of providers and facilities, reinforcing training and solving problems to improve service quality. • On the demand side, multiple communication channels—interpersonal, community, mass media— are used to provide up-to-date, understandable information about specific methods and services, such as where they can be accessed, to increase the knowledge of clients, potential clients, and communities. Freed of misconceptions and myths, the image of FP/RH is enhanced as communities are mobilized to ask more of their health service systems, so that use of FP/RH methods and services by women, men, and couples becomes the norm, and thus client demand for them is better supported. • For ACQ, advocacy is an integral part of the operational model, because an FP/RH service delivery program not only functions within a health system and a community but is also embedded in a larger sociocultural, gender, economic, and political environment that heavily influences the nature and extent of service. ACQ therefore works to (a) promote supportive and rational service policies based on the best available medical and program evidence and (b) foster allocation of human and financial resources to meet current and unmet need for FP/RH services. The improved policy and program environment for FP/RH services that leads to a better resourced, more productive, more widely supported, and more sustainable program. ACQ also considers it important to identify and nurture champions at all levels; focus messages and interventions on the benefits of FP/RH as clients and providers themselves perceive them; direct program efforts to potential early adopters and satisfied clients; and design and implement pilot efforts with replication and scale-up planned from the start and those expected to implement the replication or scale-up immediately involved. ACQ also advances gender equity through its MAP program, which works with communities, providers, and program managers to promote constructive male involvement. Three important considerations underlie the ACQ operational model because they are of central importance to maximizing access, quality, and use of services, and because they cut across supply, demand, and advocacy. It is necessary to • Ensure provision of the fundamentals of care—informed choice, medical safety, and continuous quality improvement, which are the sine qua non of FP/RH services, especially clinical services. • Use data for decision-making—both locally generated and international data—in program design, implementation, and evaluation, as the basis for sounder programs, greater transparency, and greater ownership. Strategic Appraisal of the ACQUIRE Project 25 • Foster widespread and meaningful stakeholder participation, which is critical to program ownership, success, and sustainability. Political leaders, religious and other opinion leaders, program leaders and managers, the medical community, clinic managers and service providers, advocacy groups, community organizations, and individual female and male clients are all consulted. Access quality and use is situated in the ACQ model as the final common pathway by which supply, demand, and advocacy activities bring about better quality and more use of services by more people in more places. ACQ has worked toward smooth, interrelated functioning of these program components. They come as a group; if one component is missing, the synergy is lost. If you have services but no clients, or if you have clients and no services, it does not work. In addition, for services to be effective, the policy environment must be positive. B. Integrated Interventions of Supply, Demand, and Advocacy The three matrices on supply, demand, and advocacy that are included in Annex A highlight specific activities that ACQ has taken to increase access, quality, and use through the integrated interventions of supply, demand, and advocacy. The matrices outline planned interventions, provide concrete accomplishments that ACQ achieved at both the global and country levels, discuss key findings, and offer lessons learned and recommendations. In terms of significant outcome, Table 4 highlights the service statistics for ACQ’s three focus countries— Bangladesh, Bolivia and Tanzania. Data in the table are illustrative of the positive action toward increasing access to FP/RH services by using the supply-demand-advocacy approach. ACQ’s target was to increase the number of supported sites prepared and ready to deliver quality FP/RH services. ACQ’s PMP defines “supported sites” separately for each country, but some technical or financial input is required to be counted. While the team did not have information on the nature of that input beyond the definition of supported site, the number of sites ACQ supported increased dramatically from 954 to 3,297 in Bangladesh, 189 to 357 in Bolivia, and 55 to 400 sites in Tanzania (see Table 4 below). This data illustrate to what extent ACQ reached new facilities to improve access to FP. In all three countries, ACQ promoted family planning client services with a focus on LAPM. Data from those three countries give an indication of forward movement. The number of LAPM clients served increased in all countries from 2003/4 through 2005/6. The number of injectable contraceptive method clients in Tanzania also increased during this time frame. However, because the number and type of supported sites also increased across years, the team could not rigorously evaluate the impact of ACQ’s support on the number of clients served at the sites. A rigorous evaluation of the ACQUIRE project will be presented in the end-of￾project report using the final evaluations in the focus countries and results from special studies. Meanwhile: • In Bangladesh, where there was a 3.5-fold increase in the number of sites supported, number of clients served increased 5-fold. In FY 05/06, ACQ-supported sites completed 65,992 female sterilizations, completed 49,334 male sterilizations, and inserted 72,911 implants (Table 4). • In Bolivia, the numbers of FP clients served doubled from 2003/4 to 2005/06, with a drop off from 2004/05 to 2005/06 due to a shortage of IUDs (Table 4). From 2003/04 to 2005/06, male sterilizations increased by 150% and IUDs by 90% (Table 5). • In Tanzania, where there was an 8-fold increase in the number of sites, the numbers of FP clients served increased 4-fold (Table 4). Female sterilizations and IUDs increased by 200% and 175%, respectively (Table 5). 26 Strategic Appraisal of the ACQUIRE Project Service statistics should be interpreted with care because of problems of efficient and accurate reporting by the national management information systems (MIS) from which ACQ gets its data. ACQ will undoubtedly provide some narrative on this subject in the next annual report to indicate the quality and significance of the data. In addition, there has not yet been extensive cost analysis work; it might be important to incorporate cost analysis in future reports. Strategic Appraisal of the ACQUIRE Project 27 Table 4: Summary of Service Delivery Use in ACQUIRE Focus Countries4 Bangladesh No. of sites supported 954 2,485 3,297 No. of FP clients served 85,158 247,359 423,474 755,991 Female sterilization 12,596 45,494 65,992 124,082 Male sterilization 9,781 35,640 49,334 94,755 IUDs 48,573 121,959 235,237 405,769 Implants6 14,208 44,266 72,911 131,385 Injectables7 0 0 0 0 Bolivia No. of sites supported 189 355 357 No. of FP clients served 3,150 6,959 6,339 16,448 Female sterilization 696 1,575 1,748 4,019 Male sterilization 10 15 20 45 IUDs8 2,444 5,369 4,571 12,384 Implants 0 0 0 0 Injectables 0 0 0 0 Tanzania No. of sites supported 55 138 400 No. of FP clients served 145,124 322,863 597,895 1,065,882 Female sterilization 6,092 13,449 18,150 37,691 Male sterilization 72 81 183 336 IUD 1,114 2,129 3,062 6,305 Implant 2,142 9,396 16,430 27,968 Injectables 135,704 297,808 560,070 993,582 FY03/04 (B)5 FY04/05 FY05/06 Totals 4 set up parallel, project-specific systems. The service statistics may therefore be subject to country-specific infrastructural issues. In Bangladesh, data are nationwide and the national MIS is very good. ACQ staff conduct joint field trips with local government MIS officers and USAID to assess data quality and, where necessary, take necessary corrective measures. The Tanzanian MTHUA MIS and the Bolivian SNIS MIS have major issues: lag times of several months between data collection and data entry/validation, incomplete or missing service statistics, or names or designations in the MIS system that are different from those on the ground. The Tanzania program recently adapted the Bangladesh approach and will build data quality assessment visits into the work plan. Only about 90 percent of the Tanzanian sites are fully represented in the data. 5 (B) indicates baseline 6 The Bolivian MOH does not authorize implants; supplies are not provided as part of the Bolivian Health Insurance System and are not available at health facilities. 7 Injectables are traditionally categorized as short-term rather than LAPM; they are included in the Tanzania data because over time, the project changed its focus to include many more dispensaries and health centers/maternity homes in new areas. Currently about 55 percent of supported sites are dispensaries; 30 percent are health centers/maternity homes; and only 15 percent are hospitals. It must be emphasized that the vast majority of the health centers/maternity homes and dispensaries provide short-term methods (pills, injectables, and condoms). Some provide at least one long-term method, mostly Norplant. Sterilization is not provided at most of the health centers/maternity homes or any of the dispensaries. 8 In Bolivia, there was a drop in IUD clients due to a shortage of supplies. All data are taken from national MIS because ACQ’s philosophy is to build the capacity of local staff in data management rather than 28 Strategic Appraisal of the ACQUIRE Project Table 5: Gains in Service from 2003/04 to 2005/06 Bangladesh Bolivia Tanzania FY03/04 FY05/06 %Change FY03/04 FY05/06 %Change FY03/04 FY05/06 %Change No. of sites supported 954 3,297 246% 189 357 89% 55 400 627% No. of FP clients served 85,158 423,474 397% 3,150 6,339 101% 145,124 597,895 312% Female sterilization 12,596 65,992 424% 696 1,748 151% 6,092 18,150 198% Male sterilization 9,781 49,334 404% 10 20 100% 72 183 154% IUDs 48,573 235,237 384% 2,444 4,571 87% 1,114 3,062 175% Implants 14,208 72,911 413% 0 0 0% 2,142 16,430 667% Injectables 0 0 0% 0 0 0% 135,704 560,070 313% Lessons Learned About FP/RH Service Programs • The ACQ Access Program Model has been successful and has proven that it can be replicated in different settings with different content (see Chapters VI and Chapter VII). • The model has been challenging to implement in some resource-poor settings, but the difficulty of using a comprehensive/integrated approach has been counterbalanced by gaining local empowerment, ownership, and program sustainability. • In its last program year, ACQ will complete evaluations of interventions. It will synthesize findings from across countries and disseminate key aspects of the approaches for use in future programs. Recommendations for FP/RH Service Programs 1. In future programs, USAID should encourage the holistic ACQUIRE Operational Model, which integrates supply, demand, and advocacy interventions to improve access, use, and quality of FP/RH services. More details about supply, demand, and advocacy interventions—i.e., purpose of the intervention, global and country accomplishments, lessons learned, and specific recommendations—can be found in Annex A. Strategic Appraisal of the ACQUIRE Project 29 VI. GLOBAL LEADERSHIP PRIORITIES Findings, Lessons Learned, and Recommendations Global leadership priorities are cutting-edge issues that affect FP/RH programming and require dedicated attention. Each GLP has a USAID champion and a working group that establishes a strategic direction, identifies priority areas for investment, and coordinates the activities of a number of centrally managed agreements. As noted in the sidebar, most of the GLPs focus on specific health initiatives. Because of its specialized focus on clinical methods, USAID asked ACQ to assume responsibility for GLP field activities. ACQ has been able to incorporate GLP activities easily and effectively into its on-going activities. Given the limitations of time, the team selected only five of the GLPs for in-depth review: Global Leadership Priorities 1. Family Planning/HIV Integration* 2. Repositioning Family Planning 3. Contraceptive Security* 4. Female Genital Cutting 5. Gender Issues* 6. Maximizing Access and Quality* 7. Population and Environment 8. Post-abortion Care* 9. Refugee Reproductive Health 10. Youth Maximizing Access and Quality – Implementing practical, cost-effective, and evidence-based interventions for improving access to and quality of services. Contraceptive Security – Maintaining and supporting a system that allows every person to have access to and be able to freely choose, obtain, and use quality contraceptives for FP and HIV/AIDS prevention. Family Planning/HIV Integration – Encouraging the efficient and effective blend of FP and HIV services. Post-Abortion Care – Addressing complications related to miscarriage and incomplete abortion by improving treatment and care and linking women to FP/RH services. Gender 9 Promoting gender equity and integrating gender issues into service delivery programs to eliminate barriers and create a positive environment for *Reviewed by this appraisal maximizing health outcomes. ACQ has done an excellent job of promoting GLP interventions (Annex A). Major constraints due to lack of investment or motivation, limited funding and ability to influence partner organizations, a short time frame, and shifting priorities did not prevent it from leading global efforts and crafting innovative field approaches. 9 Because of the cross-cutting nature of gender and its relevance to increased access, quality and use of FP/RH services, gender integration was expected to be mainstreamed and occur strategically within ACQ primarily through the use of core and field support funds (i.e. independent of special GLP funds). Examples of how this has occurred are included in throughout out the sections on supply, demand, and advocacy as detailed in Annex A. 30 Strategic Appraisal of the ACQUIRE Project Table 6: Major Key Findings regarding the Global Leadership Priorities GLP FINDINGS MAQ ACQ bases its service strategies on MAQ principles, and notably has led the world in field testing approaches to revitalize the IUD to “maximize access and quality” in FP. CS Many ACQ programs have engaged in CS activities, but participation has been opportunistic rather than part of an explicit central strategy. ACQ has run into critical barriers to quality service delivery, such as stock-outs and demands on service providers’ time for accurate record-keeping and supply management. FP/HIV ACQ has successfully integrated FP/HIV activities, including FP integration into antiretroviral treatment (ART) and care services in Ghana and implementing a community awareness program linked to FP for people living with HIV/AIDS in Uganda. PAC ACQ led PAC initiatives in Kenya, Tanzania, and Cambodia. Training, policy development, and organized program observation helped stakeholders learn more about implementing country-specific PAC strategies. Gender ACQ has done well in expanding the capacity of regional and country field-based staff, but has not been as effective in increasing the number of global staff able to provide TA in gender. Quality and scale-up of ACQ interventions are satisfactory where there is a close relationship between the local partner and ACQ. Lessons Learned from the GLP Experience • Planning and implementing organizations must look at supply and demand interventions, community￾based and policy interventions, clinical and non-clinical performance improvement, and changes in the behavior of clients, providers, and systems. (MAQ) • ACQ can improve services if it continues to place programmatic focus to include resupply methods as an integral part of facility-based work. (CS) • Ensuring that contraceptive commodities travel the “last mile” is critical to FP quality and access. (CS) • FP can be vital in preventing HIV/AIDS in mothers and children, though HIV programs focus little on this aspect. Because of the complex nature of integration, it is essential to allow enough time to assess the impact of FP/HIV integration activities so that ACQ can document their value. (FP/HIV) • Solid collaboration among community members, such as leaders and health personnel, results in greater commitment at the central, provincial, and district levels. A foundation of commitment among a variety of stakeholders is necessary for successful and sustainable PAC interventions. (PAC) • Mainstreaming male involvement, MAP, and gender approaches into program strategy reduces challenges to local buy-in by partners and missions; but to be effective this needs technical assistance, capacity building, and effective documentation and synthesis. (Gender) Recommendations for Global Leadership Priorities 1. USAID and ACQ should collaborate strategically with partner organizations on issues that concern improved service delivery. 2. USAID should sustain and build upon GLP integration into GH programs, making sure to institutionalize lessons learned from ACQ. 3. USAID should advocate and promote the GLPs at the national level (including missions) and make sure to incorporate GLP frameworks into National Strategies. 4. ACQ should gather and document successes and processes, such as (a) the service delivery provider perspective of CS, (b) program implementation of PAC in Bolivia and Kenya to determine ability to scale up, and (c) gender interventions. 5. ACQ should expand efforts to promote and disseminate best practices, lessons learned, and technical updates at the global and country levels. Strategic Appraisal of the ACQUIRE Project 31 VII. TREATMENT AND PREVENTION OF OBSTETRIC FISTULA Findings, Lessons Learned, and Recommendations The ACQ fistula program began in 2005 in response to an amendment to the Foreign Relations Authorization Act that tasked USAID with “establishment and operations of not less than twelve centers for the treatment and prevention of obstetric fistula at appropriate sites in developing countries.” Given facility-based FP/RH, fistula activities were assigned to ACQ as a logical home. USAID is now considering an associate award for future activities. The focus has been on building and firming up systems to address fistula repair in quality clinical surgical settings. The portfolio is gradually expanding to address prevention and reintegration. Challenges to Fistula Services • Lack of research, indicators, guidelines of care, and promising approaches • Lack of infrastructure • Lack of skilled providers to carry out repairs • Complexity of fistula repair • Broad continuum of services, from detection and repair to prevention and reintegration, incorporating technical, medical, and sociocultural issues • Differences in sociocultural issues between obstetric and traumatic fistula The program has the following key intervention areas: • Conduct prospective research on causes, severity, and outcome of treatment for fistula. • Improve public and community awareness about causes and medical and social consequences. • Improve services for prevention and treatment of obstetric fistula. • Improve advocacy and policy support for improved access to safe delivery and care for obstetric complications. (Annex A.) Accomplishments include: • Rapid roll-out and scale-up from two to eight country programs, with two regional programs. Given the limitations of clinical capacity, clinical guidelines and surgical techniques, and international consensus and collaboration building, ACQ’s ability to expand programs so rapidly is commendable. • Strategic tailoring of approaches to country circumstances (Table 7) by identifying resources, establishing partnerships, and prioritizing sites and services to maximize output. This approach built on the natural advantages and networks of each location. 32 Strategic Appraisal of the ACQUIRE Project Table 7: Tailored Country Examples Ethiopia (quality ACQ focused on collaboration to strengthen and expand services, adding a FP/RH services already focus on awareness-raising, fistula screening and referral, and strengthening in place) maternal health services at health centers. Guinea (some ACQ increased coordination between the maternity and urology departments services available in to ensure proper diagnosis and repair, while enhancing birthing and emergency the Urology obstetric care skills for fistula prevention in the maternity unit. Department) Uganda ACQ teamed public and private hospitals in collaborative sites to improve both fistula repair capability and emergency obstetric care. • Groundbreaking technical work, though the technical field is in its infancy. While strengthening systems and moving services forward, ACQ has also worked to strengthen the scientific base of its fistula work and to build international consensus on clinical procedures. • Adapting the MAP curriculum in Bangladesh and Uganda. ACQ uses MAP curricula to encourage men’s support of women to seek fistula repair and adopt behaviors that help prevent it. Challenges in implementing the fistula program include: • Lack of clarity in mandate. Congress mandated the establishment of fistula repair sites. The project attempted to address fistula along the continuum from detection and repair to prevention and reintegration. ACQ’s first priority was to build country capacity for fistula repair. Because it was not clear how much the current program should invest in prevention and reintegration, ACQ determined need based on country capacity. ACQ has worked closely with in-country partners on reintegration, but its ability to work on prevention depends on access to additional resources. Increasingly, the program has incorporated community outreach under the rubric of prevention, but it does not appear that ACQ has clearly defined prevention or set goals, articulated desired outcomes, or measured progress and impact for its prevention work. Current prevention approaches center on: • Provider training, outreach, and awareness • Community outreach events, such as “fistula week” • Community theater events • Training health outreach workers to integrate fistula prevention in health messages • Training midwives in use of partograph for monitoring labor (Ethiopia, Guinea) • Training physicians in C￾sections (Guinea) • Lack of clarity in tailoring programs to obstetric vs. traumatic fistulae. ACQ has no routine monitoring indicators that specify whether fistulae are obstetric or traumatic; these will be gathered in the prospective research study. From a clinical/surgical perspective, the issues are the same, but the differences are much more relevant when addressing interventions related to prevention, counseling, psychological rehabilitation, and reintegration. Lessons Learned About Obstetric Fistula • ACQ would benefit from collaborating with organizations that have expertise in developing economic livelihoods opportunities to help with women’s reintegration. • It is vital to build on and expand linkages with existing networks to integrate prevention and reintegration approaches quickly and more effectively without wasting resources. Strategic Appraisal of the ACQUIRE Project 33 Recommendations for the Obstetric Fistula Program 1. USAID should set clear parameters and expectations for each component (detection, repair, prevention, and reintegration) to help the program move forward and achieve maximum impact. 2. USAID should articulate clearer guidance with definition and indicators for fistula prevention. 3. ACQ should work systematically with men as partners and agents of change to promote repair and prevention. 4. ACQ should expand the GBV component of its fistula program through consultation and collaboration with organizations that have GBV expertise. A concerted effort to address prevention and reintegration would require: • A distinction between obstetric and traumatic fistulae • The design of interventions that address individual and community norms pertaining to behaviors that contribute to fistulae • Development of appropriate indicators to monitor change in attitudes and behaviors at the community level 34 Strategic Appraisal of the ACQUIRE Project Strategic Appraisal of the ACQUIRE Project 35 VIII. EVALUATION AND RESEARCH To inform its work and demonstrate results, ACQ has conducted evaluation and research studies that span its portfolio (see Annex B.4), including: • Research projects to investigate specific questions • Special activities, projects, or evaluations to assess effectiveness of an intervention or program • Assessments to determine the status of services in a country or region, generally conducted before implementation or reconfiguration of a program (e.g., needs or baseline assessments) A. Baseline and Final Studies ACQ conducts baseline and final studies in its focus countries, where technical input and resources are concentrated. These studies use four main tools—facility audit, client-provider observation, client exit interviews, and provider interviews—to provide a comprehensive picture of service delivery. The instruments were adapted from materials developed by Measure Evaluation, Macro International, and others. ACQ is conducting similar studies in Azerbaijan using different data collection instruments. The reports from the four baseline studies are well written and present detailed information systematically in tables and the text. ACQ received technical assistance from Measure Evaluation in designing the baseline and final studies, in particular determining the sample sizes for the studies (Table 8). Since the stated purpose of these studies is to “measure changes in access and quality indicators over time,” and the sample sizes appear robust, it is curious that ACQ has not taken advantage of its opportunity to demonstrate change through baseline and final studies on key indicators. Table 8: Select Baseline Study Features Length (months) Sample Size by Data Collection Instrument Country Time from Start Facilities Client- Client Exit Provider of Data Collection Provider Interviews Interviews to Report Interaction Bangladesh 24 121 240 245 193 Tanzania 30 310 773 757 681 Bolivia 11 234 200 523 524 Azerbaijan 16 76 N/A N/A 293 B. Major Challenges • The quantity of data collected has affected timeliness. ACQ collected considerable data in both its baseline assessments in focus countries and its PNAs. The amount of data to be organized may have contributed to extending the time it took to complete reports of the assessments. • The lack of initial indicators will make it difficult to determine change over time. Although the baseline studies provide detailed information that is well organized, there is no evidence that ACQ and its stakeholders attempted to identify indicators to track change over time in access, quality, and use. Because the country teams did not have indicators, they did not set benchmarks for the changes they wanted to see by the end of the project. The lack of indicators with point estimates and confidence intervals makes it difficult to measure change over time with any degree of confidence. • Varied funding streams make planning difficult. Most studies receive funding from a combination of core and field support. In the majority of the 49 studies listed in Annex B.4, core provides limited technical assistance to the design of the study. Country programs, including those with associate awards, have their own agendas and priorities, which make it more challenging to control the focus and reporting of results. ACQ may need more core funds to provide technical assistance to country programs to improve the quality of data collection and reporting. • ACQ is collaborating with other agencies. ACQ collaborated with the Frontiers Project (Population Council) on two studies (Testing the Effectiveness of Community-based Strategies in Reducing GBV and Promoting Positive Male Involvement in FP/HIV/AIDS Activities and Scaling Up PAC Services in Tanzania) and with Family Health International (FHI) on two studies (Community Awareness of and Barriers to LAPM in Guinea and the Tanzania Vasectomy Case Study). More clarity and agreement on expectations and schedules would be helpful at the beginning of such collaborations. However, the relationships are good, and the lessons learned through these joint studies will serve the organizations well in the future. Recommendations for Evaluation and Research 1. USAID should ensure that cooperating agencies have sufficient core funds to provide targeted technical assistance to field projects, including those funded in the field, especially if the global program is fully or partly responsible for the results. 2. Service delivery projects should set their own research agenda; when they collaborate with another organization, study design, roles and responsibilities, schedule, deliverables, publications, and acknowledgement should be thoroughly discussed and documented in writing. 3. Service delivery projects should have in-house expertise in monitoring, evaluation, and research and be able to access technical assistance from experts and specialty projects as needed. 4. ACQ should streamline its tools and instruments for data collection so that data are more manageable and findings can be shared more readily with project implementers and stakeholders. 5. ACQ should develop indicators based on data collected in baseline studies in the three focus countries and measure the change at the end to demonstrate results. 36 Strategic Appraisal of the ACQUIRE Project Strategic Appraisal of the ACQUIRE Project 37 IX. PERFORMANCE MONITORING The ACQ project has a well-structured system for tracking progress and results. Its main components are a performance monitoring plan, annual reports, and management reviews. A. The Performance Monitoring Plan ACQ’s results framework (Figure 1, Chapter II)—including its intermediate results (IRs), sub IRs, and global and field indicators—is responsive to the PRH SO1 results framework and indicators. ACQ’s performance indicators include field indicators that correspond to ACQ’s global results. ACQ’s PMP shows, in a series of tables, the indicators that correspond to (1) country categories (LAPM, IUD, fistula, integration, etc.), and (2) funding categories and GLPs. The PMP also lists all studies supported through ACQ. FIG U R E 2. A C Q U IR E PER FO R M A N C E IN D IC A TO R S Figure 6. ACQUIRE Performance Indicators SO : R eproductive H ealth and Voluntary Fam ily Planning Services Supported and Advanced, w ith a Focus on Facility-B ased Services # of A C Q U IR E-supported sites # of clients accepting services at supported sites # of A C Q U IR E-supported countries that have integrated M A P into current activities or are im ple m enting standalone M A P p rogra m s # of com m unications m aterials developed # of prog ram m onths dem and cam paign/prog ram im ple m ented # of tim es a T V o r radio spot on a particular m ethod is aired # of people reached through com m unications cam paign and/or co m m unity m obilization event # of tools / m odels developed to increase access to R H /FP services # of countries w here A C Q U IR E-developed tools / m odels have been im p lem ented to support increased access to R H /FP services # of tools / m odels developed to strengthen provider perfo rm ance # of countries w here A C Q U IR E-developed tools / m odels have been im p lem ented to strengthen provider perfo rm ance # of trainin g events conducted by prim ary trainin g type # of persons trained by prim ary trainin g type and cadre % of persons trained in clinical trainin gs w ho perform ed to established gu idelines and standards by the end of the trainin g, by training type and cadre # of prod ucts distrib uted by type and country program # of hits on key docum ents on the A C Q U IR E w ebsite # of m eetings convened to advocate for R H /F P services, by type Global indicators U se of m odels and to ols at field level % of sites repo rting a supervision visit in the past six m onths that included an observation of a client-provide r interaction and superviso ry feedback Menu of Field indicators # of countries w here A C Q U IR E has provided support to reduce barriers to services # of countries w here A C Q U IR E has provided support to national norm s, standards and guidelines B. Global and Field Indicators in the PMP ACQ has seven global indicators and 13 field-level indicators (Figure 6). The global indicators, which correspond to ACQ IRs, have planned values listed in the PMP. ACQ reported current values to USAID in its most recent annual report (July 2005–June 2006). The values of all field-level indicators will be reported to USAID starting in the next annual report (July 2006–June 2007), although data relating to four of the 13 indicators—those derived from service statistics—were included in the most recent annual report for the three focus countries. ACQ consults with country-level programs as they draft their own PMPs and encourages them to include field indicators from the global PMP, which primarily measure outputs, in addition to other indicators relevant to their programs, such as those that align with the mission’s program. Each of ACQ’s three focus 38 Strategic Appraisal of the ACQUIRE Project countries has a PMP. These include a few outcome indicators related to FP, such as couple-years of protection (Tanzania) and new acceptors (Tanzania and Bolivia). ACQ is not only conducting baseline and final studies in focus countries to measure change over time in the availability and quality of FP/RH services, it is measuring change in the values of certain service statistics. If ACQ had key indicators, those could have been included in the country-level PMPs and helped guide the reporting of results. As it stands, the rich data collected by the baseline studies, while useful to engage stakeholders, has not yet been used to its full potential to rally local project staff and stakeholders to determine what changes they want to make, and by how much, by the end of the project. Under current USAID policy global projects are not responsible for reporting on or making a direct link to outcome or impact measures; in a global project like ACQ, the PMP indicators are therefore mainly process. Although often recipients like ACQ report outcome information in narrative form, for the most part the global project indicators monitor output, not outcome or impact. GH/PRH is already field testing more rapid assessment instruments to link project in-country efforts to outcomes, such as contraceptive use, but it needs to accelerate this process if it is to be useful to projects now on the drawing board. C. Reporting Results through Annual Reports and Management Reviews The ACQ project, like all projects managed by PRH/SDI, submits annual reports in October, and undergoes management reviews every six months. In the annual reports, ACQ highlights its major results by IR in evidenced-based narratives that include country-level supporting data. The reports also give current values for ACQ’s global indicators, a summary of evaluation and research studies, and a summary of its funding status. The appendices include tables showing the status of major activities by IR and by country, as well as progress on field-level indicators. (The full set of 20 will be discussed in the annual report for July 2006–June 2007). Management reviews give both USAID and ACQ an opportunity to focus on a managerial or technical theme that is relevant to the project, such as the tools ACQ will create during the project, managing partners, challenges of meeting clinical training and performance improvement needs, monitoring and evaluation, and strategic vision. Management reviews also offer ACQ the opportunity to highlight progress and contributions to the global agenda and GH/PRH priorities. Recommendations for Monitoring Performance 1. USAID should encourage future global projects to include in their PMP indicators that (a) respond to PRH results and (b) measure outcome results, particularly where field activities are significant. 2. Global projects should, where applicable, direct their country programs to better align the indicators in country-level PMPs with the data they are capturing through any baseline and final studies. If indicators are developed, they can be included in the country-level PMP. The project would then be able to demonstrate change in the indicators over the life of the project. 3. Future global projects should consider the structure of ACQ’s PMP when designing their own. ACQ’s well-organized and concise PMP shows the relationship between global and field indicators that respond to the project’s activity objective. Strategic Appraisal of the ACQUIRE Project 39 X. MANAGEMENT OF THE ACQUIRE PROGRAM For the past several decades EH has maintained a long and stable relationship with USAID through a series of institutional cooperative agreements. ACQ is the largest project managed by EH; it represents about 50 percent of the total EH budget (not including associate awards). The current cooperative agreement is a leader with associate award, and ACQ has several associates (Bolivia, Azerbaijan, and Uganda), managed closely by the ACQ global team. The agreement has a total ceiling of $150 million and has so far received US$80.3 million (including associate awards). The cooperative agreement requires that EH raise a 10 percent cost share from non-USAID funding. ACQ was able to meet this requirement early in the project through a combination of foundation grants, in-kind and pro bono communication services in the field, and private individuals, raising US$16 million dollars. EH reallocated staff in 2006 to eliminate regional offices and create a flatter structure. The core ACQ project team has designated staff from EH and each technical partner. The project staff is organized around the results framework, with teams representing each IR. Some global personnel also have country management responsibilities. To plan and coordinate programs ACQ holds periodic management meetings with technical and network partners and country program staff. USAID/PRH/SDI conducts management reviews with its CAs every six months. From each meeting there results a memo outlining issues for follow-up and recommendations. In the next management review, those issues are discussed to ensure that they have been addressed. The reviews address financial, technical, and other management issues. The cognizant technical officer (CTO)/technical advisor (TA) team have used this process effectively to address issues and institute changes in the management and technical direction of the ACQ project. A. Management Strengths • The project has talented, dedicated staff both at headquarters and in the field. USAID/Washington and country missions almost uniformly describe ACQ staff as highly responsive, flexible, and technically skilled. The project has a good mix of staff with field experience and long- term institutional memory, as well as having many talented local hires in field programs • ACQ has generally had an excellent working relationship and been a fair partner with other organizations on the project. Having learned from previous experience with partners, early on it created an operating ethos that is even-handed. Signed formal memoranda of understanding make agreements explicit and the parties then hold firmly to the terms of the agreements. Where this arrangement was not possible (e.g., bringing network partners from the field into ACQ), ACQ has tried to accommodate partners in other beneficial ways, such as designating special work or accepting pass￾though funding for their activities. • ACQ has invested in institutional and staff development. It has disseminated as standard operating procedures EH compliance procedures, state of the art, and best practices in FP technical knowledge, including standardizing clinical procedures, through workshops with field and global staff. These ensure that all staff, including field programs and associate awards, benefit from global procedures, knowledge, and practices. • ACQ has streamlined its organization. The EH reorganization increased efficiency, produced cost savings, and made for more straightforward and practical lines of authority. Although closing the regional offices was a painful decision, ACQ could no longer bear their cost. 40 Strategic Appraisal of the ACQUIRE Project • ACQ staff has an excellent working relationship with the USAID management team (CTO, TA) and field missions. ACQ values the CTO/TA team’s technical, managerial, and strategic thinking and believes their contributions have enhanced the project. The CTO/TA team was instrumental in providing technical assistance on obstetric fistula and PAC, and gave direction to the strategic visioning exercise that determined their program model and the legacy of “more services to more people in more places.” Nine of 13 missions surveyed were extremely pleased with ACQ’s management and performance. The other four noted some particular concern, but all have continued support to ACQ and two went on to fund major scale-up activities. • ACQ has used the required cost accounting tool for financial and program management. The monthly pipeline analysis that is a requirement of this cooperative agreement has proven to be a useful tool for financial management and cost control despite the extra burden of conducting the analysis. Both ACQ senior management and the USAID CTO/TA team have found the exercise extremely useful for assessing charges to cost accounts, spotting errors, managing costs, reprogramming funds, and identifying and dissolving bottlenecks. Pipeline analysis monthly is generally more than USAID normally requires, but it has served a very useful purpose for both ACQ and USAID. B. Management Challenges Shifting from an institutional to a competed cooperative agreement has required EH, and especially the project, to become leaner and more efficient and has raised the requirements for accountability. Generally, ACQ has performed well on these new requirements, but EH has been slower moving. Though ACQ is the largest project within EH’s portfolio, EH financial systems have not always been as flexible as necessary to meet ACQ requirements. ACQ has had to construct parallel financial systems and hire more financial staff because EH was unable to accommodate its needs and in some cases seemed resistant to doing so. ACQ has also accepted a number of pass-throughs for non-partners that have not covered financial and administrative costs. Because their cooperative agreement is structured so that they only earn a fee on the first $25,000 of each sub-award, they have shouldered the extra costs because they benefited partners. However, ACQ has sometimes been too accommodating to partners, to the detriment of the technical agenda. For example, the partner-designated position that was recently filled had been unfilled for almost the entire project, undermining the performance improvement and training area. When the partner was struggling to fill the position, ACQ could have reclaimed it but did not. Although the position is now filled with an extremely competent person, the delay did jeopardize solid program management. ACQ (EH) takes some responsibility for the original delay because their decision to change the team leader for IR 2 halted recruitment for several months at one point. With pressure increasing from USAID to bring in new partners in large consortia, ACQ may have brought on too many. It was not aware that ADRA is very decentralized or that SWAA would view ACQ simply as a funding source. While the partnerships with CARE and Meridian have worked out well, there is some question about what organizational capacity these partnerships really bring to USAID. Their project staff have never served in the home office of CARE or Meridian but were recruited for the project from the outside, so they were not familiar with their own corporate organization and culture despite their excellent qualifications and superior performance. Nevertheless, ACQ has managed to sustain its reputation as a technically sound and competent organization. Lessons Learned About Management • USAID is not always buying organizational capacity when it insists on large partnership consortia. • Partnerships seem to work better if the terms of the relationship are negotiated in advance and “rules of engagement” are well-defined and mutually accepted. Strategic Appraisal of the ACQUIRE Project 41 • Consolidation of project management under one roof has made it more coherent and less competitive among partners. • Partnerships are best managed at the global technical level; there should be more flexibility at the field level so that the project can link with partners that have the capacity, position, and relevance for particular country or project activities. • Many FBOs and private voluntary organizations (PVOs) are highly decentralized; working with their headquarters will not necessarily be effective in the field. Engaging each field office can be time￾consuming. When working with nascent indigenous NGOs especially, CAs must be aware that they will usually need financial flows. • PVOs can cost-share if they take an aggressive approach to fundraising. Recommendations for Management 1. USAID should evaluate the advantages and disadvantages of encouraging large consortia in new awards. 2. USAID should use Mission Strategic Plans and Operational Plans (part of the new reform process) to ensure that FP programs address fundamental functional and technical areas to increase access, quality, and use. 3. USAID should ensure that the cost-share percentages required in grants and cooperative agreements are not so low that they are easily raised early in the project or so high as to be unattainable. 4. ACQ should continue with its “can do” problem-solving approach to management. 42 Strategic Appraisal of the ACQUIRE Project Strategic Appraisal of the ACQUIRE Project 43 XI. PARTNER, MISSION, AND COUNTERPART PERSPECTIVES ON ACQUIRE ACQ is implemented and managed by EH and its partners, all of which bring a range of perspectives and competences to the project. At the country level, ACQ collaborates with both USAID missions and host country counterparts, such as MOH, NGOs, community organizations, professional organizations, and the private sector. At the global leadership level, ACQ also collaborates with multilateral organizations like WHO, UNICEF, and UNFPA and bilateral development organizations like DfID and GTZ (German Agency for Technical Cooperation). ACQ works closely with USAID headquarters, missions, and working groups and with other USAID cooperating agencies, contractors, and projects. A. Partner Relationships ACQ’s partners on the project are ADRA, CARE, IntraHealth International, Meridian Group International, and SWAA. The partners have distinct technical and programmatic roles, bringing expertise in community mobilization (CARE), communications and marketing (Meridian), training and performance improvement (IntraHealth), and reaching target populations through networks of other organizations and grassroots groups (ADRA and SWAA). ACQ designed the management agreement with its partners to ensure that all have an important role to play and a stake in the project’s results. This was done to avoid the misunderstandings and negative results that sometimes happen in consortia, particularly in terms of credit and acknowledgment issues. These have been minimized because the management team set ground rules very early about identity and acknowledgement of effort. The philosophy has been that all staff working on ACQ, including EH staff, are seconded to the project and well integrated into project activities, meetings, workspace, and access to information. In USAID terms, ACQ asked all partners to “leave their uniforms at the door.” All long-term staff assigned to the project, regardless of partner affiliation, are managed as a unified team. To ensure common understanding and agreement about the values that underpin this partnership, the partners subscribed to the following values: • Commitment to shared vision and objectives • Diversity and complementarily • Transparency • Proactive communication • Respect and trust • Participation • Flexibility and willingness to compromise • Recognition and celebration This management formula has worked well, as articulated in responses to a Partners Survey designed to gauge their opinion of the management team and its relationship with them. All five partners responded. Opinions of ACQ’s management team were positive. ACQ is seen as “exceedingly thoughtful and cooperative” in its “effective,” “fair,” and “transparent” management of the partnership. Staff are “very professional, friendly…problem-solving oriented and supportive.” They have “put into place the right kind of mechanism 44 Strategic Appraisal of the ACQUIRE Project to guide/govern the partnership.” However, one partner said that “ACQ representatives [in the field] do not either understand or favor the benefits of the partnership and have therefore not invited [the partner] to the table.” Not all partners were satisfied with their level of involvement and ACQ’s use of their expertise. Two would have liked more involvement in the focus countries (Tanzania, Bolivia and Bangladesh) and “more opportunities to contribute [the] expertise” they hold in “the core technical areas defined in the ACQ proposal.” On the other hand, another respondent says of ACQ’s “linking of supply and demand” (through its use of partners with varied expertise) that “based on all my years of doing this work for USAID, it seems to me that the ACQ project really has established a better way…to increase access/use of services.” That partner is “very happy” with the work it has been able to do and with ACQ’s support. B. Mission Partnerships ACQ has worked successfully to cultivate beneficial working relationships with USAID missions. Missions have a high opinion of ACQ; of the 14 questionnaire responses (out of 21 missions queried), a large majority spoke favorably of it. To gauge ACQ’s reception in the field, country and regional missions that worked with the project were asked to assess several aspects of ACQ and its relationship with the mission, including the project’s fit with the mission strategy, issues or concerns with ACQ’s performance, reasons for selecting ACQ, and general opinion of ACQ’s work in the country. The reasons for missions working with ACQ varied: USAID asked about a quarter of them if they wanted to participate in a funded global leadership activity, and the rest chose the program because of its technical expertise, specifically in LAPM, young married people, PAC, or obstetric fistula. Again, the general opinion of ACQ’s management, program flexibility, and competence is high. The relatively few negative responses were typically from countries that had been asked by USAID/W to work on promotion of a global concern. Most missions lauded the global staff for their experience and flexibility; one mission wrote of “an excellent, strong team that is open to new ideas, easy to work with and [able] to tap into other technical assistance opportunities” if ACQ’s skill set does not meet the needs of the planned work. One respondent commented that “there is great collaboration between the mission and EH.” Another wrote “USAID mission requests for information, reports and data…were addressed by ACQ staff in a timely manner.” Still, some missions were less enthusiastic, particularly about ACQ’s management of in-country staff. Though the office staff in one country are described as “very responsive and hard working, as well as technically [and] clinically competent,” the mission also speaks of “weak program management and design,” indicating the desire for “a strong program manager, perhaps with USAID expertise, in-country.” Another mission, though, describes its country’s team as “small but efficient” with “extensive experience” and a “dynamic” and “proactive” leader. C. Country Counterpart Partnerships The team had limited exposure to country counterparts, and the information here is anecdotal at best. One mission commented in the questionnaire: “ACQ is viewed as a partner with the MOH and working with the host government has increased the project’s acceptance in the country.” During the site visit to Tanzania the team heard from national officials that “ACQ did not develop a parallel program, but worked hand-in-hand to support, nurture, and assist the MOH in repositioning FP in the national program.” A district medical officer in Tanzania described ACQ as “a reliable partner. We can count on what they said will be done.” In two telephone interviews, host country officials noted ACQ’s technical competence and the high quality of its work. Several principals of cooperating agencies described ACQ as a reliable, fair, and productive partner. Strategic Appraisal of the ACQUIRE Project 45 XII. MAJOR CONCLUSIONS In summary, the team was favorably impressed with the ACQ program and would not recommend major changes. ACQ has stayed the course with the basics of service delivery as it has defined and refined its tools, approaches, and methodologies. It has introduced many innovations to improve the state of the art relative to facility-based service delivery. There may be some refinements needed, but essentially the program is on track and meeting expectations. ACQ provides very effective global leadership that interfaces substantively with the field. It uses core funds to support the design of global approaches that it then adapts to each country. Although ACQ sometimes provides seed money to pilot approaches and tools in the field, its client missions generally fund replication and scale-up. A number of countries have initiated the process: field-generated efforts have led to global recognition, and ACQ has then elaborated them for broader use. The team believes that this approach to programming allows for cross-fertilization of best practices and practical experience around the world. ACQ has also formulated a holistic operational model that intertwines three related interventions: supply, demand, and advocacy. ACQ has found that all three components are necessary for success; the absence of one can severely hamper a program. It is successfully using this integrated approach in its three focus countries—Bangladesh, Bolivia, and Tanzania. As a result, all ACQ programs supported increasing numbers of FP clients with LAPM procedures. FP service delivery statistics indicate that the ACQ supply-demand￾advocacy approach has increased access to quality FP/RH services. The team met with many talented ACQ partners, program leaders, managers and staff who are dedicated to advancing and supporting FP/RH facility-based services. The ACQ staff are keenly committed to providing global leadership and translating applications to the field level in practical and helpful terms. Host country counterparts acknowledged the effectiveness of the technical assistance, training, and material support of the ACQ program. Several government officials noted that the philosophy of ACQ was to help, support, and facilitate the country’s efforts, not to build a parallel system outside of the government or its institutions. In this report, the team has identified lessons learned and offers a number of recommendations, both general and specific. Some are directed to ACQ for program improvements, but the majority is intended for GH/PRH so it can use ACQ’s experience to inform future programming. Since there is only a year left until project completion, the team suggests that ACQ fully document its work and seek to disseminate proven methods, tools, and approaches so they can be incorporated into the programs of development agencies and national, regional, and district programs. (The major recommendations are in the Executive Summary.) ACQ has made significant achievements that contribute to the support and advancement of clinical services in FP/RH. Many of the experiences of the ACQ project will be useful in planning new activities for service delivery. The team hopes that this report will also be of use in supporting analytical thinking for PRH visioning and decision-making about future programming. 46 Strategic Appraisal of the ACQUIRE Project For more information, please visit http://www.ghtechproject.com/resources/ Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com