ASSESSMENT OF THE BASIC SUPPORT FOR INSTITUTIONALIZING CHILD SURVIVAL (BASICS II) PROJECT John Pielemeier Stacey Ballou Alan Foose Tim Quick Jack Reynolds April 2003 Submitted by: LTG Associates, Inc. TvT Global Health and Development StrategiesTM a division of Social & Scientific Systems, Inc. Submitted to: The United States Agency for International Development Under USAID Contract No. HRN–C–00–00–00007–00 This document is available in printed or online versions (POPTECH Publication Number 2003– 106–015). To review and/or obtain a document online, see the POPTECH web site at www.poptechproject.com. Documents are also available through the Development Experience Clearinghouse (www.dec.org). Printed copies and additional information about this and other POPTECH publications may be obtained from: The Population Technical Assistance Project 1101 Vermont Avenue, NW Suite 900 Washington, DC 20005 Telephone: (202) 898-9040 Fax: (202) 898-9057 admin@poptechproject.com Assessment of the Basic Support for Institutionalizing Child Survival (BASICS II) Project was made possible through support provided by the United States Agency for International Development (USAID) under the terms of Contract Number HRN–C–00–00–00007–00, POPTECH Assignment Number 2003– 106. The opinions expressed herein are those of the authors and do not necessarily reflect the views of USAID. ACRONYMS AND ABBREVIATIONS AED Academy for Educational Development AFR Africa Bureau AFRO African Regional Office AIEPI Atención Integrada a las Enfermedades Prevalentes de la Infancia AIN Atención Integral a la Niñez ARI Acute respiratory infection ANE Asia and Near East Bureau BASICS Basic Support for Institutionalizing Child Survival project CA Cooperating agency CAPA Catchment Area Planning and Action CDD Diarrheal disease control CHANGE Behavior Change, Communication and Social Marketing Innovation CHW Community health worker c–IMCI Community-based approach to IMCI CTO Cognizant technical officer ENA Essential nutrition action EPI Expanded Programme on Immunization FANta Food and Nutrition Technical Assistance project f–IMCI Facility-focused IMCI FY Fiscal year GAVI Global Alliance for Vaccines and Immunization GH Bureau for Global Health G/PHN Bureau for Global Programs, Field Support and Research, Center for Population, Health and Nutrition GTL Global technical leadership HealthCom Communication and Marketing for Child Survival Project HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome HKI Helen Keller International IACH Integrated approaches to child health IADB Inter-American Development Bank ICC Interagency coordinating committee IEC Information, education, and communication IMCI Integrated management of childhood illness IPT Intermittent preventive treatment IR Intermediate Result ITN Insecticide-treated bed net JHU Johns Hopkins University JSI John Snow, Inc. LAC Latin America and the Caribbean Bureau MEDS Monitoring, Evaluation and Design Support Project MinPak Minimum Package of Nutrition Interventions MNH Maternal and Newborn Health Project MOH Ministry of Health MOST USAID Micronutrient Program MSH Management Sciences for Health NGO Nongovernmental organization NID National Immunization Day OER Operations and evaluation research ORT Oral rehydration therapy PAHO Pan-American Health Organization PAIN Paquet d’Activités Intégrées Nutrition PASA Participating agency service agreement PATH Program for Appropriate Technology in Health PCHC Partnership for Child Health Care, Inc. PEB Performance evaluation board PNN Perinatal and neonatal PRITECH Technology for Primary Health Care Project PRM Program results monitoring PVO Private voluntary organization REACH Technologies and Resources for Child Health Project RFA Request for Application SARA Support for Analysis and Research in Africa SANA Sustainable Approaches to Nutrition in Africa SEARO Southeast Asia Regional Office SET Strategic Experience Transfer SO Strategic Objective SOAG Strategic Objective Grant Agreement TASC Technical Assistance and Support TBA Traditional birth attendants TFA Technical focus area UNICEF United Nations Children’s Fund USAID United States Agency for International Development WARO West Africa Regional Office WARP West Africa Regional Program WHO World Health Organization CONTENTS Page Executive Summary........................................................................................................i I. Introduction..............................................................................................................1 Background and Program Setting ..............................................................................1 Objectives of the Assessment ...................................................................................4 Methodology..............................................................................................................5 II. Achievement of Flagship Functions .......................................................................7 Innovation and Operations and Evaluation Research ................................................7 Achieving Global Technical Leadership ...................................................................9 Strategic Experience Transfer to Different Audiences and Clients.........................11 Support to USAID Missions’ Child Survival Programs..........................................15 III. Increase Child Health Interventions....................................................................20 Effective and Sustainable Child Immunization .......................................................20 Nutrition and Growth Promotion.............................................................................22 Perinatal and Neonatal Health .................................................................................29 Integrated Approaches to Child Health....................................................................32 Other Key Factors that Influence Success in Scaling Up ........................................36 IV. Key Management Factors .....................................................................................43 BASICS II Management Factors .............................................................................43 Organization, Leadership, and Staffing .............................................................43 Planning .............................................................................................................46 Financing, Budgeting, and Resource Allocation ...............................................47 Program Results Monitoring (PRM)..................................................................49 Partner Relations................................................................................................52 USAID Management Factors ..................................................................................53 Program Design Process ....................................................................................53 Varying Views of Program Objectives..............................................................54 Impact of Several Funding Sources and Earmarks ............................................56 Use of Public Health Indicators .........................................................................57 Use of a Performance-Based Contract...............................................................58 CTO Management Structure and Process..........................................................60 V. Summary of Recommendations............................................................................62 BASICS II................................................................................................................62 USAID .....................................................................................................................62 APPENDICES A. Scope of Work B. Persons Contacted C. Background Documents i EXECUTIVE SUMMARY The Basic Support for Institutionalizing Child Survival (BASICS II) project was designed to be an innovative flagship1 program that would increase the use of child survival interventions in 10–15 United States Agency for International Development (USAID) countries. The program vision was to move beyond BASICS I small-scale activities to support a much broader implementation of well-known and generally accepted child health interventions. The results of this program would be measured largely by major improvements in coverage (e.g., 50 percent increase in appropriate care seeking and treatment of acute respiratory infection [ARI] in 10 countries). Increased coverage would be achieved by providing critical USAID technical and financial support and by obtaining other resources. This $79 million, five-year program design had four novel features for a Bureau for Global Health (GH) flagship program: ! the clear focus on expanding the coverage of child health interventions and concentrating program support on those USAID field programs that shared this objective, ! use of public health indicators to measure program performance, ! increased delegation of responsibility from GH to the contractor (e.g., responsibility for negotiating Mission participation in the program), and ! the use of a performance-based contract to provide incentives to the contractor. BASICS II had difficulties in the beginning years. Once the contract was signed in June 1999 (a year behind schedule), much of the first 12–15 months focused on the transition to a results-oriented program and the accompanying long-term and implementation-level planning. This lengthy and intensive planning frustrated many of the action-oriented technical staff. BASICS II staff was accustomed to BASICS I activities-level planning and could not provide a first year work plan that demonstrated clear links to the results in its five-year program strategy. BASICS II also had major senior leadership problems with four directors (including two interim directors) through the contract’s first 27 months. These planning and leadership problems were only resolved about halfway through the five-year program with more experience in the use of results-oriented planning tools and the appointment of an experienced director. Over the past 15 months, the program has operated smoothly and has begun to achieve significant results. PROGRAM RESULTS Program results to date (approximately 15 months before contract completion) are substantial. Some examples are discussed in the following sections. 1 In USAID terminology, flagship suggests a consolidation of partner agencies that share the same objective under one procurement vehicle, led by one or more of the partners. ii Technical Focus Areas (TFAs) ! Immunization: BASICS II has provided strong global leadership and country support in vaccine security, strengthening routine vaccination programs, improving vaccine safety, and encouraging the proper use of new vaccines. ! Nutrition and Child Growth: BASICS II has helped define and successfully implement a package of essential nutrition actions, fostered an effective nutrition network of 20 countries in West/Central Africa, and established community-based growth monitoring as both a key intervention integrated within other community health activities (community-based approach to the integrated management of childhood illness [c–IMCI]) and as a platform for the delivery of community and household nutrition and health services. ! Perinatal and Neonatal Health: Unlike the previous two TFAs, perinatal and neonatal (PNN) health was not ready for full-scale implementation when BASICS II began. BASICS II advocacy efforts and operations and evaluation research (OER) were needed and have been effective, but have taken longer than anticipated. At present, BASICS II is only supporting PNN programs in four countries. ! Integrated Approaches to Child Health (IACH): With GH guidance, this has effectively become support for the c–IMCI program concept. Adoption in some countries has been hampered by a continuing controversy between the World Health Organization (WHO)/Pan-American Health Organization (PAHO) and a USAID/private voluntary organization (PVO) coalition over the relative priority and sequencing of c–IMCI in relation to facility-focused IMCI (f–IMCI). BASICS II has successfully worked with WHO and other key global agencies to define c–IMCI, which emphasizes community/household action while strengthening ties to local health facilities. It is very different from the precise algorithm for case management prescribed by WHO for f– IMCI. BASICS II has successfully supported expansion of the c–IMCI model and Mission demand for BASICS’ c–IMCI support is growing. ! TFA Integration: The four TFA units in BASICS II function as separate, almost vertical programs; some observers believe that stronger integration of their efforts into a child health program is needed. Most Missions, however, only wanted BASICS II assistance for one or two selected interventions, not for a complete package of the four TFAs or an integrated approach. Use of Flagship Functions ! Support for Field Programs: BASICS II has supported child health interventions in 16 countries (only one or two TFAs in most of these countries). Most USAID Missions highly rate BASICS field staff and its programs. Many have complained, however, that BASICS II headquarters has not been flexible in meeting Mission program needs. ! Global Leadership: The BASICS II immunization and nutrition leadership is internationally known and has worked closely with USAID GH staff to iii provide global leadership. Several African members of the African regional office (AFRO) technical staff are highly respected and have provided effective regional leadership. It has proven difficult, however, to measure the impact or results emanating from global leadership activities. ! Operations and Evaluation Research (OER): OER has not been effectively built into TFA agendas or BASICS II country programs. OER has rarely been undertaken and OER objectives have not been met. ! Strategic Experience Transfer (SET): BASICS has struggled to define SET and to put it into operation; the most recent SET strategy seems unnecessarily elaborate and costly. It was difficult to determine whether SET was indeed important or essential in the successes claimed thus far (e.g., adoption of the Atención Integral a la Niñez [AIN] model in Guatemala). ! Program Results Monitoring (PRM): An effective PRM system has been established that should provide valid measures of program results. KEY FACTORS THAT CONTRIBUTED TO SUCCESS OR LACK OF SUCCESS These factors can be summarized as factors primarily under the control of BASICS II, under the control of USAID, or important outside factors that neither party might have effectively managed. BASICS II Factors ! Leadership: The contractor (including the three partner organizations) should be held responsible for a significant failure in the initial selection and slow replacement of program leadership. The absence of qualified and stable leadership had far-reaching impact on program cohesion and on the pace of program implementation. ! Grasping the Program Vision: BASICS II had great difficulty absorbing the contract’s results orientation and using the USAID concepts of “scale up, leveraging, and attribution.” ! Making Needed Operational Adjustments: The contractor (who had also implemented BASICS I) was not adept at making a timely adjustment from a level-of-effort contract designed to provide one-stop shopping for field Missions (BASICS I) to a performance-based contract designed to bring measurable improvements in child health interventions (BASICS II). Major changes were needed in staff skills, leadership vision, planning/programming tools, and cost controls. ! Strong Technical Leadership: BASICS II’s technical leadership is renowned and effective, especially in immunization and nutrition. ! Management: The headquarters structure encourages vertical programming by each of the TFAs rather than program integration. Staffing decisions and the use of subcontractors have been influenced by the three partners’ desire to iv maximize their use of program level of effort. Program financial management has been good. USAID–Related Factors ! Problem Resolution: GH set the vision for the results-based program, but the cognizant technical officer (CTO) team’s initial approach to problem resolution with the contractor was not helpful in finding ways to implement the vision. Coordination to facilitate results-based program achievements was also a new concept for USAID. ! Varied USAID Agendas: The GH program vision for BASICS II was not based on an explicit Agency child health policy/strategy and often was not shared by other USAID regional and Mission health officers. These officers required that BASICS meet its regional or country-level agendas and complained if BASICS was slow to agree. Many USAID health officers did not internalize the change in program vision from BASICS I to BASICS II and did not understand the new rigidity of program focus. ! Timeframe: USAID’s five-year contract period is very short for achieving and measuring changes in many public health indicators. The contract period, in many cases, did not coincide with the timing of USAID Mission strategies and their timetables for achieving/measuring impact. ! Funding Levels and Earmarks: The contract has a fixed ceiling so it was difficult to accommodate higher than planned levels of Mission demand. Unanticipated polio and malaria earmarks also had to be accommodated under the fixed ceiling. The latter earmark, along with anticipated micronutrient earmarks, was easily absorbed into BASICS II country programs. However, polio funds were difficult to absorb due to their magnitude and program implications, especially in Nigeria. ! Performance-Based Contract: Both the funding ceiling and performance standards have proven more difficult to change than under a cooperative agreement. The annual performance reviews have allowed USAID tight control, but have absorbed a great deal of BASICS II’s level of effort and management attention. Other Factors ! Readiness to Expand Coverage: Interviews with BASICS II, USAID, and other child health experts with field experience provided a list of essential factors that have not always been present in BASICS II countries and have hampered program expansion. Most of these factors are outside the direct management control of the contractor: • the technical intervention(s) has(ve) been tested and adapted to the satisfaction of the Ministry of Health (MOH), v • the MOH can provide strong program leadership and effective nationwide coordination, • a reasonably effective health structure is in place with trained staff, • adequate local cost funding is consistently available, • donor and international organization presence and donor cooperation is sufficient, and • there is a continuity of vision among the MOH and donors (willingness to continue). ! Complementary Donor Funds Are Available: Non–USAID donor funding for child health has gradually dwindled, even in Africa where child mortality and morbidity remains very high. USAID funds were limited, but the design assumed that other funding could be obtained from other donors and the private sector for major expansion efforts. Despite some BASICS II successes, this assumption has not been widely validated. ! Host Country Can Finance Local Costs: Most MOH budgets remain stagnant and are allocated primarily for curative care and to pay salaries. Although child health interventions are relatively inexpensive, local cost funding is needed at the health post and clinic levels and to pay for supervisory visits to outlying villages where community-based programs need periodic monitoring and resupply. Health reform and government decentralization efforts offer new options for local cost financing. RECOMMENDATIONS BASICS II ! Finalize its GTL agenda and plan a response to concerns about the appropriate balance of resources in achieving GTL. Progress during the remainder of the project should be measured in accordance with this mutually agreed plan. ! Focus on achieving the desired balance between its GTL efforts and obtaining in-country investments by other partners. Achieving this balance will help both to expand child survival efforts to achieve impact and to assist in achieving sustainability. ! Reassess the scope, purpose, cost, and utility of SET. Develop a more focused role for SET. ! Assess the demand for and use of SET materials, including the effect of having such materials available only in English and French. ! Consider separating SET functions into two categories: vi • country programs, which would emphasize replication of best practices within and outside the country; and • global leadership, which would emphasize the dissemination of lessons learned for technical and policy applications. ! Ensure that all possible resources (host country governments, other partners) are directed toward achieving sustainability in the areas in which BASICS II has been active in each country. BASICS II has only one more full planning cycle and should do its best to overcome the shortcomings in the planning process so as to leave a viable planning process in place for the successor project. ! Continue the integration of c–IMCI and other community approaches in existing country programs, with emphasis in implementing an exit strategy to help ensure program sustainability when BASICS II support ends. ! Work with partners in BASICS II countries to increase their support of c– IMCI. USAID ! Participating Missions need to be informed again about BASICS II contract requirements, especially when they conflict with Mission strategies. ! USAID should replace the performance indicator (4b) in the upcoming amendment to focus more on financial management. In addition to these recommendations, the assessment team prepared comprehensive recommendations for the design team, as called for in the scope of work. 1 I. INTRODUCTION BACKGROUND AND PROGRAM SETTING The United States Agency for International Development’s (USAID) child survival approach began in 1982 with the adoption of a health assistance policy that moved away from support for primary health care systems to the implementation of focused preventive and curative health interventions aimed at the principal causes of mortality and morbidity in less developed countries. The child survival strategy, adopted in 1986, specifically directed this approach at the reduction of infant and child mortality by focusing on the development and programmatic application of effective, low-cost technologies, principally immunization and oral rehydration therapy (ORT)—the two mainstays of health development—as well as nutrition and child spacing. An important role in the success of child survival was the technical assistance provided for the implementation of child survival activities by three global projects: the Technologies and Resources for Child Health (REACH) Project, focused on support for the Expanded Programme on Immunization (EPI) and to a lesser extent on acute respiratory infections (ARI); the Technology for Primary Health Care (PRITECH) Project, focused on support of diarrheal disease control (CDD) programs and increased use of ORT; and the Communication and Marketing for Child Survival (HealthCom) Project, focused on information, education, and communication (IEC) support of child survival programs. After almost a decade of providing such vertical support to child survival programs, USAID identified the need to bring the key elements of such programs as well as the supply and demand aspects of child health and nutrition interventions together in a single global leadership and technical assistance project. The result was the Basic Support for Institutionalizing Child Survival (BASICS) project, which began in 1993. By consolidating activities previously implemented under the three separate predecessor projects (REACH, PRITECH, and HealthCom), BASICS was expected to result in a more efficient use of human and financial resources and synergy of action with an increased capacity to take on new initiatives. A 1997 evaluation found BASICS to be a highly successful activity, substantially contributing to USAID’s global leadership and field implementation of child survival programs. However, in response to this evaluation and those of other key child survival projects (OMNI, MotherCare, and the Child Health Research Project), a new 10–year child health Results Package was written in 1998. USAID’s major vehicle for providing global leadership and technical assistance and support in the development and implementation of child survival policies and programs, the Results Package included the following activities: ! the state-of-the-art global leadership/technical assistance and partnership flagship (BASICS II) activity, 2 ! the maternal and child health Technical Assistance and Support (TASC) activity to provide on-demand implementation support to Missions and other bureaus, ! the Monitoring, Evaluation and Design Support (MEDS) activity to provide systematic support to Missions and bureaus for the development of new maternal and child health and nutrition activities, ! the state-of-the-art Behavior Change, Communication and Social Marketing Innovation (CHANGE) activity to provide technical leadership and innovation in support of other Bureau for Global Health (GH) activities and field programs, and ! occasional targeted grants and participating agency service agreements (PASAs)/interagency agreements with other organizations and U.S. government agencies engaged in international child survival activities. BASICS II was designed to assist the Center for Population, Health and Nutrition (G/PHN, now the Bureau for Global Health), other bureaus, and field Missions in developing and implementing child survival programs that make the greatest possible contribution to the impact on individuals and to advancing the state-of-the-art of global child survival programming. This goal was to be achieved through the following principal tasks: ! Technical Leadership: In close partnership with G/PHN, inform and influence global policy and program directions in key areas of child survival. ! Regional/Global Initiatives: Enter into partnerships with other USAID bureaus and organizations for the development and implementation of regional or global initiatives related to the achievement of results specified in the Results Framework. ! Country Programs: Carry out long-term partnerships that provide continuing technical assistance and support to child survival programming in a limited number of selected countries with the intention of applying project resources strategically, with the greatest chance of achieving impact. In addition to the principal tasks, the BASICS II contract specified several technical support tasks: ! Operations and Evaluation Research (OER): Design, implement, analyze, disseminate, and apply the results of high-quality OER in support of the project’s objective and results. ! Strategic Experience Transfer (SET): Develop strategic approaches to documentation, analysis, and transfer of key expertise and experience, including both document and nondocument–based strategies for influencing policy and programming. 3 ! Performance and Results Monitoring: Apply a systematic approach to monitoring progress, results, and quality of performance in all of the contract’s functions and programming to ensure maximum achievement of the project’s objective and results. To the greatest extent possible and in all areas of its work, BASICS II was also to seek to build capacity in developing country institutions and individuals and to build strategic partnerships with G/PHN, other bureaus, Missions, other cooperating agencies (CAs), and other organizations. Because BASICS II was G/PHN’s principal activity for accomplishing the objective and results of the child health Results Package, the Strategic Objective level results for the project were the same as those of the Results Package itself, specifically “increased use of effective, improved and sustainable child health interventions.” To achieve this objective, BASICS II’s activities were organized around a limited number of key technical focus areas that corresponded to critical program needs and child survival technical priorities: ! increased effectiveness and sustainability of child immunizations, ! integrated approaches to child health, ! incorporating nutrition into child survival activities, and ! neonatal survival and health. Initially, BASICS II was designed as a cooperative agreement, but upon review of the sole application submitted, the decision was made to recompete the project as a performance-based contract. It was believed that this approach would give the child survival management team more direction and control over the implementation of its technical vision and would permit increased emphasis on results, with increased incentives for accomplishment and greater involvement of subcontractors and universities. The performance results against which the contractor was assessed were the following: ! increased immunization coverage (fully immunized child) among high-risk infants and children with present EPI vaccines in at least 10 countries, ! ninety percent measles coverage achieved through sustainable methods in six countries, ! introduction and establishment of agreed-upon levels of coverage of new vaccines against major causes of morbidity/mortality of infants and children in four countries, ! ORT use increased by 50 percent or sustained at 80 percent or more of diarrhea episodes in at least 10 countries, ! fifty percent increase in appropriate care seeking and treatment of ARI in at least 10 countries, 4 ! appropriate care seeking and treatment for children with febrile illness in malaria-endemic areas increased by 50 percent in at least five African countries, ! significant increase in use by child caretakers of hand washing, food hygiene, and measures to maintain clean water at the household level in at least six countries, ! increased use of insecticide-treated materials in malaria-endemic areas in at least five African countries, ! breastfeeding at least 4 (amended to 6) months of age increased by 50 percent in five countries, ! significant increase in appropriate child feeding (frequency, quantity, and/or quality of feeding) in five countries, and ! adequate intake of vitamin A (and/or other specified micronutrients) achieved for 80 percent of children among populations identified as deficient in six countries. In addition, the following specific subresults were planned under Intermediate Result 1 (“New approaches to delivering child survival interventions developed, evaluated for effectiveness, and implemented in at least seven countries”): ! fully operating integrated management of childhood illness (IMCI) and other integrated approaches, and ! interventions to reduce neonatal morbidity and mortality. On June 15, 1999, the project was awarded to the Partnership for Child Health Care, Inc. (PCHC), a corporation formed by three contractors that had previously managed BASICS I and, before that, REACH, PRITECH, and HealthCom: Management Sciences for Health (MSH), John Snow, Inc. (JSI), and the Academy for Educational Development (AED). PCHC includes the three primary partners and six secondary subcontractors: TSL, The Manoff Group, Save the Children USA, Program for Appropriate Technology in Health (PATH), Johns Hopkins University School of Public Health, and Emory University School of Public Health. OBJECTIVES OF THE ASSESSMENT This assessment was conducted after approximately three and three fourths years of BASICS II’s implementation of a five-year contract. The annual performance-based contract evaluation process has thoroughly evaluated contractor performance; therefore, a traditional evaluation would have provided little additional value to USAID or BASICS II. This assessment is designed to provide analysis and insights on the factors that influenced program performance, rather than evaluating the performance outcomes to date. The conclusions and recommendations of this assessment, given its timing, are primarily directed towards USAID’s design of its next major GH child health program. 5 However, it is anticipated that some assessment recommendations might still be valuable for the remainder of the BASICS II implementation period. The major objectives of this midterm assessment of BASICS II, as stated in the assessment scope of work, were as follows: ! Identify the strengths, weaknesses, successes, and shortfalls of BASICS II in the following functions: • innovation and operations and evaluation research, • achieving global technical leadership (GTL), • support to USAID Missions’ child survival programs, and • strategic experience transfer to different audiences and clients. ! Identify the strengths, weaknesses, successes, and shortfalls of the project’s use of these functions to achieve programming and use of key child health interventions at significant levels. ! Assess key management functions in relation to performance and achievement of the project in these key functions and in achieving outcomes at meaningful levels for targeted beneficiaries. ! Provide recommendations in the design, process, and organization which could improve the impact and operations effectiveness of GH’s approach to carrying out its key functions in relation to child survival. METHODOLOGY The assessment team was composed of three consultants and two USAID/Washington officers. The team assessment was conducted over a 4–week period from March 18 to April 11, 2003. Questionnaires were e-mailed by USAID to the Missions and by BASICS II headquarters to BASICS II field offices. The team used the following information as the basis for its analysis, conclusions, and recommendations: ! review of key documents (including basic project documents, self-assessments by all partner agencies, internal management assessments, reviews of selected project elements carried out by the PCHC, annual performance reports, annual surveys from field Missions where BASICS II works, and USAID’s evaluation and fee award memoranda regarding annual performance); ! indepth interviews in person or by telephone with USAID officers in Washington and field Missions, BASICS II staff, former BASICS and USAID staff, donor officials, and recognized international child health experts; ! survey results from 12 USAID Missions that have participated in the BASICS II program; 3 USAID Missions that program USAID child survival funds, but chose not to utilize BASICS II; and 12 BASICS II field offices; and 6 ! site visits to two BASICS II implementation countries (Senegal and Honduras) to conduct direct observation, review country-specific documents, and interview Mission, Ministry of Health (MOH), and implementing partners. 7 II. ACHIEVEMENT OF FLAGSHIP FUNCTIONS INNOVATION AND OPERATIONS AND EVALUATION RESEARCH The project design anticipated a good deal of innovation together with practical OER to “identify and resolve critical constraints on the use, quality and sustainability of key child health and nutrition interventions; develop innovative child survival policy and program approaches; and test the feasibility, effectiveness, efficiency and cost of these approaches.”2 The contract also states that “approximately 15–20 percent of the Flagship’s total effort” would be allocated to OER. In addition, the contract requires high technical capability in OER, involvement of researchers from developing countries, integration of OER activities into the project’s country programs (as opposed to independent studies), and significant G/PHN oversight.3 Findings Expectations for this component have not been met. To date, the project has allocated 3 percent of total project resources to OER. The development of the research agenda was delayed for over two years, in part because of delays in the development of the project implementation plan. Equally important was the decision by BASICS to wait for the design teams to identify OER needs. OER staff was not involved in the design teams as was expected, which led to strained relations between BASICS and Johns Hopkins University (JHU), which was responsible for this activity. Since the JHU staff was on fixed levels of effort, this resulted in high expenditures for little productivity. Delays in staffing also contributed to a slow start. The deputy director for OER was not in place until February 2000; a full-time staff was not in place until January 2002—20 months after the contract began. USAID was not involved, as required by the contract, and noted in the first performance evaluation review that the project had fallen critically behind in two of its most important technical support tasks: performance and results monitoring (PRM) and operations/evaluation research. The criticism was even stronger in 2001 when the annual review noted that productivity in the second year of the project had continued at an unacceptable level that did not seem to justify its extremely high core budget and its high use of G/PHN funds. The proposed agenda was considered to be inadequate and the design material had not been provided. The criticism was so strong that the partners decided not to fix OER but to drop it. BASICS merged OER with PRM, reduced the combined budget 37 percent in fiscal year (FY) 2002, and shifted the funds to the technical focus areas (TFAs). JHU, which believed that it was effectively dropped, was told that OER resources had been reprogrammed. The research agenda was approved in principle in 2001 but its feasibility was uncertain; therefore, it was agreed to define methodologies and identify resources and capabilities 2 Prime contract, p. 27. 3 The contractor was to develop a set of priority OER topics that would be approved by G/PHN. The designs, implementation plans, analysis plans, and dissemination plans were also to be approved by G/PHN. Ibid., p. 28. 8 for each of the studies. Within 2 months, the OER unit was merged with PRM, the budget was reduced, and many activities were shelved. As late as January 2002, there was still concern among the partners that the OER component was “not adequately defined and operational.”4 By the time of the third performance review, the reaction from USAID had improved. The performance evaluation board (PEB) lauded the integration of OER and PRM as a way to reduce costs and focus the research. However, it also criticized BASICS for not having produced much. The PEB also pointed out that most of the topics proposed were evaluation research, not operations research, and that the documentation did not show that the results of the research would influence the way in which child health interventions are applied. The assessment team also learned that needs assessments were rarely incorporated into OER studies, largely due to lack of time and resources. BASICS lists 10 OER studies in its 2002 self-assessment, whose research results met the performance standard of “influencing the way in which child health interventions are applied.”5 Most of these are either in the planning or implementation stage; therefore, it is difficult to conclude that they have influenced anything yet. However, important evaluation and OER studies have been implemented. In Senegal, for example, two such research projects were examined by the assessment team. Evaluation Research on Community-Based ARI Treatment (Feasibility Study) MOH regulations do not allow community health workers (CHWs) to distribute antibiotics. The MOH wants to demonstrate that CHWs can be trained to treat ARI with antibiotics. The MOH has two concerns: whether the antibiotics can be made available to and distributed at the community level and whether this will lead to increased use of antibiotics to treat ARI. The MOH is already convinced that this intervention will reduce mortality and is planning to conduct a follow-up survey. This is an MOH initiative that BASICS II and the United Nations Children’s Fund (UNICEF) are supporting during the feasibility phase. Operations Research on a Neonatal Service Package The MOH is developing a new approach to address peri and neonatal (PNN) mortality, which is very high in Senegal. BASICS II is supporting this activity. A local university has been contracted to conduct both qualitative and quantitative analyses to identify and test new approaches at the facility and community levels to improve the delivery of a package of PNN services (from delivery to the 28th day of life). This study includes community and facility needs assessments, selection or development of interventions, testing, and validation. Despite the difficulties with OER in this project, most people interviewed during this assessment believe that OER is needed and should be built into country programs. 4 Minutes, BASICS Partners’ Meeting Memorandum, January 3, 2002, p. 2. 5 BASICS II Self-Assessment Report for Project Year 3, June 15, 2001–June 14, 2002, p. 45 and annex D, table D–1. 9 Conclusions ! OER has had a slow and unproductive start and has rarely been undertaken in BASICS II. The project objectives for OER have not been achieved. OER was not built into technical focus area (TFA) or country program activities. ! USAID and BASICS did not agree on the purpose, scope, and procedures for OER. Many technical staff members were not interested in OER, even to the point of feeling threatened by it. Missions were largely unaware of the OER component. BASICS started spending money to retain OER staff before there was consensus on its role, which led to large expenditures with no products. ! OER is needed and worthwhile; the few OER studies that have been undertaken have been valuable. ACHIEVING GLOBAL TECHNICAL LEADERSHIP Within the package of flagship functions, BASICS II is tasked with the following GTL functions:6 ! developing state-of-the-art strategies and approaches that can be utilized by other countries and organizations, ! supporting USAID and other agency initiatives for global/regional child health policy and programming, and ! providing technical support to nonpresence countries in child health and nutrition. Findings In each of the annual award fee memoranda of the PEB, the board has cited a number of BASICS II contributions to GTL, covering all four TFAs. These included providing guidance, technical assistance, and documentation to a wide variety of international and regional agencies and organizations as well as leadership in developing approaches that will strengthen key child survival interventions. These contributions to GTL are documented by BASICS II in a series of 1–2 page summaries describing these 6 BASICS II Global and Regional Technical Leadership Agenda and Plan, December 27, 2002 (draft). Factors Affecting Performance ! Clarity of definition of the role of OER ! Timing and sequencing of OER activities ! Acceptance and recognition of the value of OER among project staff and Missions ! Flexibility and responsiveness of OER at the global and Mission levels ! Performance incentives for OER 10 contributions and the leveraging of other resources to achieve them at national, regional, and global levels. According to the 2001 PEB memorandum, BASICS II’s focus and concentrated effort has reportedly resulted in 70 percent of technical resources being invested in nine key technical areas across the four TFAs. This projectwide focus and concentrated effort, conducted in early 2001, resulted in a GTL agenda that identified priority agenda items, countries essential to the demonstration of the GTL item, and specific end-of-project results. These findings are also supported by examples in the two countries visited by the assessment team. In Senegal, BASICS II assistance focuses primarily on using the governmentwide decentralization program to provide the foundation on which sustainable, community-based health services can be developed. Emphasis is on IMCI at the community level (c–IMCI) and supports the priorities of the MOH’s national development program. The Senegal model, including strong links with local government and collaboration with the private sector, has become a useful framework for adoption in other countries. In Honduras, BASICS II has provided leadership and technical assistance in designing and implementing practical programming approaches that will improve infant and child nutrition. This effort is based on the government’s own initiative, begun in 1994, to develop a national community IMCI program, Atención Integral a la Niñez (AIN). AIN has been promoted as a model strategy for initiating a basic community IMCI program that is low cost and that with modest local government financial assistance can be expanded to the national level. As additional resources are available through government and other channels, additional interventions can be added to the package and expanded nationwide. The importance of the AIN model as a major success in GTL is clear from the fact that the model has been adopted and adapted by other Central American countries (Guatemala and Nicaragua); in addition, several African countries have visited Honduras to learn about the AIN program and how it might be adapted for use in their countries. The PEB consistently found that for a number of activities that BASICS II reported under GTL, it was difficult to distinguish between whether the project’s role represented technical leadership or simply participation (for example, in various working group meetings). In addition, for a number of activities, the strategic relationship of the activity to achieving improved child health outcomes was not clear. The PEB also concluded that BASICS II’s documentation of progress generally emphasized activities rather than progress in achieving plans. This finding is corroborated by the fact that the recently revised and updated BASICS II GTL Agenda and Plan still exists as a draft document prepared less than 3 months ago and does not identify end-of￾project results. Conclusions ! There is substantial evidence indicating that BASICS II has made major progress toward achieving both performance indicators for GTL. 11 ! According to the PEB, BASICS II has not yet achieved the appropriate balance in allocating global core resources and GH–supported efforts to achieve GTL versus obtaining in-country investments by other partners. Recommendations ! BASICS II should finalize its GTL agenda and plan with specific end-of￾project results to respond to the concern about the appropriate balance of resources in achieving GTL. Progress during the remainder of the project should then be measured against these results in accordance with this mutually agreed-to plan. ! During the final project year, BASICS II should focus on achieving the desired balance between its GTL efforts and obtaining in-country investments by other partners. Achieving this balance will help both to expand child survival efforts to achieve impact and to assist in achieving sustainability. STRATEGIC EXPERIENCE TRANSFER TO DIFFERENT AUDIENCES AND CLIENTS The BASICS II contract calls for a significant role in documentation and transfer of experience. The contract is quite clear that the strategy should not be limited to passive information dissemination: …the Flagship shall develop strategic approaches to documentation, analysis and transfer of key expertise and experience. Since the assumption that documents alone influence policy and practices proves to be increasingly unfounded, these strategic approaches shall include both document-based and non document-based strategies for influencing policy and program of countries, USAID itself, other important organizations and other relevant target groups involved in child health and nutrition work.7 It was intended that the strategy be integrated into all of the TFAs and follow a systematic, multistep sequence, including the definition of key audiences, data needs, products, and dissemination methods. The principal objective was to “maximize impact and advancement of global policy and programs.”8 Thus, the strategy was to be linked closely with the global leadership and program expansion functions. Findings The initial attempts to implement this function were problematic. As with other components, BASICS had trouble developing an acceptable plan. USAID commented in its review of the first work plan (July 2000) that the narrative captured the principles but that an operational approach and substantive set of activities needed to be developed. The plan put unnecessary emphasis on marketing BASICS rather than on documenting and transferring key experiences. The PEB award fee memorandum noted that SET was not on track; that it seemed to be typical documentation and information dissemination. The next PEB report (August 2001) was equally critical of cost as well as relevance. The report questioned SET’s priorities, products, and links to results. Although much useful 7 BASICS II Prime Contract, p. 29. 8 Ibid., p. 29. 12 communication activity was underway, the partners agreed (December 2001) “that these functions are being carried out in an ad hoc manner and most of the time independent of one another. There is no formal communication strategy that ties these activities together or that determines what is to be communicated.” The partners agreed to have a communication strategy developed.9 A strategic communications strategy was outlined in an overall operational strategy in March 2002. Key activities were to include experience transfer, linkages to global leadership, advocacy, and documentation.10 In April 2002, BASICS staff and consultants prepared guidelines for documentation teams to “systematically document lessons and best practices in USAID programs to learn how to plan, implement and share lessons from community-focused, large-scale and multiple intervention programs.”11 BASICS planned to document activities in each of its 15 countries. In August 2002, the PEB applauded the effort to systematically document and transfer major experiences in community-based programming in USAID–supported countries. However, it remained critical of the lack of a strategic approach linking SET with results. BASICS hired a consultant to prepare a SET strategy, which was drafted in August 2002. The strategy emphasizes selective documentation of interventions that support key messages about what works in general and in each TFA. This document acknowledged past failures and proposed a rather complex approach to documentation and experience transfer. A leadership review committee would establish priorities for SET products based on 13 criteria, one of the most important of which would be cost. The strategy was never provided to the USAID cognizant technical officer (CTO) for comments and was not adopted formally, although some elements were put into practice (e.g., the committee and documentation). However, the document on the project’s evolution (prepared for the assessment team),12 states that there is a SET strategy now, it addresses multiple audiences, and it uses a variety of methods to achieve “this successful transfer.” It also states, “In effect, all activities carried out by BASICS II staff can be considered as Strategic Experience Transfer.” This includes formal presentations to targeted audiences, informal discussions with peers and colleagues, and systematic and structured documentation and dissemination. In addition, BASICS II “makes extensive use of non-print media and channels (e.g., study tours, participation…at meetings, electronic and video formats…” It also states that “all SET materials are produced using cost-effective and state-of-the-art media, such as CD–ROMs…and the BASICS II website.” In general, the emphasis seems to have been more on materials preparation and dissemination rather than replication. The organization structure reinforces this perception. The SET unit sees itself as a communication/dissemination entity. Replication is left to the technical staff. SET terminology gives the impression of high￾level information dissemination rather than replication of successful interventions. There are some good examples of implementation and replication:13 9 Minutes of December 19 Partners’ Meeting, January 3, 2002, p. 3. 10 BASICS II Operational Strategy, March 2002, pp. 24–26. 11 Tina Sanghvi and Mary Taylor, BASICS II Documentation and Strategic Experience Transfer, April 23, 2002. 12 The Evolution of BASICS II, document prepared for the assessment team, 2003. 13 The Evolution of BASICS II, p. 12. 13 ! Community-based growth promotion developed in Honduras has been transferred to several countries and agencies such as the World Bank. ! Routine immunization monitoring approach and tools have been transferred globally through WHO. ! Injection safety interventions are being transferred across Asia and Africa. ! Essential nutrition actions, developed under BASICS I as the Minimum Package of Nutrition Interventions (MinPak), were transferred from Benin and Senegal to India and 16 African countries and to nongovernmental organizations (NGOs), private voluntary organizations (PVOs) (e.g., CARE and the World Health Organization [WHO]). ! Increasing access to ARI treatment is being transferred to Francophone Africa through OER in Senegal. ! Emerging essential newborn care interventions and tools are being transferred across Asia and Africa. BASICS management acknowledges that the project does not have a systematic approach to replication. Senegal provides good examples of this; it replicated Nepal’s ARI approach. This was an expensive, major undertaking for which BASICS/Senegal had to include a specific line item in its annual budget. A Senegal team traveled to Washington, D.C., to meet with UNICEF staff from New York and USAID staff with Nepalese experience to develop a protocol. Senegal also developed the idea for a child health week as a primary mechanism for vitamin A supplementation every 6 months from a study tour to Zambia, which was relatively inexpensive. The Senegal team developed the idea, tested it, and then expanded it to other districts. SET seems to focus on transfer of experiences to other countries rather than replication within a country, although the evolution document states that SET includes local target audiences. An exception is Honduras, where SET developed an AIN video that has been used for advocacy at the local and regional levels and for program expansion. A number of PVOs and donors have incorporated AIN into their programs. BASICS/Honduras has been instrumental in this expansion with assistance from BASICS headquarters. The SET strategy seems to put little emphasis on replicating best practices, study tours, or other technology transfer devices that are more direct and less expensive. BASICS management has noted that it has been difficult for the technical staff to think strategically about transferring technology but this is gradually changing and key messages are now being developed within each TFA. BASICS II management has come to realize that replication and expansion often require significant capacity development and funds. Merely providing materials and advice is not enough. Even within Honduras, where AIN has been expanded, there are significant gaps because of the lack of staff, transportation, scales, and other resources. BASICS II 14 has not had much success to date in obtaining representative feedback or in assessing the impact of SET because of its structure and scope. Conclusions ! SET is important for several BASICS II functions, especially global leadership, expansion of coverage, leveraging, and responding to Mission support requests. ! SET is a process, only a part of which has been organized (the SET unit). ! The development and inauguration of SET was delayed for over two years. ! BASICS II has struggled to define and implement SET. Despite the development of various SET strategies, the project has not been able to articulate clearly what SET is and does. ! The view that “all BASICS II activities are SET” only leads to greater confusion about what distinguishes SET from global leadership, expansion of coverage, leveraging, and other functions. ! SET audiences seem overly broad, giving the impression of a generalized approach to dissemination and making it difficult to generate representative feedback or to assess impact. ! The 2002–2004 SET strategy seems to be unnecessarily elaborate, cumbersome, and costly. The plan to document BASICS II activities in all 15 countries seems to be excessive. ! It is difficult to determine whether the successes claimed for SET are valid. For example, the adoption of the Honduras AIN model in Guatemala seems to have been initiated by Guatemala (a non–BASICS II country), and BASICS II roles seems to have consisted of hosting a delegation from Guatemala for 1– week and providing it with the AIN materials. ! The SET unit’s materials are normally only published in English and French, which limit their audiences and utility. Factors Affecting Performance ! Clarity of SET objectives, structure, and procedures ! Awareness of and interest in utilizing SET materials and experiences among SET audiences ! Political commitment to replicating such experiences ! Adequate descriptive and step-by-step instructions and guidelines for audiences ! Access to relevant materials and technical assistance by audiences ! Relevance, potential impact, and cost to the audiences of utilizing the experiences 15 Recommendations ! Reassess the scope, purpose, cost, and utility of SET and develop a more focused role for SET. ! Assess the demand for and use of SET materials, including the effect of having such materials available only in English and French. ! Consider separating SET functions into two categories: • country programs, which would emphasize replication of best practices within and outside the country; and • global leadership, which would emphasize the dissemination of lessons learned for technical and policy applications. SUPPORT TO USAID MISSIONS’ CHILD SURVIVAL PROGRAMS Findings Country Selection and Implications According to the prime contract, the two principal criteria for selection of countries as sites for flagship long-term country programs were the potential for a contribution to child health and nutrition impact and the opportunity to develop, implement, and evaluate improved and innovative approaches. In fact, most initial BASICS II countries were the same as for BASICS I. These countries were assigned by USAID/Washington after a careful selection process that included review of their potential to be expanded to meaningful levels. These Missions had already committed field support funds that would help to ensure a quick startup for BASICS II. One country (Mozambique) did not meet the BASICS II criteria and USAID/Washington advised that Mission to use the TASC mechanism for its child health support. As the program progressed, some new countries were added (Guinea, Mali, Madagascar, and Nepal). According to the BASICS II matrix of child survival interventions in the countries selected,14 most of these countries continued the same child survival interventions that they had been implementing under BASICS I. BASICS II headquarters staff members stated that they had limited ability to choose countries that could be expected to help them meet the required performance targets specified in the contract. This problem has persisted throughout the project and has compromised the ability of BASICS II to maximize its impact in achieving either their performance objectives or Mission child survival objectives. 14 BASICS II presentation to the assessment team, February 20, 2003. 16 In the eventual BASICS II operational strategy, three categories of countries were established:15 category A (comprehensive package and significant resources), category B (selected technical interventions and moderate resources), and category C (limited resources/involvement). Most BASICS II countries chose a limited number of child survival interventions from among the 10 Strategic Objectives (SOs) within the BASICS II Results Framework. Of the 16 countries in the matrix presented, only Senegal is implementing all 10 interventions; 11 countries (two thirds) are implementing 4 or fewer. Of the latter, five countries are only implementing one or two interventions. Thus, based on the range of interventions among countries actually participating in BASICS II, the potential impact on child health was not optimal. There are several important resource implications of the number of countries and the number of interventions per country, given the overall budget ceiling for BASICS II and the limited budgetary resources available among the three categories of funding sources (Bureau for Global Health, regional bureaus, and individual Missions). A major factor was the decision to establish a BASICS II regional support office to provide technical assistance to West and Central African countries whose child survival resources were not large and where there were very few interventions. USAID’s Africa Bureau (AFR) strongly supported and fully funded the establishment of the West Africa Regional Office (WARO) based in Dakar. This office provided technical assistance to countries with no BASICS II staff and provided supplemental technical assistance to countries with BASICS II staff. While some child survival progress was made in some of these countries, it is not clear that the cost of the regional support office has been justified by its resulting impact. In countries with BASICS II country program staffs, the size of those staffs varied significantly. Obviously, the nature and intensity of BASICS II support (and potential level of impact) will have an impact on the size of each in-country staff. However, during the assessment team’s field trips to Senegal and Honduras, the contrast was noted between the two BASICS II in-country staffs: 25 in Senegal and 6 in Honduras (of whom 3 are field based). Although it was beyond the scope of the assessment team to consider the cost implications of the country staffing levels in relation to the potential impact of the respective programs, the size of the BASICS II staff in Senegal creates a significant financial burden on the project. Not all countries initially selected are currently participating in BASICS II, and others have since been added.16 While the reasons for this vary, the fact that some original countries and some added countries will not have participated in BASICS II for the full period of field implementation will detract from the impact that could have been achieved. This has obviously had an effect on achievement of the “sustained increase” performance indicator and related performance standards. 15 BASICS II Operational Strategy, March 2002, pp. 19–21. 16 Nine of the original 13 program countries providing field support funds are still active BASICS II countries, 3 are no longer providing funding but have some BASICS II involvement, and 1 never participated. Additions to the original list include the Africa, Latin America and the Caribbean, and the Asia and Near East bureaus, Senegal, Guinea, and Mali. Madagascar and Nepal were added using GH core funds. 17 Timeframe One of the difficulties faced by every USAID project is the fact that considerable time must be allowed for project startup and closeout. Thus, the five-year limit on the BASICS II contract really results, at best, in a three-and-a-half to four-year timeframe for actual field implementation. Added to that problem is the fact that in any project environment, various factors contribute to either expediting or slowing down implementation. In the case of bilateral Mission projects, these factors include the critical interaction with the host country government, the absorptive capacity of the country’s health system (including both human and financial resources), the political implications of the project within the country, and the degree of collaboration among implementing (and funding) partners in the country. Flagship projects similar to BASICS II have the additional complication of relationships and negotiations with USAID/Washington, which may or may not affect the desires of USAID Missions that use the flagship project to help achieve their own child survival objectives. These factors clearly have a major impact on the time available to implement the project and consequently on the resulting impact. While these factors are always present and can even be anticipated, it is seldom possible to expect a quick and positive outcome for a complex global project. Added to these timeframe complications is the slow startup of BASICS II. Given the difficulty in preparing and gaining acceptance of a first-year work plan, the timeframe for field program implementation was effectively reduced even further. However, there were notable exceptions where countries were identified and work began in certain TFAs (e.g., EPI) even though the annual work plan had not yet been approved. BASICS II versus Mission Perspectives on Country Support In order to obtain additional perspectives from field Missions and BASICS II country offices on the real or potential value of BASICS II as well as a sense of the degree of need for further flagship project support in child survival, three separate surveys provided additional information that would help in the assessment. In each case, there were standardized questions to which Mission or BASICS II staffs were asked to respond. One survey requested responses from Mission health staff members that have participated and/or are still participating in BASICS II. The second requested information from Missions that do not have BASICS II programs, and the third sought feedback from BASICS II field personnel themselves. According to the preliminary results of these surveys of BASICS II and non–BASICS II Missions, there were a number of instances where the Mission’s desires for BASICS II support were compromised to varying degrees. These included both lack of BASICS II agreement on desired interventions (e.g., Ghana) and dropping of countries for lack of contribution to coverage expectations (e.g., Benin). 18 Resource Constraints ! The survey results from Missions with BASICS II programs indicate that from a Mission perspective, core GH funds were rarely available to supplement Mission-funded BASICS II activities. The BOOST initiative, which focused on major increases in immunization coverage, was a very positive exception that Missions reported had strengthened their immunization programs. ! Some Missions indicated that they had received useful support in several key flagship areas: new approaches and innovations, global and regional technical leadership, and strategic experience transfer. However, most countries stated that operations research has not been adequately funded nor used effectively.17 ! In some instances, AFR funds were made available to assist some countries in carrying out programs under BASICS II that would not have been possible otherwise (e.g., Nigeria and Benin). However, according to the current USAID/Benin staff, funds were provided to continue the Benin program even though the Mission had decided to close out the activity. While this may be an isolated example, it helps to reinforce the lack of consensus among the three partners. Conclusions ! The selection of countries to participate in BASICS II has led to a degree of dissatisfaction among all three stakeholders: GH, BASICS II, and the Missions. ! In the BASICS II design, there is a conflict between the desires of Missions for assistance in one or more of the four TFAs and the performance-based nature of the contract. In some cases, this conflict has led to missed opportunities in assisting country programs to achieve their own objectives, rather than being expected to contribute to BASICS II achieving its objectives. ! The overall dilemma is using the resources provided by Congress to improve child health in view of several competing factors: • Congressional earmarks and targets and their allocation; • competing demands for funds among GH, the regional bureaus, and the Missions in order to meet their respective objectives; and • the continuing need for strong country child survival programs that can be sustained. 17 One Mission reported that BASICS II headquarters was viewed as following interests that did not affect the Mission’s program (conducting studies to triangulate findings from other parts of the world when the information sought was available but not standardized or detailed enough to allow comparison with other studies). 19 This dilemma is intensified by the anticipated reduction in child survival funding to compensate primarily for the greatly increased funding for HIV/AIDS. Recommendations In the remaining year of project implementation, BASICS II should ensure, while continuing to achieve SO results, that all possible resources (host country governments and other stakeholders) are directed toward achieving sustainability in the areas in which BASICS II has been active in each country. 18 18 Unfortunately, owing to what could be considered as a fault in the project design, desirable outcomes, such as sustainability of technical interventions, were not included in the indicators and targets. Since BASICS II would not be rated for such outcomes, they were not emphasized. 20 III. INCREASE OF CHILD HEALTH INTERVENTIONS EFFECTIVE AND SUSTAINABLE CHILD IMMUNIZATION BASICS II has three Strategic Objectives and four Intermediate Results (IRs) for the immunization technical focus area (TFA): Strategic Objectives ! Increased immunization coverage (fully immunized child) among high-risk infants with present EPI vaccines in 10 countries ! Ninety percent measles coverage achieved through sustainable methods in six countries ! Introduction and establishment of agreed-upon levels of coverage of new vaccines against major causes of morbidity/mortality of infants and children in four countries Intermediate Results ! Routine immunization coverage increased sustainably through systems strengthening ! Comprehensive approaches to disease control designed and implemented ! Underutilized and new vaccines introduced into country programs ! Quality of services improved Use of Flagship Functions Global Technical Leadership and Strategic Experience Transfer BASICS II has developed a routine monitoring chart that has been adopted by WHO for inclusion in a global training module. WHO produced a BASICS II project checklist for strengthening routine immunization through polio eradication efforts and this checklist is now being issued by WHO and UNICEF to all polio endemic countries. BASICS II developed a tool for injection safety that has been adopted by WHO and used in 24 countries. The BASICS II immunization technical team leader served on the Independent Review Committee for the Global Alliance for Vaccines and Immunization (GAVI) applications. WHO officials in Africa have submitted immunization plans to BASICS II for review and feedback for an informed and unbiased opinion on their documents. BASICS II has helped create policy documents on measles strategies and donor agency guidelines that have served to influence national level and donor organization decision￾making processes. 21 BASICS II Country Selection BASICS II selected countries based on several factors, reflecting the wide range of options and USAID–supported programming. Several countries had been with BASICS I and continued with BASICS II. Other countries were chosen through negotiation with USAID’s BOOST initiative. Through the BOOST initiative, BASICS II provided technical input into USAID country Missions within their child survival strategies. BASICS II Successes and Shortfalls Successes On a global level, BASICS II has been recognized as a key stakeholder in the field of immunization and has often been the only one to address routine immunization in a time of increasing attention to supplemental immunization activities. BASICS II participates on the GAVI Application Review Board and contributed to the creation of immunization tools and instruments that have been adopted by WHO and UNICEF. BASICS II technical guidance to other donors on establishing policy related to disease control has showcased BASICS II and USAID participation in immunization globally. Within countries, BASICS II has influenced the rejuvenation of interagency coordinating committees (ICCs). Moreover, BASICS II provided technical assistance in USAID Mission planning (DR Congo and BOOST countries), supported ministries of health in addressing issues of injection safety (Guinea and Senegal), and created innovative ways to link vertical disease management programs, such as polio and measles initiatives, into routine systems strengthening (DR Congo and Nigeria). Shortfalls While BASICS II has shown improvements in some countries (e.g., DR Congo, Uganda, Nepal, and Senegal), given the above influences on immunization at global and national levels, BASICS II has not yet shown substantial positive results in immunization coverage rates in all country programs. Factors That Influenced Successes and Shortfalls BASICS II has consistently provided strong technical leadership at the global level and has shown a strong ability to draw upon regional and local resources to support immunization programming at the country level. Country-level issues had a significant role in BASICS II’s inability to achieve national￾level coverage or impact. For example, the MOH in Nigeria has not ordered vaccine for two years, thereby necessitating BASICS II to pull back on increasing demand for vaccination services. Additionally, the immunization has been overtaken by the polio eradication program. 22 Conclusions and Recommendations Conclusions ! BASICS II has been able to participate at the global level in immunization and is recognized as a key technical leader in this area. ! BASICS II has had difficulty in expanding the measurable impact of country￾level immunization programming in the program timeframe. This is an inherent problem in short-term programs for long-term intervention, although it also reflects BASICS II’s difficulty moving demonstration or pilot projects to the national level in certain countries. Recommendations ! In collaboration with the partners, identify concrete ways to expand coverage of current BASICS pilot projects to achieve greatest impact for the investment (e.g., in Nepal, liaise with UNICEF and perhaps provide funding to obtain direct UNICEF support to the BASICS initiative and move that to the national level to achieve the greatest impact). ! On global initiatives (e.g., GAVI) bring pressure to bear on countries that are not responding within the established goals (e.g., Nigeria is not currently ordering vaccine and has none in the immediate future; BASICS, working with USAID, can influence GAVI to bring pressure to bear on the government of Nigeria). ! Work nationally to identify gaps in immunization programming and urge local and community partnerships to create concrete action plans to address these issues to achieve coverage (e.g., in Uganda, obtain bilateral buy-in for community-level BASICS programming to move to adoption at the national level). ! Strengthen linkages with other partners, including NGOs and USAID bilateral organizations in-country, to facilitate transition and phaseout. NUTRITION AND GROWTH PROMOTION The BASICS II strategic plan identifies nutrition as one of four technical focus areas (TFAs), and includes three specific nutrition results (although these are variably and inappropriately described in other documents as SOs): 1. Prevalence of appropriate breastfeeding through at least 4 months of age increased by 50 percent in five countries; 2. Significant increase in appropriate child feeding (frequency, quantity, and/or quality of feeding) in five countries; and 23 3. Adequate intake of vitamin A (and/or other specified micronutrients) achieved for 80 percent of children among populations identified as deficient in six countries. Nutrition is also a key, integral element of the fourth TFA, Integrated Approaches to Child Health (IACH). Flagship Functions Global Technical Leadership and Support to Missions The three primary nutrition results fit within a framework of six essential nutrition actions that BASICS has defined and promoted on a global and country level, first as the Minimum Package of Nutrition Interventions (MinPak) under BASICS I, and then evolving as the essential nutrition actions (ENAs) under BASICS II. The complete set of ENAs includes the following: 1. Appropriate breastfeeding practices (early initiation and exclusive breastfeeding to about 6 months, continued breastfeeding to 2 years); 2. Appropriate child feeding (introduction of complementary foods at about 6 months; frequency, quantity, and/or quality of feeding with continued breastfeeding; feeding of children during/following illness); 3. Prevention of vitamin A deficiency (postpartum maternal vitamin A supplementation; breastfeeding; routine vitamin A supplementation of children 6–59 months; vitamin A supplementation for case management of measles, protein energy malnutrition, xerophthalmia, and severe diarrhea; dietary strategies, including fortified foods); 4. Prevention of anemia (maternal and child iron/folic acid supplementation, prevention of vitamin A deficiency, malaria control, deworming, consumption of iron/folic acid-rich foods, including fortified foods); 5. Prevention of iodine deficiency (universal salt iodization, targeted iodine supplementation where iodized salt is unavailable); and 6. Improved maternal nutrition (increased food intake during pregnancy and lactation, 3–5 above, child spacing). BASICs II has appropriately focused on the first three of these ENAs as those that will have the maximum impact on health and survival of children under 2.19 UNICEF is the 19 There are a number of problems with the measurement and achievement of these results as defined by BASICS II. First, measuring the percentage change in prevalence of exclusive breastfeeding over baseline rather than percentage change from baseline distorts the actual progress, especially when starting from a very low baseline (e.g., going from 2 to 3 percent [50 percent increase over baseline] versus 30 to 45 percent [also a 50 percent increase over baseline]). There is also a lack of consensus whether exclusive breastfeeding prevalence should be measured just to 4 months or the recommended 6 months. It may be better to simply measure the shift in mean duration of exclusive breastfeeding up to 6 months of age. Second, it is not clear what constitutes a significant increase in appropriate child feeding (2 above) and 24 primary agency focusing on universal salt iodization, with some funding support directly from USAID. Maternal nutrition and anemia (both maternal and child) are areas of great need, but these areas have been widely neglected by the international community and will require integrated approaches and substantially greater resources from the international community in the future. BASICS has not only championed the definition of the ENAs and their advocacy at the global, regional, and national levels, it has been central to working out how to put them into operation as an integral component of public health programs. This has focused on community-based growth promotion or community-based growth monitoring and promotion, where children are weighed monthly; c–IMCI and f–IMCI; antenatal/postnatal care; EPI; delivery of vitamin A and other nutrition/health services during National Immunization Days (NIDs)/Micronutrient Days/Child Health Weeks; and community-based rehabilitation of malnourished children (positive deviance/Hearth). The key has been to promote small, feasible actions at the community level while strengthening community links to and the capacity of health facilities to support the community and provide essential services that cannot be provided at the community/household levels. It should also be recognized that BASICS II, as USAID’s flagship child survival project, is uniquely positioned among the primary USAID nutrition CAs, which also include the Food and Nutrition Technical Assistance project (FANta) (food security and food aid), the USAID Micronutrient Program (MOST), and LINKAGES (early child feeding), to advocate for positioning nutrition as a central public health priority in working with international agencies (WHO, UNICEF, World Bank) and with governments (MOHs, ministries of finance). Nutrition has always been a priority within its own community, but it has only begun to be elevated as a priority (e.g., vitamin A supplementation) as its impact has been demonstrated and recognized by public health and political leaders. In a number of countries, the other nutrition CAs (particularly LINKAGES and MOST) can now expand the impact because of earlier/ongoing advocacy and foundational work by both BASICS projects. For example, BASICS was the key advocate for the inclusion of vitamin A supplementation with NIDS with WHO and UNICEF at the global level and with MOHs and ICCs at the country level. As a result, more than 60 countries worldwide have been providing vitamin A supplementation with NIDs, and MOST has followed BASICS in working with many of these countries (e.g., Madagascar, Zambia, Ghana, Uganda, DR Congo) to introduce a mechanism (e.g., Maternal Nutrition Days/Child Health Weeks) for non–NIDs delivery of vitamin A. Similarly, a number of countries where LINKAGES is expanding incidence of breastfeeding and appropriate child feeding again, this should be measured as a percentage increase from baseline rather than percentage increase over baseline. Third, while populations in which vitamin A deficiency is a public health problem at a national or subnational level can be identified (prevalence of serum vitamin A <0.7umole/L in more than 15 percent of children 6–59 months of age), invariably the vitamin A status of those who are actually provided the interventions is not known (e.g., semiannual vitamin A, nor whether semiannual vitamin A supplementation (with or without other vitamin A interventions) will be adequate in fully meeting a child’s needs for vitamin A. What is known is that vitamin A supplementation as infrequently as every 6 months is efficacious in reducing the risk of all-cause mortality between 6–59 months of age, even when vitamin A supplements are not sufficient to achieve vitamin A adequacy in children. Finally, there is a question across these results as well as other BASICS II results about whether the target percentages are for the country as a whole or just for BASICS II program areas. 25 are countries where BASICS worked with the MOH and other partners to elevate nutrition (including breastfeeding and appropriate child feeding) as a programmatic priority (e.g., Madagascar, Ghana, Zambia). Strategic Experience Transfer and Operations and Evaluation Research SET has made been intertwined with GTL and direct support to Missions, strategically focusing on a limited set of key nutrition interventions, especially vitamin A and breastfeeding and appropriate child feeding, and how to implement them at the community level in the context of decentralized health systems with minimal resources and capacity. PROFILES has been an important policy and resource advocacy tool for nutrition in many countries, although it is not always linked to a specific set of proposed policy changes or other actions. Another example of SET is the establishment of the Nutrition Focal Points Network of 16 West and Central Africa countries, which have largely adopted the ENA framework and elevated it as a priority within public health programs, even in those countries where BASICS II has not been active. It is not clear to what extent operations research has been integrated within and has contributed to the nutrition TFA. Community-based approaches to maternal anemia/low birth weight were identified as needed and were to be addressed within at least one country, but they evidently have not been to date. There is also a need to fully assess the program costs and the impact of a number of the more successful nutrition programs, particularly AIN in Honduras (ongoing) and Paquet d’Activités Intégrées Nutrition (PAIN) in Senegal. Other important OER issues include the cost-effectiveness of including infant weighing at monthly community-based growth promotion sessions, focusing community-based growth monitoring and promotion up to 12 months versus 18 months or 24 months, and various strategies for non–NID vitamin A supplementation (and how these may serve as an effective platform for other health services). Because community-level interventions (e.g., community-based growth promotion, c–IMCI) are critical to achieving improved nutritional and health outcomes nationally, operations research is essential for assessing the support and compensation that is needed to attract and sustain volunteer community workers (e.g., the relais in Senegal and monitoras in Honduras). BASICS II published a monograph, Community Health Worker Incentives and Disincentives: How They Affect Motivation, Retention and Sustainability,20 which provides recommendations that could be tested in BASICS’ community nutrition activities with regard to program performance and sustainability, but it is not clear that this is presently being conducted. Choice of BASICS II Countries BASICS II has been most active and achieved the greatest expansion of nutrition activities in countries where BASICS I worked, emphasizing the need for more continuity and duration of USAID support than can be provided within the timeframe of typical five-year projects. The most successful program in Latin America and the Caribbean has been in Honduras, where more than 50 percent of the population is covered by the AIN program. This effort 20 K. Battacharyya et al., 2001. 26 is based on the government’s own initiative, begun in 1994, to develop a national community IMCI program called Atención Integral a la Niñez (AIN). AIN focuses on community-based growth promotion as the principal intervention, but also promotes exclusive breastfeeding, ORT use, and detection/treatment of pneumonia. BASICS II’s support through the AIN program reaches approximately 50 percent of children under 2 years of age. AIN works closely with the facility-focused IMCI program (Atención Integrada a las Enfermedades Prevalentes de la Infancia [AIEPI] in Spanish) to ensure that first-line health facilities are capable of supporting the AIN program. AIN is a good example of the flexibility in establishing the initial package of child survival interventions. Based on poverty and health statistics, the AIN interventions target the poorest regions and within them, the most needy area. AIN focuses on adequate weight gain of infants at monthly community-based growth monitoring and promotion sessions and promotes other key nutrition and health interventions. AIN is also being extended to El Salvador and Nicaragua, where it is being linked to c– IMCI and to a number of African countries. Countries in Africa with significant nutrition program emphasis include ! Senegal (ENA, community-based growth promotion and vitamin A): covering 15 USAID districts plus 23 additional districts through partners (World Bank, UNICEF); ! Benin (ENA, breastfeeding, and vitamin A): focusing on Borgou; DR Congo (ENA, vitamin A): in BASICS and SANRU districts; Ghana (ENA, community-based growth promotion, and vitamin A): linked to World Bank project areas; Uganda (community-based growth promotion and vitamin A): with the World Bank Nutrition and Early Development Project in 25 districts and MOST nationally; ! Madagascar (ENA); ! Nigeria (ENA, breastfeeding, and vitamin A): in 20 local government areas in three states (national vitamin A supplementation through NIDs); ! Zambia (ENA and community-based growth promotion); ! Guinea (ENA and vitamin A); and ! BASICS II’s only nutrition programming in the Asia and Near East region is in India, working with CARE in Uttar Pradesh in more than 100,000 villages with more than 10 million beneficiaries. Successes and Shortfalls As mentioned above, BASICS II’s greatest contributions on global and country levels have been to define a package of ENAs and demonstrate that it can be both integrated within other health delivery systems (e.g., antenatal/postnatal care, EPI, NIDs) and serve as a platform for the delivery of multiple nutrition and health interventions (community- 27 based growth promotion, Micronutrient Days and Child Health Weeks). Programs like AIN in Honduras and PAIN in Senegal now reach more than 50 percent of their respective populations, and those countries are committed to expanding these programs nationwide. Taken together, the experiences of BASICS, MOST, and LINKAGES provide examples of ENA expansion of impact across more than a dozen countries across Latin America and the Caribbean, Africa, and Asia and the Near East, and should be the basis for USAID’s strategic direction regarding nutrition programming. However, BASICS II’s strategic plan raises the issue that few community-based nutrition programs, particularly those implemented at meaningful coverage levels, have been sustained once external donor assistance has been withdrawn. The need is not to continue targeted funding for such projects, but to provide broader, more integrated MOH programs that have support by donor nondesignated funding for the health sector. BASICS has contributed substantially to this by getting nutrition and the ENA framework adopted as government policy as well as integrating it within many World Bank and other sector programs. While BASICS II’s work on breastfeeding, appropriate child feeding, and vitamin A can at least be considered a success in progress, it is disappointing that so little has been done on maternal nutrition and anemia, especially since these have a key role in peri/neonatal and infant survival. This includes malaria interventions (insecticide-treated bed nets [ITNs], intermittent preventive treatment [IPT], appropriate treatment) and iron/folic acid supplementation through antenatal/postnatal care contacts (especially given the recent results from the Nepal Nutrition Intervention Project Sarlahi (NNIPS3) in Nepal that showed that antenatal iron/folic acid supplementation reduced infant mortality by more than 20 percent, primarily in the first months of life). Factors Affecting Performance Resources The nutrition TFA has received about 20 percent of BASICS II’s total budget (9 percent of total core funds) over the first four years of the project, somewhat less than immunization (34 percent) and IACH (24 percent), but substantially more than the peri/neonatal TFA (4 percent). Partners Key partners for nutrition include USAID Missions, ministries of health, UNICEF, WHO, the World Bank, the Inter-American Development Bank (IADB), LINKAGES, Support for Analysis and Research in Africa (SARA), Sustainable Approaches to Nutrition in Africa (SANA), MOST, CARE, and Helen Keller International (HKI). Government Commitment and Capacity BASICS II has had a key role in securing political commitment and resource allocation for nutrition as a health sector priority at both global and country levels. 28 Policy Environment BASICS II has been instrumental in the broad adoption of the ENA framework as well as other specific policy steps for nutrition, including linking vitamin A with NIDs, the development of non–NID vitamin A supplementation policies/strategies, and strengthening nutrition as a component of c–IMCI/f–IMCI. Community-Level Organization This has been shown to be the key element to achieve impact, but expansion and sustaining community programs remains a great challenge. It is not clear whether this can be successful through a limited network of indigenous community health workers (CHWs) and traditional birth attendants (TBAs) largely trained by the government and supported locally, or to what extent it requires the additional support of PVOs/NGOs (e.g., World Vision in Senegal) or purpose-built local NGOs (e.g., Catchment Area Planning and Action [CAPA] in Nigeria). Household Behavior Constraints/Barriers Household-level behaviors continue to be the key to improved infant and child health outcomes, even in food insecure households with limited resources. Timeframe Achieving large-scale community/household-level impact, especially for an integrated package of basic nutrition and health interventions, is more likely to occur in a five to eight–year timeframe (e.g., Nepal’s eight-year expansion of the vitamin A supplementation program) than a three to five–year timeframe, irrespective of resources. Complexity of Life-Cycle Approach Maximizing the impact of nutrition on health will require a life cycle approach to the ENAs, beginning with maternal nutrition and continuing through childhood, adolescence, and adulthood (especially for women). To date, BASICS II has largely focused on interventions from birth to 2 years of age. Food Insecurity and Disease Burden The links among food insecurity, malnutrition, and disease (e.g., HIV/AIDS, malaria, and tuberculosis) are clear, but this does not appear to have been a focus of BASICS II’s work in nutrition. While BASICS II has worked closely with LINKAGES and MOST in a number of countries, there may be a need to develop a closer link to FANta in looking at ENAs in the context of food insecurity and HIV/AIDS as well as the transition from emergency situations to the reestablishment of basic nutritional and health services. 29 Recommendations !Continue engagement with the other primary nutrition CAs—FANta, MOST, LINKAGES, SARA/SANA—to identify/harmonize best practices within the ENA framework. !Focus on USAID (USAID/Washington, new entry professionals, Missions) as well as USAID’s primary health and nutrition partners (UNICEF, WHO, World Bank) as primary audiences for the ENA best practices with a continuing emphasis on achieving national impact among targeted populations (GTL/SET). !Work with partners on implementing ENA in the context of HIV/AIDS, prevention of mother-to-child transmission, and food insecurity. PERINATAL AND NEONATAL HEALTH BASICS II has established one Intermediate Result and four governing principles for the perinatal and neonatal health (PNN) technical focus area (TFA): Intermediate Result (as specified in the BASICS II contract) ! Develop interventions to reduce neonatal mortality and morbidity Principles governing the neonatal health strategy ! Improving the quantity and quality of data for decision-making ! Focusing on community actions related to neonatal health ! Developing and testing integrated approaches ! Using partnerships to the maximum, particularly those that strengthen the maternal and child health continuum Use of Flagship Functions Within the PNN technical focus area, BASICS II’s primary role early in implementation was advocacy at the global, regional, and national levels to obtain sufficient interest and buy-in from partners to proceed with PNN programming. BASICS II’s primary line of work within this TFA is to “design, implement and evaluate an essential newborn care package in a variety of settings.” Pulling together a myriad of proven interventions into one newborn care delivery package, BASICS II took on the role of GTL to create a new, integrated PNN model for introduction. BASICS II also promotes activities relevant to newborn health through its other TFAs, notably maternal tetanus toxoid through the immunization TFA and breastfeeding through the nutrition TFA. In some countries, a special attempt has been made to target mothers in the early 30 postnatal period for promotion of breastfeeding by utilizing relevant community workers, such as TBAs. The strategic experience transfer of key components of the PNN program is being shared across BASICS II PNN implementation countries, helping to influence newborn programming at all levels within each of the selected countries and to integrate into total child health care delivery. BASICS II is also able to share experiences within regions (e.g., the WHO Southeast Asia regional meetings that have been held to assist countries with strategy development and sharing of strategic experiences). BASICS II may have neglected to emphasize and adequately implement a number of PNN interventions with established effectiveness in improving PNN survival, for example, malaria interventions (ITNs, IPT, case treatment), ante/postnatal iron/folic acid supplementation, early initiation/exclusive breastfeeding. Where these were part of BASICS II’s activities, it does not appear that they were promoted as part of the overall approach to PNN survival. BASICS II has been able to demonstrate two approaches to implementation of a new program: a regional or national-level approach and a community-level approach. The regional/national approach has been demonstrated in the Asia and Near East Bureau (ANE) in collaboration with the Southeast Asia Regional Office (SEARO), where BASICS II has advocated for the adoption of PNN programming and has worked with regional and national level partners to draft strategies and policies. In Senegal, BASICS II is working at the community level to create evidence-based programming to use in national-level policy and protocol creation. BASICS II Country Selection BASICS II worked in collaboration with Missions to identify countries (although it was a long and time-consuming process for BASICS II to obtain interest from Missions) or at country level, for the PNN programming. Countries selected were those where BASICS II country programs expressed interest, BASICS II was able to obtain Mission support for the PNN program, and it was able to secure buy-in from local partners. BASICS II Successes and Shortfalls Successes BASICS II has been able to influence key decision-making partners (including Missions and ministries of health) in four countries to begin implementation of the neonatal health program. BASICS II has been able to obtain regional-level support and buy-in of the program from ANE and has been instrumental in helping to craft a regional health strategy for newborns in collaboration with WHO SEARO. In India, BASICS II has been successful in integrating the PNN program into a preexisting nutrition and health project implemented by CARE in eight states. 31 In Honduras, BASICS II developed and has supported an MOH action plan for maternal and child anemia. The program also is developing a protocol to be used by midwives and community volunteers. In Senegal, although it took a long time to obtain buy-in from the country level, the MOH now considers this an MOH initiative and is utilizing BASICS II to evaluate the package of care delivery in one district. This evaluation will be used by the Senegalese MOH to draft national-level policies and protocols. BASICS II is using the evidence of successful programs and approaches and demonstrating how to replicate them within the PNN context. Shortfalls The length of time needed to establish PNN programming has made it difficult for BASICS II to achieve program expansion potential within its focus countries. BASICS II will not be able to show impact results within this project timeframe; mortality statistics are not attainable within this timeframe. BASICS II will be able to show behavior change within this timeframe and it is hoped that this information will have an influence on future PNN programming within the selected countries. BASICS II has had only limited links with the Maternal and Newborn Health Project (MNH) and with the Saving Newborn Lives (Save the Children) activities. Factors That Influenced Successes and Shortfalls ! The length of time needed for advocacy in the neonatal area surpassed initial expectations. Moving forward without buy-in from global (and especially national) level partners was not possible. National-level partners included Missions as well, which hampered BASICS II’s ability to establish programming within USAID countries. ! BASICS II also required long lead times to develop tools and resources within the program and this took a long time to obtain buy-in and support from all levels involved in the program. ! BASICS II was confronted with a dearth of qualitative research for addressing PNN issues within countries. These qualitative studies needed to be completed before BASICS II could begin designing certain program components. ! BASICS II had difficulty in recruiting highly qualified technical staff for this relatively new area of technical emphasis. The size of the staff was relatively small and there was limited staff support. 32 Conclusions and Recommendations Conclusions ! PNN is gaining recognition as a critical area of focus in the global area and BASICS II has been a key proponent of discussion and recognition. ! BASICS II has been able to take on the advocacy role of PNN intervention and obtain interest and buy-in at all levels in four countries. Through BASICS II’s work in these four countries, the PNN program is ready to be expanded now or in the near future. Recommendations (to address country-level issues over the life of the project) ! Continue to work with high-level partners to obtain best practice models adopted for broader global use. (Utilizing links with PAHO, UNICEF, and WHO, use research to help establish best practices and have them adopted.) ! Continue advocacy role with national-level partners, using available tools to exercise leverage with established linkages to address PNN issues within a larger context. (Linking ITNs and iron/folic acid supplementation within the maternal framework and PNN survival, work to link to MNH within USAID.) ! Work with global, regional, and national-level partners to share policies, protocols, and tools drafted or created within the BASICS II project to have them adopted and adapted for broad utilization within the broad maternal and child health context. ! Promote interventions with demonstrated impact on low birthweight and neonatal survival (e.g., ITNs and IPT for malaria in pregnancy, tetanus toxoid immunization, and antenatal iron/folic acid supplementation) while defining/refining the peri/neonatal care package. INTEGRATED APPROACHES TO CHILD HEALTH Annex I of the BASICS II contract includes performance indicator 1b: “new approaches to delivering child survival interventions developed, evaluated for effectiveness and implemented in at least seven countries.” There are only two related performance standards: fully operating IMCI and other integrated approaches and interventions to reduce neonatal mortality and morbidity. The only performance indicator for IACH specifies only that integrated approaches should be developed in at least seven countries. However, the BASICS II strategic plan includes the following three Intermediate Results: ! Approaches to improve household and community health, nutrition, and child development behaviors adapted, tested, and “taken to scale” in 10 countries; ! Health system capacity to support integrated approaches to child health and nutrition improved in five countries; and 33 ! Case management and preventive actions at first and referral-level facilities improved in 10 countries. IACH is a somewhat different type of TFA than are the other three TFAs discussed above. This does not detract from the importance of IACH because implementing effective strategies for delivering health services to children through an integrated mechanism is more likely to have an impact on child morbidity and mortality and is typically more cost-effective. This is especially the case when these mechanisms focus on the underserved communities that do not have access to first-line health facilities for routine prevention and treatment of childhood illness. The fact that initially there was only one “number-of-countries” indicator for IACH within the monitoring and evaluation system described in the BASICS II contract and related documents may reflect the difficulty of establishing indicators and targets for this intervention. In fact, by using effective integrated approaches, it becomes easier to deliver the other critical interventions and to have an even greater impact on morbidity and mortality. In 2001, a BASICS II external partners’ review team defined a narrower focus for this TFA. Until that time, there had been some confusion within BASICS II about the role of one technical team having the integrative role, and this configuration did not work. As a result, the partners’ team recommended that the IACH team be renamed c–IMCI, and that its work be focused on achieving the four SO results related to malaria (2), diarrhea (1), and ARI (1). In February 2002, the partners’ review team further recommended that “the technical focus of this TFA is the treatment and prevention of the primary causes of childhood mortality and morbidity—in particular, malaria, ARI and diarrhea.” While this shift in focus of the IACH TFA helped to provide a programmatic vehicle for achieving the prescribed performance standards in malaria, ARI, and diarrhea, the focus on IACH was deemphasized. Use of Flagship Functions The concept of IACH has many dimensions. One of the most widely used frameworks is the strategy for IMCI, which has three components: ! increasing the skills of health workers, ! improving the delivery system, and ! improving household and community practices. Initially, as a result of promotion by WHO, emphasis was placed on the first two components, and for some years the household and community component was typically not addressed in a major way. The USAID CTO ordered BASICS II to transfer its initial focus from health worker training to the more innovative c–IMCI. The previous discussion of GTL highlighted the fact that BASICS II has been at the forefront technically and invested considerable resources in the development and promotion of c– IMCI, which in many countries has been the missing segment in improving child health in areas where people cannot reach first-line health facilities. 34 BASICS II has documented this integrated approach by developing a framework for c– IMCI21 and promoting it through BASICS II activities at global, regional, and national levels. BASICS II has also promoted the c–IMCI approach through the SET flagship function. In addition, the popularity of the AIN program in Honduras has resulted in an informal but highly effective SET–type activity. A number of countries have either visited Honduras or have had Hondurans visit their countries in order to learn about AIN and then develop similar programs—on their own—within their own countries. Choice of BASICS II Countries The choice of countries by BASICS II for IACH was made from within the overall countries with which BASICS I was already working. For example, Honduras had been implementing AIN since 1994 but was in need of technical assistance and resources to expand the program into other areas of the country. BASICS II is providing technical assistance and supervision in three regions to help institutionalize AIN so that it can become sustainable. Similarly, in Senegal, BASICS II has used the IACH community￾based approach in developing and expanding the decentralized child survival program. BASICS II Successes and Shortfalls In general, BASICS II has worked closely with USAID Missions, host country governments, and country partners to design and/or implement a minimum package of child health interventions that match the needs of the respective countries. While the interventions are not always the same, they typically are based on the critical factors in the country environment (e.g., epidemiology, implementation capacity) in order to help ensure that they target the population groups most in need and that they will become sustainable over time. The team’s visit to Honduras highlighted the importance of BASICS II’s technical assistance to help expand the AIN program that focuses on growth monitoring and promotion, including links to first-line health facilities for support and referral. BASICS II created the document about reaching communities to provide both a rationale and a strategy for developing c–IMCI approaches in a variety of country settings. This publication has been made widely available through the BASICS II SET program. In retrospect, not having the opportunity to work in a large number of countries to develop integrated approaches to child health was a shortfall. There are several important factors contributing to this reasoning: ! Most of the initial BASICS II countries had participated in BASICS I; essentially, they continued to implement the same traditional child survival interventions with BASICS II assistance through three of the four BASICS II TFAs. Since IACH was a new (and less well understood) TFA than the other three more technical interventions, it was seldom included in Mission child health programs. The AIN program in Honduras is a notable exception, and other Central American countries have already begun to implement the AIN model. 21 “Reaching Communities for Child Health and Nutrition: A Framework for Household and Community IMCI,” April 2001. 35 ! The BASICS II Reaching Communities document was only produced in April 2001, and thus a clear rationale and a strategy for implementing IACH programs were not available for general use during the time most BASICS II country assistance programs were developed. ! The limited remaining timeframe within BASICS II at that time—less than three years—left little time to develop and implement new IACH programs. This is especially true since the factors affecting performance described in the following section emphasize that significant investments of both time and resources are usually required. ! In many countries, a large proportion of the population does not have access to primary health facilities and could benefit significantly from community￾level child health interventions. The fact that IACH was not attempted more frequently seems to be a missed opportunity in terms of putting in place effective mechanisms that could eventually have a major impact on improving child health, especially in underserved areas. Factors Affecting Performance There are many factors that have influenced BASICS II work in IACH. Many are not within the control of BASICS II; positively affecting others often requires both a major investment and a relatively long timeframe. Several important examples are highlighted below. ! If relevant policies and guidelines already exist within the framework of the host country government, BASICS II was able to use these as a platform for developing and helping to implement IACH. Again, Honduras provides a good example, because in 1994, the government had established a policy to implement the AIN program, and BASICS II assistance helped to reinforce and speed up its implementation. ! The BASICS II c–IMCI approach is flexible in that it encourages a minimum package of services that are likely to be sustainable within the country environment. Although the AIN approach predated BASICS II, it is a good example of a limited package (focusing primarily on growth monitoring and promotion) that has a good chance of eventually increasing to the national level and being sustainable. ! The active involvement of other country partners is critical. WHO/PAHO and USAID/BASICS have different organizational philosophies regarding the importance of c–IMCI and its relationship to the training and health facility components of IMCI. These differences have been reflected in different recommendations to the MOH in Honduras from the same organizations. ! Capacity building is almost universally an important factor, since typically there are insufficient skills among health workers, communities, and local partners to achieve IACH on a large scale. Linked to capacity building is the 36 obvious need for sufficient human resources, as noted above. Often, limited BASICS II resources, as well as the relatively short project timeframe, have constrained the level of capacity building. ! The fact that IACH was developed by USAID as an Intermediate Result (IR) rather than an SO (and has no programmatic indicators and targets) may have been a significant factor in the extent to which integrated approaches have been supported in BASICS II countries. Conclusions ! The BASICS II Reaching Communities framework (c–IMCI) has provided both a rationale and a strategy for community-based child survival programs. ! The IMCI–based approach has led to worldwide acceptance of the framework for reaching communities. ! BASICS II has had major successes in implementing child survival packages that match the needs of the respective countries. ! “Scaling up” is often constrained by most if not all of the factors described above. Recommendations ! Continue the integration of c–IMCI and other community approaches in existing country programs, with emphasis in implementing an exit strategy to help ensure program sustainability when BASICS II support ends. ! Work with other partners in BASICS II countries to increase their support of c–IMCI. OTHER KEY FACTORS THAT INFLUENCE SUCCESS IN “SCALING UP”22 Findings Several crosscutting factors have influenced BASICS ability to support scaling-up efforts in the four TFAs: ! uncertainty in the definition and approaches to achieving scale, ! country selection, ! timing and funding constraints, ! relationship with other donors, and ! readiness to move to scale in a particular country. 22 “Scaling up” and “move to scale” are terms used by USAID. The discussion in the following section attempts to define the terms. 37 Uncertainty in the Definition of Scale and Approaches to Achieving Scale During the initial years it became clear that the term “scale” had an assortment of meanings to key stakeholders in the program. The contract stated that the SO–level result for the Child Health Results Package is “increased use of effective, improved and sustainable child health interventions.”23 The contract also states that “in some settings, the most important strategy for increasing use of child health and nutrition interventions will be wider and more effective application of existing approaches.” In other circumstances, the contract anticipated that the “development and application of new approaches” might be needed before increased application. Also in this case, “these approaches will also be evaluated in terms of their effectiveness in increasing use” (e.g., low cost and replicability) and “if effective, the Flagship will aim for broadened application of these new approaches.” The contract includes language about both “contributing to scale” and “achieving scale of intervention.”24 BASICS was sometimes uncertain about what scale “was enough.” Was working in the “three USAID regions” in Honduras enough? Or would the contract be judged on whether TFA interventions were also being introduced in regions where the host government had given other donors the lead? USAID Missions were normally content with BASICS helping them achieve the first objective and would place priority on reaching the largest population within “their regions.” The contract, however, was often interpreted to require that the ideal “scale” for BASICS was nationwide scale. Therefore, “breadth” of geographic coverage would be appear to be more important for GH than “depth” of population coverage. Some performance standards encouraged this “breadth of coverage” (90 percent measles coverage achieved in 6 countries). Most BASICS II technical staff had used the BASICS I activity approach to planning. They had difficulty with three new concepts in BASICS II: ! how to program contract resources to explicitly move to scale, ! how to obtain other resources to help achieve scale, and ! how to measure and report attribution. BASICS leadership (the chief of party had little prior experience with USAID programs) did not supply a clear strategic approach. USAID could not or did not provide any successful approaches from other USAID programs. BASICS II provided an acceptable long-term strategic plan within 6 months of the contract’s inception. The BASICS II team had a much more difficult time translating the vision of the strategic plan into an acceptable first year implementation plan. A much more useful program framework was provided by a consultant, near the end of the contract’s first year. Based on this program framework and the extra effort invested during the latter part of the first year, a greatly improved work plan was submitted to USAID in July 2000.” However, the project continued to struggle with the practical application of the concepts of scale and linking activities to results. In early 2001, the project adopted a simplified 23 BASICS II contract. 24 BASICS II contract, annex 1. 38 systems model (inputs–processes–outputs–outcomes–impact) to identify the linkages between program resources and activities to results. Called a country program design, this tool was used by all country program teams to identify activities to be completed by the completion of the project; expected improvements in policies, programs, or resources (interim results); and eventual changes in the use of child health interventions (Strategic Objective–level results). More recently, USAID’s CTO received good results by simply asking BASICS II to outline what would be in place at the end of the contract (e.g., end-of-project indicators). Country Selection Country selection for BASICS has a clear impact on the contractor’s ability to work at scale or to achieve scale. Twelve of 16 BASICS II countries were preselected by USAID, but only after a careful review of their fit with the BASICS II objectives. Many Missions transferred field support funds to Washington for buy-ins to the contract even before the contract was signed. BASICS II is quick to point out that moving to scale in countries with the size and complexity of DR Congo or Nigeria is extraordinarily difficult. By contrast, BASICS was in the opposite situation in India when it was invited to work with CARE’s well-established, mature, nationwide program. Timing and Funding Factors BASICS was required to move to scale or work at scale within a five-year contract timeframe. Effectively, this translates into less than four years for measuring impact: the time between the baseline survey, normally carried out late in the first year or early in the second year, and the final survey in the last year. To complicate the timing issue, some USAID Missions have required their own timetable for BASICS to achieve impact. USAID/Senegal’s SO agreement continues two years beyond the BASICS contract, so it is most insistent on BASICS achieving desired coverage scale by 2006. The amounts of USAID funding and its timing are other factors that have complicated BASICS’ efforts to scale up. USAID Mission funding for BASICS II normally has been based on the funding that the Mission has available for child survival within its total health (and Strategic Objective Grant Agreement [SOAG]) budget. It is rarely based on an estimate of what BASICS might need to achieve scale. Examples include: ! USAID/Honduras provided approximately $300,000 a year for BASICS in FY 2001–2003. In FY 2004, HIV/AIDS funds are replacing child survival funds within the Mission’s static health budget; therefore, BASICS will receive less funding. ! USAID/Nigeria: In FY 2002–2003, polio earmarked funding replaced Mission child survival funds, limiting BASICS ability to implement its long￾term country plan. These funds will only be replaced late in the fourth year of the contract. 39 ! USAID/Nepal provided only $300,000 for BASICS II work over a three-year period. Scaling up requires not just robust funding but also assured funding for more than one year. To help resolve this problem, BASICS II developed long-term agreements (Memoranda of Understanding) with Missions to achieve some continuity for planning and implementation. The memorandum typically specifies annual funding contributions over several years from both the Mission and GH. Relationships With Other Donors BASICS can be proud of several excellent examples of using other donor programs and supporting other USAID–funded programs. ! Senegal: The BASICS/Senegal director and African Regional Office (AFRO) nutrition specialist worked with other Senegalese nutritionists to design a nutrition program that now has World Bank funding and implements the BASICS MinPak in many non–USAID (and therefore, non–BASICS) regions of Senegal. ! Honduras: A child health program funded by the IADB in Honduras implements AIN in several non–USAID (non–BASICS) provinces. The director stated that he “totally depends” on using BASICS/Honduras to provide technical guidance and training materials for his program. ! GAVI: The BASICS senior immunization adviser is a key member of the GAVI working group that reviews and approves all GAVI immunization grants. ! PVOs: BASICS hosts periodic meetings and provides technical guidance to six USAID child survival grant recipients (PVOs) that provide AIN to hard￾to-reach populations in Honduras. Working with other donor programs, however, can often be difficult. In many countries (Senegal, Benin, Nigeria, Honduras), the host country directs donors to work only in specific geographic regions, thereby limiting BASICS II to USAID regions. Attempts to reach beyond these geographic limitations to use other donors first requires government review or acquiescence. The absence of agreement on a child health intervention at the global leadership level can affect a country’s scale-up efforts. WHO/PAHO and USAID/PVO differences of approach for IMCI has trickled down to Honduras, where PAHO and BASICS vie for government of Honduras approval of their respective c–IMCI and f–IMCI approaches. Readiness to Move to National Scale in a Particular Country How automatic is the move to national scale for mature child health interventions? Interviews with senior BASICS staff and other child health experts with field experience 40 provided a short list of essential factors needed before supporting a successful nationwide program. ! The technical intervention (e.g., the nutrition MinPak) has been tested and adapted to host country conditions to the satisfaction of the MOH. Guatemala plans to incorporate the Honduras AIN model in its national program, but only after adaptation to its particular health structure and experience with community health programs. ! The MOH can provide strong program leadership and effective nationwide coordination. In Guinea, BASICS did not believe that the MOH had the stability and leadership to support scale up. ! A reasonably effective health structure (facilities and personnel with basic equipment, materials, and operational costs) is in place with trained staff. Necessary policies and norms are in place to support the intervention. In Nigeria, the national counterpart to BASICS (National Public Health Care Development Agency) has not been given authority for nationwide child health programs and has no operational budget. ! Adequate local cost funding is consistently available. Mature child health interventions depend on a mixture of foreign currency (to purchase vaccines, micronutrients) and local cost funding for program operations. Too often, local costs are simply not available for basic expenses, such as health personnel salaries, vehicle maintenance, gasoline, per diem (for field supervision visits), and basic commodities. Recent government decentralization programs in many countries have passed local cost funding responsibilities to local governments that have limited human resource capacity and limited capacity to generate local revenues. Examples include the following: • In Senegal, a USAID–supported matching grant program for local governments is being tested. The grant structure places priority on social services (including health) and provides incentives for local authorities to finance health services. • In Honduras, 80 percent of the MOH budget pays personnel salaries. Health center nurses have no funds to pay for supervision visits by community volunteers implementing AIN in 50–100 villages. An additional nurse for each health center is desirable to ensure health center coverage while supervision visits are made. MOH hiring is frozen with most personnel providing curative services in urban areas. BASICS hopes to find ways to convince municipal and local governments to hire and pay the additional nurse. ! Sufficient donor presence and donor cooperation: Even in small African countries, a single donor (such as USAID) rarely has the resources to support 41 nationwide scale-up programs. Donor agreement with the MOH on their roles in a nationwide program strategy is essential. ! Continuity of vision among MOH and donors: Nationwide scale-up of child health interventions requires a permanent MOH priority for child health and donor willingness to help create sustainability. • BASICS has provided very strong support to the AIN program for six years. Scale-up efforts have not yet resulted in nationwide coverage, especially in hard-to-reach regions. The existing program is described as fragile. AIN’s effectiveness and sustainability depend on continued program adaptation/improvement (e.g., discovering effective incentives for community volunteers, ensuring local cost funding, adding a neonatal component). This is not the time for donors to terminate their child health program support in Honduras. Conclusions ! The definition of scale has different meanings for BASICS, GH, and USAID Missions. ! The BASICS program framework has assisted BASICS staff in understanding the dynamics of working at scale. The country program design tool has served the project for its practical applicability in linking country program activities to interim and SO–level results. Leadership vision is key to providing a common set of concepts, terms, and tools and is an important factor in planning how to attain scale. ! Scaling up requires robust funding. In many countries, BASICS was not provided sufficient funding to work at scale. ! Mission SOAG periods sometimes did not coincide with the BASICS contract period, complicating long-term planning and funding in those countries. ! The five-year contract period limits BASICS’ ability to measure impact to only three years. For many indicators, this period is too short to demonstrate significant change in impact. ! BASICS can be proud of several excellent examples of using donor programs and provision of effective technical support to child survival grant recipients and other NGOs. ! Multidonor programs lead inevitably to issues of how to determine attribution. USAID provided no clear written guidance on how a contractor should measure and report attribution. 42 ! Lack of consensus among key international partners on a few technical issues (especially IMCI) has sometimes been reflected at the country level and has hampered effective donor cooperation in achieving program objectives. 43 IV. KEY MANAGEMENT FACTORS BASICS II MANAGEMENT FACTORS Planning, management, and administrative support of contract services supports the greatest possible application of project resources to achievement of results. Five key management functions were identified: ! organization, leadership, and staffing; ! planning; ! financing, budgeting, and resource allocation; ! program results monitoring (PRM); and ! partner relations. The contract does not specify performance standards for organization and leadership but does for the other functions. It also includes a section on management functions, tasks, and capabilities of the project that consists of sections on planning, PRM, and technical reporting. Organization, Leadership, and Staffing BASICS II is a project of the Partnership for Child Health Care, Inc. (PCHC). This is a 501 (c), nonprofit joint venture corporation established in 1993 by AED, JSI, and MSH. PCHC was awarded the first BASICS contract in 1993 and the BASICS II contract in 1999. There is a board of directors made up of the chief executive officers from each of the principal partners (AED, JSI, and MSH). The BASICS II project director reports directly to the board. There is also a Project Management Committee that meets 2–4 times a year; it is made up of representatives from each of the nine partners. The partners bring a variety of skills and expertise to the project to the degree that most of the technical capability required by the project is present in one or more of the partners. Most project staff and consultants have been recruited from staff of the nine partners. Initially, most staff were carried over from BASICS I. The first director was a technical specialist. There have been four directors in the first three years. The current director is a public health management specialist. Initially, there were four deputy directors, one each for technical and country programs; finance, management, and administration; performance and results monitoring; and OER. There have been a few organizational changes, notably the merging of PRM and OER and the establishment of a SET unit. BASICS II Partners and Areas of Expertise 1. AED: Behavior change, communication, nutrition 2. JSI: Public health and management 3. MSH: Project administration, public health, management 4. Emory: Neonatal health, immunization, nutrition 5. JHU: Operations research 6. Manoff Group: Nutrition 7. PATH: Support of appropriate technologies 8. Save the Children: Field-based operations, research, and linkages with PVOs 9. TSL: Nutrition 44 Findings Organizationally, the project began as a continuation of BASICS I, with the exception that USAID insisted that the field support program had to report to the technical director to ensure the technical quality of field operations. The initial BASICS II structure was later restructured to fit the new results orientation of BASICS II. This restructuring was based on the key flagship functions and core technical focus areas. This led to the development of TFA plans that were not clearly linked to flagship functions, especially country program plans. The restructuring left the group of staff known as crosscutting resources without a clear place in the organizational chart of funding source. By the third year of the project, at least five of these staff members had left the project. Initial delays in filling all senior staff positions (the full complement was on board by February 2000, 6 months after contract initiation) were followed by the departure of the original project director after 18 months and his replacement by two interim directors for another 9 months. In 2001, the PEB noted that the contractors had not even “provided highly qualified candidates” to replace the original director. The inadequate performance of the original director and the subsequent rapid turnover in contract leadership led to delays in startup activities and made it difficult for BASICS staff, especially field staff, to be certain of the strategic course that headquarters leadership wanted to provide. An interim director rewrote the job description of the deputy director (a key remaining point of continuity among the senior staff) who then resigned in January 2002. A permanent replacement for the original director was appointed in October 2001. There was also much turnover in the technical staff, partly due to the reorientation of the project toward achieving results, which required a difficult adjustment period for a number of technical staff. These leadership difficulties and uncertainties, therefore, continued through the first half of the five-year contract. The next PEB noted that the project’s new leadership had made substantial and effective efforts to improve and streamline management of the project. The PEB also noted the increasing, although belated, involvement of the managing partners in resolving the program leadership issues. USAID/Washington has been generally satisfied with the quality, capability, and performance of the BASICS II senior technical staff. However, the advantage that the partnership was supposed to provide in terms of staffing has not worked out as well as expected. The project has noted that “in spite of the potential, it has been hard to fully engage partners in BASICS II. Partner organizations have full plates of work and the staff is not always available when needed. In addition, managing in the involvement of PCHC, Inc. and nine partner organizations has been a complex challenge.”25 Because overhead is only charged on full-time BASICS II staff (and not on consultants), there is no incentive for the partners, especially the minority partners, to propose consultants. Smaller partners, such as Emory, have very few full-time staff and cannot afford to assign them full time to the project. Because they are not encouraged to hire and use consultants to carry out their tasks, this severely restricts their participation in the 25 “The Evolution of BASICS II,” p. 16. 45 project. Majority partners have a built-in incentive to assign their full-time staff to the program rather than to lend them to the project as short-term consultants. When field staff reductions were needed at the initiation of BASICS II, the partners reportedly objected to retrenching of their full-time staff. WARO was established with AFR funding. This seemed like a good idea initially, given the large number of country programs in Africa. Over time, however, the number of countries that AFRO supported has dwindled and the office has largely become an expensive second source of technical assistance to the Senegal country program, which the country program no longer needs. WARO, following AFR guidance, also emphasizes capacity development rather than results, which contrasts with BASICS II’s primary objective. BASICS II headquarters argues that it has been forced to utilize precious funds ($1,750,000 each year) within the contract’s fixed ceiling to support an AFR regional objective that does not directly lead to the achievement of performance contract targets. Conclusions The partnership (PCHC, Inc.) has not been as effective a management mechanism as expected. The board of directors is not as involved as expected and the partnership mechanism has not been particularly useful in providing needed staff and consultants. The organizational structure of BASICS II does not appear to promote cohesion. Each TFA operates independently, as do the country programs and the regional office. Rather than promote integration, the structure sometimes encourages independent vertical programming.26 BASICS II had significant problems with staffing over the first two years, especially at the senior leadership levels. This resulted in serious delays in planning and implementation as well as frustration among BASICS II staff and the USAID CTO team. The major organizational and staffing challenge of BASICS II has been to merge program and technical staff—to have the technical staff become technical managers rather than technicians. USAID originally was looking for technical leadership from the director when what was needed was management leadership. WARO now has a much reduced regional support role in West Africa and has become largely redundant to Senegal’s needs. Transfers of two senior technical staff to BASICS II headquarters has weakened its ability to provide regional leadership in nutrition and immunization, according to WARO, AFR, and USAID/Senegal officers.27 26 BASICS II set up headquarters country teams for the major country programs to encourage program integration. 27 BASICS II believes that its African regional interests are even better served with their two senior WARO technical officers in BASICS headquarters. 46 Planning Planning is a critical function of the contract, which calls for the development of both a strategic plan and “detailed implementation plans for all components.”28 A transition plan was to be completed in the first month and a strategic plan within the first 6 months. The strategic plan was to “define the work to be accomplished over the life of the activity in the four Technical Focus Areas…The plan shall define specific lines of work that the contractor will undertake to achieve the results described in the G/PHN Results Framework.”29 Also within the first 90 days, G/PHN and the contractor were to agree upon an annual planning cycle and program review cycle. Based on that cycle, the contractor was to develop a detailed annual work plan that the CTO would approve. Findings BASICS II had no trouble preparing the transition plan and the strategic plan. However, it had a very difficult time developing a first annual work plan that linked first year activities to life-of-program results. The PEB noted that during the first year of the contract, the contractor did not produce an acceptable results-oriented work plan for its TFAs and country and regional programs. The first work plan was rejected; this continued to be a problem in the second year. Among the principal reasons was the underestimation of the time and effort required to develop a results orientation within the project. The tools needed to carry out multiyear planning and linking activities to results were developed, introduced to staff, and used projectwide. USAID and BASICS staffs both acknowledge this problem, which was characterized as “never-ending planning” for the first two years. For some staff, the creation of a five-year overview of activities to guide single year work planning was a new approach. Frustration built within the project as technical officers preferred to move forward with a series of technical activities while project management sought to clearly delineate the sequence of activities that would lead to results. This problem was not limited to BASICS headquarters; it affected all of the country programs as well. One country noted that planning was complex, not useful, continual, and frustrating. The solution came in the second year with the development by BASICS management of a number of planning tools that were eventually embraced by BASICS staff. The plans are now results oriented, integrated, and linked to budgets. One Mission noted that the plans now are much better, shorter, and more useful. Despite this 28 Prime contract, p. 35. 29 Ibid., p. 35. Factors Affecting Performance ! Leadership, especially the managerial leadership of the project director and majority partners ! Organizational structure, which can promote integration or discourage it ! Technical and managerial expertise of project staff ! Ability and willingness of staff (central, country, and regional) to buy into the results-based approach to programming 47 improvement, planning still has several shortcomings, according to USAID. Among the criticisms raised in the 2002 PEB report are inadequate strategies to achieve significant coverage or impact and inadequate levels of impact planned. BASICS II is still having problems linking activities to results, although this has improved. In addition, the structure of the TFAs results in vertical rather than integrated plans for child health. There is still tension between central and Mission objectives. For example, the objective of USAID/Senegal’s SO agreement with the government of Senegal is not to expand the coverage of interventions but to improve access, quality, private sector involvement, and increased local funding for child survival services. In Honduras, the BASICS II SO objectives fit those of the Mission to a large degree. However, USAID/Honduras concentrates on certain geographic regions. It does not plan to achieve comprehensive national coverage, although the program expanded into two new regions in response to the devastation of Hurricane Mitch and the availability of special Hurricane Mitch funds. Conclusions ! BASICS II had significant problems in preparing work plans that were results oriented. This had serious effects on project progress and staff morale. ! Technical staff, in particular, had a very difficult time switching from an activities orientation to a results orientation. Quite a few technical staff left the project as a result. ! Although much better now, there are still several weaknesses in planning, especially planning to achieve national-level impact, vertical planning, and plans that reconcile the different perspectives of central, regional, and Mission objectives. ! One fundamental planning issue is the merging of central and Mission objectives. To the degree that there is a compatible fit between the two, then the country program will not only be more successful but the experience gained may also contribute to global technical leadership. Financing, Budgeting, and Resource Allocation The BASICS II contract does not specifically address financial management issues. The performance indicator and standards are found in annex 1, but without a narrative that describes what USAID meant. The performance standards are relatively straightforward but the performance indicator is not. In fact, there is no linkage between the two. As phrased, the performance indicator, “high quality services are delivered at the lowest possible cost,” is practically impossible to meet unless “high quality services” is defined. A general rule for this type of indicator is that either the numerator or the denominator Factors Affecting Performance ! Leadership in introducing a new planning process ! Willingness of staff to buy into the new process ! Compatibility of central, regional, and Mission objectives 48 must be held constant. The objective could be to find the least expensive way to provide a given quality of service, or, for a given amount of money, find the highest quality of service that can be provided. The vagueness of this indicator did not deter the PEB from applying it in the award reviews. The first PEB review stated that the project had not successfully delivered its services at the lowest possible cost. No mention of this item was made in the second PEB, but it was raised again in the third PEB review as a “significant concern.” However, the reviewers seem to have misunderstood what this meant, since they referred to project costs rather than service delivery costs. The PEB reacted negatively to the initial high expenditures when the project was still largely focused on strategic and first year plans. On the positive side, the PEB has been generally pleased with the project’s efforts to control costs and to keep the allocable cost factor below a certain level. Findings Financial control: Over time the project has made significant progress in budgeting and cost containment. A financial management system was put in place to provide detailed information on cost categories. Costs can be monitored by Intermediate Result, TFA, country program, and task. The project has also developed a five-year financial overview system to keep staff focused on multiyear activities and end-of-project results. Staff members note that they now assess each potential activity and opportunity that arises in terms of its cost and potential contribution to the achievement of SOs. Cost containment has become a project norm. Management and staff have identified a large variety of ways to save money, including reducing per diem rates, eliminating business class travel, and changing vendors. BASICS reduced telephone costs, for example, by 75 percent by changing vendors. BASICS has also eliminated staff positions and merged OER with PRM, which also reduced costs. However, some of these cost reductions have diminished or even eliminated key functions. OER is a good example. Allocable cost factors have been one of the lowest among USAID centrally funded contracts due to the indirect cost system set up by the partners. Overhead is taken only on full-time staff. No overhead is taken on consultants or operating costs, which have helped to keep allocable costs low. Many of the financial difficulties that the project has faced are well beyond its control; many are due to USAID’s own contracting and programming procedures. Chief among these are the following: ! Budget ceilings and earmarked funds: The contract ceiling could not accommodate a greater than expected Mission demand for BASICS II support. Core funding and regional bureau funding have been provided at expected levels, but Mission field support funding has exceeded the planned 40 percent of total contract funding and is now at 54 percent of total funding. A substantial portion of the additional field support funding is linked to Mission absorption of approximately $23 million in earmarked funds for polio 49 and malaria (not anticipated in the program design). The contract ceiling will be raised by $23 million via a contract amendment in FY 2003. ! Annual funding: The project has a five-year duration and USAID has insisted on multiyear work plans linked to results. However, funding is received on a year-by-year basis from 21 Missions and bureaus, including seven types of earmarks. This annual funding from multiple sources constrains the project’s ability to make multiyear commitments. Annual negotiations require large amounts of staff time. Conclusions ! Overall, BASICS II has performed well in controlling costs and keeping USAID informed about financial matters. The early problems with uncontrolled expenditures and unrealistic budgets have been largely overcome. ! The USAID–generated constraints on resource allocation, budgeting, and cost containment remain to be resolved. These are internal USAID issues and cannot be resolved by contractors. Program Results Monitoring (PRM) Findings The contract emphasizes results monitoring, as the performance standards indicate. But, as noted previously, the results orientation was new to many of the staff and there was quite a bit of resistance to it. The managers responsible for planning and PRM had a better understanding of this approach and eventually were able to develop planning and monitoring procedures that focused clearly on results. The first PEB review was critical of the lack of progress in staffing the PRM unit as well as in developing indicators and carrying out monitoring, but it also acknowledged that these delays were largely due to the absence of an acceptable implementation plan. By the second PEB review, much progress had been made. Plans for program monitoring Factors Affecting Performance ! Contractor understanding of USAID contracting and financial procedures ! Contract ceilings ! Amount and availability of flexible global funds ! Mission buy-ins ! Special uses attached to funding accounts ! Level of funding for country programs and central functions ! Annual funding ! USAID understanding of the rationale and calculation of overhead ! Financial monitoring and reporting systems 50 had been produced for many countries, and the project had identified available sources of data and information that could be used to monitor progress. Surveys or other data collection approaches have been designed and some baseline studies have been conducted. However, USAID was still concerned about inadequate benchmarks that would allow BASICS II and USAID to assess progress toward the ultimate coverage indicators. This concern carried over into the third PEB review. USAID has also been critical of BASICS II’s emphasis on pilot or demonstration projects that are not expanded because of the need to collect endline data for the same geographic area as the baseline. Although this is a design issue rather than a PRM issue, BASICS II reports that this is not a constraint. The project area can expand and the endline data can still be collected in the original geographic area. This is the case in Ghana where the program has expanded to 16 districts but data are only going to be collected in the original 4 districts. BASICS II has baseline data for all countries and plans to collect endline data for all of them as well. However, there will be gaps, especially where BASICS II was relying on data from other agencies. In Uganda, for example, BASICS II was relying on a WHO– sponsored study to provide baseline and endline data. However, the study was canceled, which left BASICS II without endline data. Most of the baselines are limited to core SO variables. Few needs assessments have been undertaken because of costs. BASICS II did add questions on beliefs and practices to a baseline in Senegal. This information was used to identify information and behavior gaps at the community level, which was used, in turn, to develop intervention strategies and behavior change communication messages. BASICS II has noted that the endline surveys may not be worth the effort in some countries, especially those where the investments were very small or where there has not been enough time to see a significant change (DR Congo, for example). Overall, BASICS II expects to see significant changes in about 5 of the 16 country programs: Honduras Senegal, Nigeria, Uganda, and El Salvador. The contract includes a large number of SO and IR–level indicators. The PRM concentrates on the SO–level indicators. The IR–level indicators are more qualitative than quantitative and are not monitored as closely. As a result, BASICS II is not able to measure the effects of such key project components as global leadership, SET, OER, support to Missions, and even PRM. There are serious technical problems with many of the targets of the SO–level indicators but the contract has not been modified. USAID states that it offered to make changes during the second year of the contract once BASICS II provided clear country plans with five-year targets and the original director agreed that the targets should not be modified until the trajectory of country programs was defined. BASICS II indicates that these plans and targets were produced by the second year; verbal assurances were given that the targets established in the contract would be modified. USAID indicates, however, that BASICS II never provided complete and acceptable revised targets. In the fourth year, the CTO agreed to review the performance standards as one element of a contract amendment that was needed to raise the contract ceiling (reflecting unanticipated earmarked funding). However, the USAID contracts officer refused to take these combined actions, which could be interpreted as adding money to a contract while reducing the contractor’s responsibilities. 51 The contracts officer interpreted the original contract language to cover the evolution in the field of these standards (because BASICS II is a performance-based contract, fifth￾year activities are expected to continue to aim toward the achievement of the anticipated contract impact as described in C.2.c. of the original contract). This language states that “the results described above, combined with other efforts by USAID, other development partners and host countries themselves will be associated with improvements in the health of children.” In addition to the main indicators specified in the contract, BASICS is required to develop process benchmarks so that progress toward the indicators can be monitored throughout the contract period. These benchmarks were not included in the contract; therefore, they have been negotiated with USAID and included in BASICS’s work plans. This appears to be a reasonable and helpful managerial device that seems to be acceptable to both USAID and BASICS. BASICS/Senegal noted that its work plan includes too many of these benchmarks and it plans to reduce them in the next work plan. Conclusions ! After a slow start, the PRM unit has been able to create a useful and responsive performance-oriented monitoring system for SO–level results. ! PRM uses a streamlined systems model to aid country results teams in identifying key process and outcome indicators as well as the activities that need to be carried out over the life of the project to achieve SO–level results. This has been an important tool for reorienting staff toward achieving results as well as linking activities to results. ! The current system appears to meet all of the performance standards listed above; it provides an analytical basis for planning, measuring, and monitoring results. ! The impact and coverage indicators are important as guides to performance. However, some of the indicators are vague and the targets embedded in them often appear to be arbitrary, unrealistic, and/or even meaningless. They promote vertical rather than integrated child survival approaches and seem to be inflexible. ! The IR–level indicators are much less useful and specific than the SO–level indicators. As such, quite a few components of the project are not monitored or measured very well. ! Baseline and endline surveys will have been conducted in all 16 country programs. However, endline surveys may not be worth the effort in several countries that had very small programs or too little time to demonstrate any statistically significant change. 52 Partner Relations This contract relies heavily on partners to achieve results. These start with USAID offices in Washington and in field Missions, but also include other CAs, PVOs, multilateral and bilateral donors, and host country ministries of health as well as international technical agencies. In fact, “Working Through Partnerships” is a specific component in the scope of work: For all functions, the Flagship shall operate through partnerships with G/PHN, other bureaus, USAID field Missions, other USAID cooperating agencies and other organizations…This term is meant to indicate a strategic relationship in which the Flagship shall work with appropriate partners to identify opportunities for achieving impact and advancing the state-of-the-art of child survival policy and programming, while at the same time meeting concrete country, regional, or global needs, especially in the Flagship’s technical focus areas.30 Findings BASICS II had the advantage of existing relationships with most of the above countries through BASICS I. The first PEB review rated the project high on coordination and communication with many USAID central, regional, and country offices. In fact, this was the highest rated component of the first award review. The few complaints from Missions and regional bureaus related to the clash between local and central objectives. This carried over to the second PEB review, which was also generally positive about relationships, but noted that several Missions complained that BASICS was “doing its own thing,” rather than working with partners. Relationships had improved by the third PEB review, but again, a few countries still complained that partners were not adequately consulted or involved in project planning and implementation. BASICS also has strong relationships with such international technical agencies as WHO and UNICEF, which is critical to its global leadership function. This is especially true in immunization and nutrition. In Honduras, the assessment team found that BASICS had excellent collaborative relationships with the Mission, the MOH, UNICEF, the Pan-American Health Organization (PAHO), PVOs, and other CAs. BASICS has provided training and other 30 Prime contract, p. 34. Factors Affecting Performance ! The procurement mechanism: contract or cooperative agreement. ! The degree of contract emphasis on results and the specificity of targets for SO–level indicators. ! Willingness and ability of staff to accept results-based performance ! Flexibility in selecting indicators and setting targets ! The experience and analytical capacity of PRB staff ! Quality of planning and monitoring tools ! Enough scope and time to achieve statistically measurable improvements 53 materials as well as technical assistance to a number of these partners, thus helping to expand AIN to other areas of the country. A key reason for Mission support in Honduras is that BASICS/Honduras fits exactly into the Mission’s strategy. The same is true in Senegal. However, this is not always the case (e.g., Benin). Conclusions ! In general, BASICS II has been able to maintain and build on its positive relationships with other partners. ! These positive relationships have enabled BASICS II to influence global, regional, and country-level policies and programs, sometimes in very significant ways. ! The principal cause of friction, when it does occur, seems to be due to differences in agendas, especially at the country and regional levels where USAID and international agencies may not have the same objectives as BASICS. USAID MANAGEMENT FACTORS Program Design Process The design of the scope of work for BASICS II was led by the USAID technical officers who had managed the BASICS I contract. An independent evaluation had concluded that BASICS I was successful as a program that responded to the needs of field Missions and that advanced the child survival technical agenda. However, GH leadership and the CTO team were disappointed that the BASICS I team could report only in terms of activities and could not demonstrate accomplishments in terms of changes in public health indicators on child survival. USAID reengineering, which began in 1996–97 (after BASICS I was designed and initiated), had challenged USAID officers to begin viewing their programs more in terms of results or outcomes and less in terms of inputs and outputs. The design of the BASICS II scope of work squarely focused on “results” and “scaling up” child health programs in participating countries. Several major design changes were made from the BASICS I program: ! Smaller numbers of countries were expected to use BASICS II as it would not provide one-stop shopping for all USAID Missions. Factors Affecting Performance ! Staff expertise in technical areas, management, and communication ! Awareness and acceptance by various partners of BASICS II contract requirements ! Relationship/leadership role in relation to other GH CAs ! Flexibility on the part of BASICS II and partners 54 ! Technical content was given primacy over field support in the management structure suggested in the request for application (RFA); the field operations would report through technical offices to the program director. ! USAID delegated to the contractor the responsibility of negotiating the fit between BASICS program objectives and Mission program objectives. ! Results were defined in terms of public health indicators or impact measures (e.g., 15 percent reduction in under 5 morality in 10 countries) and performance standards (e.g., ORT use increased by 50 percent or sustained at 80 percent of diarrhea episodes in 10 countries). The program cost was estimated at $79 million, with 40 percent to be provided by GH (core funding), 20 percent by regional bureaus, and 40 percent by USAID Missions through the field support mechanism. The design was later described as “elegant, but daunting.” It was the first time that a flagship GH program had established “scaling up” as a major program objective, used public health indicators to measure program performance, and used a performance-based contract for a flagship program. At the same time, the design process was adventuresome since the members of the small design team were technical officers whose previous design experience was limited. This design process contrasts with traditional USAID design teams for Mission and regional bureau programs where project design officers provide design skills and experience as part of the core design team. Conclusions ! The design was carried out by a small, dedicated group of health officers who would have benefited from the participation of experienced project design officers who have a much broader view of USAID design options. ! The costs of the program were not considered during the design. There was an assumption that $79 million (along with obtaining other resources) could lead to the desired program results. ! The failure of the RFA process resulted in a significant delay and raised tensions between the single bidder and USAID. Careful procurement planning and involvement of a USAID procurement officer in the initial design might have eliminated the delay. ! The design incorporated an operational strategy for child health (“scaling up”) that had not been articulated in any explicit agencywide child health strategy document, nor had it been vetted among and accepted by USAID health officers in field Missions and regional bureaus. Varying Views of Program Objectives The change in program approach from BASICS I to BASICS II was significant. BASICS I provided one-stop shopping that tried to meet the needs of all USAID programs, while 55 BASICS II was to support only those 10–15 Missions that had “scaling-up” objectives. When a Mission’s request was not compatible with the BASICS II agenda, the BASICS II leadership was supposed to reject that Mission’s request for support. The BASICS II “scale-up” objectives were compatible with several USAID Missions (e.g., Senegal and Honduras). However, the compatibility was far from universal and many USAID Missions reported major problems with what they characterize as the lack of responsiveness or lack of flexibility of the BASICS headquarters office. Some examples are discussed in the following sections. USAID/Guinea’s request for BASICS II assistance was initially accepted by BASICS II after a visit by the program director. USAID/Washington questioned this decision because it appeared that Guinea was not a “scale-up” candidate due to lack of Mission funding for IMCI. The USAID Mission director insisted, with AFR support, that BASICS II could not leave the country and this view eventually prevailed. USAID/Benin’s strategy anticipated that a new bilateral program would incorporate and continue work that BASICS had begun. BASICS II was viewed as reluctant to collaborate in this plan, and USAID suspected that BASICS II wanted to use Benin’s expansion results to help meet the contract objectives. In frustration, the Mission eventually asked BASICS II to close its Benin office. The West Africa Regional Program (WARP) placed field support funds in BASICS II and asked for the procurement of oral rehydration salts packets to complement other aspects of their Family Health International program. These funds were unused for three years because this small activity did not coincide with BASICS II program objectives. USAID/Senegal requested that BASICS procure a large quantity of refrigerators and child weighing scales. BASICS II headquarters demurred, stating that procurement of equipment was not allowed under the contract. The Mission insisted that it had no alternative to BASICS II providing the refrigerators and scales, essential elements written into the USAID–government of Senegal SO agreement. After a significant delay, BASICS II relented and did purchase the scales, while USAID/Senegal purchased the refrigerators itself. USAID/Senegal has recently asked BASICS II to expand its field operations into a new province, reflecting a new U.S. embassy and USAID country strategy. BASICS II headquarters was perceived to be reluctant to comply because this transfer of its limited Senegal resources in the fourth year of the program would reduce the likelihood of achieving results in existing regions where BASICS II was working and would not allow time to achieve clear results in the new province before the BASICS II contract would end in June 2004. USAID/Senegal was surprised because its SO period continues through 2006 and the Mission’s timeframe for demonstrating impact is the latter date. USAID’s Africa Bureau (AFR) also had a different view of how BASICS II should support its program interests, although AFR’s health chief had participated in the BASICS design. AFR provided approximately $1.7 million a year for the establishment of a regional BASICS office (WARO) in Dakar. The AFR health strategy focuses on capacity building rather than expansion of impact and AFR insisted that WARO focus 56 part of its time and energy on capacity-building activities. Reflecting these tensions, midway through the contract GH suggested that WARO be funded through a TASC work order. AFR and GH both eventually decided that splitting WARO from BASICS would have more costs than benefits. Although BASICS II had complied and has tried to meld the two objectives, BASICS II complains that $5 million of its contract ceiling is not available to meet expansion objectives. These examples clearly illustrate that most Mission and regional health officers want to use BASICS II to meet Mission and regional objectives, and not necessarily to meet GH and contract objectives. A number of these USAID health officers were surprised to learn from the assessment team that the objectives and scope of BASICS II were significantly different from BASICS I and that BASICS II was a performance-based contract, rather that a more flexible fee-for-service contract. At the time of contract initiation, the objectives of the new BASICS II contract and its differences from BASICS I were clearly announced through USAID–wide announcements. However, these objectives are either not well understood or not accepted by many Mission and regional officers who continue to believe that GH programs should serve their needs and not vice versa. Conclusions ! The major strategic differences between AFR and GH have not been resolved. ! There have been major tensions between many Mission programs and BASICS II throughout the program, reflecting lack of agreement on whether Mission objectives or GH objectives have primacy in Mission settings. Most Missions have been single-minded in demanding that BASICS II be sufficiently flexible to fit into Mission country strategies. ! BASICS II headquarters, delegated the responsibility of negotiating country and regional programs by GH, has found it difficult to respond to the various objectives of USAID global, regional, and Mission health programs. Impact of Several Funding Sources and Earmarks Congressional earmarks are a reality for all USAID officers and USAID CAs. BASICS II has absorbed funding from earmarks or sub-earmarks for polio, malaria, and micronutrients. BASICS II has also absorbed funding from four different Washington￾managed funding sources (GH, AFR, LAC, and ANE) and from 16 different USAID Missions through the field support mechanism. In most cases, BASICS II has absorbed these funds without major problems. Malaria and micronutrient funds have been used to finance activities that were already planned as part of a BASICS country program or to finance new activities that are compatible with long￾term country program objectives. 57 Earmarked funds for polio eradication have been much more difficult to absorb. This is especially true in Nigeria, where $11 million for polio was added to the BASICS program during the third year. Several significant problems resulted: ! The polio eradication effort that BASICS has been asked to support in Nigeria is much broader geographically than three BASICS II program regions. ! The polio effort is collocated with the BASICS program in only one of the three BASICS regions. ! The polio eradication program is delivered through National Immunization Days, while the conventional BASICS program has emphasized routine immunizations at health facilities (this problem also occurred in Ghana). ! The polio funds replaced some of the nonpolio funding BASICS had anticipated for the Nigeria program. ! The polio funds have replaced other funds within the BASICS II contract ceiling. ! The massive level of polio funding provided (45 percent of the total BASICS program) has required that BASICS country staff dedicate much of its time to the polio program even though polio does not contribute meeting contractwide results. Recognizing these incompatibilities, the USAID CTO, BASICS II, and the Nigeria Mission attempted to channel these polio funds into a TASC work order, rather than through BASICS. This effort failed, unfortunately, due to the high cost of establishing a new polio-only TASC CA in Nigeria. Conclusions ! BASICS has performed excellently in absorbing various funding sources and most earmarks into the long-term program agenda. ! The addition of malaria and micronutrient funds has not led to significant program distortions because of their flexibility in use and their compatibility of program objectives with the BASICS II program strategy. ! The addition of polio funds led to significant distortions and inefficiencies in Nigeria. Use of Public Health Indicators BASICS II is a performance-based contract that strives to have an impact on child health by expanding coverage of proven interventions. Impact indicators are those that measure changes in mortality, morbidity, and health. Coverage indicators (also known as effect or outcome indicators) are those that measure behavioral change and/or utilization of child survival services. Both of these have been called public health 58 indicators, which is confusing. The contract does not require achievement of any impact indicators, so that public health indicators should be used to mean coverage. It is also important to distinguish between the indicator and its target. An indicator is an indirect measure of a variable (such as the number of children immunized). A target specifies the quantified achievement desired (such as 15 children immunized, a 15 percent increase in children immunized, increased immunization in 15 countries). Most USAID field Missions believe that this may serve as an incentive, but that many of the more potent factors influencing success are outside the manageable interest of BASICS II or any one implementing agency. The indicators in the contract are fine but the aggregate or global targets are not and they should be negotiated with each country program. In the future, USAID should specify the indicators (reduce neonatal mortality, increase ARI treatment) but leave the targets to be negotiated in each country with the Mission and MOH. Use of a Performance-Based Contract GH decided to use a performance-based contract for BASICS II after the failure of the RFA (cooperative agreement) bid process. This was viewed as a positive alternative to the level-of-effort contract used in BASICS I, which the CTO team believed provided very few opportunities to influence grantee decisions. There had been only one bid for the RFA and only one bid (from the same partnership) was anticipated for the request for proposal. Therefore, changing the procurement mechanism was one of the very few options USAID had to provide incentives for an improved proposal during the second bid. Performance-based contracting was relatively new to USAID but seemed to fit well with USAID’s new results orientation. BASICS II was the first GH flagship program to use a performance-based contract. Key elements of a performance-based contract are: ! a fee award structure (in this case, heavily weighted to the developmental impact of services and products); ! an annual performance evaluation board review; and ! an annual fee award at the end of the PEB review. By all accounts, this process was taken very seriously by BASICS and USAID. The process was very time consuming for both parties and quite expensive for BASICS. Several key concerns are evident from the use of a performance-based contract to implement this complex and groundbreaking global program: Measuring Performance: The timeframe for measuring impact for a five-year program is short because the time between the baseline survey and the end-of-project survey is 59 probably less than four full years. Measuring annual changes for many performance indicators is even more difficult. ! Attribution: Attribution is especially difficult to measure for a contract that uses leveraging for an intervention, but has a significant weight in the PEB judgment of how performance targets are being met. ! Setting and Resetting Performance Standards: Eventually both USAID and BASICS agreed that the original contract performance standards were not attainable within the funding and timeframe of the contract. Agreeing on new targets and amending the contract have proven to be very difficult. ! Cost: Measuring and reporting information needed for the annual performance review process absorbed enormous contractor time and senior management attention. • Monetary Incentive: The fee award was not significant as an incentive. The relatively modest award was not consequential when divided among the three partner subcontractors and the six secondary subcontractors. (The award was not passed on to BASICS staff.) • Ceiling: Raising the financial ceiling of any USAID contract is often difficult with the number of protests from competitors rising. Field support buy-ins for BASICS were significantly higher than anticipated in the $79 million contract. However, the only rationale accepted by the USAID contracts office for eventually raising the contract ceiling in FY 2003 was to absorb unanticipated and earmarked polio and malaria funding. Conclusions ! The award fee was not an incentive for performance. However, the desire to achieve contract targets was an incentive to staff. ! There were no serious problems in the management of the PEB reviews (continuity of objectives, GH use of surveys to gauge Mission views), but the PEB process was time consuming and quite costly to BASICS. ! The contract fee structure (with relatively low weighting for flagship functions and high weighting for successful field programs) clearly influenced partnership decisions on the allocation of resources. ! The performance contract probably increased the contentiousness (of both the CTO team and later the USAID contracts office) of adjusting program objectives. 60 ! Most Missions surveyed do not believe that the use of performance contracting for BASICS II has provided any special incentives to the contractor. CTO Management Structure and Process The USAID CTO team for BASICS was reduced over the course of the contract. The original team consisted of five GH staff but was reduced to three after the initial year of the contract. As the contract moves into its final stages, the formal USAID team has been reduced to one individual, who spends approximately one third of her time on BASICS II. This CTO, who is an experienced manager but not a child health specialist, has been successful in involving other GH staff in technical reviews and problem-solving when needed. Her management style, described as “direct but helpful” and “solution oriented” has been effective. Until the recent reduction, the membership of the CTO team was unusually consistent (the same two individuals, including the CTO throughout BASICS I and II). However, the GH CTO team did not incorporate members from other units of USAID, unlike USAID Mission CTO teams that often include nontechnical staff (program office or controller staff). Regional bureaus, such as AFR and LAC, had their own mini–CTOs, who saw themselves as responsible for monitoring how BASICS utilized regional bureau funds and who contacted BASICS personnel to resolve any issues concerning these funds. BASICS responds globally to multiple CTOs: USAID Mission officers responsible for monitoring BASICS field activities within the context of the Mission’s health program, USAID regional bureau officers responsible for monitoring BASICS regionally funded actions, and the official GH CTO team. The lines of authority between BASICS II headquarters and the GH CTO team were reported to be generally clear and consistent. BASICS II personnel knew who they should contact on strategic program and technical issues. Since the reduction of the CTO team, BASICS II technical people continue to contact their former USAID technical counterparts and the CTO also consults these same technical staff when needed. The USAID design team reduced the number of routine contract approvals to the minimum required by USAID’s overall contract regulations (e.g., international travel, key personnel, and annual work plans). These routine approvals were provided in a timely fashion. No quarterly reports were required due to the burden of the annual performance review. The USAID performance review board was also remarkably consistent over the first three years of BASICS II. This consistency has been important in ensuring continuity in USAID expectations and in USAID requirements during the first three reviews. The PEB’s reviews of BASICS’s lengthy self-assessment reports were extremely thorough; the performance board’s written comments to BASICS II were also lengthy and detailed. However, the style and content of the CTO team’s interaction with BASICS II, especially for problem-solving, contributed to strained relationships between the CTO team and 61 BASICS II leadership. Shortly after the contract was signed, BASICS II asked to renegotiate what they termed unrealistic performance targets. USAID refused to negotiate until BASICS II could clearly show through country strategic plans that these standards were unreachable. BASICS II’s first annual work plan was rejected. The CTO team put the onus for meeting the requirements for this new and innovative program on the contractor. BASICS reports that the CTO team did little to provide detailed guidance on how to define contract terminology (“scale up” versus “working at scale”) or how to improve their plans. “USAID could have been more supportive,” especially in providing written direction, according to former senior leadership. When BASICS II asked USAID to pay for training of technical staff to learn new skills, such as programming and financial management, USAID initially refused, stating that providing adequately trained staff was the full responsibility of the organization that accepted the contract. Conclusions ! CTO approvals for operational requirements were well structured to be limited in scope and decisions were provided in a timely fashion. ! The CTO team did not effectively incorporate wider USAID regional and Mission interests; BASICS II found it difficult to broker agreements between competing USAID interests. ! The CTO team was clearly disappointed at contractor acceptance of the contract and perceived an immediate retreat from program targets. This contributed to a less than helpful problem-solving attitude from the team during early contract planning. ! Inability to agree on a program approach significantly delayed contract activities and had severe repercussions on BASICS II staff morale. 62 V. SUMMARY OF RECOMMENDATIONS BASICS II ! Finalize its GTL agenda and plan a response to concerns about the appropriate balance of resources in achieving GTL. Progress during the remainder of the project should be measured in accordance with this mutually agreed plan. ! Focus on achieving the desired balance between its GTL efforts and obtaining in-country investments by other partners. Achieving this balance will help both to expand child survival efforts to achieve impact and to assist in achieving sustainability. ! Reassess the scope, purpose, cost, and utility of SET. Develop a more focused role for SET. ! Assess the demand for and use of SET materials, including the effect of having such materials available only in English and French. ! Consider separating SET functions into two categories: • country programs, which would emphasize replication of best practices within and outside the country; and • global leadership, which would emphasize the dissemination of lessons learned for technical and policy applications. ! Ensure that all possible resources (host country governments, other partners) are directed toward achieving sustainability in the areas in which BASICS II has been active in each country. BASICS II has only one more full planning cycle and should do its best to overcome the shortcomings in the planning process so as to leave a viable planning process in place for the successor project. ! Continue the integration of c–IMCI and other community approaches in existing country programs, with emphasis in implementing an exit strategy to help ensure program sustainability when BASICS II support ends. ! Work with partners in BASICS II countries to increase their support of c– IMCI. USAID ! Participating Missions need to be informed again about BASICS II contract requirements, especially when they conflict with Mission strategies. ! USAID should replace the performance indicator (4b) in the upcoming amendment to focus more on financial management. APPENDICES A. SCOPE OF WORK B. PERSONS CONTACTED C. BACKGROUND DOCUMENTS APPENDIX A SCOPE OF WORK (from USAID) BASICS II ASSESSMENT ACTIVITY Scope of Work IDENTIFICATION OF THE TASK One impact of the performance-based design of the BASICS II contract is that extensive data on contract results has been collected on a continuous basis since the contract's beginning in 1999. Because this data will be available to evaluation team members, this assessment offers a unique opportunity to use the data and other methodologies to go beyond a traditional evaluation and examine additional questions about the efficacy of a "flagship" project mechanism in meeting USAID's goals in child health. Thus, the results of this evaluation will not only focus on what BASICS II has achieved, but will contribute to the redesign of the USAID child health portfolio. BACKGROUND The BASICS II Project Despite major gains in child survival in the last 25 years, more than 10 million children die each year before the age of five - often from diseases and conditions that are preventable or easily treated. The U. S. Agency for International Development (USAID) is a leader in the quest to improve child health. Its flagship project is BASICS II, or Basic Support for Institutionalizing Child Survival (1999 - 2004). BASICS II builds on the lessons of its predecessor, BASICS I (1994 - 1999). The global project is charged with achieving the greatest possible country-level impact on major threats to child health and providing technical leadership in policies and programming. BASICS II focuses on four areas that promise the greatest reduction in mortality. These include: • Effective and sustainable child immunization • Perinatal and neonatal health • Nutrition and growth promotion • Integrated approaches to child health The project aims to: integrate programs at the community level; strengthen planning and management at the district level; and, increase program scale and impact. BASICS II is a performance-based contract. Activities under BASICS II are geared to achieving measurable increases in the use of child health and nutrition interventions. The global scale of project efforts to improve child health are gauged against expected levels of change in 14 specific areas defined in the contract. The definition of these areas and contract performance is covered by performance reports and not by this assessment; performance reports will be made available to the team. Bureau of Global Health Flagship Projects "Flagship" projects have been developed by USAID's Bureau for Global Health (BGH) to be major mechanisms in carrying out the bureau's functions within the Agency. These functions include: - Development, evaluation, and introduction of new approaches and interventions aimed at the major causes of infant and child morbidity, mortality, and malnutrition; - Providing global technical leadership, that is, helping to set the analytic, policy, and program agenda for global child survival efforts; - Providing technical assistance and support to USAID field programs, in order to inject state-of-the-art technical content into these programs, improve their effectiveness, and support approaches that yield greatest scale and impact of USAID's investments in its country programs; - Facilitating the transfer of successful program experiences (not just those developed by GH or its cooperating agencies) in ways that make them available to USAID missions and countries where they might be applied effectively. BGH is tasked with carrying out these different functions in ways that best support USAID's overall Agency Goal in child survival and its mandate from Congress, that is, achieving greatest possible child health and nutrition impact at greatest possible scale through the increased use of key child health and nutrition interventions (BGH's Strategic Objective 3, under which the BASICS II Project was designed). Achieving this objective requires different approaches in different circumstances. As stated in the BASICS II contract (scope of work), "In some settings, the most important strategy for increasing use of child health and nutrition interventions will be wider and more effective application of existing approaches. When this is the case, the Flagship will focus its expertise and resources on increasing effective implemention of these proven approaches, with the final evaluation criteria being increase in use of key interventions for improving child health and nutrition. "Under other circumstances, increasing use of these interventions will require development and application of new approaches. If so, these approaches will also be evaluated in terms of their effectiveness in increasing use; if effective, the Flagship will aim for broadened application of these new approaches." To allow Flagship Projects to carry out these functions, BGH and USAID provide these projects with: - BGH "core funds," which can be used for supporting technical experts and their activities at the global level (for example, for interaction with representatives of other international organizations), as well as for supporting strategic activities such as operations research and policy analyses to inform the global technical agenda, or "seeding" field programs to stimulate the implementation or expansion of an effective program approach. - access (through field support funding and MAARDS) to funding from regional and other bureaus, and especially from USAID field missions; - the opportunity to participate directly in USAID country programs implementing child survival interventions; - through these funding and field program opportunities, the opportunity to influence ("lever") broader child survival resources and to develop and evaluate program approaches in the field and globally; - as implementing partners of USAID in its program implementation and technical functions, representatives of flagship projects are given access to other influential organizations (like UNICEF, WHO, the World Bank, and others) and decision￾makers at the country and international levels; by working in multiple countries, flagship projects also the opportunity to learn from different program approaches, and in turn to synthesize and transfer these experiences to other countries and to the global community. BASICS II Partner Organizations BASICS II is a project of the Partnership for Child Health Inc., consisting of The Academy for Educational Development, John Snow International, and Management Sciences for Health. BASICS II sub-contractors are: Emory University, Johns Hopkins University, The Manoff Group, and Program for Appropriate Technology in Health, Save the Children Federation, and TSL. PURPOSE OF THE ASSIGNMENT The purpose of the BASICS II evaluation is to: a. Identify the strengths and weaknesses, successes and shortfalls of BASICS II in the following functions: - Innovation and Operations/Evaluation Research - Achieving global technical leadership - Support to USAID Missions' child survival programs - Strategic experience transfer to different audiences and clients b. Identify the strengths and weaknesses, successes and shortfalls of the project's use of these functions to achieve programming and use of key child health interventions at scale. c. Assess key management functions in relation to performance and achievement of the project in these key function and in achieving outcomes at scale.. These management functions include: planning, allocation of funds, monitoring of implementation and progress toward outcomes, and relationship with corporate partners and other stakeholders; and d. Provide recommendations in the design, process, and organization which could improve the impact and operations effectiveness of BGH's approach to carrying out its key functions in relation to child survival. It is important to note that this assessment is not intended to evaluate the performance based contract mechanism specifically; however, the team will certainly want to consider this mechanism, and the way the contractor reacted to it, among the factors related to the project's performance of the key functions. The team should also consider the other various factors that influence the success of a USAID global project like BASICS II. These can be categorized into three types: internal USAID factors, external global factors, and internal project factors. The first is under the control of USAID but not of the project; the second is not controlled by either USAID or the project. However, both of these can affect how the project succeeds in carrying out its functions, what it can achieve, and how it relates to USAID missions and development partners. The internal project factors, on the other hand, are in the "manageable interest" of the contractor and can be controlled by the project. The evaluation team should consider the role of these factors in how BASICS II has worked in carrying out its key functions, and indirectly, their importance in the design of any follow on effort. Internal USAID factors The way USAID designed BASICS II, the way it manages the project, the availability of funding, the way funds flow, and the strings attached to those funds all affect how a "flagship project" like BASICS II performs. These factors are: - The overall performance contract design - As a performance-based contract, this project requires deliverables that, in theory, are under the control of the contractor. - Technical content of the project - The technical content of a project obviously influences implementation. - SO targets - The choice of public health deliverables that the project is aimed to achieve and the expected level of achievement and the expectations of achieving that level at a predetermined geographic and demographic scale. - Source of funds and the strings attached - BASICS II receives funds from USAID Global Bureau, Africa Bureau, LAC Bureau, ANE Bureau, and USAID missions, in addition to funds earmarked for infectious diseases, malaria funds, and polio. Each source of funding comes with strings attached, regardless of whether the project is performance-based or not. As a project funded by the Global Bureau, ultimately performance is judged by this Bureau; however, other parts of USAID responsible for these earmarked funds care principally that the project works within the specifications of the earmark and meets their needs. This underlying conflict might affect project implementation. - The amount of global funding - Global funds allow the project to do things that are directly in line with the contract. Global funds provide more flexibility and better ability to leverage other resources, policies and programs. - Perception, thinking, professional interests, and management style of CTOs - Until recently, the project was managed within USAID by three CTOs. Their interests, management style, perceptions of "correct" project implementation all have a tremendous amount of influence on how the project unfolds, who it works with, what it focuses on, how project leadership works within USAID, and other aspects of project implementation. Global environmental factors - Global trends and interests - During the life of BASICS II, the global environment has changed and continues to change, particularly in relationship to IMCI, ARI, and peri-/neonatal health (PNN). These trends determine what USAID and other organizations are interested in. These may evolve so they differ from project SOs and direction, which were set several years ago. - Willingness of global partners to collaborate - What BASICS II brings to the table in terms of money and technical resources affects collaboration. The project having global money to "leverage" partner collaboration may facilitate working relationships; on the other hand, having more "tied" funding may result in the project's having less to bring to the table as incentive for collaboration by other groups. Internal BASICS II and Partnership for Child Health, Inc.(PCHC Inc) factors These final factors are fully under the control of PHCH Inc and the leadership and management of the BASICS II project. - Project structure - PCHC Inc. reports that this partnership was formed to provide the best value for money to USAID: MSH, JSI and AED report that they created the PCHC Inc in order to seamlessly bring the core partners' expertise to BASICS (I and II). Under BASICS II, PCHC Inc also has an additional six subcontactors. The structure, function, use of partners' resources, and method of involving partners (including subcontractors) are under the control of PCHC Inc. - Project leadership and management - The project presently has a director and four deputy directors, overseeing technical and administrative staff. The performance of project functions clearly depends substantially on the skills, qualifications and relationship amongst these individuals in positions of leadership. It also depends on their technical and managerial competence, how they organize themselves to work, and how they communicate with each other to focus the project and get the job done. - Project technical staffing - The actual technical work, in headquarters and in the field, is done by staff hired from partner companies or from outside. Performance will be affected by whether PCHC Inc and project leadership found and hired the best people for the job. This in turn is affected by the process of recruitment, as well as by whether this partnership project structure offered staffing advantages over a more traditional prime/sub-contractor arrangement. - Resource allocation and use - Project leadership, under the overall guidance of the PCHC Inc Board, obtains resources from USAID, and allocates those resources across all functions of the project. Performance will be affected by: how resource allocation decisions were made, as well as the balance between technical, administrative, and overhead costs. Equally important is the proportion of resources spent on staff versus those available for direct operational costs, as well as the proportion spent in headquarters and non-country specific activities, versus those spent in the field. Additional resource questions include which resources flowed to what partners; whether the "strings" attached to specific funding sources were complied with, and the effect of this; and whether resource allocation decisions supported effective programming. - Choice of implementation approach - Within the constraints of contract and project structure and of resource availability, the project still has substantial latitude in choosing how it will invest its resources and take advantage of its opportunities to achieve the results specified. The "BASICS II Program Framework," developed in Year 2 of the project to guide these choices toward achievement of greatest possible impact and scale, illustrates the types of implementation choices available to the project. What approaches did the project take, how did they line up with the "Program Framework," and what forces and considerations drove the choice of implementation approaches in the technical areas and country programs? - Monitoring and evaluation - To what extent did the performance results monitoring (PRM) function serve to provide information on achievement of SO results, as well as on intermediate outcome and management indicators of performance of the contract (so that progress toward overall results could be judged, and management action could be taken in a timely manner)? METHODOLOGY OF THE ASSIGNMENT The assessment will include the following: 1. Self-assessments by all partner agencies and surveys of BASICS II country representatives and of USAID field Missions (mission surveys conducted by USAID) 2. Review of relevant documents, including basic project documents, self￾assessments by all partner agencies, internal management assessments and the reviews of selected project elements (SET, O.R., IACH) carried out by the Partners, annual performance reports by the project, annual surveys from field missions where BASICS II was working, and USAID's evaluation and fee award memoranda regarding annual performance; 3. In-depth interviews with USAID/GH and USAID Missions working with BASICS II, BASICS II partners, technical experts in the field; other donors, and BASICS II staff. The team will do site visits and in-depth interviews with the three main BASICS II implementing partners; telephone interviews and a review of self-assessment data will be used to cover the six additional members of the BASICS II partnership. 4. Site visits to at least two BASICS II implementation countries to do direct observation, review country-specific documents, interview Mission, MOH, and implementing partners on the ground. The team should focus on the following questions during these visits: - Did BASICS II focus on appropriate problem areas, and was the project the correct mechanism for the in-country functions the project was asked to play? - What were the unique features of BASICS II that promoted, or inhibited, progress toward achieving goals? - How did BASICS II approach its work in countries - was the "BASICS II Program Framework" approach (the "Mary Taylor framework") or its principles applied in developing the project's approach? What other (project or non-project) factors conditioned the project's choice of approach to the country work? - Was the approach designed to achieve impact at scale? To increase the influence of USAID's investment more broadly in the country? To extract and transfer key experiences to inform other countries' or regional/global experience? - How did the mandate of the BASICS II contract mesh (or not mesh) with specific mission needs? - How effective were the relationships of the project staff (in-country representatives, headquarters and regional project atsff, and consultants) with the USAID mission and with ocunterparts and other organizations working in the country? - What was the apparent contribution, and the scale and impact of that contribution, to the country's child survival efforts and to increased use of key interventions? NOTE: The Team Leader will begin the assignment by assisting in designing the self-assessment and mission survey documents, and finalizing the workplan for the assignment before other team members are on board. The team will have the following sources of information available to them to complete the scope: 1. The USAID RFP for the BASICS II contract and the BASICS II bid and contract 2. BASICS II Self Assessment Reports for Project Years One, Two, and Three 3. USAID performance evaluation ratings of BASICS II for Project Years One, Two, and Three 4. BASICS II Monitoring and Evaluation data 5. BASICS II work plans for Years 1 - 4 6. Relevant BASICS II publications 7. BASICS II self-assessment by Partners and BASICS country representatives for this assignment 8. Results of the USAID Mission survey. TEAM COMPOSITION The five-member team will be made up of: POPTECH Project technical consultants: 1. Team Leader with USAID programming knowledge, understanding of how USAID Missions operate, French language skills 2. Specialist with management skills and Child Survival experience 3. Technical specialist with developing country experience and knowledge of other donors and Child Survival experience Other team members: 4. One GH USAID staff member 5. One Child Health Fellow POPTECH will provide all technical, administrative, logistical, and editing support for completion of the assignment. USAID technical direction will be provided primarily by Elizabeth Fox, the BASICS II CTO (202) 712-5777, EFox@USAID.gov) SCHEDULE AND LOGISTICS The BASICS II evaluation should begin in mid-February, 2003 to enable the evaluation to be completed by June, 2003. USAID GH will make arrangements with the selected countries for the site visits. BASICS II will arrange the logistics in-country for the team. POPTECH will provide space in the POPTECH offices for the team. Non-DC based team members will require hotel reservations, which will be the responsibility of POPTECH. DELIVERABLES USAID expects an evaluation report addressing questions listed under the "Purpose of the Assessment" section of this SOW incorporating both an evaluation of the existing mechanism/functionality (ie - not performance evaluation against indicators) and recommendations to USAID for the redesign of the child health portfolio. The document will be in English and no longer than 30-50 pages, not including appendices and tables. While USAID is interested in having a well presented final document, a key factor in this assignment is timing. USAID expects a final draft by the end of May, and a finalized document by June 2003. APPENDIX B PERSONS CONTACTED PERSONS CONTACTED WASHINGTON, DC U.S. Agency for International Development (USAID) Bureau for Global Health, Office of Health, Infectious Disease and Nutrition Infectious Disease Division Dennis Carroll Mary Ettling Maternal and Child Health Division Al Bartlett Carol Dabbs Elizabeth Fox Richard Greene Murray Trostle Nutrition Division Frances Davidson Sheila Lutjens Bureau for Asia and the Near East, Office of Strategic Planning and Operations Lily Kak Bureau for Africa, Office of Sustainable Development Hope Sukin-Klauber BASICS II Serigne Diegne, Team Leader, Nutrition Technical Focus Area (TFA) Dan Kraushaar, Director Indira Nayaranan, Team Leader, Peri and Neonatal TFA Beth Plowman, Deputy Director, Performance Results Monitoring and Operations, and Evaluation Research Tina Sanghvi, Technical Deputy Director Ian Sliney, Deputy Director, Country and Regional Programs Robert Steinglass, Team Leader, Expanded Programme in Immunization (EPI) TFA Fred White, Deputy Director, Finance, Management, and Administration Partnership for Child Health Care, Inc. Marcia Griffiths, The Manoff Group, Inc. Carrie Hessler-Radalet, John Snow, Inc. Joel Lamstein, John Snow, Inc. Jack LeSar, Academy for Educational Development (AED) Mireille Mather, Team Leader, Strategic Experience Transfer (SET) Steve Moslely, AED Ron O’Connor, Management Sciences for Health (MSH) David Oot, Save the Children Christine Whalen, MSH SENEGAL Ministry of Health Eva Marie Coll-Seck, Hygiene and Prevention Moussa Diakhate, Director of Health Guelaye Sall, Director, Department of Food and Nutrition USAID/Senegal Felix Awantang Brad Barker Matar Camara BASICS II/Senegal Aboubacry Thiam, Country Team Leader Biram Ndiaye, Coordinator, Programme de Renforcement de la Nutrition (PRN) BASICS II/Senegal staff Decentralization and Initiatives of Community Health (DISC) Farba Lamine Sall Plan International Winnie Tay, Representative United Nations Children’s Fund (UNICEF) Ian Hopwood, Representative Fatoumata Diawara, Program Officer, Health World Health Organization (WHO) Dr. Yankalbé, Representative World Vision Banda Ndiaye, Representative Kebemer Ibrahima Wone, Prefect, Department of Kebemer Balla Mbacke Mboup, District Medical Officer Thiolom Fall Wally Ndiaye, Health Post Nurse HONDURAS USAID/Honduras John Rogosch, Office Director, Human Resources Development (HRD) Meri Sinnitt, Chief, Health, Population and Nutrition Division Pan-American Health Organization (PAHO) Luis Amendola BASICS II/Honduras Victorio Vivas de Alvarado, Country Team Leader Partners for Health Reformplus (PHRplus) Gustavo Corrales, Chief of Party Comayagua Hospital Arturo Gutierrez UNICEF Luis Robert Escoto Hector Espinal Canadian Red Cross Ian Stein, Director, Community Public Health Catholic Relief Services (CRS) Judith Galindo Technical Resource Contacts Jean-Marie Okwo Bele, UNICEF Yves Bergevin , UNICEF Robert Black, Johns Hopkins University Mariam Claeson, World Bank Nils Dalair e, Global Health Council Laura Hoeneke, USAID/Benin (former) Kate Jones, USAID/GH (former) Julie Klement, Director, Monitoring, Evaluation and Design Support (MEDS) Project Karen Leban, Communities Responding to the HIV/AIDS Epidemic (CORE) Richard Moore, John Snow, Inc. Joy Riggs-Perla, USAID/GH (former) Sally Stansfield, Bill and Melinda Gates Foundation Ron Waldman, Columbia University APPENDIX C BACKGROUND DOCUMENTS BACKGROUND DOCUMENTS Ag Bendech, M., A. Acakpo, V. Aguayo, S. Baker, S. Mbaye Diène, L. Lathen and A. Ouédreogo. Les Pratiques Prometteuses et les Leçons Apprises dans le Lutte Contre la Carence en Vitamine A dans les Pays de l’Afrique Subsaharienne. Arlington, VA: Basic Support for Institutionalizing Child Survival (BASICS II) Project for the U.S. Agency for International Development (USAID). 2000. The Basic Support for Institutionalizing Child Survival (BASICS II) Project. “2002– 2004 Set Strategy.” ___________. “Achieving Impact on Child Health at Scale with the Support of the BASICS II Project.” Draft paper for discussion. ___________. BASICS II Presentation to the Assessment Team. February 20, 2003. ___________. “BASICS II Global and Regional Technical Leadership (GRTL) Agenda and Plan.” December 27, 2002. (Draft). ___________. BASICS II PEB Award Fee Memoranda. 2000–02. ___________. “BASICS II Response to Performance Evaluation Board.” 2002. ___________. BASICS II Self-Assessment Reports. 2000–02. ___________. “BASICS II Self-Assessment Report for Project Year 3, June 15, 2001– June 14, 2002.” ___________. “BASICS II Global and Regional Technical Leadership Agenda and Plan.” December 27, 2002. ___________. BASICS II Prime Contract. ___________. BASICS II Operational Strategy. March 2002. ___________. BASICS II Country Program Briefs for Africa, Latin America and the Caribbean, and Asia. ___________. BASICS II Key Contacts List. 2003. ___________. “BASICS II Experience with Performance-Based Contracting.” ___________. “BASICS II FY 03 Work Plan (October 1, 2002–September 30, 2003).” ___________. “BASICS II Strategic Plan, January 2000.” ___________. “BASICS Leveraging Narratives in 12 Technical Areas.” ___________. “BASICS II Operational Strategy, 2002.” ___________. “BASICS II Documentation and Strategic Experience Transfer: Providing Essential Child Health Services at Scale in Diverse Settings.” April 23, 2002. ___________. “BASICS Priority-Setting Exercise: Global Technical Leadership Priorities.” ___________. BASICS II Work Plans FYs 2000–03, including country programs and technical focus areas. ___________. BASICS/Ghana Performance Monitoring and Evaluation. (Draft) ___________. BASICS/Nigeria Performance Monitoring and Evaluation. (Final) ___________. “Child Health and New Developments in the Health Sector.” PowerPoint presentation for a meeting on November 19, 2001. November 16, 2001. ___________. “Community Integrated Management of Childhood Illness Monitoring Indicators Report.” ___________. Country Program Designs. ___________. Country Program Frameworks. ___________. “Consultant SOW: Achieving Scale.” ___________. Country SO Indicator Tables (Benin, Bolivia, Ecuador, El Salvador, Guinea, Guinea, Honduras, Nicaragua, Nigeria, Uganda). ___________. El Salvador Performance Plan. ___________. “Five-Year Progression of Results in Technical Focus Areas.” ___________. Flagship Activity Contract HRN–C–00–9900007–00. ___________. Flagship Request for Proposal M/OP–99–208. ___________. Global Alliance for Vaccines and Immunizations (GAVI) Acknowledgment Letter. February 2003. ___________. Healthy Newborn Partnership Information Sheet. May 2002. ___________. Honduras Work Plans, FY 2000–03. ___________. Implementation Benchmarks Review FY 2002. ___________. “Improving Neonatal Health in South-East Asia Region.” Report prepared for a regional consultation, New Delhi, India, April 1–5, 2002, World Health Organization (WHO). ___________. List of documents produced by BASICS II/West Africa Regional Office (WARO) . February 7, 2003. (in French) ___________. Materials for Ghana Integrated Child Health Survey (ICHS), 2001. ___________. Memorandum on Progress since December 19, 2001, Partners Meeting. February 20, 2002. ___________. Memorandum of Understanding between BASICS II Project of the Partnership for Child Health, Inc., and Rational Pharmaceutical Management Plus Program. Management Sciences for Health, Inc. December 6, 2002. ___________. Memorandum of Understanding between West African Health Organization and BASICS II. July 2002. ___________. “Memorandum to Partners in the BASICS II Project: Findings of Partners’ Panel on Operations and Evaluation Research.” February 15, 2002. ___________. Minutes of BASICS II Partners Meeting, December 19, 2001. ___________. Minutes of BASICS II Partners Meeting Memorandum, January 3, 2002. ___________. “Neonatal Health in South-East Asia.” ___________. “Nicaragua Trip Report, July 2002.” ___________. “Performance-Based Programming for Child Survival.” PowerPoint presentation to the Bureau for Global Health. ___________. Performance Results Monitoring Plan. ___________. PNN Bibliography. September 26, 2002. (Draft) ___________. PowerPoint presentation prepared for the Senegal Neonatal Health Program on Study Design. Technical Exchange Group. March 19, 2002. ___________. “Proceedings: Workshop on Vitamin A Supplementation in Africa.” Paper prepared for vitamin A supplementation on Goree Island, Senegal. June 4–7, 2001. ___________. “Program Monitoring and Evaluation for Growth Monitoring and Promotion Technical Exchange Group.” Agenda for meeting held at BASICS II headquarters, November 29–30, 2001. ___________. “Reaching Communities for Child Health and Nutrition: A Framework for Household and Community IMCI.” April 2001. ___________. “Report of Annual Performance Review and Award Fee Recommendations, BASICS II Project (936–3096.01).” July 25, 2000. ___________. “Report of Annual Performance Review and Award Fee Recommendations, BASICS II Project (936–3096.01).” August 6, 2001. ___________. SET Evaluation Team Documents. ___________. Senegal Work Plans FY 2000–03. ___________. “Strategic Objectives SO Tables.” ___________. Strategic Plans for Honduras, Years 1–4. ___________. Strategic Plans for Senegal, Years 1–4. ___________. Summary of Household Surveys in BASICS. ___________. Summary of Recommendations and Proposed Actions from the BASICS II Partners’ Meeting. December 22, 2001. ___________. Technical Proposal. ___________. “The Evolution of BASICS II.” ___________. “TOC Global and Regional Technical Leadership and Leveraging.” February 7, 2003. ___________. Trends in SO Level Indicators (Mali, DR Congo, Ghana, Senegal). ___________. “Update on BASICS II Global and Regional Technical Leadership Results.” February 22, 2003. ___________. West Africa Regional Office Materials. Baume, Carol. A Guide to Research on Care-seeking for Childhood Malaria. Arlington, VA: Support for Analysis and Research in Africa (SARA) Project and BASICS II, for USAID. April 2002. Bhattacharyya, K., P. Winch, K. LeBan, and M. Tien. Community Health Worker Incentives and Disincentives: How They Affect Motivation, Retention, and Sustainability. Arlington, VA: BASICS II for USAID. October 2001. de Maza, I.C., M. de Oliva, S. L. Huffman, R. S. Magalhase, M. Stone-Jimenez, and B. Burkhalther. Sustainability of a Community-Based Mother-to-Mother Support Project in the Peri-Urban Areas of Guatemala City: A La Leche League Study. Arlington, VA: BASICS II for La Leche League International and USAID. LaForce, F.M., R. Steinglass, M. Griffiths, I. Narayanan, T. Sanghvi, R. Sogunro, P. Ickx, and K. Cervantes. A Better Future for Children: Progress Toward World Summit Goals for Health and Nutrition. Arlington, VA: BASICS II for USAID. June 2001. Narayanan, I. PowerPoint presentation prepared for Senegal Neonatal Health Implementation Program. March 2002. Nelson, Dan, and Lora Shimp. The Immunization Inter-Agency Coordination Committee Model: Examples from DR Congo. BASICS II Project. December 2002. Saade, Camille, M. Bateman, and D. Bendahmane. The Story of a Successful Public￾Private Partnership in Central America: Handwashing for Diarrheal Disease Prevention. Arlington, VA: BASICS II, the Environmental Health Project, UNICEF, USAID, and the World Bank. September 2001. Samba, S.N., N. S. Maty, M. Ntiru, and S. M. Diène. Best Practices and Lessons Learned for Sustainable Community Nutrition Programming. June 18, 2001. __________. Pour un Programme de Nutrition Communautaire Durable: Pratiques Prometteuses et Leçons Tirées de L’expérience. June 18, 2001. Sanghvi, Tina. “The World Health Organization Global Consultation on Complementary Feeding. Department of Child and Adolescent Health and Development. Department of Nutrition for Health and Development.” Report on meeting on complementary feeding in Geneva. December 10–13, 2001. Sanghvi, Tina and Mary Taylor. “BASICS II Documentation and Strategic Experience Transfer.” April 23, 2002. Sanghvi, Tina, et al. Essentielles en Nutrition: Guide Pour les Responsables de Activités de Santé. Arlington, VA: BASICS II for UNICEF and the World Health Organization (WHO). 1999. __________. Nutrition Essentials: A Guide for Health Managers. Arlington, VA: BASICS II for UNICEF and WHO. 1999. Schumacher, R., E. Swedberg, M.O. Diallo, D.R. Keita, H.D. Kalter, and O. Pasha. Mortality Study in Guinea: Investigating the Causes of Death in Children Under 5. Save the Children, Inc., and BASICS II. 2002. Tain, Frances and Bendahmane, D. The Story of a Successful Public–Private Partnership in Central America: Handwashing for Diarrheal Disease Prevention. 2001. UNICEF, WHO, USAID, and BASICS II. Communication Handbook for Polio Eradication and Routine EPI. November 2000. Van Roekel, Karen. “Technical Exchange Group: AIN Program and Evaluation Overview.” PowerPoint presentation for meetings August 27–28, 2002. Winch, Peter, K. LeBan, B. Kusha, et al. “Reaching Communities for Child Health and Nutrition: A Framework for Household and Community IMCI.” Paper prepared for the workshop “Reaching Communities for Child Health: Advancing PVO/NGO Technical Capacity and Leadership for Household and Community Integrated Management of Childhood Illness (HH/C IMCI),” Baltimore, MD, January 17–19, 2001. The Child Survival Technical Support (CSTS) Project. April 2001. POPULATION TECHNICAL ASSISTANCE PROJECT 1101 Vermont Ave., NW Suite 900 Washington, DC 20005 Phone: (202) 898-9040 Fax: (202) 898-9057 www.poptechproject.com