ASSESSMENT OF THE PARTNERS FOR HEALTH REFORMplus (PHRplus) PROJECT Jaime Arias Tina Cleland Peter Connell August 2004 Submitted by: LTG Associates, Inc. Social & Scientific Systems, Inc. Submitted to: The United States Agency for International Development Under USAID Contract No. HRN–C–00–00–00007–00 This document is available in printed or online versions (POPTECH Publication Number 2004– 190–030). To review and/or obtain a document online, see the POPTECH web site at www.poptechproject.com. Documents are also available through the Development Experience Clearinghouse (www.dec.org). Printed copies and additional information about this and other POPTECH publications may be obtained from The Population Technical Assistance Project 1101 Vermont Avenue, NW, Suite 900 Washington, DC 20005 Telephone: (202) 898-9040 Fax: (202) 898-9057 admin@poptechproject.com Assessment of the Partners for Health Reformplus (PHRplus) Project was made possible through support provided by the United States Agency for International Development (USAID) under the terms of Contract Number HRN–C–00–00–00007–00, POPTECH Assignment Number 2004–190. This report was submitted to USAID in February 2005. The opinions expressed herein are those of the authors and do not necessarily reflect the views of USAID. ACKNOWLEDGMENTS The team would like to thank the entire staff at PHRplus in Bethesda, MD, for preparing many very useful and informative briefing materials as part of this assessment and for spending so much time with the team. The considerable efforts on logistics for the field trips were much appreciated. In that regard, appreciation is extended to staff at PHRplus in Albania and Peru and to staff of the National Institute for Medical Research in Tanzania for hosting the field trips and resolving all the usual logistical challenges. The team is mindful of the enormous effort at the beginning stages by POPTECH and USAID/Washington staff in conceptualizing and organizing the assessment and would like to express its gratitude for the smooth progress that resulted from all that hard work. ACRONYMS AND ABBREVIATIONS ATC AIDSTreatCost software BCC Behavior change communication CPG Clinical practice guideline CRHCS Commonwealth Regional Health Community Secretariat CTO Cognizant technical officer DFID Department for International Development (United Kingdom) DHS Demographic and Health Survey FY Fiscal year GAVI Global Alliance for Vaccines and Immunization Global Fund Global Fund To Fight AIDS, Tuberculosis, and Malaria GH Bureau for Global Health (USAID) GH/HIDN Bureau for Global Health, Office of Health, Infectious Diseases, and Nutrition GH/OHA Bureau for Global Health, Office of HIV/AIDS GH/PRH Bureau for Global Health, Office of Population and Reproductive Health HCD Human Capacity Development (USAID–funded project) HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome HPSS Health Policy and Systems Strengthening (USAID Results Package) IDSR Integrated disease surveillance and response IMCI Integrated management of childhood illness IR Intermediate Result M&L Management and Leadership Program (USAID funded) MH Maternal health MHO Mutual health organization MIS Management information system MOH Ministry of Health NHA National health accounts NIMR National Institute for Medical Research (Tanzania) PHN Population, health, and nutrition PHRplus Partners for Health Reformplus (USAID–funded project) POPTECH Population Technical Assistance Project (USAID funded) QAWD Quality Assurance and Workforce Development (USAID–funded project) REDSO Regional Economic Development Office for Eastern and Southern Africa RH Reproductive health RPM Plus Rational Pharmaceutical Management Plus (USAID–funded program) SCM Supply Chain Management (USAID project) SIS Integrated Insurance System (Peru) SO Strategic Objective T Sh Tanzania shilling WHO World Health Organization WHO/AFRO WHO Africa region WHO/EURO WHO Europe region CONTENTS Page EXECUTIVE SUMMARY .............................................................................................i I. BACKGROUND AND OBJECTIVES ...................................................................1 II. PERFORMANCE ASSESSMENT .........................................................................3 Performance Against Intermediate Results.................................................................3 IR 1: Appropriate Health Sector Reforms Are Implemented ...............................3 IR 4: Health Financing Is Increased and More Effectively Used .........................5 IR 5: Health Information Is Available and Appropriately Used...........................7 IR 2: Health Workers Deliver Quality Responsive Services................................8 IR 3: Commodities Are Available and Appropriately Used.................................9 Performance Against GH Strategic Objectives...........................................................9 Activities Undertaken for the USAID SO Teams.................................................9 SO Clients’ Satisfaction With PHRplus Performance ........................................12 Performance on Crosscutting Tasks..........................................................................13 Technical Leadership and Responsiveness.........................................................14 Training and Capacity Development ..................................................................15 Health Systems Research....................................................................................16 Monitoring and Evaluation and Performance Tracking......................................17 Performance on Project Management and Planning.................................................19 Internal Organization and Management..............................................................19 Market Positioning..............................................................................................21 Managing Competing Priorities..........................................................................21 Adapting to Change ............................................................................................22 USAID Oversight and Support...........................................................................23 III.FUTURE PROSPECTS FOR HEALTH SYSTEMS STRENGTHENING......25 Conclusions to Date and Lessons Learned ...............................................................25 Portfolio and Positioning ....................................................................................25 Results and Impact..............................................................................................25 Strategic Thinking...............................................................................................26 Alternative Approaches to Health Systems Strengthening.......................................28 Future Directions for Health Systems Strengthening in USAID..............................31 A Clear Market Definition..................................................................................32 Strengthening the Process for Identifying Country-Level Needs.......................33 Controlling the Proliferation of Health Systems Strengthening Initiatives........33 TABLE PHRplus Staff Complement, May 2004 ...........................................................................1 FIGURES 1. From Health Sector Reform to Improved Health Status...........................................29 2. Paths to Improved Health Status...............................................................................30 3. USAID’s Current Organizational Approach to Health Systems Strengthening .......31 APPENDICES A. Scope of Work B. Persons Contacted C. USAID Strategic Framework and Organizational Context for the Assessment D. Summaries of the Field Visits E. The Product Development Ladder F. A Systems-Oriented Bureau for Global Health G. References EXECUTIVE SUMMARY BACKGROUND AND OBJECTIVES The Partners for Health Reformplus (PHRplus) project is a five-year cost plus award fee contract that began in October 2000. The project was funded through the U.S. Agency for International Development (USAID) Bureau for Global Health, Office of Health, Infectious Diseases, and Nutrition (GH/HIDN) and was designed to be the bureau’s flagship health systems strengthening initiative, responding to the five Intermediate Results (IRs) in GH/HIDN’s Health Policy and Systems Strengthening Results Package. PHRplus is being implemented by Abt Associates Inc. in collaboration with eight partners. The project draws on a pool of about 150 staff (124 full-time equivalent), split approximately 40:60 between the United States and overseas operations in 30 countries. The ceiling budget is $98 million (including potential award fee); obligated funding as of June 2004 was $62.3 million and expenditures were $54.8 million. Obligated core funding through fiscal year (FY) 2004 was $20.3 million, of which $10.6 million was allocated to common agenda activities. GH/HIDN requested that the Population Technical Assistance Project (POPTECH) provide a three-person team to conduct an assessment of PHRplus, beginning in July 2004, with the following four objectives: ƒ assess PHRplus’s progress in meeting its objectives, ƒ identify lessons learned to date, ƒ document health system strengthening approaches potentially useful to USAID, and ƒ recommend future directions for USAID’s health system strengthening activities. The team gathered information in July and August 2004, including visits to three PHRplus project countries: Albania, Peru, and Tanzania. This report summarizes the team’s conclusions and recommendations (see appendix A for the scope of work). PHRplus PERFORMANCE ASSESSMENT Performance Against Intermediate Results i PHRplus has generally performed well with respect to each of the IRs. IR 1 (health sector reform) was expected to dominate and the project has worked on decentralization and its consequences, hospital strengthening, and central health policy design and support in many countries. The focus has been mostly on implementation issues; PHRplus and USAID itself are rarely involved in high-level policy formation. Work on IR 4 (health financing) has probably been the single largest activity to date; this remains the area where the project’s activities and reputation are strongest. IR 5 work (health information) has focused on national health accounts and infectious disease surveillance; this constitutes a relatively narrow focus that does not provide an information platform for integration across the bureau’s Strategic Objectives (SOs). Work on both IRs 2 and 3 (quality and commodities) has been modest and only in the context of health sector reform, as expected, since both are the focus of other USAID projects. Performance Against the Bureau’s Strategic Objectives The Bureau for Global Health’s Strategic Objective 1 team (family planning and reproductive health) represents a satisfied and growing client for PHRplus’s core-funded work; the focus has been on contraceptive security. The SO 2 team (maternal health) is much smaller but also well satisfied to date with the work on how insurance schemes affect the use of skilled birth attendants. The SO 3 team (child health) has been the largest provider of common agenda core money for the project; it is largely pleased with the way individual subprojects have been handled. The SO 4 team (HIV/AIDS) has provided modest funding to date, and recent work on costing (using the project’s AIDSTreatCost software) in particular has been well done. The SO 5 team (infectious diseases) has provided the largest source of all core funding to PHRplus and has made the broadest use of the project’s services, mostly on infectious disease surveillance. The SO clients collectively recognize that PHRplus has a range of strong products and offers technical depth in its areas of expertise. There are some concerns about timeliness of delivery and that the project is more expensive to use than most other centrally contracted projects. There is also concern that the priorities for use of common agenda core funding are insufficiently transparent; generally, the project is seen as being too remote from its clients. In addition, there is probably insufficient understanding of the project’s full scope, making it somewhat esoteric and unfocused to some clients. Performance Against Crosscutting Tasks PHRplus has demonstrated strong technical leadership (tasks 1 and 6) in health systems work through its range of products, research, published materials, and web site. It has amassed enormous intellectual capital, which redounds to USAID’s credit, both internally and externally. The range of series publications is probably a little wide in relation to the volume of original material and tends to diffuse further the project’s image. Expenditures on dissemination, about 14 percent of core-funded expenditures in the third year, seem reasonable. Extensive and good work is done on training and other capacity building (task 5) at the country level; use of local personnel and institutions makes this effort generally very sustainable. However, there is no clear coordinating strategy on these topics at the center. Originally, applied research and monitoring and evaluation (tasks 2 and 4) were expected to enhance significantly the evidence base underpinning systems strengthening work. Scaled back early in the project due to funding constraints and lack of interest from Missions in participating in field research, both activities assumed a secondary role in PHRplus’s work, consistently operated within a narrow scope, and failed to use their allocated budget fully. Performance on Project Management and Planning ii The internal management of the project generally has been good. Overseas marketing and client relationships appear strong, although additional work is needed to improve communication and mutual understanding with SO clients in Washington. Product development has resulted in a wide range of tools, consulting products, and methodologies. There is a need for more universally applicable and relatively mechanistic tools in the product portfolio (since these are the most marketable and cost-effective) and the project is apparently working on more of these. Financial management and coordination with USAID brought the pipeline to just 12 percent of obligation in June 2004, but budgeting skills are questionable in a project that seems to spend consistently one third less than the budget in most subprojects (although this may have been rectified in the fourth year). It would be preferable to have less bulky annual plans and reports; the sheer volume of writing and preparation each year represents a waste of resources. PHRplus seems to have managed field versus headquarters office tensions well; the internal matrix structure (SO versus thematic area versus geography) seems to be a good approach for fostering internal communication and cross-fertilization. Partnership issues have not been major problems, despite little being documented initially on partners’ roles and Abt Associates dominating the senior management team membership. PHRplus is one of the strongest and best recognized entities in an ill-defined market for systems strengthening. It does not dominate the market because there are too many segments and organizations, but it has a comfortable niche in health economics and would certainly be missed if it no longer existed. PHRplus has followed the usual route for a USAID centrally contracted project and is now over 80 percent field funded. This means that it has been highly responsive to buy-ins. However, 30 field-based clients with differing needs and understanding of sector reform and system strengthening have contributed to fragmenting the project’s efforts and image. Neither PHRplus nor the cognizant technical officer (CTO) team has done enough to prevent this fragmentation. The project has adapted well to environmental changes centered on funding. It has established strategic relationships with new funding organizations for HIV/AIDS and vaccines. The project has not been actively engaged in other medically related opportunities (e.g., Roll Back Malaria or the 3 by 5 Initiative in HIV/AIDS) and is not obviously in the forefront of the debate over the switch from intervention-driven to systems-driven development. USAID’s oversight of the project has been generally supportive and positive, but the CTO team is probably too large to be sufficiently directive on a project as complex as this. LESSONS LEARNED FROM THE PHRplus EXPERIENCE PHRplus has generally been stronger on tactics than strategy, which seems to be partly attributable to the general confusion over the market it serves and therefore the business it is in. The project has a strong but fragmented portfolio of products focused mainly on the financing aspects of health sector reform and health systems strengthening and does not constitute a flagship because there are too many competing projects active in other areas of systems strengthening and PHRplus is narrowly based, even within the financing segment. The project’s contribution to health impact is distant at best; USAID expects a contribution to the increased use of services that will have long-term impact but only 1 of the 13 PHRplus end-of-project indicators addresses service volume. Most focus on systems efficiency and effectiveness (i.e., at least one step further removed). It seems that health systems strengthening work represents a fairly lengthy route to meeting the bureau’s impact objectives. Having a USAID/Washington champion for each of the major PHRplus products would help keep a tight focus on the contribution to impact. iii Looking to the future, five lessons can be drawn from the PHRplus experience to date. ƒ USAID needs to address the tension between interventions and systems as the driving force of development. The expanded role of health sector reform and health systems strengthening is now a clear trend and a matrix is forming by default within the USAID/Washington structure. ƒ The markets for health sector reform and health systems strengthening need clearer definition and segmentation. There is little consensus on and limited knowledge of these subjects, both inside and outside the USAID community. The lack of clarity is hindering the discussion of the future and undermining PHRplus’s image and contribution. ƒ Categorization of countries’ readiness for health sector reform/health systems strengthening would optimize aid investments. Different regions are more readily absorptive of sector reform and systems strengthening than others, and this should be recognized. ƒ A single flagship health sector reform/health systems strengthening project makes little sense. PHRplus has struggled with the trade-off between depth and breadth, and the sheer scale of health sector reform/health systems strengthening, once clearly defined, will almost certainly prohibit a single project response. ƒ Knowledge and complexity will remain as barriers to the acceptance of health sector reform and health systems strengthening for the medium term. This will put a premium on orientation and education as well as on ensuring that health sector reform/health systems strengthening activities are as concrete as possible. FUTURE DIRECTIONS FOR SECTOR REFORM AND SYSTEMS STRENGTHENING Five approaches to determining a country’s needs for health sector reform/health systems strengthening support are evident: ƒ USAID’s current approach: generally Mission and/or contractor driven and usually intervention oriented; ƒ prescreening of country capacity: as pioneered by the Millennium Challenge Corporation, which ranks a country’s readiness for health sector reform/health systems strengthening against standard benchmarks; ƒ directed, external assessment: the traditional approach used by the World Bank; ƒ self-assessment and request for assistance: likely to become more common as more countries adopt sectorwide approaches; and iv ƒ response to a request for proposal: as pioneered by the Global Fund. The needs for health sector reform/health systems strengthening work require some perspective on definitions. Figure ES–1 From Health Sector Reform to Improved Health Status (Assessment team’s view) HEALTH SYSTEMS Information Commodities Logistics Facilities Procurement Structure (includes community boards, decentralized governance, public-private partnerships) Equipment Personal Behaviors Research (includes science and operations research) Financing (includes funding and distribution) Management (includes accountability, business management, clinic management, monitoring and evaluation, planning, quality protocols, quality assurance) Public Health Education and Training Water Sanitation Health Sector Reform = Policy Change + Health Systems Strengthening Use of Services Health Status Efficiency/ Effectiveness Health sector reform is a higher order concept than health systems strengthening; health sector reform is in fact achieved through health systems strengthening work plus work on policy change. Health systems strengthening in turn is defined broadly and can include many systems. Health systems strengthening work should favorably affect systems efficiency and effectiveness, which improves the use of services, which in turn produces health impact. Once needs are defined, there are two main paths (or combinations thereof) for providing assistance that will eventually yield impact on health: ƒ top-down donor assistance, wherein intervention-specific assistance is provided directly to beneficiaries or governments, sometimes on an emergency basis (e.g., social marketing without any local institution building, which is rarely sustainable); and ƒ bottom-up donor assistance through systems strengthening; this usually focuses on service delivery but may also involve public health interventions or improved self-care initiatives; the systems component makes this approach much more likely to be sustainable. v Figure ES–2 Paths to Improved Health Status (Assessment team’s vision) INCREASED USE OF SERVICES Public Health Interventions Health Care Delivery Population oriented, largely through government Family/individual oriented, through various providers Self-Care Initiatives Family/individual oriented, through self-responsibility Donor Assistance (national and global) Top down, possibly emergency; usually nonsustainable Bottom up through systems strengthening; more sustainable IMPROVED HEALTH STATUS In shaping its future approach to health systems strengthening work, USAID needs to adapt some of the best practices that others have developed in identifying country-level needs (see alternative approaches in section III) and ensure that field programs build an appropriate systems perspective into their designs. Inside GH today, a matrix-based approach to systems strengthening is already emerging as different SOs develop their own approach to health systems strengthening within their intervention-specific scopes. Figure ES–3 USAID’s Current Organizational Approach to Health Systems Strengthening SO 1 SO 2 SO 3 SO 4 SO 5 POLICY X X X M&L* X X MEASURE/DHS * X X X RPM Plus* X X X DELIVER X X QAWD* X SCM* X X X X X X PHRplus* X X X X X . . . *DHS: Demographic and Health Survey M&L: Management and Leadership QAWD: Quality Assurance and Workforce Development PHRplus: Partners for Health Reformplus RPM Plus: Rational Pharmaceutical Management Plus SCM: Supply Chain Management (not yet funded) X X vi In this context, there are five options for GH to respond to this new reality. ƒ Encourage the current trend toward SO–driven health systems strengthening initiatives, which could lead to suboptimality. ƒ Fold all (or as many as possible) of the health systems strengthening initiatives into a single, real flagship project. (Such a project would be difficult to manage and control.) ƒ Add a sixth SO focused on systems strengthening and have it develop and manage either a flagship or a small set of health systems strengthening projects. Congress seems unlikely to agree in the near future and a sixth SO without its own funding would probably be worse than the current situation. ƒ Radically reorient GH’s structure around systems strengthening. This would mean turning the matrix above on its side and is probably too radical in today’s context. ƒ Maintain but improve the status quo. This is probably the only realistic option in the planning timeframe of five or six years. In order to obtain the improvements sought under the fifth option, the following need to occur: ƒ There needs to be a collective effort to agree on and disseminate a clearer market definition. ƒ The process for identifying country-level needs should be strengthened. A country classification system is needed, along with a standard assistance package for each class of country and a more rigorous, objective, and holistic approach to country assessment. ƒ The current proliferation of health systems strengthening initiatives within GH needs to be brought under better coordination and control. SUMMARY OF RECOMMENDATIONS Many of the recommendations may require incremental core funding or redistribution of existing core funding with USAID’s approval to be achievable. PHRplus and USAID are urged to make sure that funding is secure before proceeding. All of these recommendations are worthy of effort in the remaining period of the PHRplus contract. For Action by PHRplus ƒ Review how the current balance between methodologies and tools can be improved, given that tools are highly marketable and cost-effective. ƒ Focus additional attention on identifying, assessing, and transferring innovative indigenous approaches to health sector reform and health systems vii strengthening in developing countries and regions to add to the existing strong efforts on transferring international approaches. ƒ Expand the scope and agenda of health information work to build a strategic approach that can support all SOs. ƒ Reduce the range of publication series titles to help focus the project’s image. ƒ Review subproject proposal guidelines to ensure that anticipated time charges by participating staff members are clear to the client. For PHRplus Action, With Explicit USAID Support ƒ Develop a more comprehensive approach to sector reform and systems strengthening at the country level to offer a wider, integrated array of products with a less segmented impact. ƒ Institute a process for bringing PHRplus and its GH SO clients together more regularly, especially to ensure that common agenda expenditure plans mesh closely with SO objectives. ƒ Identify a short list of existing project countries where a more comprehensive health sector reform approach can be proposed and implemented with an appropriate budget. ƒ Encourage the creation of additional regional/global health sector reform/health systems strengthening alliances and forums as part of the wider education effort and to identify third parties capable of handling project products. ƒ Identify a USAID/Washington champion for each of the project’s existing and new tools and consulting products, and develop strategies for achieving product impact with that champion. For USAID Action ƒ Organize a stakeholder workshop to clarify the market definitions and segmentation of both health sector reform and health systems strengthening. ƒ Make financial analysis and management skills an important and stand-alone piece of any future health systems strengthening program. ƒ Classify countries by their receptivity to health sector reform and health systems strengthening work and design a standard assistance package that can be tailored by country category. ƒ Devise a holistic and objective approach to assessing the health sector reform/health systems strengthening needs of countries before major USAID investment. viii ƒ Review and possibly redistribute roles and responsibilities among the various health systems strengthening projects that already exist in the GH portfolio. ƒ Review the size, composition, and time commitment of the PHRplus CTO team to improve the focus on PHRplus oversight. ƒ Implement a stronger communication/orientation process to increase understanding of health sector reform/health systems strengthening throughout USAID and to change mindsets from intervention-specific approaches as the default option for development, substituting a greater role for systems-driven work. ƒ Ensure that Missions and USAID/Washington include an appropriate systems strengthening dimension in field program designs to avoid overreliance on unsustainable direct assistance. ix I. BACKGROUND AND OBJECTIVES The PHRplus project was awarded in September 2000 and the award was confirmed in March 2001 after resolution of a protest. The project was scheduled to end in September 2005 but has been extended through September 2006. The project is funded through the U.S. Agency for International Development (USAID) Bureau for Global Health, Office of Health, Infectious Diseases, and Nutrition (GH/HIDN). It is designed to serve clients drawn from all five of GH’s Strategic Objectives (SOs) and responds in varying degrees to all five Intermediate Results (IRs) in GH/HIDN’s Health Policy and Systems Strengthening Results Package. (Appendix C summarizes the strategic framework and organizational context for the project.) PHRplus is being implemented by Abt Associates Inc. in collaboration with eight partners: ƒ Development Associates, Inc., ƒ Emory University Rollins School of Public Health, ƒ Philoxenia International Travel, Inc., ƒ Program for Appropriate Technology in Health, ƒ Social Sectors Development Strategies, Inc., ƒ Training Resource Group, ƒ Tulane University School of Public Health and Tropical Medicine, and ƒ University Research Co., LLC. The project draws on a pool of 150 staff globally, equivalent to a full-time complement of approximately 124 staff: PHRplus Staff Complement, May 2004 Location Professional Support Total U.S.–based Full time 14 12 26 Part time 37 12 49 Full-time equivalent of part time staff 19 4 23 Total (full-time equivalent) 33 16 49 Field-based (all full time) 47 28 75 Total (full-time equivalent) 80 44 124 PHRplus has worked in 30 countries to date. The project is a performance-based, cost plus award fee contract, with a budget ceiling of $98 million (including potential award fee); obligated funding as of June 2004 was $62.3 million and expenditures were $54.8 million. The obligation remained stable through the end of fiscal 2004; $20.3 million (33 percent) was core funding and $10.6 million (half of core money and 17 percent of the total obligation) had been allocated for common agenda purposes. GH/HIDN requested that POPTECH provide a three-person team to conduct a midterm assessment of PHRplus, beginning in July 2004, with the following four objectives: ƒ assess PHRplus’s progress in meeting its objectives, 1 ƒ identify lessons learned to date, ƒ document health system strengthening approaches potentially useful to USAID, and ƒ recommend future directions for USAID’s health system strengthening activities. The team gathered information from PHRplus and USAID staffs as well as external stakeholders (clients, other donors, universities, other contractors) over a 4–week period in July and August, including field trips to three countries—Albania, Peru, and Tanzania. (The scope of work is in appendix A, a list of persons contacted is in appendix B, and appendix D contains summaries of the three field visits.) This report summarizes the team’s conclusions and recommendations. 2 II. PERFORMANCE ASSESSMENT PERFORMANCE AGAINST INTERMEDIATE RESULTS IR 1: Appropriate Health Sector Reforms Are Implemented IR 1 was expected to be the largest component of the PHRplus project, to which about 30 percent of core funding would be directed. Sub–IRs include improving implementation of reform, empowering stakeholders to participate in reform decisions, monitoring the effects of reform, and building global consensus on reform guidelines. Most of the project work has focused on sub–IR 1.1—improving the design, adoption, and management of reforms—and empowerment through training (sub–IR1.2), with relatively little attention to the monitoring or consensus issues. The main activities have been in the following four areas: designing and implementing sector management decentralization, strengthening hospital and other facility management in a decentralized system, central health policy design and support, and training stakeholders on reform principles, policies, or implementation issues. Examples of the country initiatives follow. Designing and Implementing Sector Management Decentralization (IR 1.1) PHRplus has provided assistance to structure a framework for decentralization in Eritrea (assisting the Ministry of Health [MOH] to develop decentralization policy) and Peru (a decentralization matrix). Support to clarify roles and functions was implemented in Malawi (roles of the central and district levels defined), El Salvador (management contracts between the MOH and 28 basic health systems), and Peru (formation of regional health directorates). PHRplus has supported new regional planning processes in such countries as Honduras (regional health plans and monitoring), Peru (regional participatory plans), the Democratic Republic of the Congo (DR Congo) (management support systems to health zones), and Senegal (decentralization management schemes). Strengthening Hospital and Other Facility Management in a Decentralized System (IR 1.1) Assistance was provided in Jordan (strengthening hospital systems by improving public sector contracting plus a hospital policy forum), Albania (clinical performance improvement), Peru (coordination of public and Essalud hospitals), Eritrea (referral systems in three pilot hospitals), Malawi (financial capacity strengthening at the central hospital), and El Salvador (financial management for Sibasis). Central Health Policy Design and Support (IR 1.1) 3 Policy mapping has been conducted in such countries as Benin (donor mapping) and Egypt (stakeholder analysis on policy advisory committee for participation in health reform). Legislation support was provided in Honduras, Malawi (legal workshops to help design reform legislation), Ghana (memorandum to Parliament proposing amendments to a national health financing bill and a concept paper on the National Health Financing Act), and Albania (health insurance laws). Political consensus building has been carried out in Honduras (health plans for regions) and Peru (decentralization policy). Support policies have been designed in Guatemala (national hospital policy); Egypt (steering committee to create, advocate for, and manage reform agenda); the Philippines (PhilHealth national insurance single payer system); and Peru (monthly meetings to discuss decentralization issues). Training Stakeholders on Reform Principles, Policies, or Implementation Issues (largely IR1.2 but also general capacity building) Training was provided in Eritrea (national hospital policy), Egypt (improving planning in human resource management and budget tracking), West Africa (reinforce capacity of mutual health organizations [MHOs] in risk management), and Albania (seminars on primary care and family medicine). There seem to be few countries where PHRplus has been able to cluster its various products into a comprehensive level of assistance to support sector reform; Egypt, Honduras, Jordan, Peru, and possibly Albania may be examples. The project has generally been invited into countries without a clear agenda on the range of needed interventions and as the balance between core and field support funding has shifted, the tendency to a piecemeal approach seems to have strengthened. There is a need to develop a more clustered approach to sector reform (and thereafter systems strengthening) to gain a systems perspective. To support this, PHRplus should develop a more comprehensive approach to sector reform and systems strengthening at the country level, enabling it to offer a wider, integrated array of products. Both USAID/Washington and local Missions will need to lend support in encouraging a more strategic approach at the country level. In the areas of sector reform and decentralization, at least three different stages of country development can be identified: early (mainly in some parts of Africa and Asia), medium (mainly in Central Asia and some Latin American countries), and highly developed (countries that are sophisticated users of sector reform assistance). PHRplus has rightly been flexible in offering different products, depending on the needs and conditions of each country. The natural categorization of countries should be explicitly recognized in PHRplus’s conceptual thinking since this will help all parties produce the right response to a given country situation. As a corollary to the above two recommendations, there is also a need for formal assessment of the country situation prior to making major investments in sector reform. PHRplus is usually doing this in the context of its country assistance plans but that then limits the scope of PHRplus assistance. Given the acknowledged complexity of sector reform, there will need to be a redoubled effort in the future on building consensus around sector reform guidelines, processes, and definitions. The team was struck by the general lack of consensus, even on basic terminology, both within and outside USAID; this is one area (IR 1.4) where PHRplus has made little progress. Clearly, considerable debate on the principles is needed. Thereafter, there can be more informed communication among all the parties on what needs to be done. In the interim, PHRplus needs to increase its communication facilitation activities to ensure that USAID/Washington and Mission staffs appreciate the wide concerns of sector reform, its components, timeframe, and implementation implications—all within the confines of today’s understanding of the principles. 4 IR 4: Health Financing Is Increased and More Effectively Used PHRplus’s involvement in health financing is deeply rooted in two predecessor USAID contracts focused on health financing that were held by the prime contractor, Abt Associates. It is not surprising, therefore, that health financing has dominated the project’s activities and that much of its work is either a continuation of or variation on earlier initiatives in the financing area. Within GH, PHRplus is closely identified with work in health financing; it is noted that four of the six thematic areas that are the reference point for monitoring project performance include indicators related to work in financing. In all five sub–IR areas, PHRplus has made contributions and built capacity that has the potential to have lasting impact on both host country performance as well as future donor activities and approaches. The project has provided technical assistance, developed tools, conducted analyses, and initiated research that together represent a significant resource for the international community. Furthermore, PHRplus has documented its work in easy￾to-use formats and made it available globally through its web site. Almost all clients have been highly satisfied with the work in financing. By requiring the project to be responsive to each of the Strategic Objective (SO) areas, USAID has successfully promoted the development of substantial intellectual capital that has greatly enhanced the capacity of country leaders, USAID SO team leaders, and other donors to grasp the financial implications of service delivery management decisions. Particularly noteworthy are the ƒ AIDSTreatCost (ATC) model and software, ƒ guidance on the development of financial sustainability plans and cost￾effectiveness analyses for The Global Alliance for Vaccines and Immunization (GAVI), ƒ analysis of the systemwide effects of The Global Fund To Fight AIDS, Tuberculosis and Malaria (Global Fund), ƒ impact of new funding mechanisms (sectorwide approaches and poverty reduction strategy papers) on country decisions related to reproductive health, ƒ promotion of national health accounts (NHA) and development of tools for NHA implementation and subanalyses, and ƒ studies related to the delivery of priority services by community-based health financing programs. Information derived from the analyses is serving and will continue to serve as entry points for the next generation of efforts to improve health systems in the developing world. 5 Almost all of the countries that are using PHRplus have a financial component in their country assistance plans. The range of initiatives includes improving the policies and capacity for health financing at the national and local levels; improving financial management skills in hospitals, community-based health financing programs and other government service delivery units; developing cost analyses for priority services; and increasing the sustainability of community-based health financing and NHA. PHRplus also conducted special assessments around financing issues that country leaders and USAID Missions requested to understand better the country’s ability to respond to broad sectoral financing challenges. Examples include ƒ how financing reforms have affected the ability of rural health centers to serve the poor (Philippines), ƒ expenditures on elderly health care and the resulting impact on the financing of priority health services (Jordan and the Philippines), ƒ equity in accessibility to priority health services and out-of-pocket costs (Rwanda), ƒ local-level delivery of services (Albania), and ƒ improved operations of MHOs (Mali). Some important areas of financing that one would expect to see in a system strengthening portfolio have been addressed in a minor way, if at all. These include ƒ institutional planning and budgeting; ƒ the link to planning, budgeting, and financing reform that flows from the NHA analysis, ƒ national health insurance, and ƒ other broad national financing strategies oriented to increasing sources of funds and more rational allocation of funds. The contract indicates that health financing should represent about 25 percent of the project’s core funding. The financial reports do not track funds by IR; therefore, the actual allocation could not be determined. However, estimations based on a combination of core-funded work on NHA, the portions of the applied research funding that focused on community-based health financing, and a portion of the expenditures on production and dissemination of products suggest that the actual share of common agenda funding allocated to health financing efforts was above 25 percent. 6 Financial analysis and management skills should be the centerpiece of USAID’s future health system strengthening efforts. A goal of development assistance is to create the capacity for indigenous populations to manage for themselves. When countries have managers at national, local, and clinic levels who can understand the costs of prevention and treatment and how those costs can be funded, they are better able to request resources for specific areas and allocate limited resources to maximize impact. In this context, the role of a USAID contractor working on system strengthening should evolve into one of counselor to decision-maker in solving specific problems. PHRplus now effectively fills this role in Jordan and Peru. However, in most countries where PHRplus is providing assistance, the initiatives are fragmented and do not represent sufficient assistance in the financial area needed to promote self-reliance. IR 5: Health Information Is Available and Appropriately Used PHRplus has made a contribution to all three of the information sub–IRs through its assistance to countries in strengthening integrated disease surveillance and response (IDSR) and in implementing and institutionalizing NHA. Four countries were engaged in IDSR activities (Cambodia, Georgia, Ghana, and Tanzania), and at least 14 were involved in NHA (which is also considered responsive to the health financing IR). Indicators of project performance in the area of health information were not developed so it is not possible to determine the quantity or quality of performance against expectations. While the scope of health information project activities has been narrow, both IDSR and NHA entail much more than the production of health information. They are solid contributors to an information culture within countries. For instance, IDSR builds capacity for mapping the flow of data, establishing roles and responsibilities for local staff, conducting data collection, analysis, reporting, and response. NHA requires the development of a data plan and extensive data collection from various ministries, donors, households, providers, and industry groups (e.g., private insurers, employers, and pharmaceutical companies) along with careful attention to data standardization, validation, and interpretation. For true health systems strengthening to be possible, countries where IDSR and NHA are being implemented will have to broaden the agenda for health information capacity building and transfer their knowledge to other areas where information can be used to improve performance. Country leaders must have reliable information on population health and the health care system in order to support decision-making at all levels, and this connection is clearly made in the Health Policy and Systems Strengthening (HPSS) Results Framework. Future work should build on and expand the project’s various isolated health information initiatives at the country level, such as primary care quality and use data (Albania), integrated cost and surveillance data (Asia/Near East), consumer satisfaction data (Jordan), household survey data, immunization financing database, management information systems for immunization (Georgia), and MHOs. Defining the components of a basic information infrastructure (that is tied to the SOs) would be a good starting point for establishing priorities. Before providing assistance in this area, a demonstration of financial and institutional commitment by the country would provide more assurance that the interventions would be sustainable. 7 There is a standard methodology for NHA that enables not only a deeper understanding of the flows of funds within a country’s health care system but also comparisons across countries and regions. The NHA methodology and initiative were not originated by PHRplus, but the project has seized the topic as an appropriate component of a country’s information infrastructure and aggressively pursued its adoption in USAID countries. NHA is included in the work plans of at least half of the 30 countries in which PHRplus is working. In several countries in which PHRplus has worked, the NHA process has become institutionalized and has been effectively used to analyze needs and change policies (Egypt, Guatemala, Honduras, Jordan, and Yemen). As international interest in NHA has grown, PHRplus has positioned itself to offer high-quality assistance to interested countries and donors. The fact that PHRplus is motivated largely by field support has marginalized the potential for USAID to have significant impact in any one area of health information, including IDSR and NHA. Even with common agenda funding, there does not appear to have been a coordinated approach to identifying clients for NHA. For example, one of the achievements of the project was the formation of an NHA network in USAID’s Europe and Eurasia region. However, despite the fact that Albania had begun work on NHA in earlier years, the Mission decided to ask the World Bank to fund the continuation of this. The contract envisaged that this IR would receive 25 percent of common agenda funding but the actual allocation cannot be determined because costs are not tracked by IR. In the future, a strategic approach to building broader health information capacity that would support all SOs as a good use for common agenda funds is recommended. USAID’s efforts could be enhanced by coordination and collaboration with other donors, particularly the World Bank, which provides loans for information technology. IR 2: Health Workers Deliver Quality Responsive Services This IR was expected to represent no more than 15 percent of the project’s core funding stream and, as anticipated, has not featured strongly in PHRplus’s activities. The sub– IRs are associated with regulation, compliance with clinical guidelines, accountability, institutionalizing quality improvement, and consumer participation in design and delivery. Of these, the most visible have all been at the field level. ƒ Quality improvement (IR 2.3): Work has been conducted to improve clinical practice guidelines in Albania, Jordan, and Peru, always as part of a broader effort in sector reform. Clinic-level diagnostic skills have been improved through the dissemination of standard case definitions as part of IDSR work in Ghana and Tanzania. Quality manuals and individual service quality modules have been prepared for MHO managers in West Africa as well as quality self￾assessment tools for service providers to MHOs. Referral guidelines have been prepared in Malawi and Peru. ƒ Client participation in program design (IR 2.4) has been facilitated in Jordan by seeking views on client perceptions of the quality of services to be included in the insurance scheme and through participatory approaches to district-level health planning in Malawi. MHO members have been encouraged to participate in MHO management decisions in selected West and Central African countries. The project’s stakeholder participation activity, which would have supported this sub–IR directly, was terminated by agreement with USAID in the second year. ƒ There have been modest efforts to improve the regulation (IR 2.1) of health services, particularly in the context of sector reform, for example, checklists for district health teams in supervising the operation of community￾based health financings (for the Regional Economic Development Office for Eastern and Southern Africa [REDSO]) and facility licensing procedures and licensee supervision in Honduras. 8 The PHRplus activities related to accountability (IR 2.3) were prominent in annual work plans in the second and third years under applied research. One conceptual framework technical report was completed (Albania), but a second (Senegal) was canceled for lack of core funding. The recurrent theme is that PHRplus has become involved in quality improvement activities primarily as part of a national effort (not necessarily involving PHRplus directly) in sector reform. This seems appropriate since quality improvement at the service delivery level was never intended to be a strategic focus for the project, which also explains the low level of core support for it. Including small-scale quality improvement efforts through PHRplus in such circumstances has probably been more cost-effective than bringing in a specialist contractor. In Albania (which the team observed in detail), the scale of effort has probably reached the limit for a nonspecialist project like this; much greater effort would probably justify subcontracting to a clinical quality assurance specialist. IR 3: Commodities Are Available and Appropriately Used This IR is peripheral to PHRplus’s activities and was expected to account for 5 percent or less of core funding. The team found four references to the IR over the project’s first three years: ƒ the most extensive has been the work for SO 1 on various financing aspects of contraceptive security; ƒ cooperation in Albania between PHRplus and the Rational Pharmaceutical Management Plus (RPM Plus) Program on a drug use manual, which is keyed to the clinical practice guidelines developed by PHRplus; ƒ efforts to promote and expand community drug funds in Honduras; and ƒ analysis of centralized or coordinated drug procurement and distribution in both El Salvador and Peru. Given that both RPM Plus and the DELIVER project overlap with PHRplus in this area, the level of activity is not surprising. As with IR 2, the modest effort by PHRplus under this IR makes sense because the small scale of effort would not justify the implied separate overhead and start-up costs associated with other more specialist projects being awarded to provide assistance. PERFORMANCE AGAINST GH STRATEGIC OBJECTIVES PHRplus’s clients for core-funded work are drawn from GH’s SO teams. The SOs and their relationship to the bureau’s organizational structure are described in appendix C. Activities Undertaken for the USAID SO Teams SO 1: Family Planning and Reproductive Health SO 1 is growing in importance as a client for PHRplus. Since it is not managed through GH/HIDN, SO 1 does not contribute common agenda core funds but its directed core 9 funding has risen from $100,000 in FY 2001 to an anticipated $600,000 in FY 2004— making it probably the largest contributor of directed core money in the current year. Three main topics have been funded. ƒ Contraceptive security: PHRplus has joined the Contraceptive Security Working Group and has contributed to strategic planning on contraceptive security methodology, especially the financing aspects. It has field tested the analytical tools under the Strategic Pathway to Reproductive Health Commodity Security initiative in two countries (Madagascar and Nigeria) and is preparing a study of the impact of new development assistance mechanisms (especially sectorwide approaches and poverty reduction strategy papers) on contraceptive security. ƒ National health accounts: PHRplus has undertaken reproductive health (RH) subanalyses of NHA in Egypt, Jordan, and Rwanda and developed and disseminated generic guidelines on tracking RH expenditures through NHA analysis. ƒ Impact of health sector reform on RH programs: Largely through publications and forums, PHRplus has examined how various health sector reform initiatives have especially affected the financing and use of RH services. RH programs have also been explicitly included in the sectorwide effects analysis of the Global Fund. SO 2: Maternal Health and Nutrition As the SO with the lowest funding stream, SO 2 has contributed only small amounts of directed core funding (less than $600,000 in total to date) to PHRplus and has also been the smallest contributor of common agenda core money. The work with SO 2 has focused generally on the impact of different funding mechanisms (especially insurance schemes) on the use of maternal health (MH) services, particularly the use of skilled attendants at births. Some work has also been funded in Mali on the impact of behavior change communication (BCC) programs on the use of MH services. The financing and BCC analysis streams are coming together in the current year to examine whether BCC or financing through insurance has the more powerful impact on the use of skilled attendants. SO 3: Child Health and Nutrition SO 3 enjoyed considerable funding growth during the 1990s and has been the largest source of common agenda funding to PHRplus ($4 million, which is 40 percent of the total to date). Its contribution to directed core funding has been more modest. The funding has largely been used for the following three areas of PHRplus effort: 10 ƒ Assistance to GAVI and recipients of GAVI support: This has been by far the largest effort, with PHRplus deeply involved in the funding side of GAVI’s activities, including the development of a database of immunization financing activities conducted through GAVI to date. PHRplus staff also has been central to the development of methodology and indicators for financial sustainability for immunization programs and has helped six African countries to date (Ghana, Kenya, Malawi, Rwanda, Tanzania, and Uganda) to prepare financial sustainability plans as part of their applications for GAVI support. PHRplus is now expected to give most of the GAVI relationship to the new BASICS immunization project. ƒ A study of the relative economics of oral versus injectable vaccines for polio in the posteradication phase of polio control: This study was completed in the second year of the project. ƒ A study of MHO priority services in Senegal, including child health services in general and immunization in particular: The study is designed to illuminate how priority services are selected and the role of pricing in selection. SO 4: HIV/AIDS The Office of HIV/AIDS (GH/OHA) has been created relatively recently and is no longer making common agenda contributions to PHRplus core funding; before FY 2004, it had been a modest contributor. It remains a modest contributor of directed core funds. The work has, however, been spread across a relatively wide spectrum of activities, including the following: ƒ Analysis of the cost of different aspects of HIV/AIDS care: PHRplus has developed its ATC software for analyzing the cost of introducing antiretroviral treatment into new countries. The software has been applied to five countries (Cambodia, Ethiopia, Nigeria, Uganda, and Zambia) in detail using field-developed data, to all of the President’s Emergency Plan for AIDS Relief countries (except Vietnam) through desk research, and to Mexico in a separate study. The ATC software should prove useful to countries in making their future applications to the Global Fund. Separate work has analyzed the cost of home-based care (Rwanda and Uganda) and the broad human capacity implications (especially costs) of introducing antiretroviral therapy, using Ethiopia as an initial study site. ƒ The use of NHA subaccount analysis to track and then predict the sources and uses of HIV/AIDS funding and expenditures: PHRplus is working with 10 countries on introducing this new and urgent subanalysis methodology. ƒ A small study of the impact of HIV/AIDS on community health funds in one district in Tanzania: The study is not yet finalized but is showing how much more intensively community funds are used by HIV/AIDS patients, with consequent impact on overall funding needs (confirming findings from a PHRplus predecessor project). SO 5: Infectious Diseases 11 SO 5 has been the largest contributor of all core funding to PHRplus since the project began (accounting for $7.5 million, just under 40 percent of the total), largely as a result of large infusions of directed core money in the early years. Tanzania alone has received $1.8 million in directed core money—a relatively large contribution of core money to a country program. As in SO 4, the funding has been spread across a large number of initiatives related to infectious disease surveillance, many of which are more varied compared with those funded through other SOs. This may be because the infectious disease surveillance team leader in Washington is a member of the PHRplus cognizant technical officer (CTO) team and therefore is much closer to the project than are other SO team members. These include the following: ƒ Management information system (MIS) improvements: This work has been done in cooperation with the World Health Organization’s Regional Office for Europe (WHO/EURO) and focuses on strengthening the MIS for vaccine-preventable diseases (e.g., measles, rubella/mumps) and ensuring compliance with WHO MIS guidelines. ƒ A strategic framework for an infectious disease surveillance system: The framework includes determinants of a functioning infectious disease surveillance system, a strategic approach to ensuring that these determinants are in place, and a monitoring and evaluation plan. This work draws together previously separate efforts on strategy and monitoring and evaluation for infectious disease surveillance systems. ƒ An operations research agenda for infectious disease surveillance: Four topics were identified in the third year: infectious disease surveillance data analysis and response (to be carried out in Georgia and Tanzania); costs and financing of IDSR (Georgia, Ghana, and Tanzania); communications technology for IDSR (Tanzania), which was subsequently dropped; and strengthening IDSR in decentralized management environments (global). The work is ongoing. ƒ A documentation and dissemination strategy for infectious disease surveillance: This has included adding an infectious disease surveillance component to the PHRplus web site and the development of a planned program of attendance at meetings and workshops as well as production of infectious disease surveillance–specific publications. SO Clients’ Satisfaction With PHRplus Performance In the course of discussions with the SO clients at USAID, the team probed for overall performance issues and formed four main impressions of how PHRplus is viewed: excellent technical skills, effective products, weaknesses in client relationship management, and outlook or practice that is not strategic. Excellent Technical Skills 12 The PHRplus staff is viewed as having strong technical competences delivered through professional consulting processes. The project as a whole is considered responsive to client requests once a subproject scope has been agreed to and has shown both willingness and an ability to collaborate broadly with third parties. There are mixed views on the timeliness of delivery and a general sense that PHRplus is more expensive (both overhead and direct labor charges) than other centrally contracted projects from which respondents procure work, but overall subproject management is rated highly. Subproject proposal processes should be reviewed to ensure that planned time charges by individuals are fully understood. Effective Products A thorough review of every PHRplus offering was not conducted, but the ATC software, IDSR processes, and NHA techniques were often mentioned as well conceptualized and well documented. Weaknesses in Client Relationship Management Regarding planning and setting priorities, PHRplus is often viewed as being too remote from its clients and often out of accord with clients’ objectives and internal plans. It is routinely compared unfavorably with other centrally contracted projects in this regard. The use of common agenda core funding—an important preoccupation of the SO teams—is seen as being determined in an opaque manner. There is a feeling that project identification too frequently turns into selling and that initial responses or eventual proposals tend to be overengineered and expensive. The CTO team should institute a process for bringing PHRplus and its SO clients together more regularly, especially to ensure that common agenda expenditure plans mesh closely with SOs. Outlook or Practice The project is not very strategic in outlook or practice. This is partly a perception problem because few of the clients understand the full range of PHRplus’s services or the overall context of health systems strengthening within which those services are offered. It is therefore considered somewhat esoteric, having a lack of focus or a sense of incomplete product lines (e.g., strong on community-based health insurance but less obviously involved/interested in user fees), causing the project’s capabilities on health financing capability to be viewed as incomplete. Perhaps more concerning is that the project is seen to be stronger on theoretical solutions and studies and weaker on practical implementation. However, this could be a Washington perception based on the proximity to the project’s studies; Missions see PHRplus as a practical implementer. The team concurs with some of these impressions, as noted above. Having spoken not only to clients in Washington but also having observed some field operations and surveyed Mission impressions of PHRplus, the team particularly agrees with the project being technically strong and very responsive at the subproject level but also fragmented—probably more fragmented than even its multiple client base would suggest—and lacking a coherent framework. PERFORMANCE ON CROSSCUTTING TASKS The original PHRplus contract called for performance against six tasks that cut across the IRs described above. Five of those tasks are analyzed below; the sixth—field support—is implicit throughout this section. 13 Technical Leadership and Responsiveness The project’s performance against two crosscutting tasks from the original contract— technical leadership (task 1) and strategic documentation and transfer of experience (task 6)—is summarized in this section. Technical leadership was originally translated by PHRplus to include ƒ building global consensus on methods for tracking health sector performance; ƒ development of NHA techniques; ƒ ways of increasing stakeholder participation; ƒ documenting and transferring experience, initially through global syntheses (i.e., including task 6); and ƒ assisting USAID’s SO teams to maintain technical leadership. However, the first of these components was transferred to the project’s monitoring and evaluation work in the second year. In the third year, the global syntheses were separated from the dissemination and transfer efforts. Therefore, these are the two aspects on which the team focused. The remaining elements of technical leadership above are discussed in other parts of section III. The intellectual capital amassed through the project to date is highly impressive. The publications seen and read are excellent; the publications and the project’s web site receive universally high marks from external stakeholders. Overall, the project’s output of materials and information reflects great credit on USAID and helps to establish the Agency’s credentials in the fields of health sector reform and health systems strengthening. Less impressive is the management of the global synthesis and dissemination elements of the technical leadership objective. The evolution of the contents of the task(s) is confusing and the terminology remains obscure. The global syntheses seem to overlap with the technical dissemination efforts and both produce a range of documents (e.g., Policy Primers, Executive Summaries, Insights for Implementers) that seem similar in nature. USAID has now reduced or eliminated further funding for the global synthesis work. The team heard nothing but praise for the documents themselves but was unable to judge true readership levels. There is a wide range of series titles for a relatively narrow range of topics and original material generated by PHRplus. It is well understood how this might raise concerns about project fragmentation, lack of strategic overview, and cost. The range of series titles should be reduced to focus PHRplus’s image more tightly. The analysis shows that dissemination activities accounted for about $660,000 (14 percent of total core spending or 25 percent of common agenda core spending) in the third year of the project, which seems reasonable compared with the probable $920,000 (21 percent or 45 percent) in the second year, which was probably excessive. Setting a firm benchmark compared with other USAID projects of similar size and with a similar technical leadership brief may be possible, but true comparison would doubtless be very 14 difficult. Good dissemination is worth allocating about 10–15 percent of the core budget or 25 percent of the common core budget. Training and Capacity Development PHRplus activities in most countries include a great amount of capacity building at different levels. Most stakeholders receive training, ranging from on-the-job training or coaching to educational programs, which are usually described in the annual implementation plans or country assistance plans. Most of the planned capacity-building work seems to have been implemented. There is no overarching global strategy on capacity building but country-level work still seems well directed. There is a senior training advisor in Bethesda in charge of collating country-level training activities through the TraiNet database and providing advice and support on training programs. An annual meeting is conducted with USAID/Washington staff to provide information and orient new staff to the purposes and strategies of sector reform and systems strengthening. State-of-the-art workshops are conducted when SO team leaders request them. Capacity building goes beyond training and training materials. It can include ƒ sponsorship of education programs; ƒ creating new information structures at central and facility levels; ƒ technical assistance; ƒ pilot programs designed to hand over skills and responsibility to local staff; ƒ study tours to demonstration sites, such as Albania, Egypt, and Peru; and ƒ development of local intellectual capital (such as methodology guidelines for use of tools or clinical practice guidelines as in Albania). Many capacity-building activities are conducted by indigenous professionals and often through local, country-level institutions. The use of host country nationals in training of trainer programs and the use of indigenous educational institutions is notable. Three were observed by the team: in Tanzania, the Centre for Educational Development in Health, Arusha; in Albania, the Tirana Medical School, Department of Family Medicine; and in Peru, Universidad del Pacifico. In these and other countries, most training is planned and designed locally and is carried out by local teachers or indigenous institutions. This approach allows for the adaptation of courses to cultural and other conditions and to the sustainability of interventions as a sufficient number is created. PHRplus seems to be strong in building sustainability considerations into such efforts. 15 PHRplus has a good policy of using local people and institutions to conduct teaching and training in order to create multipliers that maintain motivation and knowledge when project activities end. To the degree possible, continuous education should be planned to continue after programs end so that individuals are constantly motivated through training and systems improvements are sustained. Health Systems Research The contract envisaged an extensive research effort, placing USAID at the forefront among donors in advancing knowledge of how health systems function in resource poor environments. Under task 2, PHRplus was required to ƒ identify an agenda of six to eight research topics, ƒ establish at least three intensive research and demonstration sites to test and validate new tools and methodologies and identify the most effective approaches to system strengthening, and ƒ work with WHO and others to gain agreement on the indicators to assess health sector performance. The health systems research agenda for the project was scaled back significantly after the first year of the contract. PHRplus attributes the reduced emphasis to a combination of limited core funding with consequent enforced revisions to its plans at several junctures and the inability to generate interest from Missions and field staff. The research effort, however, has used less than its budget in all years to date (this is to be rectified in the fourth year). A close linkage between the technical assistance activities of the project and the research agenda was preserved. There are two areas of research being pursued and one planned. First, five related studies focus on different aspects of community-based health financing in West Africa. This interconnected research agenda will create a solid evidence base of knowledge and understanding of community-based health financing to guide future champions and increase the effectiveness of technical assistance. Second, the sectorwide effects study of the Global Fund addresses the distortions in the health care systems that are created by new large sources of funding for categorical programs. This is a timely and important inquiry that will inform donors and host country leaders about the contingency factors that need to be considered when awarding or receiving these funds. In Georgia, operations research will assess the effectiveness of job aids in improving IDSR. Due to lack of funding and sufficient size of the country programs, there is only one intensive research and demonstration site—Albania. This activity represents an intersection of monitoring and evaluation and research. Findings will lead to an understanding of how specific interventions have affected availability, quality, and use of priority services, contributing to decision-making about replicability. The findings also will add to the understanding of methodologies best used for measuring the impact of interventions. In the first few years of the contract, the research group developed cross-country syntheses of experience related to particular interventions or country-led health system changes. The usefulness of the information to practitioners in the field of health reform was significant and utility was maximized because of wide dissemination through the project web site, a now well-known resource. This activity was subsequently curtailed and the narrow resulting focus of the project on several thematic areas has limited its knowledge-building potential. 16 Curtailment of the PHRplus research agenda represents a missed opportunity for USAID. Nonetheless, this type of effort is an essential component of any health systems work. The model provided by PHRplus of combining technical assistance with research protocols provides an innovative approach that has the potential to increase the return on donor investments in both technical assistance and research. Every intervention should include a query about how and what USAID will learn from the work. Monitoring and Evaluation and Performance Tracking The original concept of monitoring and evaluation under the contract was quite ambitious. Required to conduct monitoring and evaluation as part of task 3 (field support) and task 4 (performance monitoring and results tracking), PHRplus envisioned a monitoring and evaluation program with three dimensions. First, as part of crosscutting strategies for achieving results, the project planned to use a strategic framework and sentinel indicators to track health system performance and priority services. Second, monitoring and evaluation activities were intended to track internal program results of country-specific and global activities and perform knowledge-building evaluations. These two dimensions of monitoring and evaluation activities were to be closely coordinated with the third dimension, the applied research program. By producing an interconnected system of measures, the project planned to create evidence-based knowledge and understanding of system strengthening in the most effective way. The work plan for the second year included internal project monitoring and knowledge￾building activities that included three studies aimed at evaluating health system performance, the impact of donor-supported programs on the poor, and the impact of health reform on population, health, and nutrition (PHN) priority programs. The first of these studies was not pursued, but work is ongoing on the other two. By the end of the second year (FY 2002), the CTO team decided to narrow the focus for performance monitoring and concentrate on areas in which PHRplus has a manageable interest. This decision facilitated communications with potential clients because it identified the primary areas of emphasis in the work plan. Ten thematic areas were identified as the areas of concentration for project work and a subset of five areas was selected for monitoring project results. These included community-based health insurance, NHA, global alliances, infectious disease surveillance, and HIV/AIDS work. Measurable targets were established for each. A sixth indicator (other) was also included, without a measurable target. Results monitoring has been based on these five thematic areas since then, with some additions and subtractions along the way. Hospital strengthening was added in the third year and removed in the fourth year; decentralization was also added in the third year. The end-of-project indicator areas include the original five areas plus decentralization. All of the indicators, which measure the impact of health systems strengthening, were revised annually to broaden the project’s horizons progressively, although the targets seem to have been modest. Early in the project, PHRplus discontinued its work on indicators to track health system performance. During interviews, the team learned that the shift to thematic areas was made because the USAID CTO team and PHRplus agreed that the original contractual language was too broad and could not be implemented with the limited core funding available. However, the monitoring and evaluation activity consistently did not use its available budget (this is to be rectified in the fourth year). The decision not to track health 17 system performance was taken because it was not possible to attribute changes (or lack thereof) in health systems to PHRplus alone. The finding that the project’s legacy is identified with a quite narrow segment of the many possible health system strengthening initiatives probably stems from the decision to track only six thematic areas for reporting purposes. Tightening the focus for the contractor under a scope of work that is broad and all encompassing maximized the potential for a sufficient amount of impact. But it may have compromised the project’s broader health systems strengthening mandate, skewing the focus toward financing. Conclusions ƒ The six thematic areas used as the basis for measuring results under the monitoring and evaluation plan are clearly linked to three of the five HPSS IRs, with a particularly heavy orientation to IR 4 (financing). Given the breadth of the contract scope of work, PHRplus results are being tracked along a fairly narrow selection of system strengthening initiatives. ƒ Indicators are measurable and rely on external data sources to a great extent. ƒ Country reporting was inconsistent in the level of detail and format. ƒ Only one of the evaluation studies (knowledge building) was timely relative to the schedule in the plan. Difficulties encountered included a lack of approval from USAID to proceed with further development or finalization as well as delays within PHRplus. ƒ The number and variety of documents in different formats presented great difficulty for auditing purposes for the indicators and targets, which reduced transparency—an important part of performance assessment. Examples include the following: • annual plans and reports do not address performance indicators and targets (these are addressed in annual performance reports); • the assessment scope of work indicated that there are 12 thematic areas but the project claims there were only 6; • research and monitoring and evaluation activities peculiar to a country program do not cross-reference the plans and reports for tasks 2 and 4; and • the fourth year implementation plan (FY 2004) uses a different format than the plans for the first three years and does not address tasks 1, 2, 4, 5, and 6, which obscures the reference point for reporting in the FY 2004 annual report and performance assessment. ƒ Clear distinctions have not been made between the final/approved work plan with indicators and targets and the intermediate submissions. 18 ƒ The monitoring and evaluation function has consistently used less than the budget, which suggests lost opportunities (which are planned for the fourth year). The project’s contribution to health impact is distant at best. USAID expects a contribution to increased use of services that will have long-term impact but only one of the 13 PHRplus end-of-project indicators addresses service volume. Most focus on systems efficiency and effectiveness (i.e., at least one step further removed). Health systems strengthening work represents a long-term investment in achieving GH’s impact objectives in a sustainable way. System strengthening and service delivery strengthening need to proceed simultaneously to protect the Agency’s short-term investments. It is incumbent upon the Agency to find appropriate measures for system strengthening that are tied to impact. To date, these measures have been elusive. For this reason, it is regrettable that the original monitoring and evaluation plans were curtailed. A guide for assessing the priority assigned to monitoring and evaluation in a project is that the budget should represent about 10 percent of core funding. Expenditures for monitoring and evaluation through the end of the third year (FY 2003) were a little less than 10 percent of common core funding for the period and about 5 percent of total core funding. Since three of the six thematic areas directly relate to GH SOs, the budget appears low. However, the slow pace of completion of evaluation studies raises the question of whether increased funding would enhance the commitment to monitoring and evaluation. PERFORMANCE ON PROJECT MANAGEMENT AND PLANNING Internal Organization and Management The internal management of the project was considered from six perspectives: marketing and client relationships, product development, finance, planning, organization structure and subproject management, and partnership management. Marketing and Client Relationships Both marketing and client relationships seem to be strong in the field, with PHRplus having attracted a large number of relatively stable buy-in relationships. Field support is expected to be 82 percent of total funding in FY 2004. Three satisfied USAID clients were observed during the field visits, and the results of the Mission survey as part of this assessment are generally very positive. Relationships with Washington-based USAID clients are less robust; there are persistent concerns about common agenda funding and the matching of SO priorities with PHRplus activities because the project is considered more remote and less accessible than other centrally contracted projects. The communications side of marketing, including dissemination, is very high quality but could have more focus, as discussed above. Product Development 19 PHRplus has developed a range of tools, consulting products, and approaches. A tool is defined as being fairly mechanical in process and universal in application; NHA and ATC are probably the best examples. A consulting product is based on a well-codified approach but still requires methodological adaptation to country settings and client needs; IDSR, community-based health financing, and sustainability planning for GAVI applicants are examples. An approach is based on a body of knowledge but has not yet been refined or packaged into specific tools or consulting products; analyzing costs (variously applied to IDSR, integrated management of childhood illness [IMCI], home￾based care for HIV/AIDS sufferers) and managing the consequences of decentralization would be two examples. Cost analysis (whether an approach or packaged as a tool) represents a fundamental building block for the future since systems strengthening is often based on choices and those choices are strongly shaped by costs. It is impressive that PHRplus has managed to make costing a base technology for its business globally. PHRplus should review its activities to identify how it can improve the balance between methodologies and tools, given that tools are highly marketable and cost-effective (see appendix E). The project is in fact working on a number of new tools (e.g., facility-based information systems in Albania and Egypt), which may be coalesced into a new tool in the near future). Finance The project’s financial performance has been mixed. PHRplus has operated within budget and the pipeline trend is now favorable, having fallen from over $20 million last year (38 percent of obligation) to $7.5 million in June 2004 (12 percent). Of the 72 subprojects separately budgeted in the second and third years, 63 spent 35 percent less (on average) than the budget compared with 9 projects, which spent 45 percent (on average) more than the budget. This could reflect poor budgeting judgment by subproject managers or unexpected changes in the field or both. PHRplus management has stated that this apparent underuse of available funding will have been rectified by the end of the fourth year. Continuous pressure on core funding since project startup has forced many savings (especially in research and dissemination). The project is now coping with the mismatch between funding and the original scope of work, which should reduce the cost pressure that USAID has had to exert in the early years. Planning Annual plans and reports generally match well, although some activities (e.g., research and technical leadership) seem to have been planned only sketchily at times. The level of effort that clearly goes into annual plans and especially annual reports is of concern. Both are bulky documents and the high level of detail provided is of questionable value. Organization Structure and Subproject Management PHRplus seems to have managed well the tensions between field and headquarters and between core and field objectives. The internal matrix structure (balancing SOs, thematic areas, and geography) seems well designed to foster synergies and cross-fertilization. Only in Tanzania were some communication and logistical problems noted. Partnership Management 20 The PHRplus partnership seems to work remarkably smoothly given the informal nature of the original agreements. Partner affiliations are correctly invisible to a third party. Abt Associates dominates in terms of size and name and also controls five of the six positions on the senior management team; decisions on division of labor do not seem to be totally transparent to all partners. Market Positioning The team did not identify a single, recognized authority in the field of health systems strengthening that specializes in the developing world, although the World Bank is the largest donor and has the most comprehensive approach. There are contractors that have the capacity to address various aspects of health systems strengthening (e.g., DELIVER, POLICY, and the Management and Leadership Program [M&L] in the United States and the Institute for Health Sector Development in the United Kingdom). PHRplus, because of its level of funding, geographic coverage, the quality of its resource center, and the specialized experience of its staff, is definitely in the forefront internationally. However, its capacity is uneven. It is best positioned and most often identified with the health financing niche of systems strengthening, where it would clearly be missed if it no longer existed. In USAID/Washington, where it coexists with several other systems strengthening projects, it seems to have a sector reform image that differentiates it from the logistics, policy, and management niches of the other projects. Within the USAID system, it seems to overlap most with POLICY although the POLICY project is much more oriented toward reproductive health, and with M&L, especially in the implementation of decentralization, where both projects are involved in splitting obligatory functions and determining performance standards for newly decentralized districts. Outside the USAID system, there is only significant overlap with the Institute for Health Sector Development, the Department for International Development’s (DFID) equivalent of PHRplus. PHRplus is less well known in the medical/technical aspects of sector reform and has to hire staff when such topics arise in the field. USAID and its contractors taken together are also relatively weak on top-level sector reform or systems strengthening policymaking. Compared with the World Bank in particular, USAID rarely is present when these topics are on the agenda and thus, with a few exceptions, PHRplus is not usually engaged in policy discussions or broad questions of health reform. Instead, PHRplus is usually present once the policy direction is set and implementation assistance is needed, although its implementation experience does often help to shape subsequent policy development. Managing Competing Priorities PHRplus was viewed as an opportunity for GH to create broad-based systems capacity (financial, organizational, analytical, logistical) at the country level to manage service programs. Instead, the project has become a collection of specialized interventions that have little relation to each other. These interventions seem to be viewed by the country￾level clients more as ends in themselves and not as part of an interconnected set of capacities that improve delivery capability. The failure of the project to live up to its original purpose can be linked to the following: 21 ƒ The project has had to balance the interests of SO team leaders, regional bureaus and 30 field Missions. However, there is a strong presumption that a centrally contracted project is only successful if it proves its worth by having the field Missions significantly buy in to the project. PHRplus has been highly successful in attracting field support, which has caused the project to be pulled in many directions, facilitating the piecemeal approach that the team has observed. ƒ Core funding has been lower than originally anticipated, giving PHRplus less ability to develop products that it can lead with in the field. ƒ There was little opportunity at the outset to explain fully the project concept and the benefits of a coherent health systems approach, and the project has subsequently not found enough ways to convey successfully the benefits of system strengthening and health reform. Many PHRplus clients still seem to have little understanding of the project’s conceptual foundation. ƒ USAID itself gives little priority to health reform at either the central or field level, which marginalizes the potential for PHRplus to engage in this area. It is difficult for PHRplus to gain entry to a country with its full program unless it is actively sponsored by USAID. ƒ USAID’s focus on short-term results conflicts with the long-term nature of many systems strengthening activities. These need to be designed as part of a continuum so that the strategic value of the efforts is not lost and initiatives are not abandoned when Mission or GH staff changes. The PHRplus project has clearly been more responsive than strategic but it has had few options, with the amount of Missions buying in to the project being a measure of success and core funding less than expected. This is not to say that the project has not made important advances in many areas of health sector development. But the potential for the work to lead to a progressively more sophisticated development effort is not currently apparent. Adapting to Change PHRplus has coped well with the many uncertainties that have arisen at the country level, including changes in the minister that caused changes in policy direction and changes in economic fortunes. In Albania and Peru, the project had in fact become an invaluable source of continuity. The high proportion of local staff on the project country teams has helped this adaptation because of their grasp of local realities and political changes. Despite these many close and successful working relationships at the country level, it appears as if little work has been done to transfer good indigenous models of sector reform or systems strengthening into the international community (except for community-based health insurance); the traffic has typically been in the other direction. A lot can be learned from host countries and more attention should be paid to identifying, assessing, and transferring innovative, indigenous approaches in developing countries and regions. 22 In the international area, PHRplus has been both successful and strategic in adapting to changes in the environment that affect funding channels. The project is working with GAVI on its sustainability; through the sectorwide effects, it is grappling with whether the Global Fund will truly swamp funding going to interventions outside the AIDS, tuberculosis, and malaria scope. With the President’s Emergency Plan for HIV/AIDS, by use of the ATC model, it is examining whether the budget and the objectives realistically match. By contrast, PHRplus has not been called upon by other medically related opportunities, such as Roll Back Malaria or the 3 by 5 Initiative. More importantly, it does not seem to be in the lead in guiding or focusing the increasing intellectual debate about whether interventions or systems will guide development in the future. In the latter area, there is a need to generate more debate and this meshes with the overall need for better communication and education on systems strengthening and sector reform topics in general. Compared with other segments of the health sector, these topics have fewer international organizations or forums dedicated to them. One useful adaptation would be for USAID to support PHRplus in encouraging new global or regional alliances and forums in the areas of sector reform and systems strengthening. This is being done to some extent under the shared global agenda on health system strengthening in the fourth year project work plan. USAID Oversight and Support The PHRplus contract is among the most all encompassing and complex that the team has seen. The CTO team in Washington has maintained close surveillance of the project and receives high marks from PHRplus staff for its technical knowledge of the project’s many and varied activities. Missions also seem very satisfied with the CTO team’s positive and responsive approach to field relationships (e.g., in work plan coordination, contract modifications, and funding management). Such good relationships have helped to build the strong buy-in from the field to PHRplus. The CTO team needs to work on fostering communication and coordination between PHRplus and Washington-based clients, as mentioned in a previous section. The CTO team may wish to conduct a comparison of its approach with that of other GH projects that have been cited for their effectiveness in this area. PHRplus expresses concern that the CTO team has been unable to obtain sufficient core funding for the project, although this appears to be a problem faced by all of the centrally contracted projects in the current environment. PHRplus has obtained its share relative to others. The project has also commented on the availability and involvement of the CTO team from time to time. The size of the CTO team may be an issue here. With seven part￾time members, communication is at a premium and responsibility can be diffused. Other centrally contracted projects seem to have smaller oversight teams; PHRplus would benefit from a smaller group and more focused responsibility. In this regard, it is striking how quickly the project moved away from some of the provisions of the original contract, with full cognizance of both parties (e.g., the approaches to monitoring and evaluation and applied research, and the accountability and stakeholder participation activities). While this suggests some dissatisfaction with the original design or possibly inadequate core funding, it also calls for strong and continuing direction and support to a project that lost some of its contractual direction at an early stage, which was more difficult with a large CTO team sharing multiple responsibilities. Reviewing the size (and therefore the composition and time commitment) of the CTO team to achieve more focused interface with the project is recommended. 23 Since PHRplus is a performance-based contract, there is now an annual review of the project’s performance by a fee award panel, a review process that follows standard USAID guidelines. This review is made harder by the sheer volume and detail of reporting made by the project, which is probably prompted in part by fear of the performance-based nature of the contract. It has taken the assessment team 15 person￾weeks of concentrated effort to obtain a rudimentary understanding of the range and variety of PHRplus activities, not including detailed performance considerations. The award panel follows a similar track but in much less time and without the benefit of field visits, which constitute an important check on reality. GH may wish to conduct its own review of the effectiveness of the fee award process with projects of such complexity as this. 24 III. FUTURE PROSPECTS FOR HEALTH SYSTEMS STRENGTHENING CONCLUSIONS TO DATE AND LESSONS LEARNED This assessment of PHRplus has generated a wide range of performance conclusions that can be used to learn lessons about USAID’s approach to health systems strengthening to date. The analysis and findings in section II lead to conclusions about PHRplus under three main headings: portfolio and positioning, results and impact, and strategic thinking. Portfolio and Positioning The project’s portfolio of tools, consulting products, and methodologies contains many strengths that make PHRplus a unique asset to USAID. But it also remains very fragmented. PHRplus is not offering a complete service in any of the major segments of the health systems strengthening or health sector reform markets. It is probably positioned most strongly in the financing segment of health systems strengthening/health sector reform (especially mutuelles, other community-based financing approaches, and NHA) but has less experience in developing major policy change on national financing approaches for health and seems to do little work on user fees, a topic which is still of interest globally. Such fragmentation can be explained in part by the project’s multiplicity of clients but a piecemeal portfolio like this produces narrow impact and belies a systems perspective. USAID and PHRplus should work closely over the next 18 months to identify a short list of existing project countries where a more comprehensive health sector reform approach can be proposed and implemented within an appropriate budget. PHRplus has comfortable niches in the health systems strengthening and/or health sector reform markets, clearly in health economics and possibly in health sector reform overall; it would be missed in both if it no longer existed. However, it does not constitute a flagship health systems project because it is too narrowly based, too focused on financing issues, and faces too many other USAID projects that are also health-systems oriented (e.g., DELIVER, M&L, POLICY, RPM Plus). It comes close to offering one-stop shopping (GH’s definition of a flagship) in financial systems but not in health systems strengthening overall. Results and Impact USAID/Washington has conceded that a health systems strengthening project such as PHRplus should have direct impact on the use of services and through increased use, indirect impact on health status. In that context, of the 13 end-of-project indicators currently being finalized between PHRplus and USAID, only one is closely related to the impact on service use (addressing use of services by MHO members). The others are more process oriented in a global health context but remain highly appropriate to a health systems strengthening initiative: institutionalization of NHA, policy or strategy changes within the Global Fund, development of health plans in newly decentralized district management systems, and impact on HIV/AIDS resource allocation policies. 25 PHRplus is generally well on schedule toward meeting its end-of-project indicator targets and those associated with more specific indicators used at the country level. This indirect path to health impact needs to be more lengthy than those pursued by intervention￾specific (e.g., RH or child survival) projects, since the EOP indicators are often at least one step removed from increased service use, and the opportunities for breakdown in the causal chain between what PHRplus delivers and how the local government or client then proceeds are legion. This distance problem is probably reinforced by PHRplus not having a single, directive client for many of the tools and consulting products it provides (except CTO team members), for example, NHA, community-based financing, and decentralization impacts. While these tools/products are invariably satisfying Mission￾level objectives, they would benefit from a global owner (based in Washington) who focuses on long-term impact across different countries. The CTO team and PHRplus should review the portfolio and ensure that product development activities over the remaining 18 months each involve a USAID/Washington champion. All new core activities proposed by the project have to identify a Washington-based client or interested parties; the champions can be drawn from this pool but their role in advocating for health impact needs strengthening. Investment activities can still be identified but with a CTO team member as the client promoting impact. Strategic Thinking PHRplus has successfully adapted to the changing environment in health care funding. It is working strategically on improving GAVI sustainability, examining the potential swamping effect of the Global Fund, and assessing the reasonableness of the long-term objectives of the President’s Emergency Plan. It has done less to advance the state-of-the￾art at the strategy level in other areas of health systems strengthening/health sector reform. More importantly, the project has been more successful tactically than strategically, even given the environment in which it has had to work (e.g., limited core funding and more than 30 clients with different needs): ƒ a wide range of good tools, products, and methodologies but only two real tools that are relatively quick to implement, can quickly move a country along the spectrum from pilot to institutionalization and resonate with USAID’s need to limit its long-term funding commitments (additional ideas are summarized in appendix E); ƒ a range of excellent publications but probably too many series titles, which exacerbates the project’s diffused image; ƒ a solid IDSR product being installed in Tanzania but not drawing attention to the extremely weak malaria response (part of the IDSR system) that continues to make it the number one killer of children under 5 and an enormous drain on resources; and ƒ an unclear definition of the overarching business that PHRplus is in: health sector reform, health systems strengthening, health financing, or some combination of the above. These concepts as well as the project’s image are vague; many USAID/Washington staff members categorize PHRplus as GH’s health sector reform project to distinguish it from the other health systems 26 strengthening activities. The stepping stones paper attempted to clarify this but was very academic and failed to elicit clear resonance from USAID. These conclusions lead into the five lessons on health systems strengthening/health sector reform that were identified from PHRplus to date. 1. USAID needs to address the tension between interventions and systems as the guide of development. USAID/Washington is organized around and funded through five intervention-specific SOs. PHRplus is one of the first major projects to cut across all five, while other systems strengthening projects, mostly based in the Office of Population and Reproductive Health (GH/PRH), are increasingly developing a multiple SO clientele. The expanded role of systems and systems strengthening is clear and a matrix is forming by default. USAID urgently needs to decide how it intends to deal with this trend. 2. The markets for health sector reform and health systems strengthening need clear definition and segmentation. There is little consensus on and limited knowledge of these subjects, both inside and outside the USAID community. The lack of clarity is hindering the discussion of the future and undermining PHRplus’s image and contribution. 3. Categorization of countries’ readiness for health sector reform/health systems strengthening would optimize aid investments. It is already clear that different regions are more receptive to health sector reform and health systems strengthening than others. For example, many of the former Soviet states and now parts of Latin America are deeply involved with these subjects because they have both the need and the absorptive capacity; Africa and Asia generally have less absorptive capacity, although needs are as high or higher. The assistance has to be tailored to the situation, especially in balancing systems strengthening with urgent service delivery improvement in the countries with less well-developed health sectors. 4. A single flagship health sector reform/health systems strengthening project makes little sense. PHRplus has struggled with the tradeoff between depth and breadth. The sheer scale of health sector reform and health systems strengthening, once clearly defined, will almost certainly prohibit a single project response. The flagship concept was appropriate as long as USAID’s thinking was dominated by the intervention-specific dimension; now there seems to be more balance between interventions and systems. 5. Knowledge and complexity are going to remain as barriers to acceptance of health sector reform and health systems strengthening for the medium term. Even after it is better defined, health sector reform and health systems strengthening will remain large, complex, and relatively esoteric in a bureau still dominated by intervention-specific structures and funding. This will put a premium on orientation and education and on making sure that health sector reform/health systems strengthening activities are as concrete as possible. 27 ALTERNATIVE APPROACHES TO HEALTH SYSTEMS STRENGTHENING In examining alternative approaches, the starting premise is that health systems strengthening is seen by those who invest in it as a means of increasing the efficiency and effectiveness of systems as a means of improving quality, raising use of health services and, ultimately, as a means of improving health status. In that context, through the assessment of PHRplus and conversations with outside experts in health sector reform and health systems strengthening, the ways in which the needs for health systems strengthening can be identified and how those needs can be addressed have been explored. The first approach to identifying needs is the approach predominantly followed by USAID itself. Missions typically lead the process since they are the closest to country needs and priorities for improving health status; they may or may not invite in a contractor at an early stage to participate in discussions and help to shape the problem and a response. The needs are typically defined in an intervention-specific context (e.g., enforcement of minimum age of marriage policy in an RH program, information systems for vaccine-preventable childhood diseases, logistics consequences of contraceptive security policy, subanalysis of NHA to understand payment burden for HIV/AIDS treatment). Occasionally, where USAID or its existing contractor has an especially strong relationship with a country government, the need might be much wider and more strategic (e.g., switching from government subsidy to national health insurance or planning radical decentralization of responsibility for health management). Depending on the need identified, a contractor will assist in providing the solution (e.g., PHRplus, POLICY, RPM Plus). Four other approaches to needs identification, either existing or emerging, follow. ƒ Prescreening of country capacity: This is being pioneered by the Millennium Challenge Corporation and consists of an external, independent ranking of countries against standard benchmarks measuring such variables as poverty level, quality of governance, public investment in human capital, and economic and political freedoms. Qualifying countries are then invited to propose areas in which they wish to receive assistance; they are reassessed each year. ƒ Directed, external assessment: This has been the World Bank’s traditional approach, sending a mission to the country and assessing needs independently. ƒ Self-assessment and Request for Assistance: Increasingly, and especially in countries that have adopted sectorwide approaches, countries negotiate a level of assistance with a group of donors and then specify their needs for specific help within the umbrella-assistance level. The amount of negotiating room the country has in preparing its detailed request for assistance seems to vary. ƒ Response to a Request for Proposal (RFP): This approach is being used by the Global Fund. A new round of funding is opened by a broadcast RFP that requires country-coordinating mechanisms (especially created as the potential recipient of Global Fund money) to submit proposals in a specified format. 28 Successful bidders then have to meet a series of conditions precedent laid down by the Global Fund and verified by its locally appointed agent. These five approaches vary widely; some are more dirigiste or egalitarian than others. USAID can learn from these different approaches in determining its own future approach to health systems strengthening. Before discussing how a funding agency can best respond once needs have been identified, the terms need to be defined. If a country’s needs include health sector reform, then health sector reform usually comprises national policy change and health systems strengthening work (i.e., health sector reform is a higher order concept than health systems strengthening). In many cases, only health systems strengthening work is needed, possibly involving internal changes of procedure but not national policy. Health systems strengthening is defined broadly to include most of the systems components that can affect the success of service delivery, including delivery systems as well as public health and self-care systems and issues. Figure 1 From Health Sector Reform to Improved Health Status (Assessment team’s view) HEALTH SYSTEMS Information Commodities Logistics Facilities Procurement Structure (includes community boards, decentralized governance, public-private partnerships) Equipment Personal Behaviors Research (includes science and operations research) Financing (includes funding and distribution) Management (includes accountability, business management, clinic management, monitoring and evaluation, planning, quality protocols, quality assurance) Public Health Education and Training Water Sanitation Health Sector Reform = Policy Change + Health Systems Strengthening Use of Services Health Status Efficiency/ Effectiveness The two main groups of responses to a country’s health improvement needs are either direct, short term, and less sustainable, or through improved health systems, long term and more sustainable. 29 Figure 2 Paths to Improved Health Status (Assessment team’s vision) INCREASED USE OF SERVICES Public Health Interventions Health Care Delivery Population oriented, largely through government Family/individual oriented, through various providers Self-Care Initiatives Family/individual oriented, through self-responsibility Donor Assistance (national and global) Top down, possibly emergency; usually nonsustainable Bottom up through systems strengthening; more sustainable IMPROVED HEALTH STATUS The bottom-up approach through systems strengthening is preferable on sustainability grounds. It involves a response that not only addresses the intervention(s) that has(ve) been identified, but also, in parallel, the main support systems that sustain that (those) intervention(s). Examples include health insurance schemes, evidence-based medicine, continuous medical education requirements, and hospital accounting standards. Every effort needs to be made to ensure that the systems themselves are left in a sustainable condition when aid support ends, for example, for a program assisting service delivery through government providers, training improvements have been embedded in curricula of courses offered by government staff and/or institutions, a line item for commodity procurement has been included in MOH budgets, quality improvement has become a process of continuous self-assessment, or adequate government supervisory staff is in place. In this way, the intervention itself is more likely to be sustained and lead to improved health status in the long term. The alternative approach of direct, top-down assistance is less preferable because it will probably lead to the all too familiar problem of projects ending before the work is completed—the donor leaves and the benefits stop. Examples include a social marketing program that succeeds in promoting commodity volume to clients but does not develop any indigenous institutional structure with a hope of financial sustainability and an NGO– led program that does not enable the NGO partners to build financial reserves for the long term. This approach may still be necessary in some situations, such as when health indicators exhibit crisis or support systems simply do not exist and have to be built. However, this should not be the default option as it has often been in the past. 30 FUTURE DIRECTIONS FOR HEALTH SYSTEMS STRENGTHENING IN USAID Figure 3 illustrates the current matrix structure of GH’s organizational approach to health systems strengthening. Figure 3 USAID’s Current Organizational Approach to Health Systems Strengthening SO 1 SO 2 SO 3 SO 4 SO 5 POLICY X X X M&L* X X MEASURE/DHS * X X X RPM Plus* X X X DELIVER X X QAWD* X SCM* X X X X X X PHRplus* X X X X X . . . *DHS: Demographic and Health Survey M&L: Management and Leadership QAWD: Quality Assurance and Workforce Development PHRplus: Partners for Health Reformplus RPM Plus: Rational Pharmaceutical Management Plus SCM: Supply Chain Management (not yet funded) X X PHRplus is one of many projects with a health systems strengthening objective that cuts across two or more of the five GH SOs. Many of these projects began with a family planning/reproductive health orientation, were awarded by GH/PRH, and were designed to serve SO 1 objectives; gradually, they have started to serve multiple SOs. GH/HIDN has awarded at least three projects (PHRplus, QAWD, and RPM Plus) to serve its three SOs, although PHRplus is one of the few with a bureauwide scope. Looking to the future, five possible strategies were identified and analyzed for USAID in its future organizational approach to health systems strengthening. Option 1: Encourage the current trend toward SO–driven health systems strengthening initiatives. The SO teams know best their systems strengthening needs and it seems clear that GH/OHA, with a substantial funding advantage, is now embarking on its own approach to health systems strengthening. Suboptimality is bound to result if this proliferation continues. Furthermore, developing systems individually on an SO platform militates against the beneficial effect of systems work in breaking down barriers between SOs and helping countries to develop delivery systems catering to the entire population’s needs. 31 Option 2: Fold all (or as many as possible) of the health systems strengthening initiatives into a single, flagship project. This would improve strategic direction and minimize overlap. However, there are currently so many vested interests that implementing such a change would be challenging, and the resulting project would have such a wide scope as to be difficult to manage internally and control. Option 3: Add a sixth SO focused on systems strengthening and have it develop and manage either a flagship project or a small set of health systems strengthening projects. This approach should have the benefit of providing a dedicated funding stream for health systems strengthening and eliminating the constant pressure to find funding from other SOs as long as Congress can be persuaded of the separate importance of systems work. A flagship project may still not be practicable but a smaller set of projects within a new SO would be manageable and focused. The consensus among the team and its interlocutors is that Congress would not currently accept this idea (although it may remain something for long-term consideration), and a sixth SO without its own funding would probably be worse than the current situation. Option 4: Radically reorient GH’s structure around systems strengthening. USAID has traditionally organized along intervention-specific lines but now crosscutting health systems strengthening initiatives are becoming more prominent and a new matrix is emerging. This option would make health systems strengthening the primary force of sustainable development, with SOs the subsidiary and crosscutting functions. Such an approach would be motivated by the need to reinforce sustainability, as in figure 2 above. The team is aware that GH is currently giving thought to its future structure and strategy and this could be an input to that process, although it its doubtful that there would be an appetite for such wrenching change in even the medium term. Option 5: Maintain but improve the status quo. This option would maintain the currently evolving matrix-based approach but would seek to improve it in a variety of ways. Since there are clear dangers or impediments involved in all of the other four options within the timeframe being considered (five or six years), this option was selected for more detailed thought. Based on the status quo represented in figure 3 and the findings and conclusions presented, three main, interconnected improvements appear to be needed in USAID’s future approach to health systems strengthening: a clear market definition, strengthening the process for identifying country-level needs, and controlling the proliferation of health systems strengthening initiatives. A Clear Market Definition 32 A clear market definition needs to be agreed to and disseminated. A contribution to how the definition and segmentation may be clarified has been provided above. USAID should convene a workshop to discuss and clarify the issue. That workshop should be organized by USAID rather than one of its contractors, since there is bound to be intercontractor conflict on the definitional issues. The objective of the workshop should be practical, not academic, to obtain consensus on ƒ how health systems strengthening fits within health sector reform or vice versa, ƒ whether all of the topics included in figures 1 and 2 belong within the definition of health systems strengthening or health sector reform, and ƒ how health systems strengthening may be usefully segmented as input to how the different USAID systems strengthening projects may be separately classified (see appendix F for additional detail). Strengthening the Process for Identifying Country-Level Needs USAID’s approach to identifying needs is compared with alternatives above. One lesson learned from the PHRplus experience is that countries vary widely in their readiness for health systems strengthening and health sector reform. USAID should classify countries by readiness for health systems strengthening/health sector reform and prepare a standard menu of assistance for each category to help guide Mission decision-making. (See appendix F for additional detail.) Another concern is that the process for identifying country-level health systems strengthening/health sector reform needs in USAID is fairly informal and contrasts with other organizational approaches, for example, the World Bank mission approach. While the mission-based approach might be too directive for many overseas USAID staff, a more formal assessment of country needs is desirable by those who are familiar with the complexities of health systems strengthening/health sector reform—skills only rarely found in USAID Missions—and who can more accurately assess the length of a planned program. Therefore, a holistic and objective assessment of a country’s needs should be made before investing in health systems strengthening/health sector reform to any significant degree. To ensure objectivity, it is suggested that this role be separated from any specific contractor and assigned to a separate unit within GH, possibly the Office of Strategic Planning, Budgeting and Operations. This work would be straightforward in countries where poverty reduction strategy papers are already in place, since the analysis will have been largely completed already. The assessments need to be short and low cost and rank systems development priorities to encourage Mission support. Controlling the Proliferation of Health Systems Strengthening Initiatives 33 Further significant proliferation of systems strengthening projects within USAID will be harmful and lead to increasing duplication and suboptimization of effort. There are many barriers to the second option above but some progress can be made not only to arrest proliferation but also to reverse it somewhat, without restricting healthy competition. On the surface, candidates for examination might include DELIVER versus RPM Plus versus SCM, POLICY versus PHRplus, and QAWD versus Human Capacity Development. The bureau should conduct a formal review of its portfolio of health systems strengthening/health sector reform projects with a view to some redistribution of roles once the health systems strengthening/health sector reform definition has been clarified. It will be important in that review to maintain an emphasis on systems as a unifying force among SOs, especially if the future of the bureau may lie in redressing the balance between intervention-driven and systems-driven development. 34 APPENDICES A. Scope of Work B. Persons Contacted C. USAID Strategic Framework and Organizational Context for the Assessment D. Summaries of the Field Visits E. The Product Development Ladder F. A Systems-Oriented Bureau for Global Health G. References APPENDIX A SCOPE OF WORK (from USAID) ASSESSMENT OF PARTNERS FOR HEALTH REFORMplus PROJECT SCOPE OF WORK July 7, 2004 I. BACKGROUND A. Project Overview The Partners for Health Reformplus (PHRplus) Project is funded by USAID’s Bureau for Global Health’s Office of Health, Infectious Diseases and Nutrition (HIDN). This five￾year project was awarded to Abt Associates Inc. and its partners1 in September 2000 and is planned to operate through September 2005. The project builds on and expands the scope of its predecessor Partners for Health Reform (PHR) Project (1995-2001) as well as other USAID projects that have provided assistance in health financing, management, service delivery organization, pharmaceuticals, quality assurance, policy development and decision-making. PHRplus is a performance-based, cost-plus-award fee contract that has operated in approximately 21 countries, four USAID regional bureaus and globally as a counterpart to USAID’s Bureau for Global Health. In the four years of implementation from October 1, 2000 through the present, it has received at least $70million in funding. Contractually it was designed to implement up to $98 million through September 2005. A USAID panel reviews and assesses PHRplus’ progress every year, at which time an award fee is determined for that particular annual period. The project was designed to address problems in the financing, organization and provision of health services, including weak health sector stewardship, inadequate resources, poor organizational performance and poor quality services. The development hypothesis was that these problems hindered performance of developing country health systems in implementing Population, Health and Nutrition (PHN) priority interventions and that expert technical assistance could help overcome these problems. The project thus sought to improve the performance of host country health systems in delivering PHN priority interventions by providing: • USAID operating units with access to expert technical assistance in health finance, policy and information (including infectious disease surveillance) and to a lesser degree in commodity management and quality and human resources, and • USAID’s Bureau for Global Health with an instrument to provide global technical leadership and to advance the state of technical knowledge through health system research A–1 1 The project’s prime contractor is Abt Associates, Inc. Partners are Development Associates, Inc.; Emory University Rollins School of Public Health; Philoxenia International Travel; Program for Appropriate Technology in Health; SAG Corp; Social Sectors Development Strategies Inc.; Training Resources Group; Tulane University School of Public Health and Tropical Medicine; and University Research Co. LLC. B. Impact Measurements The impact measurements for the PHRplus contract are the same as that for the overall Health Policy and Systems Strengthening (HPSS) results package, which was approved – and is expected to operate through September 30, 2009. The PHRplus contract, in combination with other efforts by USAID, other development partners, and host countries, contribute to improvements in health sector performance related to the delivery of PHN priority interventions. These impact measurements are listed below. (1) ≥ 10 countries increase the percent of overall public sector funding allocated to finance PHC priority interventions (2) ≥ 5 countries increase the percent of the poor with access to affordable PHN services (3) ≥ 5 countries increase the percent of clients obtaining PHN services from private sector providers (4) ≥ 10 countries with sustainable policies and funding mechanisms for the financing of EPI vaccines (5) ≥ 10 countries increase coverage (fully immunized child) among high risk infants and children with present EPI vaccines (6) ≥ 10 countries increase the percent of recent live births for which women report having the assistance of a medically trained health attendant at delivery Major Thematic Areas Given that these indicators are difficult to attribute, project management and the CTO team have recently completed a process to define different end-of-project indicators that can more accurately and easily be attributed to project performance. Part of that process included the identification of major thematic areas of work, as follows: • Community-Based Health Financing • Decentralization • Hospital Strengthening • Accountability and Quality • HIV/AIDS • Infectious Disease Surveillance • Reproductive and Maternal Health • Child Health • National Health Accounts • Monitoring and Evaluation • Global Alliances • System wide Effects of the GFATM and other funding mechanisms (SWEF) C. Results and Performance Indicators As with the impact measurements, the Strategic Objective (SO) for PHRplus is the same as that for the overall HPSS Results Package: Improved health system performance in delivering PHN priority interventions. It is expected that PHRplus will make a A–2 significant contribution to the achievement of the SO by supporting the achievement of three of the five HPSS intermediate results: IR 1: Appropriate health sector reforms are effectively implemented. IR 1.1: Design, adoption and management of reforms that affect PHN priority interventions improved. IR 1.2: Policymakers, providers, communities and clients empowered to participate in health reform. IR 1.3: Monitoring of the effects of health reform is carried out, and used by stakeholders in the reform process. IR 1.4: Global consensus on appropriate guiding principles of health reform achieved. IR 4: Health financing is increased and more effectively used. IR 4.1: Rational financing policies enacted. IR 4.2: Alternative financing schemes to improve affordability of services implemented. IR 4.3: Economic analysis, resource allocation, budgeting and financial management practices improved. IR 4.4: Partnerships to mobilize and leverage additional resources established. IR 4.5: Mechanisms for stakeholder input to health financing decisions expanded. IR 5: Health information is available and appropriately used. IR 5.1: Policies for effective application of information management and processes enacted. IR 5.2: Capacity to design, develop and maintain information enhanced. IR 5.3: Community knowledge of health care practices, quality and options increased PHRplus will also contribute to a lesser extent to Results 2 and 3: IR 2: Health workers deliver quality responsive services. IR 2.1: Effective strategies for regulation of public and private health services implemented. IR 2.2: Measurement of compliance with clinical guidelines increased. IR 2.3: Accountable programs and incentives to improve quality and efficiency institutionalized. IR 2.4: Consumer participation in design, delivery, and evaluation of health services increased. IR 3: Commodities are available and appropriately used. IR 3.1: Commodity system diagnosis and strategic planning capabilities enhanced. IR 3.2: Selection, forecasting, procurement, and distribution of commodities improved. Global vs. Field Impact Measurements Although PHRplus is a global project, it receives at least 75 percent of its funding from the field. Each mission has its own impact measurements and results packages, which are often somewhat different from USAID/Washington impact indicators. This presents an overall challenge for the project to achieve global impact with disparate field mission priorities. Additional indicators to measure field impact can be located from the PHRplus-developed Country Assistance Plans (CAPs) A–3 D. USAID’s Bureau for Global Health Strategic Objectives Along with other USAID Office of HIDN projects, PHRplus is intended to contribute to achieving the Bureau for Global Health’s five Strategic Objectives: SO 1: Advance and support voluntary family planning and reproductive health programs worldwide; SO 2: Increased use of key maternal health and nutrition interventions; SO 3: Increased use of key child health and nutrition interventions; SO 4: Increased use of improved, effective and sustainable responses to reduce HIV transmission and mitigate the impact of the HIV/AIDS pandemic; and SO 5: Increased use of effective interventions to reduce the threat of infectious diseases of major public health importance. II. PURPOSE OF ASSIGNMENT The purpose of this assignment is to assist the USAID Bureau for Global Health’s Health Systems Division in conducting an assessment of USAID-supported health system strengthening activities. The assessment results will provide a key input to the design of any future USAID instruments to support health system strengthening, beginning with a workshop for stakeholders in the Fall of 2004. Accordingly, the objectives of the assessment are to: 1. Assess the progress of the PHRplus Project in meeting its objectives, especially factors associated with the level of success for each technical area; 2. Identify lessons learned from the successes and failures in implementation of the PHRplus Project; 3. Document health system strengthening approaches, including those not addressed by the PHRplus Project, which could potentially improve the access to and quality of USAID’s priority health services in developing country settings; and a. Make recommendations about future directions for supporting health system strengthening. Specifically, the assessment team should be cognizant of the following points: • Provide a balanced picture of what positive progress and impacts were achieved and why they occurred, in addition to listing deficiencies and negative gaps that were found and why they occurred. • Examine PHRplus thematic areas to determine the extent to which they are contributing to achieving the HIDN “ladder of progress” and the strategic objectives and intermediate results, as tracked by the annual workplans, annual reports, CAPs and measured by the monitoring and assessment plan, where applicable) of the HPSS results package. (Re the “ladder,” in what areas is the project proceeding to proof in principle to piloting large scale application and how?) A–4 • Discuss factors and areas for improvement that have contributed to the success or failure in strengthening the health system or improving the policy environment of the provision of priority health services. Examine the project’s relationship with integration into the broader portfolio of HIDN office projects. Address gaps in the technical areas that are currently being implemented by the PHRplus project. • Balance assessment of the PHRplus Project by taking into account the tension between global and field impact indicators. • Identify health system strengthening activities currently being used with success by other projects that show promise for implementation at the Mission level, inclusion in the research agenda, or as a new initiative suitable for core funding. (Annex A: selection of TASC I and other bilateral health reform activities.) • Discuss both the scope and relevance of the mandate of the HPSS results package, particularly as it relates to family planning/reproductive health, HIV/AIDS, infectious diseases and child survival activities and potential technical approaches or activities, which could be used to achieve results. • Consider how USAID can play a bigger role strengthening and promoting health systems at the global level. III. QUESTIONS TO BE ADDRESSED Key questions have been formulated to address the above points. These questions are listed below. 1. Reform and health impacts: Unlike most health reform projects, PHRplus has been expected not only to contribute to fundamental changes in health systems but also to contribute to demonstrable health impact in terms of the Bureau of Global Health’s five strategic objectives (SOs). Why has PHRplus been more successful in working with some SOs than with others? What factors have been associated with greater success? What are some ways the project and USAID can achieve the SO and IRs under the HPSS results package as well as how USAID can address broader health systems issues. How can the health system best achieve the IRs and SO of the HPSS results package, and/or reach? 2. Market niche and alternative mechanisms: Broadly, what role does PHRplus play from various perspectives—USAID GH, USAID missions, developing country health policymakers in USAID countries, policymakers in non-USAID countries, other bilateral donor agencies, private sector foundations, international lending institutions and think tanks, universities? Does it duplicate other players or fill a unique niche? Is there any major role that it leaves unfilled? Does PHRplus’ work contribute to A–5 USAID’s capacity to influence health systems in developing countries to improve PHN priority services and if so, how? In the absence of PHRplus, would anything be lost in the global health system strengthening arena? In its absence, where would current clients obtain the same type of technical assistance? How well, if at all, does the project integrate into the broader portfolio of HIDN office projects? 3. State of the art and global leadership: What impact has the project had in advancing the state of global knowledge on key health system parameters (financing, information, commodities, quality, policy, etc.) so that we are closer to having standards, norms, guidelines for developing countries to improve health system performance? How has the project managed to achieve global impact and how did it influence other donors and developing countries? What factors were associated with successful global leadership? What is the right combination of tool development, training, dissemination and advocacy? How will project activities in key thematic areas that have been started under the project be sustained? (This category addresses progress on the “ladder.”) 4. Project management and structure: Has the project been managed in such a way as to promote synergies and cross-fertilization (application of lessons from one site to others)? Has it been managed in a cost-conscious way? Has the project managed core and field support, choices on staffing, choices on program tradeoffs, financial management, reporting and monitoring and evaluation effectively? In an era of increasing scrutiny of USAID spending on information dissemination, has the project managed this function cost-effectively and in ways appropriate for its audiences in developing countries? How does the structure of the project enhance or detract from the results achieved? How can the project be structured to achieve the best result with the most cost-efficiency? What cost savings measures has the project employed? 5. USAID oversight: Has the USAID CTO team participation been effective in monitoring the technical aspects of the project? In what ways has it contributed to or detracted from project performance? Has the USAID CO been effective in oversight of the contractual aspects of the project? Have interactions among operating units (missions, regional bureaus), contractor, contracts officer and technical officer been effective? 6. Host country capacity: Beyond the period of direct assistance, sustained impact depends in part on the capacity of in-country individuals and institutions to continue the work. To what extent has PHRplus used its work as a means to develop in￾country partner capacity, at both the individual and at the institutional levels? Where has PHRplus’ capacity building worked best and why? Has the project built developing country institutional capacity? What are the institutions that have benefited and how has their capacity been built? 7. Emerging context: How has PHRplus’ environment changed over the course of the implementation period? How have new developments such as GFATM, Roll Back A–6 Malaria, Stop TB, The Emergency Plan, 3x5, Millennium Challenge Corporation, etc. affected PHRplus’ role? How has the project responded to these changes/new opportunities? How do these new initiatives affect USAID and PHRplus’ health systems strengthening activities? Has the project and USAID taken advantage of partnering opportunities created by these initiatives? If not, how can they form better partnerships? 8. Strategic or responsive: The project has been asked to be both strategic and at the same time responsive to operating units. Responsiveness to operating units through field support could distract from the project’s ability to be strategic in its global leadership and applied research agendas. Alternatively the knowledge gained by responding to operating units could enrich the global leadership and applied research agendas. Which has been the case in practice and why? What can we learn from this about how to balance these competing objectives in the future? 9. Visionary approach: The project is required to implement activities in order to help achieve the HPSS Results Package as well as develop a vision for addressing health system strengthening issues. However, the project is often constrained by limited core funding. What changes should be made to ensure that the project can meet these two objectives given the current funding situation? IV. RESOURCES AND METHODOLOGY a. Data sources/documents Documents that the assessment team will review for this assignment include but are not limited to: • The HPSS Results Package • Request for Proposal (RFP) • PHRplus proposal and appendices • PHRplus contract* • PHRplus workplans* • PHRplus Annual Reports* • PHRplus Award Fee Documents* • PHRplus Country Assistance Plans (CAPs)* • PHRplus Thematic workplan* • PHRplus Client Relations Strategy for USAID Global Bureau • PHRplus Client Relations Strategy for Field Support Activities • Relevant PHRplus publications • USAID comments on PHRplus technical components and future design • Overview of HSD health system strengthening activities • PHRplus self-assessment* • Mission surveys* A–7 USAID will ensure delivery to POPTECH of the above documents. The priority documents are followed by an asterisk. B) Self-Assessment Prior to the assessment starting date, USAID will ask the PHRplus Project to complete a self-assessment questionnaire and send it directly to POPTECH. The self-assessment will provide raw data for the assessment team. To foster collaboration and encourage candor, the information contained in the self-assessment will not be included in the main body or annexes of the final report. Rather, the assessment team will analyze and synthesize the data in the process of developing its findings and recommendations. C) Mission Surveys As part of the preparation for this assessment, USAID will send surveys to selected missions. Mission responses to the surveys will be sent directly to POPTECH, and a summary report will be prepared for review by the assessment team. 2 D) Team Planning Meeting A Team Planning Meeting will be held in Washington, D.C. for USAID, POPTECH and the assessment team to ensure that the team members understand the assignment objectives. The team will be briefed by the CTO/TA and POPTECH on the purpose, strategy and current status of activities. Background materials and other data sources will be provided, the timeline finalized, and the team member responsibilities assigned. Report preparation guidelines will be provided and discussed. The team will review the proposed outline of the report (Annex B) and agree on any revisions with the CTO. b. Interviews While this assessment incorporates a variety of methods, emphasis should be given to consultations with key informants. The broad set of clients and contacts spread throughout the developing world and beyond poses special challenges for information gathering. USAID will provide a list of organizations and individuals for the team to interview (Annex C). Consultants will gather information from: • representatives from the PHRplus Project, leading experts in the field and USAID staff – both within the Offices of HIV/AIDS, PRH, and HIDN as well as in missions, regional bureaus and other offices within the Bureau for Global Health, • host country policymakers, and • technical personnel and representatives from other organizations and foundations, including the World Bank, WHO, universities and think tanks working on A–8 2 A consultant has been contracted by POPTECH to assist USAID in drafting the self-assessment questionnaire and the mission surveys. The consultant will compile mission responses into a report. developing country health system performance issues. In addition to USAID’s PHRplus project, its predecessors and other USAID health system projects (including M&L, DELIVER, QAWD, RPMplus, POLICY), other sources of our growing understanding include the World Health Organization’s Evidence and Information for Policy unit, the World Bank’s sector and project work; universities, other institutions and projects such as those supported by SIDA (IHE) and DFID (IHSD), and initiatives such as the WHO Commission on Macroeconomics and Health. E) Field Visits The assessment team will conduct field visits in a minimum of three PHRplus implementation countries to do direct observation, review country-specific documents, and interview Mission, MOH and implementing partners on the ground. Criteria for country selection will be based on the amount of field support, length of time project has been present in-country and scope of activities. The probable candidate countries include Albania, Egypt, Jordan, Tanzania, Ghana and Peru. IV. PROPOSED LEVEL OF EFFORT It is estimated that up to six weeks of effort will be required for each of the POPTECH consultants, and possibly an additional two weeks for the team leader. The consultants will perform some of the work at home prior to the team’s arrival in Washington, D.C.. The consultants are authorized to work a six-day week when in the field. The assessment will begin in mid-July. A total of six weeks will be needed for data collection and report writing, and approximately 13 weeks to complete the entire assignment. V. TEAM COMPOSITION The assessment team will consist of 3-4 professionals who have the qualifications and expertise described below. One will serve as the team leader. Health Sector Reform Specialist: • Advanced degree in public health, business management, economics or other relevant course of study • Expertise in the design, implementation and/or assessment of substantial health sector reforms in developing countries • Experience managing international health technical assistance, policy dialogue • At least 10 years of experience in health sector reform • International reputation in the field of health reform desirable • Excellent oral and written communication skills in English • Demonstrated knowledge of USAID’s policies and priorities in PHN • Skills in designing qualitative research instruments and methodologies • Regional expertise A–9 Economist: • Advanced degree in economics • Track record of successful oversight of complex international technical assistance projects, preferably in health • At least 5-10 years of experience providing economics TA, ideally in health • Experience/credibility in USAID priority program areas • Understanding of how international donors operate • Regional expertise Management and M&E Specialist: • Advance degree in public health, business management, economics or other relevant course of study • At least 5-10 years of experience managing and conducting M&E in developing countries • Expertise in project management and management systems • Expertise in monitoring and evaluating health sector reforms in developing countries • Expertise in health reform research may be helpful • Skills in designing qualitative research instruments and methodologies • Experience/credibility in USAID procurement process • Regional expertise VII. DELIVERABLES A) Debriefings The assessment team will conduct separate debriefings for USAID and PHRplus team in Washington, D.C. to discuss major findings and recommendations. B) Draft Assessment Report The draft assessment report will be submitted to the CTO on or about August 25, 2004. The CTO will share the draft report with the TA and PHRplus team for corrections and comments, and will consolidate the collective feedback to the team leader. The draft assessment report will follow the Report preparation guidelines, contain clear findings, conclusions and recommendations, and address the priority questions above. The draft will be submitted in PDF format via email and, if so requested, in hard copy. C) Final Assessment Report The final assessment report will be no longer than 30 pages total, excluding Annexes (Times New Roman font 12 point). The report will follow the attached outline, and any modifications to the outline will be discussed with USAID/GH/HIDN. The report will be a public report, edited by POPTECH, with 20 hard copies and 20 CD-ROMS delivered to the CTO. A–10 VIII. FUNDING AND LOGISTICAL SUPPORT All funding and logistical support will be provided through POPTECH. POPTECH activities will include recruiting and supporting the assessment team (including travel, per diem and related team expenses), providing logistical support, including setting up meetings in Washington and the countries visited, possible translation and secretarial support, and producing and distributing the final report. The PHRplus Project team will assist POPTECH in making arrangements for the country site visits and the scheduling of in-country meetings. A–11 Week Activity Weeks 1 – 2 (July 12 – 23 ) a. Preparation (three days) at home b. Arrive in Washington DC on July 18 c. Interviews with PHRplus team on July 20 – 21 d. Interviews with USAID & others on July 22 – 23 Week 3 (July 26 – 30) a. Depart for field visits on July 24 b. Field visit to selected mission Week 4 (Aug 2-6) a. Field visit to selected mission b. TL arrives in Washington DC on Aug 8 Week 5 (Aug 9 – 13 ) a. Field visit to selected mission b. Interviews with key informants & selected missions c. Additional interviews with PHRplus team Week 6 (Aug 16 -20) a. Interviews continued b. Team drafts report and prepares for debriefings Week 7 (Aug 23 – 27) a. Debriefing with USAID Aug 24 b. Debriefing with PHRplus Aug 25 c. TL submits draft report to USAID/CTO on Aug 27 Weeks 8 – 9 (Aug 30 – Sept 10) a. USAID/CTO sends consolidated comments to TL by Sept 10 TL revises draft report Weeks 10 – 12 (Sept 13 – Oct 8) a. TL submits final draft to POPTECH by Sept 20 b. POPTECH edits report c. POPTECH sends clearance copy to USAID/CTO by Oct 8 Week 13 (Oct 11 – 15) POPTECH prints and delivers final assessment report within 3 days of receiving clearance by USAID A–12 ANNEX A: SELECTION OF TASC I AND OTHER BILATERAL HEALTH REFORM ACTIVITIES TASC I Activities Task Order Date Contractor Country Task Order Completion Date Mission Task Manager Short Title 05/14/1999 MSH El Salvador 12/31/2001 Raul Toledo TA for Health Reform 05/04/2000 JSI Eritrea 12/31/2003 Linda Lou Kelley Health Reform TA 06/15/2000 MSH Philippines 01/31/2003 Marichi de Sagun Health Sector Reform TA 06/22/2000 JSI Morocco 09/30/2003 Taowfik Bakkali Decentralized health reform 08/15/2000 DA Senegal 08/15/2003 Matar Camara Health Financing (decentrzn) TA 02/15/2002 C&M Philippines 05/11/2002 Jed Meline Design on strengthening family Plannin & Health Servies through Local Governments Bilateral Health Reform Activities • Albania • Armenia • Central Asia – Zdrav Plus Project • Dominican Republic – REDSALUD • Philippines • Zambia – ZIHP Project A–13 Annex B: Draft Outline for Final Assessment Report I. Table of Contents II. Executive Summary (3 pages) – The Executive Summary should convey the important points of the report clearly and concisely. Because it may be distributed to a wider audience, it should be written as a stand-alone document which contains findings, conclusions and recommendations related to all priority questions listed in the scope of work. III. Background IV. Methodology V. Program Description VI. Summary Findings, Conclusions and Recommendations A. Results and Accomplishments B. Lessons Learned C. Future Strategic Directions Annexes A. Scope of Work B. List of Interviewees C. Summary of Mission Responses D. References A–14 Annex C: List of Key Informants “Luminaries” & USAID and Field Personnel Name Email Phone World Bank - Jacques Baudouy - Alex Preker - April Harding WHO - Kei Kawabata - Tim Evans - Surveillance contacts? PAHO - Daniel Lopez Acuna - Hernan Montenegro GAVI - Steve Landry - Logan Brenzel GFATM - DFID - Stuart Tyson Swedish SIDA - Par Erikson IHSD - Veronica Walford? Other - Richard Skolnik - Bill Savedoff USAID/Washington staff CTO team HSD? SOs • Margaret Neuse, Tanvi Pandit, Joan Robertson, Ellen Starbird • Mary Ellen Stanton • Murray Trostle, Susan McKinney, Ellyn Ogden, Al Bartlett • Dennis Carroll, Mary Ettling • Vic Barbiero, David Stanton, Estelle Quain, Amanda Gibbons • Richard Greene Regional Bureaus • Kelly Saldana • Billy Pick, Andrew Clements, Sonali Korde • Forest Duncan • Hope Sukin, Ishrat Husain A–15 APPENDIX B PERSONS CONTACTED PERSONS CONTACTED UNITED STATES Centers for Disease Control and Prevention Helen Perry,* Team Leader, Integrated Disease Surveillance and Response (IDSR), National Center for Infectious Diseases George Washington University Richard Skolnik, Director, Center for Global Health Inter-American Development Bank Alfredo Solari, Senior Health Adviser PHRplus, Bethesda, Maryland Nancy Pielemeier, Project Director Sara Bennett, Head, Applied Research Paurvi Bhatt, SO 4 Team Leader Manjiri Bhawalkar, Asia–Near East Regional Coordinator Catherine Connor, Deputy Project Director Susna De, National Health Accounts Coordinator Yann Derriennic, Technical Adviser, Systems/Financing Mary Diehl, Peru Coordinator Tania Dmytraczenko, SO 1 and 2 Team Leader Zuheir Al-Faqih, Senior Associate, International Health Lynne Franco, Technical Adviser Jack Galloway, Latin America/Caribbean Coordinator Allison Gamble Kelley, Associate Regional Coordinator, West and Central Africa Debbie Gueye, Tanzania Team Leader David Hotchkiss, Head, Monitoring and Evaluation Team Natasha Annual, Associate Technical Officer, SO 3 Marty Makinen, SO 3 Team Leader Steve Mason, Finance and Administration Manager Kathleen Novak, Infectious Disease Surveillance Coordinator Melinda Ojermark, Regional Coordinator, East and South Africa Nadwa Rafeh, Chief of Party, Egypt Julie Urban, Albania Coordinator Social Insight William Savedoff,* Director Social Sectors Development Strategies, Inc. Margaret Saunders,* Research Associate Training Resource Group Margaret Morehouse,* PHRplus Senior Adviser Note: An asterisk (*) denotes that the interview was conducted by telephone. B–1 USAID/Washington Richard Greene, Director, Office of Health, Infectious Diseases, and Nutrition (GH/HIDN) Margaret Neuse, Director, Office of Population and Reproductive Health (GH/PRH) Vic Barbiero, Chief, Implementation Support Division, Office of HIV/AIDS (GH/OHA) Al Bartlett, SO 3 Team Leader Dennis Carroll, Head, Infectious Diseases Karen Cavanaugh, Cognizant Technical Officer (CTO), PHRplus Project Andrew Clements, Bureau for Asia and the Near East (ANE) Robert Emrey, Supervisory Health Development Officer, GH/HIDN Emily Gardiner, PHRplus CTO Team Ishrat Hussein, Senior Technical Adviser, Bureau for Africa Irene Koek, Chief, Infectious Disease and Environmental Health Division, GH/HIDN Sonali Korde, ANE Bureau Susan McKinney, SO 3 Team Ellen Ogden,* SO 3 Team Tanvi Pandit, SO 1 Team Billy Pick, ANE Bureau Estelle Quain, Human Capacity Development Adviser, GH/OHA Mark Rilling, SO 1 Team Joan Roberston, SO 1 Team Kelly Saldana, Bureau for Latin America and the Caribbean (LAC) Russ Scarato, PHRplus CTO Team Elizabeth Schoenecker, CTO, POLICY Project Mary Ellen Stanton, SO 2 Team Leader Ellen Starbird, SO 1 Team Murray Trostle, Head, Integrated Disease Surveillance and Response (IDSR) World Bank Institute Tom Merrick* ALBANIA Health Insurance Institute Gjergji Leka, General Director Elda Shera, Deputy Director INSTAT Milva Ekonomi, Director General Dhimiter Tole, Director of Information Technology Institute of Public Health Eduart Takariczi, Director, Department of Epidemiology Ministry of Health Eduart Hashorva, Deputy Minister Sokol Dede, Ministerial Adviser on Sector Financing B–2 Doctor-in-charge: Ndroq Primary Care Center Havaleas Primary Care Center Kashar Primary Care Center Llukan Prifti Primary Care Center Muzakaj Primary Care Center Dervish Hoxha, Director of Public Health, Berat District Vladimir Kumaraku, Director, Berat District Hospital Neritan Kurteshi, Director of Primary Care, Berat District Arqile Mjeshtri, Director of Primary Care, Kucova District Agim Shehi, Director of Primary Care, MOH PHRplus Margaret Cook, Chief of Party Altin Malaj, Technical Officer Zamira Sinoimeri, Deputy Chief of Party Jan Valdelin, Former Chief of Party University Research Co., LLC (URC) Victor Boguslavsky, Chief of Party USAID Zhaneta Shatri, Project Specialist, Health PERU Andean Health Organization Mauricio Bustamante, Director CATALYST Project Milka Dinev, Country Representative Ministry of Health Pilar Mazzeti, Minister Eduardo Zorilla, Vice Minister Janeth Bouby, Advisory Group on Decentralization Lyda Desulovich, Advisory Group on Decentralization Eva Guerrero, Advisory Group on Decentralization Percy Minaya, Advisory Group on Decentralization Luis Podesta, Hospitals Division Lambayeque Regional Government Nery Saldarriaga, Vice President Regional Health Council Marco Burba, Communication Unit Coordinator Jorge Campo, United Nations Representative Miguel Gonzalez, Member Carmen Gutierrez, Dean of Medical School B–3 Luis Heredia, Member Ronald Montes, Member Elizabeth Neciusup, Social Insurance Organization Juan Samillan, Regional Health Accounts Beatriz Solis, Chiclayo City Council Victor Torres, Management Information Systems Miguel Vega, Member Regional Health Directorate Alejandro Cabrera, Director Enrique Altamirano, Management Information Systems Roberto Larrea, MCHC Director Jolonny Portocarrero, Nutrition Elsa Reyes, Economist Flor Rojas, Epidemiologist Juanne Rojas, CRS Vilma Salazar Raquel Sanchez, Legal Advisor Angel Seruza, Management Unit Luis Solano Irma Suarez, Council Secretary Luis Teran, Environmental Health Antonieta Zuloeta, Planning Office Lima Municipality Victor Choquehuaca, Health Director Edith Matias, Health Advisor Walter Menchola, Member of Regional Health Board PHRplus Midori de Habich, Chief of Party Ricardo Alania Gustavo Arica Javier Linares Miguel Madueno Nancy Ordinola Ada Pastor Alfredo Sobrevilla Oscar Ugarte Susana Valle POLICY Project Patricia Mostajo, Country Representative Public Health Insurance Organization Moises Acuna, Chief USAID Richard Martin, Chief, Population, Health, and Nutrition (PHN) Office Susan Thollaug, Deputy Chief, PHN Office B–4 Luis Seminario, PHRplus CTO TANZANIA Ministry of Health Everest Manumbi, Acting Permanent Secretary Ali Mzige, Director of Preventive Services Mohamed Seha, Head of Epidemiology Unit Joseph Baynit, Director, Babati District Hospital Resituta Gama, Health Officer, Dodoma Rural District Edward Ganja, Regional Health Officer, Dodoma Region Catherine Kalege, Matron, Hanang District Hospital Elicia Kishimbo, Health Officer/IDSR Trainer, Dodoma Rural District E. Kombe, Regional Health Officer, Manyara Region Gift Kumbakumba, District Health Secretary, Babati District Festo Masai, District Medical Officer, Hanang District Silivester Mboya, District Reproductive and Child Health Coordinator, Hanang District Tom Mtoi, District Medical Officer, Babati District Joshua Muna, Head of District Laboratory, Hanang District Emanual Mziwanda, District Health Officer, Dodoma Rural District Theobury Ngowi, District Health Officer, Hanang District Stanislaus Sinkazi, Health Officer, Hanang District Crispin Shayo, Regional Health Officer, Arusha Region Joseph Suluo, District Community Health Fund (CHF) Coordinator, Hanang District S. M. Toure, Regional Medical Officer, Manyara Region Centre for Educational Development in Health, Arusha S. S. Ndeki, Principal R. A. Mboya, Lead Trainer Commonwealth Regional Health Community Secretariat Mark Bura, Coordinator, Health Systems Development National Institute for Medical Research Peter Mmbuji, Program Manager, IDSR Leonard Mboera, Senior Scientist, IDSR Team PHRplus Kathy Banke, Technical Leader, IDSR USAID Patrick Swai, Project Management Specialist, Health World Health Organization (WHO) Mohamed Amri, Disease Prevention and Control Officer United Kingdom Institute for Health Sector Development Veronica Walford,* Director B–5 APPENDIX C USAID STRATEGIC FRAMEWORK AND ORGANIZATIONAL CONTEXT FOR THE ASSESSMENT USAID STRATEGIC FRAMEWORK AND ORGANIZATIONAL CONTEXT FOR THE ASSESSMENT Figure C–1 Organizational Structure of the Bureau for Global Health AAAA//DDAAAA Office of Professional Development and Management Support Office of Professional Development and Management Support OOff fificcee o off HHII V/AV/AIIDDSS Office of Population and Reproductive Health Office of Population and Reproductive Health Office of Regional and Country Support Office of Regional and Country Support Office of Health, Infectious Diseases and Nutrition Office of Health, Infectious Diseases and Nutrition Office of Strategic Planning, Budgeting and Operations Health Systems Division Policy, Evaluation and Communication Division SO1 SO4 SO2 SO3 SO5 The above schematic shows the overall structure of the Bureau for Global Health (GH), its four offices, and the organizational location for each of GH’s Strategic Objectives (SOs): ƒ SO 1: Advance and support voluntary family planning and reproductive health programs worldwide; ƒ SO 2: Increased use of key maternal health and nutrition interventions; ƒ SO 3: Increased use of key child health and nutrition interventions; ƒ SO 4: Increased use of improved, effective and sustainable responses to reduce HIV transmission and mitigate the impact of the HIV/AIDS pandemic; and ƒ SO 5: Increased use of effective interventions to reduce the threat of infectious diseases of major public health importance. Three of these SOs (2, 3, and 5) are the responsibility of GH/HIDN. SO 1 is covered by GH/PRH, and SO 4 is covered by GH/OHA and the President’s Emergency Plan for AIDS Relief, based in the U.S. Department of State. Each SO has its own funding stream and since PHRplus comes under GH/HIDN, it receives support for carrying out the GH common agenda activities, which fall within its scope from SO C–1 2, 3, and 5 core fundsdefined as common agenda core funding. In addition, all SOs (including SOs 1 and 4) can specify and fund PHRplus activities that they consider to be directly supportive of their resultsdefined as directed core funding. GH/PRH, GH/OHA, and GH/HIDN each have three subsidiary divisions (not shown). The Office of Regional and Country Support has four region-specific support teams and a strategic support team (not shown). The Health Systems Division has a GH–wide scope, as does the Policy, Evaluation, and Communication Division; one reports for administrative purposes to GH/HIDN, the other to GH/PRH. The Health Systems Division is the home of the Health Policy and Systems Strengthening (HPSS) Results Package; the SO and five Intermediate Results in that package are described in appendix A. PHRplus is managed through the Health Systems Division and contributes directly to the HPSS Results Package. C–2 APPENDIX D SUMMARIES OF THE FIELD VISITS SUMMARIES OF THE FIELD VISITS I. FINDINGS FROM THE ALBANIA VISIT DATES OF VISIT July 25–30, 2004 TEAM MEMBERS VISITING Tina Cleland, Peter Connell, Jaime Arias PHRplus ACTIVITIES AND FUNDING The government of Albania has a strategic plan for the health sector, “Health System Strategy 2000–2010 for Albania,” that identifies an integrated approach to personal health services. At the Mission’s request, PHRplus oriented its project in Albania toward supporting and enhancing the government’s vision. It specifically provides technical assistance to strengthen primary health care. The objective is to demonstrate a sustainable, primary health care delivery model in two pilot districts (Berat and Kucova), with one rural and one urban health center from each district participating. Based on the experience in the pilot sites, the model will be modified as appropriate and replicated across Albania. The three-year program (October 2001 to October 2004) has interventions in four interdependent areas: ƒ primary health care service delivery, including training; ƒ quality assurance and regulation; ƒ health information systems; and ƒ finance, planning, and budgeting, including community participation. Albania is also an intensive demonstration site for PHRplus. The project’s monitoring and evaluation unit has completed a baseline survey and is conducting a number of studies to make it possible to measure the impact of project interventions. The project is cofunded by core and field support funds. Total obligations since the beginning have been $5.3 million, of which $253,590 for the demonstration site work has come from core monitoring and evaluation funding. Over the life of the project, the budget has included funding for two resident, long-term advisers, one residing in the pilot districts and the other located in Tirana. At the time of the field visit, the project was starting to phase out and only one resident adviser was still on staff. Along with the resident advisers, the project uses both expatriate and local consultants in the area of quality assurance. The Albanian Institute of Statistics is a subcontractor. About a year before the assessment, USAID/Albania awarded a bilateral project to the University Research Co., LLC (URC) to advance health sector reforms, building on and expanding the work of PHRplus. URC’s transition plan has not yet been developed and it is of concern that the experience of the project will possibly not be incorporated into future health reform efforts in Albania. To provide a smooth transition, PHRplus has D–1 been extended to March 2005. The project seems to be making every effort to inform URC about progress to date to facilitate transition follow-on planning. RESULTS TO DATE The project has developed an excellent approach to linking its interventions and results to the USAID Mission’s Results Framework. Unlike many projects, a baseline survey was conducted at the beginning of project implementation because the project had been identified as an intensive demonstration site. Building on the baseline, the project identified a good set of indicators for tracking various aspects of the changes in the delivery system that it hopes will occur as a result of the interventions. Even though no targets were set, it will be possible to see trend lines regarding the use of services. Some of the indicators are oriented to yes/no responses. This is a reasonable approach in the earliest stages of project implementation if the indicators become more differentiated as experience evolves. Indicators are identified for all four areas of project activity but most relate to primary health care service delivery. The return on investments in system strengthening will be much higher where USAID gains evidence regarding what has and has not worked. The impact study planned for Albania has the potential to provide much needed insights that can guide future strategy development. Primary Health Care Service Delivery The team visited three of the four pilot health centers (Muzakaj, Havaleas, and Llukan Prifti) and three government facilities not in the pilot program for comparative purposes. With the exception of one of the nonpilot clinics, all four had been refurbished and were clean. However, patients tended to be in the clinics only in the morning hours. In the facilities taking part in the pilot program, the team found that the providers had been trained in the use of clinical practice guidelines (CPG) for primary care services and in modern methods of medical recordkeeping. The new medical records system has been implemented and is organized by family unit. Providers and midwives had also been trained in education and counseling about modern family planning methods. Discussions with providers revealed that they were all pleased with the training but it was sensed that they had not taken full ownership of the approaches as yet. For example, during an interview with three physicians at one health center, a spirited discussion was witnessed in response to a question about whether they thought the CPGs had improved their practices. Finally, the senior physician responded that they all agreed that the guidelines had improved their practice. On another occasion, a physician told the team that it was not necessary for her to record all of the information required by the new medical records system because she knows all the families and remembers their circumstances. In preparation for phaseout, the project has transferred its technical assistance materials and functions to the government’s field office and has facilitated collaboration between the offices responsible for public health, primary care, and regional health insurance management. D–2 Quality Assurance and Regulation The project collaborated with physicians from the United Kingdom and the Chief of the Family Medicine Department of the University of Tirana Medical School to develop a curriculum and train general practitioners in family medicine. Twenty-five CPGs were produced, providing primary care physicians with an accurate and up-to-date clinical management plan for a priority list of common clinical conditions. As CPGs are introduced into training programs, feedback from practicing physicians guides the revisions and adaptations to the Albanian setting. Working through the Department of Family Medicine, the project held training programs on CPGs for practicing physicians. Eighteen general practitioners were trained in weekly sessions as part of a 10–week course. The team met with the department chief to discuss his department’s role in the introduction of CPGs. It was learned that he had been involved in this initiative since its beginning, and as a result, he was personally committed to full implementation. He has already planned to incorporate the guidelines into the standard curriculum. By working through a credible local institution, it is expected that the impact on quality of care will be sustainable. To address the need to provide training of nurses in modern practices, the project worked with the Viore Nursing School. Forty nurses and midwives completed the series of weekly courses. Quality boards have been organized to review the level of compliance with the CPGs, based on the results of medical chart audits. The team interviewed some prominent members of the boards and found them giving high praise to PHRplus but not responding to questions about the deliberations of the boards. Since it was known that they had been regularly attending the board meetings, it was concluded that there was a breakdown in the translation process. Health Information System A primary care patient encounter form was introduced at health centers to make it possible to track service use and to estimate the costs of care. Forms designed by PHRplus are submitted to a local office of the MOH, where data entry is completed. Reports on service use have been produced on the basis of data collected through the patient encounter forms. The project now has information on well over 100,000 encounters, which is being used to assess changes in use since the beginning of the project. The team met with the data entry staff, found them knowledgeable, and was impressed with the ability of the system to detect incongruent information signaling any recording or entry errors. The unresolved issues at the time of the site visit included how to cover the cost for the encounter forms in the future, and who would have the technical knowledge to make system improvements or correct problems. At the present time, the project is relying on the project director’s spouse for volunteer information technology assistance. Finance, Planning, and Budgeting In October 2002, PHRplus completed an assessment of the organization and financing of primary health care in Albania. The analysis revealed that there was significant fragmentation of financing and management as a result of decentralization in 2000. A D–3 single payer model was proposed, which has been accepted in concept by the government of Albania, the Health Insurance Institute, and the World Bank. However, the proposal has not advanced beyond the concept stage because the project was not able to field test an alternative scheme. USAID/Albania is disappointed that PHRplus was not more assertive in convincing the government to issue a waiver, which would allow field testing of the approach. In a meeting with the head of the Health Insurance Institute, the team learned that he fully supports the PHRplus concept and is preparing to propose a new insurance law that reflects the PHRplus proposal. Perspectives on the Global Assessment PHRplus’s work in Albania contributes to four of the five HPSS IRs from the original contract: ƒ health sector reform (IR 1): project initiatives have strengthened local capacity to provide primary care and have promoted a more rational approach to primary care financing, ƒ health financing (IR 4): the PHRplus–sponsored analysis and proposal regarding health care financing has opened new options to the government, ƒ health information (IR 5): patient encounter forms and the analysis of use based on these forms will provide important management information, and ƒ quality, responsive services (IR 2): the introduction of CPGs and the training of providers contribute to improved quality of primary care service delivery. With respect to the thematic areas, Albania has focused most of its attention on improving the quality of care and the provision of more responsive health care at the local level. Thus, the project work in Albania touches both decentralization and quality/responsive services. Surprisingly, PHRplus did not include national health accounts (NHA) in its program. Albania initiated NHA with USAID assistance before the PHRplus effort. Given the strong capacity of PHRplus in this area, the team believed that it had missed an opportunity to help the country institutionalize NHA. On the nine assessment questions in the team’s scope of work, Albania provides insights on reform and health impact and market niche and global leadership. Reform and Health Impact In Albania, PHRplus is engaged in a classic example of health systems strengthening. The government reform plan provided the platform for the program, and the project designed an integrated program to demonstrate the systems needed for the reforms to be effective. The integrated program included medical management, information technology, financing, training, and public administration. Preliminary reports that were provided showed changes in the use of services, although not in all areas. It is possible to analyze the impact of the interventions because baseline data were collected at the beginning of the project’s tenure in Albania. D–4 Market Niche and Global Leadership In all the team interviews, it was clear that the project staff and PHRplus were very highly regarded within Albania. In particular, the government has been impressed that the primary care financing proposal developed by PHRplus appears to be rational and feasible. The fact that the Mission chose to award a bilateral project for future health sector reform work rather than continuing to use PHRplus indicates that the Mission believes there must be other organizations capable of performing the same work as PHRplus. Generally, PHRplus has a niche in health care financing and the extent of its work in quality improvement in Albania is an aberration. This is not to detract from the accomplishments, which have been substantial. People are being inspired to think differently about patient care, and a lasting impact has been made on the way that educators will teach primary care skills to physicians, nurses, and midwives in the future. Through the intensive demonstration site experience, the Albania project should also provide an excellent resource to inform future health system strengthening initiatives. D–5 II. FINDINGS FROM THE PERU VISIT DATES OF VISIT August 8–15, 2004 TEAM MEMBER VISITING Jaime Arias PHRplus ACTIVITIES AND FUNDING USAID/Peru began buy-ins to PHRplus in the third quarter of 2002, asking for exploration of possible health sector reform activities that might continue work carried out by the predecessor PHR project between 1995 and 2000. Total funding to June 2004 has been US$ 3.1 million. The main activity has been support for decentralization, carried out at both the center and in four northern regions. Expected results include improved capabilities of regional governments to exercise decentralized competencies and to conduct and monitor participatory strategic health planning based on evidence and needs, and increased efficiency in health services to the poor. RESULTS TO DATE Decentralization Peru began a decentralization process from central to regional governments in the early 1990s, based on a constitutional amendment that was later reversed by President Fujimori. The new Toledo administration has defined decentralization, including the health sector, as one of its main policies. PHRplus conducted an initial study of supply and demand for health services that helped both the MOH and the project team identify the problems and issues to be addressed. PHRplus has conducted the following work: ƒ at the center, supported the MOH in the development of an analytical framework for decentralization, achievement of consensus on objectives, outline of the overall health sector reform process, and mapping of competencies, functions, and responsibilities to be shared by the different levels; and ƒ in the four pilot regions (La Libertad, Lambayeque, San Martin, and Ucayali) provided capacity building; technical assistance and training to the regional authorities, the health services directorates, and the regional councils; and assistance in 12 municipalities organizing demonstration projects that link MOH/Essalud hospitals to health centers. One of the initial activities was to develop a system to identify evidence-based health needs. Regional health accounts and analyses of epidemiological data are now used by the Regional Health Council and are complemented with community decision-making (through ballots) on the priorities that should be adopted in the health plans. D–6 Other Health Sector Reform Initiatives A number of other activities have been conducted to support specific government projects or ongoing programs that assist the decentralization process. A pilot public health networking project, which links a hospital to a health center in Trujillo, La Libertad, will help define the respective roles and services to be provided by hospitals and health centers. Cost studies, management information systems (MIS) analysis, and development of a referral and referral/return system are also included. PHRplus is providing technical assistance to the Integrated Insurance System (SIS) to design a user identification system that allows identification of the poorest clients. In Lima, the project is helping the regional authority strengthen the city’s emergency services by creating a new Prevention and Referral Center for Emergency Services. PHRplus is also working with the MOH on the Mandatory Transit Accidents Insurance scheme. Activities in the area of social sector policy were dropped. To assist with capacity building, PHRplus has held a series of policy meetings in Lima, which different stakeholders attend, and the MOH participates in discussion of issues related to health reform. Nonhealth Sector Reform Activities In coordination with two other USAID projects (CATALYST and POLICY), PHRplus has designed and is conducting a series of courses in health services management in association with Universidad del Pacifico. These courses aim to cover more than 500 individuals from the four selected regions. A number of on-the-job training programs are being carried out in each region and facility. Perspectives on the Global Assessment PHRplus’s activities in Peru mainly cover IR 1, since the bulk of interventions are concentrated in health sector reform, decentralization, and health planning. Work on regional budgeting and using regional health accounts is a contribution to IR 4 (health financing); designing information systems at hospitals and regional directorates is related to IR 5 (health information); and training local workers at sites and designing an emergency services fall under IR 2 (quality, responsive services). PHRplus has established a strong technical leadership role in-country on the topic of decentralization: supporting policymaking, responding to the MOH and regions on key reform questions, organizing technical meetings to discuss reform issues, building networks, developing education and training programs, documenting activities, and following up reform progress. Under training and capacity building, the project has devised a very comprehensive plan that covers individuals and institutions in all four regions as well as in central institutions. On the questions posed in the scope of work, there are five observations. ƒ The project will make substantial contributions to health sector reform, assuming consistent follow through by the Peruvian government. The four selected regions are convinced that decentralization should be implemented, and at least four other regions are willing to join this effort. D–7 ƒ PHRplus has a clear niche in Peru, although other contractors may exist. The MOH and USAID consider that the project should develop into a research group that maintains the institutional memory of health sector reform and helps the country solve some difficult specific technical problems, which are currently hindrances to progress in the reform process. ƒ Regarding project management and host country capacity, working through a local team that is composed of talented Peruvian professionals is an important, positive lesson. The project is well respected at the central and regional levels; it has become a permanent consulting resource for the MOH, SIS, and Lima municipality in the health sector reform area. ƒ PHRplus is providing excellent country capacity building at the central level, in the four regions, and in a number of facilities. ƒ The project has been more responsive than strategic. In a country such as Peru, with already good intellectual and institutional capacity, it is difficult to offer packaged solutions that are outside the country’s agenda. D–8 III. FINDINGS FROM THE TANZANIA VISIT DATES OF VISIT August 1–6, 2004 TEAM MEMBERS VISITING Jaime Arias, Tina Cleland, Peter Connell PHRplus ACTIVITIES AND FUNDING There have been four main activities to date affecting Tanzania: ƒ improvements to the national integrated disease surveillance and response (IDSR) system in 12 pilot districts; ƒ development of new financial management tools for community health funds, piloted in Hanang district; ƒ study of the impact of HIV/AIDS on the operation of community health funds, again using Hanang as the study site; and ƒ assistance to the MOH in preparation of a sustainability plan in support of its application for Global Alliance for Vaccines and Immunization (GAVI) funding. Total funding since the beginning of the project has been approximately US$ 4.7 million; $2.2 million came from the Tanzania Mission, $1.8 million came from directed core funding, and $700,000 came from the Regional Economic Development Office for Eastern and Southern Africa (REDSO). IDSR accounted for 85 percent of the total. Tanzania is unusual in not having a PHRplus resident presence, given the size and range of its activities. The National Institute for Medical Research (NIMR) is the subcontractor for IDSR work; the rest is conducted by visiting staff from Bethesda or consultants. RESULTS TO DATE Assistance with IDSR Improvement PHRplus/NIMR’s work on IDSR focuses on two results: enhancing system capacity to collect, process, and analyze data in a complete and timely manner, and ensuring that data are used appropriately for planning and response. It was found that the 12 districts have markedly improved their ability to collect data on the 13 priority diseases. All 12 are now reporting using the standard forms, although reporting remains less than 100 percent complete in some districts on either a weekly or monthly basis. Seven diseases are supposed to be reported weekly (acute flaccid paralysis, cholera, measles, meningitis, plague, rabies, and yellow fever), and six are to be reported monthly (bacillary dysentery, diarrhea, malaria, neonatal tetanus, pneumonia, and typhoid). The need for timeliness has been well established, and the country is on a path to improvement. Diagnosis of these 13 D–9 diseases is apparently now fairly accurate, with most suspected cases being confirmed through secondary investigation, often using laboratory analysisalthough laboratory facilities remain too sparse for reliable confirmation in most districts. Most of the reporting forms that the team reviewed recorded a nil response on all diseases except diarrhea, malaria, and pneumonia. It appears that the response capability remains weak because staffing and financing remain major constraints. As explained to the team, facilities reporting more than the threshold number of suspected cases (set at one case for most of the diseases) are required to conduct their own secondary investigation, and their district office is also expected to conduct its own confirmatory visit, in addition to laboratory confirmation, where feasible. Most of this is reported to be happening fairly reliably, and all MOH respondents agreed that diagnostic skills down to the lowest facility level (dispensary)aided by the PHRplus/NIMR training programare adequate for the identification and reporting tasks. But by far the dominant number of cases comes from malaria and pneumonia, where reporting is only monthly and the response capability seems vestigial. Beds in the medical centers and district hospitals that the team visited were occupied mostly by adult malaria and pneumonia cases who were short-term inpatients, being administered limited drug doses to stabilize them before discharge. Little evidence was seen of an impact of the IDSR in the community below the health dispensary level. The PHRplus program has successfully trained a number of traditional healers in IDSR techniques, but community-level alertness is hampered by a lack of paid or volunteer health-oriented staff in villages. As a result, it is not clear that all cases are identified, since there remain many pockets of population well beyond the average 10 km distance from a health facility. Also, the facilities themselves seem to offer low-quality care, which would be unlikely to motivate client travel over long distances. Sixty percent of deaths among children under 5 occur without any contact with a health facility. There seems to be some ambivalence about the PHRplus/NIMR IDSR effort on the part of MOH staff, both at the center and in the field. The work is clearly valued, especially the 10–module training program. The refresher training that the program provides on diagnostic skills alone is very useful in the Tanzanian context. One district was found (not among the 12 pilots) to be independently conducting the training with its own resources. However, all attempts to compare the IDSR performance of NIMR districts with non–NIMR districts (where the preexisting MOH system still prevails) met a similar response: identification and reporting performances are similar. The team was unable to confirm this with external evidence, but it is conceivable that cases of cholera, for example, may well be as readily identified and reported in either location because of their rarity. This still leaves the response to malaria and pneumonia in particular as the major issue in both locations, and it is wondered if a sophisticated surveillance system can be justified if the response capability is so poor, as indicated by the continuing high incidence rates. Malaria kills 80,000 children under 5 (their main cause of death) and 20,000–45,000 adults in Tanzania annually. IDSR is not mentioned in the new USAID/Tanzania Country Strategic Plan. D–10 Despite these concerns, NIMR’s performance and its obvious high standing and good working relationships with the MOH, both centrally and in the field locations visited, are impressive. To expand the PHRplus/NIMR approach to Tanzania’s remaining districts, of which there are more than 100, will require districts to buy in (under the decentralized management system now in place), MOH endorsement of the approach based on the 12 pilots, and funding. NIMR in particular is working in all three areas; the Global Fund is a possible future funding source. MOH endorsement is currently stymied by the repeated failure of the IDSR Task Force to meet, but both NIMR and USAID are confident that this is temporary. Financial Management Tools for Community Health Funds Community health funds are becoming a formal part of the MOH’s policy on health funding. The funds are voluntary, and in Hanang District, where PHRplus has been working, families pay 10,000 Tanzanian shillings (T Sh), about US$ 10, for a year’s coverage for parents and children under 18. The alternative is to pay an all inclusive T Sh1,000 per visit to a dispensary or T Sh1,500 at a health center or district hospital, which have many exemptions, including family planning, maternal care, children under 5, and many adult chronic diseases. The funds are not insurance but rather prepayment, with a risk of overpayment if family visits to a facility are less than about 10 annually, and a benefit if use is higher than this. Furthermore, there does not seem to be any properly recorded cost side to the fundsthey are simply revenue to the districts in the form of advance user feesso there are no concepts of surplus/deficit or mutual ownership. PHRplus’s limited role has been to assist in improving the management accounting aspects of the fund in Hanang (i.e., the design of membership registers, patient registers, facility-level daily status reports and financial ledgers, and monthly reporting formats). These have now been incorporated into 1 module of a 10–module training manual, which the Commonwealth Regional Health Community Secretariat (CRHCS) is preparing for the MOH with funding from USAID/REDSO. That manual will be used by the MOH to improve community health fund management in the 88 districts where funds have either been launched (44 districts) or district managers have been oriented (44). CRHCS may also use the manual in other community member states outside Tanzania. CRHCS has been pleased with PHRplus’s contribution, although the district medical officer in Hanang commented that PHRplus has been slow to contribute its component (CRHCS agreed) and that transparency (e.g., in communication between USAID and PHRplus) could be improved. Study of Community Health Funds and HIV/AIDS The team was unable to meet anyone in Hanang who could comment on this work. It is understood that the study protocol was agreed to only recently and that a PHRplus representative had recently visited Hanang to collect facility-level questionnaires sent out as part of the baseline assessment. Assistance with Sustainability Planning for GAVI The team was unable to meet with the Expanded Programme on Immunization program manager for the MOH, who is based outside Dar es Salaam. From information collected in Bethesda, it is understood that assistance has been provided to prepare a sustainability analysis in support of an application for GAVI funding. D–11 PERSPECTIVES ON THE GLOBAL ASSESSMENT PHRplus’s work in Tanzania is a good example of activities contributing to all five of the HPSS IRs named in the original contract: ƒ health sector reform (IR 1): both IDSR and community health fund efforts help strengthen district-level management capacity; ƒ health financing (IR 4): community health funds represent a new approach to revenue collection; ƒ health information (IR 5): IDSR is an essential information system component; ƒ quality, responsive services (IR 2): IDSR training has made a small contribution to improving service providers’ diagnostic skills; and ƒ commodities (IR 3): the GAVI work has enhanced commodity strategic planning. The IDSR work also represents good technical leadership in the country (task 1 in the contract), with one of PHRplus’s best developed products. NIMR is seen as being the technical leader as much as PHRplus, but this reflects the strong contribution that PHRplus has made to local capacity development (task 5). With respect to the thematic areas, PHRplus reports community health funds under the heading of community-based health financing, but the community element is weak at best and community health funds should not be confused with the West African mutuelles, which seem to have a much stronger role in promoting service use by the poor. As in Albania, PHRplus is acting well in support of decentralization, but its role is not strategic. This is a disappointment in both countries for USAID’s flagship health sector reform project in such a critical element of reform. The contribution is strong with respect to infectious diseases but marginal with respect to the other four GH SOs, although there may be useful lessons learned from the community health funds–HIV/AIDS study in the current year. PHRplus’s work in partnership with CRHCS is a good example of cementing global (or regional) alliances, although it was sensed that this was done more at USAID/REDSO’s behest than PHRplus’s initiative. CRHCS is also 30 years old, so this is a weak example of adapting to a changing market. On the nine questions in the scope of work, Tanzania provides four main insights. ƒ PHRplus is clearly working on systems strengthening, which already supports two elements of the Tanzanian government’s health sector reform agenda: decentralization and financing. However, the health impact is not apparent; at least one of the target diseases (malaria) remains a major killer. This could be a timing issue, but it seems that it is because of Tanzania’s weak response capabilities, especially in malaria and pneumonia, and especially for children under 5. There could be an important lesson about the priority order for the ‘S’ and ‘R’ in IDSR; a developing country needs a good, basic response D–12 capability before there can be much added value from investment in improved surveillance. ƒ PHRplus is in a clear health systems strengthening niche in Tanzania (information and financing) but in less of a health sector reform niche, because it is not engaged in policy and strategy setting at the MOH level. It is also not alone in its niche; CRHCS is at least as visible in sector financing, and the WHO is strong and influential in IDSR. PHRplus’s role is not sufficiently large that it would be critically missed at this stage. ƒ On balance, working through NIMR on IDSR has been cost-effective and may represent an important lesson learned from the projectthat working through the right local counterpart from the outset can be less expensive, still effective, and much more sustainable by developing capacity from the start. NIMR, and especially its PHRplus local counterpart team, are well informed and respected and can have a strong role in efficiently replicating the IDSR model nationally. This proven strength and further potential outweigh the criticisms heard of PHRplus’s slow response on community health fund workcaused at least in part by the NIMR relationship on IDSR having finessed a PHRplus residential presence in-country. ƒ Tanzania represents a highly responsive model for PHRplus, as perhaps most field projects do. PHRplus has been able to use its strategic position and knowledge internationally to develop models and products rapidly for the local market. It has been responsive to Mission and REDSO requirements, even though those requirements do not really add up to a strategic role in￾country. Above all, it has been working (in both IDSR and community health funds) to improve existing government of Tanzania systems: developing particularly a stronger skill and training base in IDSR as well as a stronger management capability for community health funds. While working on subcomponents of existing systems may be less satisfying for some of the staff, it has avoided prolonged selling of new policy concepts by an outside party and thereby facilitated speed and acceptance. D–13 APPENDIX E THE PRODUCT DEVELOPMENT LADDER THE PRODUCT DEVELOPMENT LADDER There has been considerable discussion of a ladder approach to both product development and country graduation in the Bureau for Global Health (GH). The concept is that a project such as PHRplus should be able to develop product concepts, refine and test them in a particular country, demonstrate them through country-level pilot programs, replicate them nationally as appropriate in the country, institutionalize them, and then be able to leave the country and move on to different products. The aim would be to have a well￾balanced portfolio of country activities at different stages on the ladder with different products. This is good for innovative thinking and for graduating countries once institutionalization is finished (and thereby finessing projects’ tendencies to remain in a country). The team agrees with this strategic concept, but has two additional thoughts prompted by the PHRplus assessment. First, ascent of the ladder is easier with a well-defined tool than with a less well-defined approach. Figure E–1 A Product Evolution/Country Graduation Matrix Product development Pilot Replication Institutional￾ization Consulting product Tool Zero investment Modest investment QUIC K SLOW VERY SLOW/E XPEN SIV E Product development Pilot Replication Institutional￾ization Approach Consulting product Tool Zero investment Modest investment Heavy investment QUIC K SLOW VERY SLOW/E XPEN SIVE Product development Pilot Replication Institutional￾ization Approach Consulting product Tool Zero investment Modest investment Heavy investment QUIC K SLOW VERY SLOW/E X PEN SIVE Product development Pilot Replication Institutional￾ization Approach Consulting product Tool D evelopment ladder Product evolution Little investment Modest investment Heavy investment QUIC K SLOW V E RY SLOW/E XPEN SIVE Country 1 Country 2 Country 3 E–1 In this report, tool is defined as being fairly mechanical and universally applicable (such as national health accounts [NHA]), a consulting product as being well codified but still needing adaptation to different country/client needs (such as community-based health insurance), and an approach as being broad knowledge but as yet relatively unpackaged (such as general cost analysis). It seems that a tool allows the fastest progression up the development ladder within a given country. It still may take two years to generate the data and develop a first set of NHA, but this is quicker than trying to wholly create mutual health organizations (MHOs) or have their management visibly improved. Therefore, projects such as PHRplus should be encouraged in their product development activities to have a number of tools in their portfolios. However, applying a single tool or product in one country does not constitute systems strengthening. A broad array is needed, and this takes time. Second, graduating a single country does not release GH from funding obligations; there will be other countries in which the product is demanded. Only after the product has been institutionalized in a sufficient number of countries can it feasibly be handed over to a third party to develop for additional recipients. Finding such third parties should be an important USAID objective and an integral part of the contractor’s role. PHRplus is doing well in this regard on NHA, being now well networked with various agencies who could be candidates to take over the NHA product. However, the lesson again is that passing responsibilities to a third partywhich ultimately allows USAID to stop funding and move on to new productsis much easier if the product has evolved into a tool already. Not all products can reach this stage, since they are not necessarily susceptible to mechanization or universal application. But it will be much easier to orchestrate a transfer if the product is highly packaged and therefore well documented and teachable. E–2 APPENDIX F A SYSTEMS–ORIENTED BUREAU FOR GLOBAL HEALTH A SYSTEMS–ORIENTED BUREAU FOR GLOBAL HEALTH SEGMENTING THE HEALTH SECTOR REFORM/ HEALTH SYSTEMS STRENGTHENING MARKET The team proposed that USAID convene a workshop to discuss the definition and segmentation of the health systems strengthening and/or health sector reform markets. As part of that workshop, it is anticipated that the issue of how to segment the market into useful parts will arise. This is linked to ƒ the much larger issue of whether GH wants to drive its development efforts through the current SO–based interventions or through systems; if it decides eventually to switch to systems as the driver, then segmenting the systems into useful subsets will become very important; and, ƒ the reality that a single flagship approach to systems strengthening is not feasible on sheer scale grounds. Against this background, a more systems-driven GH might look like the following: Figure F–1 A Possible Systems-Driven Activity Matrix for GH Sector Reform, Policy and Financing Information and Logistics Education and Training Facilities and Facilities Management Behavioral Change Reproductive Health and Nutrition Child Health and Nutrition Adolescent Health and Nutrition HIV/AIDS Infectious Diseases Chronic Diseases Health systems have been segmented into at least five parts: ƒ health sector reform, policy, and financing; ƒ information and logistics; ƒ education and training; ƒ facilities and facilities management; and ƒ behavioral change. There may well be other major segments (e.g., environmental health). It is recognized that research is omitted, but one suggestion is that applied research join reform, policy, and financing, while operations research is part of facilities management, and formative F–1 research is part of behavioral change. This particular segmentation was arrived at through brainstorming, assisted by a business-based approach to defining independent business units in a corporation. The clues for independence include independent sets of clients and competitors, grouping substitute activities into the same unit, and assessing divestibilityif an activity can be dropped without adverse impact on other activities, it can be seen as an independent activity for planning purposes. More thorough analysis in or before a dedicated workshop would no doubt result in an improved segmentation, although this seems to be an intuitively appealing outcome for this report’s purposes. ORGANIZATIONAL IMPLICATIONS OF A SYSTEMS–DRIVEN GH Were such an approach to USAID’s future activities ever to arise, it would also have organizational implications: Figure F–2 Organizational Implications of a Systems-Driven Approach in GH AAAA/DA /DAAA Client Health Division Client Health Division Service Delivery Systems Division Service Delivery Systems Division Other Other Program Strategy Division Program Strategy Division HSR, policy and financing group Information and logistics group Training and education group Facilities and facilities manage￾ment group Reproductive health group Child health group Adolescent health group HIV/AIDS group Infectious disease group Chronic disease group Strategic planning and research group Country needs assessment group Behavioral change group Country coordination group This would result in a wrenching change. An advantage of such rethinking is that it would provide a locale for the proposed country assessment work. It has been recommended that countries first be classified in terms of the type of health systems strengthening/health sector reform assistance needed and then be assessed in detail against a standard assistance package. The assessment should be handled objectively, possibly by the Office of Strategic Planning, Budgeting and Operations. If the organization structure were changed, then a purpose-designed unit, such as the country assessment group in figure F–2, could be created. F–2 A POSSIBLE COUNTRY CLASSIFICATION SYSTEM FOR HEALTH SYSTEMS STRENGTHENING WORK Consideration was given regarding how the classification scheme might work and to the standard assistance package. Table F–1 below provides an example of possible criteria for categorizing countries to determine the type of health systems strengthening activities that would be useful. The ultimate purpose of health systems strengthening goes beyond improving country capacity to meet the health needs of its population; health systems strengthening also helps countries achieve self-reliance in the management of their health care system. As countries become increasingly self-reliant, the role of the donor shifts to that of counselor. The table reflects the fact that general country development (economic, political, social) and health sector development are closely associated, and that even though a country may need assistance, it does not necessarily translate into an interest or capacity to receive help. An important criterion for health systems strengthening work must be probable impact or likely sustainability. For the category 1 countries, USAID’s objectives might be primarily humanitarian assistance rather than health systems strengthening, unless the motivation of country leaders to accept health systems strengthening support is high. Table F–1 Possible Country Classification Scheme and Sample Health Systems Strengthening Activities F–3 Classification Criteria Category 1: Rudimentary Health System (Example: Haiti) Category 2: Existing Health System Capacity, Beginning Reform (Example: Albania) Category 3: Active Effort to Improve Health System (Example: Peru) Country Socioeconomic Status Very low income, minimal development Low income, considering sector reforms Moderate low income, reforms actively pursued Per Capita Income Percentage of Poorest Percentage With Access to Sanitation Education Level Political Stability Under US$ 500 Over 70% Under 50% Illiteracy over 30% Minimal $500–999 50–69% 50–70% Illiteracy at 15–29% Acceptable $1,000–3,000 30–49% 20–49% Illiteracy at 5–15% Acceptable Health Sector Situation and Needs Undeveloped Medium Sophisticated with some problems Structure/Resources Minimal/very poor Number adequate but condition uneven Acceptable/some poor facilities System Performance Inadequate Acceptable but with general problems Acceptable but special problem areas Population Health Status Very poor Medium Medium to good Prevalence of Infectious Diseases High, with high mortality Transitional Transitional Access to Health Services Minimal More than 60% of population More than 80%of population Human Resources Minimal, lack basic skills Basic skills out of date Well trained, but in need of continuous education Quality of Service Poor Problems Specific problems Population’s Payment Capacity Minimal High percentage of income paid in out-of￾pocket payments General capacity to pay Management Capability Poor to minimal Limited training Acceptable/problems Understand Sector Reform Issues Not clearly Not completely Yes Classification Criteria Category 1: Rudimentary Health System (Example: Haiti) Category 2: Existing Health System Capacity, Beginning Reform (Example: Albania) Category 3: Active Effort to Improve Health System (Example: Peru) Accept Help/Guidance Not willing With conditions Yes Able to Respond At a low level With difficulty Yes Commitment to Reform Low Medium High Availability of Counterparts Nonexistent Not very good Excellent Capacity to Receive Assistance Minimal Acceptable Very good Feasibility of Successful Support Not feasible Feasible Very feasible Probable Outcome Not predictable May be good Good Impact/Effects Minimal Some specific Good chances Sustainability Difficult Some chances Many chances If USAID finds the concept of country categories interesting as an aid in targeting resources, the concept can be further developed with GH’s existing resources. This table and the suggested package of services below are provided as an illustration. POSSIBLE HEALTH SYSTEMS STRENGTHENING ACTIVITIES IN COUNTRIES IN EACH CATEGORY The activities described below address both health sector reform and health systems strengthening issues that might be appropriate areas for USAID assistance. The health sector includes the broad political, economic, policy, and structural context in which health services are delivered. Health system refers to the organization, management, financing, and provision of services to the population. The grouping of these interventions by category of country is meant to convey that health systems strengthening work is continuous and that there is a broad range of capacities that countries must develop to provide priority and sustainable services. Assistance with Health Sector Reform Assessment of Needs and Problems ƒ Surveys and other methodologies to establish population’s wanted needs ƒ Political mapping to describe policy influential institutions and individuals ƒ Epidemiological, service use, and financial data analysis ƒ Legislation and regulation analysis ƒ Focus groups Policy Development ƒ Policy analysis ƒ Consensus-building methods ƒ Framework of main reforms needed ƒ Policy advocacy ƒ Follow up on policy development Funding and Financial Structure and Broad Insurance Schemes ƒ Analysis/discussion of current funding structure, constraints, and opportunities F–4 ƒ Definition of structure and sources of funding and funding mechanisms ƒ Assessment of possible insurance policies to cover large populations ƒ Development of subsidized insurance for the poor ƒ Design and implementation of focusing mechanisms for the poor (user identification systems) Decentralization ƒ Mapping of responsibilities, competencies, and functions ƒ Design and implementation of new responsibilities at regions and localities ƒ Health needs assessment at regions and localities ƒ Health sector structure at decentralized regions and localities ƒ Budgeting, allocation, and financial management of decentralized funds ƒ Health planning for new competencies ƒ Design of supervision and surveillance systems for regional authorities ƒ Basic information and statistical systems for regions and localities ƒ Human resources planning and continuous educational policies ƒ Management tools and capabilities to direct and control new assignments ƒ Communication policies ƒ Community participation ƒ Council and health authorities’ leadership capability Central Authorities’ New Roles and Responsibilities ƒ Defining new roles and responsibilities ƒ Defining and implementation of control and surveillance systems ƒ Regulatory policies and methods ƒ Management of public health and other vertical programs ƒ MOH leadership and stewardship roles ƒ Coordination of key central-level institutions (MOH, social security, other) Assistance in Strengthening the Health Care System Financing ƒ Financial management ƒ Budgeting and control Information Systems ƒ MIS and clinical information (histories, other) Structure and Performance ƒ Reengineering processes ƒ Structural reorganization, staffing changes Management Facilities and Equipment Public Health Procurement and Logistics Education and Training Research and Evaluation Quality Assurance Systems F–5 Strengthening Operations at the Facility Level Facilities, Building (or Rebuilding), and Equipment Management and Financial Tools Staffing and Health and Human Resources Management; On-the-Job Training Quality Systems; Protocols Social or Other Marketing Planning and Controlling Package of Sector Strengthening Initiatives This approach assumes that USAID’s interventions will be complemented by capital investments from the World Bank. Category 1 Country: Focus on strengthening service delivery through the public system while offering opportunities for private sector growth. Map country capacity in the areas below (using existing evaluations by Missions and other donors, verified by onsite visits) and initiate pilot programs for testing approaches where there are gaps. ƒ Educational programs to be provided by academia and local training institutes would be designed for local and national country leaders, including legislators and academics. Topics could include • the intersecting functions within a health systemfinancing, organization, personnel, medicines, procurement, transportation, public health, sanitation, environmental factors, institutional development, personal behavior, and social security system; • national health accounts (NHA): result from other countries, action plans derived from NHA, and methodology for first time users; • burden of disease; and • social, economic, demographic, and health status trends. ƒ Public health information/education campaigns ƒ Identification and training of stakeholders; training in consensus building ƒ Methodology for targeting services for the poor. User identification systems as a starting point. Create a basic platform of capacity to produce information needed to manage health sector resources (distribution of services, staff, and funds). Information infrastructure to include reliable vital records system, surveillance and response systems, burden of disease by geographic area, service and drug use data, NHA with selected subanalyses, household survey data with income, expenditures, and health status and use updated every three years. Schedule for conducting needs assessments. ƒ Review of regulations and addition of provision for waivers to conduct demonstrations F–6 ƒ Locally adapted, evidence-based clinical practice guidelines integrated into local medical and nursing school curricula ƒ Cost analysis for treatment of conditions associated with leading causes of death and disability and other priority population issues, such as family planning ƒ Budgeting based on population, income distribution, and burden of disease as opposed to historical basis; basic accounting ƒ Analysis of roles, responsibilities, capacities, and functions of people and institutions. Focus on capacity of primary care health care providers and institutions to deliver priority services. Training to include balancing demands for patient care, recordkeeping and reporting, and clinic management, followed by training in the use of clinical guidelines for leading conditions. ƒ Management systems for pharmaceuticals, supplies, laboratories, and outsourcing ƒ Assessment of the physical condition of health care facilities at all levels Category 2 Country: Institutionalizing new methods for sector management and expanding opportunities for private sector growth. Build on foundation laid above and on experience derived from pilot programs. ƒ Training in a format for medical records (along with the provision of a supply of folders) ƒ Use NHAs to inform policy changes. Assistance to the MOH and legislators in formulating new policy directions. Institutionalizing NHA process. ƒ Framework for reform developed ƒ Legalization of NGO operations and friendly tax code to spark private sector activity. Eligibility of NGOs and private organizations to receive public funds for services provided ƒ Introduction of quality review committee structure and first-level quality improvement protocols ƒ Implementation of managerial cost accounting in hospitals ƒ Production of management reports merging use and cost data at various levels of the delivery system ƒ National health financing scheme where the level of government subsidy is based on need F–7 ƒ Evaluation of the ability to conduct health planning (using the information infrastructure described above) and sector rationalization and to implement decentralization ƒ Training in policy analysis and assistance in conducting sectorwide assessment to inform the sector reform agenda ƒ Passage of health sector reforms to increase efficiency and effectiveness of the health care system and equity of service delivery, including requirement for continuing medical education ƒ Licensing and accreditation of providers and licensing of facilities ƒ Training of trainers in the implementation of sector reforms to more effectively finance and deliver priority services Category 3 Country: Reforms adopted and implementation proceeding. Focus is on solving problems encountered in the implementation of reforms, identified through management information, or voiced by consumer, provider, or private sector groups. ƒ Strengthen the management of facilities, including physical plant ƒ Link payment system to productivity ƒ Support the country in implementation of reforms and information feedback system to monitor the impact of reforms F–8 APPENDIX G REFERENCES REFERENCES Banks, Dwayne, Mary Paterson, and Catherine Connor. “PHRplus Country Assistance Plan (CAP): Jordan.” Abt Associates, Inc., May 25, 2001. Connor, Catherine and Pia Schneider. “PHRplus Country Assistance Plan (CAP): The Philippines.” Abt Associates, Inc., June 3, 2003. Galloway, Jack, Ibrahim Shehata, and Natasha Hsi. “PHRplus Country Assistance Plan (CAP): Malawi.” Abt Associates, Inc., September 2001. Health Policy and Systems Strengthening (HPSS) Flagship Project: HPSS Management Plan. Abt Associates, Inc., January 5, 2001. Hotchkiss, David, Mary Paterson, Edward Kelley, and Sara Archibald. Internal Project Monitoring System Design and Implementation Report. December 2001. Luchitsky, Anton and David Mercer. “PHRplus Country Assistance Plan (CAP): Georgia Health Information and Disease Surveillance Reform CAP.” September 20, 2001. Novak, Kathleen, Francisco Vallejo, and Caroline Quijada. “PHRplus Country Assistance Plan (CAP): Honduras.” Abt Associates, Inc., August 2001. Novak, Kathleen and Midori de Habich. “PHRplus Country Assistance Plan (CAP): Peru.” Abt Associates, Inc., December 2003. Partners for Health Reformplus (PHRplus). Annual Reports, Years One, Two, and Three. Abt Associates, Inc., October 2001–September 30, 2003. __________. Performance Assessments One Through Three. June 1, 2001 to September 30, 2002. __________. Years Two through Four Implementation Plans. Abt Associates, Inc., October 31, 2001 to September 30, 2004. __________. “Year 3 Thematic Workplan: October 1, 2002 to September 30, 2003.” October 20, 2002. (draft) __________. Summary of PHRplus Funding and Expenditures to Date. Prepared for the assessment team, 2004. __________. “PHRplus Fourth Performance Evaluation: October 1, 2002 to September 30, 2003.” October 22, 2003. __________. “Client Relations Strategy for Field Support Activities.” August 2003. First draft. __________. “Client Relations Strategy for USAID Global Bureau.” November 2002. G–1 __________. “PHRplus Country Assistance Plan (CAP): Eritrea.” (No date) Paterson, Mary. “PHRplus Country Assistance Plan (CAP): Albania.” Abt Associates, Inc., August 27, 2001. Pielemeier, Nancy R. “Introduction to Health Systems and Reform Implementation.” PowerPoint Presentation, February 10, 2003. Posner, Stephanie, Anton Luchitsky, and David Mercer. “PHRplus Country Assistance Plan (CAP): Tanzania Infectious Disease Surveillance Assistance.” Abt Associates, Inc., October 2, 2001. Rafeh, Nadwa, Zuheir Al-Faqih, Gary Gaumer, and Catherine Connor. “PHRplus Country Assistance Plan (CAP): Egypt.” Abt Associates, Inc., January 2003. Setzer, James C. and Natasha Hshi. “PHRplus Country Assistance Plan (CAP): Democratic Republic of Congo.” Abt Associates, Inc., April 2002. Setzer, James C. and Chris Atim. “PHRplus Country Assistance Plan (CAP):” Ghana. Abt Associates, Inc., September 2001. Tien, Marie and Tammy Stewart. “PHRplus Country Assistance Plan (CAP): Kenya.” Abt Associates, Inc., September 2003. U.S. Agency for International Development. “HPSS Flagship Contract.” (No date) __________. “HPSS Results Framework.” (No date) Valdelin, Jan. “PHRplus Country Assistance Plan (CAP): Albania.” PHRplus Albania, April 10, 2003. G–2 POPULATION TECHNICAL ASSISTANCE PROJECT 1101 Vermont Ave., NW Suite 900 Washington, DC 20005 Phone: (202) 898-9040 Fax: (202) 898-9057 www.poptechproject.com