MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT REDUCING CHILD MORBIDITY AND STRENGTHENING HEALTH CARE SYSTEMS October 2007 This publication was produced for review by the United States Agency for International Development. It was prepared by Pinar Senlet, Robert Rosenberg, and Greg Becker through the Global Health Technical Assistance Project. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT REDUCING CHILD MORBIDITY AND STRENGTHENING HEALTH CARE SYSTEMS Pinar Senlet Robert Rosenberg Greg Becker October 2007 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 07-001-38) is available in printed or online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources/. Documents are also made available through the Development Experience Clearing House (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. ACKNOWLEDGMENTS The consultants would like to acknowledge the support they received from the staff of USAID/Malawi, the MSH Project, and the Ministry of Health in conducting this evaluation. The team expresses special appreciation to Catherine Chiphazi, USAID Child Health Specialist and the Activity Manager for the MSH Project, for her guidance and participation in the evaluation effort; and Rudi Thetard, MSH Project Chief of Party, for making himself, his staff, and project information so readily available to the team. Throughout this evaluation, we benefited greatly from the contributions of several dedicated health professionals. Among the most valuable contributions received were the insights offered by the MOH staff at all levels, whom graciously provided the team a better understanding of the realities of the health systems and the needs of the country. The team also greatly appreciates the contributions from representatives of the teaching institutions and non-governmental organizations who generously gave their time to talk with the evaluation team. Pinar Senlet Robert Rosenberg Greg Becker MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT i ii MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT ACRONYMS ACTs Artemisinin combination therapies ANC Antenatal clinic ARV Antiretroviral therapy CHAM Christian Health Association of Malawi CLINS Contract line items CMS Central medical stores COM College of Medicine COMREC College of Medicine Research Ethics Committee CPT Cotrimoxazole prophylactic therapy CTC Community therapeutic care CWW Concern World Wide DCOP Deputy chief of party DFID Department for International Development, United Kingdom DHIS District health information system DHMT District health management team DHO District health officer DHS Demographic and health survey DIP District implementation plan DOTS Directly observed treatment short course EHP Essential health package EPI Expanded program of immunization EPICS Electronic pharmaceutical inventory control system ETAT Emergency triage assessment and treatment GOM Government of Malawi GTZ German Technical Assistance Agency HIV Human immunodeficiency virus HMIS Health management information system HPSA Health Partners of South Africa HSA Health surveillance assistant IEC Information, education, and communication IMCI Integrated management of childhood illnesses IMR Infant mortality rate IP Infection prevention IPT Intermittent presumptive therapy IRS Indoor residual spraying ITN Insecticide-treated nets KCH Kamuzu Central Hospital LMIS Logistics management information system M&E Monitoring and evaluation MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT iii MCH Maternal and child health MICS Multiple indicator cluster survey MNH Maternal and neonatal health MOH Ministry of Health MSH Management Sciences for Health MTA Management technical assistant NAF National Action Framework NHSRC National Health Sciences Research Committee NRCM National Research Council of Malawi NRU Nutrition rehabilitation unit OFDA U.S. Office of Foreign Disaster Assistance ORT Oral rehydration therapy PEPFAR President’s Emergency Plan for AIDS Relief PHI Pediatric Hospital Initiative PMI Presidential Malaria Initiative (U.S.) PMP Performance management plan POW Plan of work QA Quality assistance QECH Queen Elizabeth Central Hospital QIST Quality improvement support teams PMTCT Prevention of mother to child transmission RETR Rapid expansion of treatment and rehabilitation of severely malnourished children SLA Service-level agreement SOW Scope of work SP Sulfadoxine-pyrimethamine SWAp Sector-wide approach TA Technical assistance TB Tuberculosis VCT Voluntary counseling and testing iv MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT CONTENTS ACKNOWLEDGMENTS.................................................................................................................................. I ACRONYMS.................................................................................................................................................. iii EXECUTIVE SUMMARY ................................................................................................................................ 1 I. INTRODUCTION AND METHODOLOGY ................................................................................................... 5 PURPOSE OF THE EVALUATION ........................................................................................................... 5 II. BACKGROUND.......................................................................................................................................... 7 III. OVERVIEW OF THE MSH PROJECT......................................................................................................11 IMPLEMENTATION APPROACH AND STRATEGIES............................................................................11 RELATIONSHIPS WITH USAID, MOH, AND OTHER PARTNERS.........................................................11 ORGANIZATIONAL STRUCTURE...........................................................................................................12 MONITORING AND EVALUATION (M&E) ..............................................................................................12 FINANCIAL MANAGEMENT....................................................................................................................12 IV. PROJECT COMPONENTS: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ......................15 HEALTH SYSTEMS STRENGTHENING .................................................................................................15 FINDINGS AND CONCLUSIONS.............................................................................................................15 PREVENTION AND MANAGEMENT OF CHILDHOOD ILLNESSES......................................................21 MALARIA PREVENTION .........................................................................................................................24 HIV/AIDS INTERVENTIONS.....................................................................................................................26 QUALITY ASSURANCE...........................................................................................................................29 CENTRAL HOSPITAL REFORM INITIATIVE ..........................................................................................31 V. FUTURE DIRECTIONS FOR USAID ........................................................................................................37 CONTINUED SUPPORT FOR STRENGTHENING HEALTH SYSTEMS ................................................37 CONTINUED SUPPORT OF USAID PROGRAMS THROUGH SYSTEMS STRENGTHENING .............38 APPENDIX A: SCOPE OF WORK ................................................................................................................39 APPENDIX B: PERSONS CONTACTED......................................................................................................51 APPENDIX C: DHMT INTERVIEW GUIDE ...................................................................................................55 APPENDIX D: MSH FIELD STAFF INTERVIEW GUIDE..............................................................................57 APPENDIX E: MSH TRAINEE INTERVIEW GUIDE .....................................................................................59 APPENDIX F: SITES VISITED......................................................................................................................61 APPENDIX G: REFERENCES ......................................................................................................................63 APPENDIX H: DRUG PRIORITIZATION MATRIX—A TOOL FOR RATIONAL PROCUREMENT DECISIONS................................................................................................................................67 FIGURES FIGURE 1: BUDGET BY USAID CLINS .......................................................................................................13 FIGURE 2: EXPENDITURE BY TECHNICAL AREA ....................................................................................13 FIGURE 3: PROGRESS TOWARD MEETING MANAGEMENT CRITERION..............................................19 FIGURE 4: PERCENTAGE OF IMR DECREASE IN MSH-SUPPORTED DISTRICTS AND OTHER DISTRICTS 2004–2006..............................................................................................................20 FIGURE 5: NUMBER OF PREGNANT WOMEN TESTED FOR HIV IN 8 MSH-SUPPORTED DISTRICTS 28 FIGURE 6: HIV TESTING COVERAGE OF NEW ANC VISITS IN MSH- SUPPORTED DISTRICTS ..........28 FIGURE 7: COMPARISON OF IP SCORES IN DISTRICT HOSPITALS......................................................30 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT v EXECUTIVE SUMMARY The objective of this evaluation was two-fold: 1) To review and analyze the impact of the Management Sciences for Health (MSH) project against project objectives and USAID/Malawi Strategic Objective 8 and evaluate its general contribution to child health in Malawi, and 2) To determine lessons learned and directions for future activities. The evaluation was conducted by a team of three consultants between July 9 and August 17, 2007, and included five weeks of in-country work. Informants from stakeholder organizations and health facilities were interviewed, and the team traveled to all eight districts the project had been working in, visiting 18 health facilities, including central and district hospitals and health centers. Both qualitative and quantitative data sources were used in the evaluation. MAJOR FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS The team found that the project implemented robust strategies to strengthen health systems and improve child health. It used a holistic approach to provide support for training, technical assistance (TA), materials, procurement, and renovations. The approach was flexible and bottom￾up, allowing timely response to local needs. Because all interventions were designed and implemented with full participation of the Malawi Ministry of Health (MOH), there was a high degree of local ownership. The project formed strong working relationships with the MOH and other stakeholders as well as with USAID. The project was decentralized, with a core team of technical experts located in the central office and project staff in the district offices, which were embedded within MOH premises in the districts. The MSH project produced substantial results that helped strengthen health systems and improve child health in Malawi. Its accomplishments contributed to USAID strategic objectives and to MOH goals. Specific findings by project component follow. Strengthening District Health Systems • MSH provided substantial assistance directly to the districts to reinforce health systems and to the central MOH to enable it to give effective support to the districts. The package of assistance included a broad array of management practices, including district-level planning and budgeting, management of essential drugs, integrated supervision, monitoring and improving data utilization, transport management, financial management, communications, human resource management, referral systems, and finally technical support to the zonal offices and Christian Health Association of Malawi (CHAM) facilities. • Qualitative and quantitative assessments reveal progress in all technical areas and in all districts. However, many elements of district health management systems require longer-term interventions, and there is room for further improvement. Prevention and Management of Childhood Diseases • Specific child health programs were integral to this project and central to the operational support, TA, and training provided at both the national and district levels. The child health programs covered were community-based integrated management of childhood illnesses MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 1 (IMCI), facility-based IMCI, the Pediatric Hospital Initiative (PHI), and nutrition services through the Community Therapeutic Care (CTC) program. • Noteworthy improvements have been achieved in all interventions, although community￾based IMCI intervention was terminated prematurely because one of the partners withdrew. The PHI led to increased access to and quality of pediatric emergency care. The CTC approach allowed the MOH to address a significant child health issue rapidly and effectively. • Continued efforts are needed to maintain and upgrade the performance and management of IMCI, PHI, and CTC. All efforts need to be expanded to achieve national impact. Malaria Prevention • MSH focused its work on distribution of insecticide-treated nets (ITNs) to increase community access to the nets and on Intermittent Presumptive Treatment (IP) for prevention of malaria among pregnant women. Both interventions have been successful, contributing to prevention of malaria in project districts. • Future malaria prevention efforts in Malawi need to increase constant and consistent use of the ITNs. A change in the drug used to treat malaria was scheduled to be rolled out in the fall of 2007. Additional systems strengthening activities are needed at the district and community levels to ensure the success of this policy change. HIV/AIDS Interventions • MSH support in the area of HIV/AIDS prevention and treatment has been targeted to the vulnerable populations of pregnant women and children; it concentrated on voluntary counseling and testing (VCT), prevention of mother to child transmission (PMTCT), and creating linkages between tuberculosis (TB) and HIVAIDS diagnosis and treatment services. The activities led to significant increases in access to and availability of HIV testing and treatment. • There is a need to do more to treat children with HIV/AIDS, which has been a relatively neglected area. There is also a need to integrate further the PMTCT program into antenatal clinics and new maternal programs. Quality Assurance • MSH assisted with an IP program to improve quality assurance at district hospitals and also helped to draft and disseminate a National Quality Assurance Policy that outlines practices to be followed at health facilities. • The evaluation team found that all hospitals significantly improved their IP practices. Continued support for the IP program is recommended. Hospital Reform Initiative • The project worked with the two central hospitals to help them establish autonomous governance and strengthen administrative systems in support of decentralized management. A broad range of highly valuable TA interventions was carried out at the hospitals and has resulted in improvements in hospital operations. • The project also provided valuable assistance at the national level to create the environment necessary for hospitals to become autonomous. However, efforts to achieve trust status for 2 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT the hospitals have been hampered by excessive turnover of MOH and hospital staff. In the future, simple and collaborative approaches are needed to achieve hospital autonomy. • The Draft National Policy on Hospital Reform and the Draft National Health Policy drawn up with assistance from the project seem to be complex, and their implementation and monitoring would be very resource-intensive. The team suggests that future drafts of these documents should be reviewed with these concerns in mind. FUTURE DIRECTIONS FOR USAID The team’s recommendations for future directions are based on the assumptions that USAID will continue to provide foreign aid in Malawi through specific programs defined in the Mission’s operational program and that USAID will not participate in basket funding of the SWAp but continue to be a discrete donor. Doing so gives USAID significant flexibility to respond to situational changes and to provide a broad range of technical support both to the MOH and to the various USAID projects. • Continue support for strengthening health systems: MSH’s TA in strengthening district health systems, linked with targeted interventions to improve child health, has been unique and valuable. There is evidence that the interventions have had impact, and the evaluation team recommends not only continuation but also expansion of this assistance model. We recommend that the MOH be given further support in systems strengthening through: o Continuous support for management training, with emphasis on cost-effective modalities like on-the-job training, which has proven successful and efficient. o Building capacity in zonal offices to better supervise and provide TA to districts. This assistance should complement rather than replace support for district-level management. o Continuing TA through such channels as embedded positions within the MOH. TA should be directed to mentoring and empowering national professionals as well as participating in MOH policy and planning activities. o Supporting the MOH through special projects to improve its planning and program oversight capabilities, consistent with the SWAp and with USAID’s expertise. • Continued support of USAID programs through systems strengthening: Specific TA should be directed to improving access to and the quality of rural health care through coordination of programs influencing rural health. In addition, a renewed focus on cross-cutting issues and a holistic approach to the delivery of care could support current efforts to develop quality services and improve access to care. • Because the staffing and human resource management issues are overwhelming in Malawi, no project will reach its full potential to improve the health situation until this issue is addressed. Future goals and targets for accomplishments need to be adjusted in recognition of this situation. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 3 4 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT I. INTRODUCTION AND METHODOLOGY PURPOSE OF THE EVALUATION The objective of this evaluation was to assess how Management Sciences for Health (MSH) has supported USAID’s efforts to reduce child morbidity by strengthening health systems in Malawi. The evaluation had two major purposes: 1) To review and analyze the impact of the MSH project against project objectives and USAID/Malawi Strategic Objective 8, and evaluate its general contribution to child health in Malawi; and 2) To determine lessons learned and recommend directions the Mission may wish to explore in designing future programs. The Scope of Work (SOW) for the evaluation, which presents an array of management practices questions, is found in Annex A. Methodology The team used both qualitative and, to the extent possible, quantitative methods in conducting the evaluation. Most of the quantitative information was derived from existing data, but some was also collected simultaneously with the qualitative information. Qualitative information was generated primarily through interviews with key informants and observations at health facilities. Data Sources The following data sources were used to inform the evaluation: • Background Documents: The team reviewed a wealth of background materials received from the Malawi Ministry of Health (MOH), USAID, and MSH. Gaps in information were filled in during meetings, interviews, and site visits. • Interviews with Key Informants: Key informant interviews were conducted with USAID/Malawi and MSH/Malawi staff and subcontractors, central MOH officials, district health management teams (DHMTs), hospital and health center staff, and faculty at teaching institutions (see Appendix B for a list of informants. To help ensure that comparable types of information were collected from the varied sources during field visits, the team drew up standard interview guides (see Appendices C, D, and E). These data collection tools were designed to reflect the questions posed by the evaluation SOW, and they were used to interview DHMTs, MSH field staff, and trainees trained with MSH support. • Existing Data: The team used a variety of data sets and databases to gain understanding of the magnitude of effort and to help assess the impact of the program. Program and financial databases maintained by MSH and surveys conducted during the project provided valuable information on many aspects of project achievements. The team also examined the results of the Demographic and Health Survey (DHS) and Multiple Indicator Cluster Survey (MICS). • Site Visits: To explore geographic variation, the team visited all eight districts supported by the project, which covered facilities in all three regions of the country (North, Central, and MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 5 South). Site visits were made to facilities in each district (see Appendix F for the list of facilities visited). Evaluation Timeline and Composition of the Evaluation Team The evaluation was conducted in country between July 9 and August 17, 2007. The team was composed of Pinar Senlet, Team Leader/ Health Services Management and Child Health Specialist; Robert Rosenberg, Health Systems Specialist; and Greg Becker, Hospital Reform Specialist. Dr. Senlet and Dr. Rosenberg concentrated on national, district, and community level activities; Mr. Becker focused on central hospital reform efforts and national policy interventions related to hospital reform. 6 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT II. BACKGROUND Malawi’s health system has long strained to meet the demands made by communicable diseases and the health needs of the poor, but it deteriorated further in the 1990’s with the rapid growth of HIVAIDS, the resurgence of malaria, and the intensification of poverty and malnourishment. The growing crisis led to a series of structural reform programs that defined the context in which this USAID-funded MSH project functioned. Malawi is one of the most underdeveloped countries in the world; per capita income is less that $180. By the early years of this decade, life expectancy had fallen to 37 years, 14.7 percent of the population had HIV/AIDS,1 and 80,000 people were dying of AIDS each year, leaving over one million orphans. Between 1992 and 2000, maternal deaths doubled to over 1,000 deaths per 100,000 live births. The current infant mortality rate (IMR) of 76 per 1,000 live births and the under-5 mortality rate of 133 per 1,000 are mainly due to malaria, diarrhea, pneumonia, malnutrition, and increasingly HIV/AIDS. While Malawi has had a long history of significant donor support for health programs, the care particularly of the poor and of vulnerable groups (e.g., mothers and children) has continued to be an intractable problem. In 2004 the health system, already weak and underresourced, was increasingly unable to either keep abreast of the growing disease burdens or serve rural and vulnerable communities. Major problems were the scarcity of resources, severe staffing shortages, sporadic delivery of essential drugs, and inefficient management systems—all of which caused both access and quality issues. It was within this context that the MOH embarked upon a series of health reforms, including the National Action Framework (NAF), to address services and structural problems and shape relationships with donors. Three critical structural changes defined the reformed Malawian health system: • decentralization of the health system with a shift from bureaucratic to political leadership • basket financing of the MOH by donors under the 2004 SWAp agreement • reformation of hospital financing and management Within the past decade, much has been achieved: • The SWAp has been defined and funding of over $700 million pledged. Two separate groups of donors are contributing to the SWAp: the pool donor group contributes to the basket, and discrete donors contribute for specific activities identified through the annual implementation plan process. • A detailed Essential Health Package (EHP) defines the MOH’s scope of responsibility. • Decentralization has begun. District assemblies, in conjunction with the DHMTs, now have authority over their budgets for capital and recurrent costs and can set their own priorities within the EHP. More recently, the MOH has established five zonal offices, each of which supervises and supports five to six districts. • National HIV/AIDS, TB, malaria, and nutrition policies have been drafted to direct activities, but a continued program focus on the access, quality, and sustainability of these services will require even greater efforts in the future. 1 UNAIDS HIV/AIDS Survey, 2005. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 7 USAID, which has long been active in Malawi, responded rapidly to the nascent structural changes and the attempts to strengthen health systems. The Mission goals were detailed in the 2001–2008 strategic plan; the general program goal is “Poverty Reduction and Increased Food Security Through Broad-Based, Market-Led Economic Growth.” The Health, Population, and Nutrition (HPN) program’s goal of Healthier Malawian Families supports work on Strategic Objective 8, Increased Use of Improved Health Behaviors and Services, in three areas: Reducing fertility and population growth, lowering the risk of HIV/AIDS, and lowering infant and under-5 mortality. Activities to strengthen the health system were folded into Intermediate Result 8.4, Strengthening Health Sector Capacity to Deliver Quality Health Services in a Sustainable Way. Because of its programmatic focus, USAID decided not to participate in the SWAp basket￾funding program but to function as a discrete donor using program-driven funding to support the EHP and certain SWAp program goals. For 2007 through 2011, USAID will operate under a U.S. Government Developmental Assistance Strategy for Malawi that has annual resource levels of $60 to $80 million. The major themes of the strategy are: 1) Sustainable economic growth 2) Social protection 3) Social development 4) Prevention and management of nutrition disorders and HIV/AIDS 5) Improving governance Health programs will be within the Investing in People strategic goal and will focus on “increased coverage and access to basic services and HIV/AIDS related services.” In 2002, the USAID project, Reducing Child Morbidity and Strengthening Health Care Systems in Malawi, was awarded to a collaborative partnership between MSH, Health Partners of South Africa (HPSA), SATELLIFE, and the American Red Cross. The original MSH proposal was to integrate “technical and managerial support to facilitate local ownership and responsibility through decentralization and autonomy, with a unified framework emphasizing outcomes and impact.” This broad and flexible focus allowed MSH and its partners to build upon and strengthen the reformation programs driving the MOH. Unfortunately, the first program years were marred by some misunderstandings between the MOH and the project; one conflict in particular was that the MOH had focused on the basket financing of the SWAp and considered the USAID/MSH programmatic approach as a threat rather than supportive. Only with a change in leadership at the MOH was this resolved; then, the MOH began to welcome the support for MSH programs. The program also evolved, becoming a vehicle for providing services to other US Government Agencies and initiatives such as the Office of Foreign Disaster Assistance (OFDA), the President’s Emergency Plan for AIDS Relief (PEPFAR), and the Presidential Malaria Initiative (PMI). It gradually carved out a significant niche as the organized source of TA to the MOH, the eight selected districts (Balaka, Chikwawa, Kasungu, Mangochi, Mulanje, Mzimba, Ntcheu, and Salima), and two of the new zonal offices. The original collaborative partnership was also modified during the program; the American Red Cross terminated its agreement to provide community IMCI services, and the SATELLIFE communication and networking programs were considered too advanced and abandoned early on as well. Thus, except for the hospital reform initiative, the project has been largely implemented 8 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT by MSH, which contracted with HPSA to work on the central hospital autonomy interventions, the national health policy, and the national research and bioethics policy. Flexibility was a hallmark of this project. That allowed MSH to respond rapidly to environmental changes, such as the famine of 2005–2006 and the emergence of new strategies in HIVAIDS, TB, malaria, and IMCI programs. Throughout the life of the project, training and management assistance activities were compromised by staffing issues, such as the transfer out of trained staff and the high turnover of managers. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 9 10 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT III. OVERVIEW OF THE MSH PROJECT This section reviews the project generally: its implementation strategies; relationships with MOH, USAID, and other stakeholders; organizational structure; financial management; and monitoring and evaluation. IMPLEMENTATION APPROACH AND STRATEGIES MSH applied a simple but robust approach to strengthening health systems in order to improve child health in Malawi. Many elements of the approach evolved from lessons learned once the project was underway. This is a stepwise approach to iron out inefficiencies in the system while improving service quality: • First, assess systems and services to identify inefficiencies and gaps. • Next, provide a package of comprehensive and structured support (training, tailored TA, equipment and materials, renovations) directly to districts and central hospitals to address the gaps. District and hospital efforts were supported through interventions at the national MOH level. • Finally, work to maintain and institutionalize gains. The MSH approach might be described as “removal of blockages to make things work.” The evaluation team considers this a practical and compelling problem-solving strategy. The team found ample evidence that it usually works; examples will be introduced throughout this report. MSH used a number of guiding principles to strengthen systems and improve quality. These principles strongly supported use of a bottom-up approach to systems improvement, allowed ample flexibility to respond to local needs, and strengthened the ownership of activities by the MOH and other partners. • Project TA has been consistently targeted to facilitate the work of the MOH in strengthening systems rather than MSH doing the systems strengthening work on behalf of the MOH. MSH describes its philosophy as providing support “through” rather than “to” the MOH. • The MOH fully participated in the design of interventions at both central and district/facility levels. • Recognizing that MOH systems and services were the basis for all interventions, at the request of the MOH the project focused on areas where districts and hospitals needed support. • The project emphasized on-the-job rather than classroom training to the extent possible; this increased the effectiveness of the training and decreased staff time away from duties. • Interventions were designed holistically; support was provided not only for training but also for other areas of need, such as the TA, equipment, and materials required to implement a given component successfully. RELATIONSHIPS WITH USAID, MOH, AND OTHER PARTNERS The evaluation team was informed that the early years of the project were slow and difficult (see Background Section), but the project has taken significant steps to improve the situation. By the time of the evaluation, the project had achieved a solid collaborative relationship with the MOH. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 11 MSH’s technical expertise, particularly in management at the district level, is highly valued by all health sector partners. Similarly, the project was responsive to USAID and has formed satisfactory relationships with USAID personnel. ORGANIZATIONAL STRUCTURE The project has an efficient and decentralized organizational structure. The chief of party (COP) has exceptional leadership and management skills as well as technical expertise; he is highly praised by partners. A central team of experts provides technical expertise as needed for implementing activities. The TA team is of high caliber and their work is much appreciated by the MOH and other partners. At the district level, the management technical assistants (MTAs) are charged with the responsibility of linking with the central team and coordinating project activities within their districts. Their technical expertise varies depending on their professional background, but they have participated in all major training initiatives supported by MSH so that they can play an active role as change agents. In the districts, MTAs have been integrated with the DHMTs, which has eased their acceptance and success in linking planning and implementation. All technical experts and MTAs report directly to the COP in a flat organizational structure. MONITORING AND EVALUATION (M&E) The project’s M&E plan evolved over time with changes in the SOW and modifications of the USAID Strategic Plan. The initial M&E plan for 2003–2005 was updated in 2005 to cover 2006. Currently the project is using a draft M&E plan covering work through September 2007. The plan defines 33 indicators sorted by activity group, data sources, and frequency of data collection; the project uses another 29 indicators to track progress on hospital reform. The indicators selected are appropriate, and the number of indicators is manageable. The M&E plan aligns with USAID Strategic Objective 8 and the indicators feed into the USAID Performance Management Plan (PMP). However, among other weaknesses, the M&E plan does not articulate the project’s overall approach to M&E. For example, it does not describe how information will be disseminated and used to provide support and feedback to the district offices, the MOH, and the implementing partners. Similarly, there is no description of how the plan it will be used as a management tool to track progress and improve performance. However, despite these weaknesses, the information collected was being analyzed and used, and feedback was provided to the field and the MOH, though on something of an ad hoc basis FINANCIAL MANAGEMENT The project maintains thorough financial/accounting systems and records, which are overseen by the deputy chief of party (DCOP). Through its experience in other countries, MSH has standardized its field accounting system to allow for decentralized management. The evaluation team examined the project budget breakdown by USAID contract line items (CLINS) and by TA expenditures at district and central levels (Figure 1). Total USAID funding for the project through September 2007 was $17,649,940. MSH has raised an additional $153,450 in cost share. About 26 percent of the budget is spent on salaries and wages; the overhead rate is 14 percent. Subagreements and contracts constitute 24 percent of the budget, and 15 percent went to training, local activities, and procurement of equipment. This is a reasonable and balanced allocation of the financial resources. 12 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT Figure 1: Budget by USAID CLINS An analysis of the budget by technical area reveals that district activities in child health, nutrition, malaria, HIV/AIDS/TB, and management assistance consumed the bulk of resources (Figure 2). The largest line item for central level activities is support for the hospital reform initiative. Given the scope and focus of the project, the distribution of funds across technical areas and between the district and central levels is appropriate. Figure 2: Expenditure by Technical Area 800 700 600 000500 , 400 $1 300 200 100 0 n a I B o nt e i c r T e H m e P i t r ti em c f a / i o r l / t Of l u DS or f n d I ta i Ma /N /QA anag s Re i upp Ch A l IP S a th / t l V ss onal i a HI A Z p M s eH Ho Central Level District Level MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 13 14 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT IV. PROJECT COMPONENTS: FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS HEALTH SYSTEMS STRENGTHENING This section summarizes project assistance to strengthen district health care systems. Over the course of the project, MSH directly introduced the districts to a broad array of management practices to strengthen health systems and assisted the MOH in giving effective support to the districts. FINDINGS AND CONCLUSIONS District Level Planning and Budgeting District implementation plans (DIPs) are the annual health plans that list specific activities to be carried out, with corresponding schedules and budgets. DIPs are the basis for proper planning of activities and resource utilization within Malawi’s decentralized health service structure. The process of drafting DIPs takes many months, and the plans need to be reviewed periodically throughout the fiscal year. It is crucial that all priority activities be included in the DIPs to ensure that they are adequately funded. MSH provided substantial assistance to the MOH Planning Directorate in drafting DIP guidelines and disseminating them to all MOH district offices and CHAM facilities.2 MSH also provided direct TA to all districts, even those not incorporated into the project, in DIP drafting and periodic review. The project also analyzed DIP processes and budgets for fiscal years 2005–2006 and 2006–2007 and disseminated the reports widely. It also helped national programs, notably on TB, to ensure that their priorities were incorporated into the DIPs This was important because separate funding for the TB program ceased when the SWAp was implemented In the field, the evaluation team observed that almost all districts were routinely developing and submitting DIPs and conducting periodic reviews. To ensure that DIPs respond to the needs of the entire district, many districts expanded the DHMTs to include not only district supervisors and coordinators but also other health providers, such as CHAM. Nevertheless, implementation of DIPs varied by district. Some were quite successful in following their plans; others drifted away from them in implementation. Integrated Supervision In the districts, supervision has been traditionally carried out by several vertical program coordinators. Because of logistics, time commitments, and the verticality of the supervision, this system never worked efficiently. With support from MSH, DHMTs introduced an integrated supervision system whereby a single district supervisor would be responsible for visiting a group of health centers (usually about five) once a month for multipurpose supervision. MSH helped put together a package of tools for supervisors to use, such as checklists. The project also helped to ensure that enough resources were set aside to ensure that the supervisors could do the work. At the same time, it also helped the DHMTs to design effective mechanisms for reporting back after supervisory visits. 2 See page 13 for more information on CHAM and the importance of involving CHAM facilities in the DIP process. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 15 The integrated supervision system worked well in most districts; supervision coverage improved dramatically, from 14 percent to more than 80 percent. The checklists have been useful in drawing attention to areas that otherwise would not have received adequate attention. As a result, a wide range of improvements has been made at health facilities, such as facility repairs, equipment provision, renovations, and outreach activities.3 Health Management Information Systems (HMIS) In 2002 the MOH introduced a new HMIS, and a district health information system (DHIS) is being used to capture and aggregate district data from the health centers and hospitals for report to the center. An assessment of the HMIS at the beginning of the project identified major problems: although the system was operational at all levels and the instruments were available, data quality was poor, and the data were not being used. MSH worked with assistant statisticians and the DHMTs to improve the timeliness and quality of data collection, reporting, and analysis, supplementing a variety of tools with on-the-job training. The system for data review was reorganized to actively involve health center staff as well as coordinators and supervisors. Statisticians and DHMT members were also trained in how to interpret and use data for decision-making. The timeliness of health facilities reports to the district has improved markedly The project designed an HMIS Cluster Review Model that has proved effective; in this model, staff from a cluster of facilities meet routinely with coordinators and DHMT members to review HMIS data. Most promisingly, the HMIS and DIP review meetings have been linked in most districts, which has improved utilization of data for planning and budgeting as well as for monitoring performance against the DIP. Drug Management Keeping drugs available in districts and health facilities has long been a challenge in Malawi. Many studies show that at any given time there are high levels of stock-outs, especially at the health facility level. Those interviewed identify drug shortages at the Central Medical Stores as the main bottleneck in the drug management system. MSH helped to reinforce drug supply management in individual facilities by working with staff to reorganize drug stores and pharmacies; training them in effective ordering systems; and ensuring appropriate district supervision of drugs. The work was conducted in collaboration with the DELIVER Project to ensure that the results of the two programs complement each other. The evaluation team observed that drug management had improved in the districts. The project reports that drug stores in 227 facilities have been upgraded and reorganized. The Logistics Management Information System (LMIS) was upgraded, and reporting rates increased from 30 percent in 2004 to 90 percent in 2007. An analysis comparing the percentage of health centers without stock-outs of IMCI tracer drugs in 2003 and 2004 in project districts found significant improvements in drug availability. A Logistics Indicators Assessment conducted in 2005 with assistance from the DELIVER Project found that drug management, record keeping, and supervision improved more in districts supported by MSH than in other districts. However, stock￾outs were still high and did not differ between the groups. This finding points up the fact that improved logistics management at the facilities does not mean improved availability of drugs. 3 MSH gave the evaluation team a list of outcomes of supervision activities in each district, which was too extensive to incorporate in the report. 16 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT Financial Management In decentralized health systems, the importance of district financial management practices and skills is magnified. MSH’s approach to financial management has been to train DHMT members so that they have more insights into what district financial management requires. Training was based on a needs assessment for accounting staff and on the distribution of responsibilities within the accounts section. Further support was provided to computerize the accountancy department and train staff in use of computers and spreadsheets. After the initial training, a mentor sponsored by MSH, spent one or two weeks providing on-the-job training to make the accounts section more functional. In many districts, the evaluation team found, the DHMTs were confident in their ability to analyze financial statements and were able to make decisions based on the analysis. On-the-job mentoring and training in computer skills also seemed to improve the timeliness and accuracy of financial functions. Financial management requires considerable commitment from the DHMTs. Because some DHMTs did not give this area enough emphasis, improvements were limited. Improving financial management may be seen as a threat and a disincentive because it makes it easier to detect irregularities. Transport Management Among the inefficiencies of the district health care system, transport management is one of the most challenging. High costs of vehicles, maintenance, repairs, and fuel, exacerbated by ineffective management, drain away a significant portion of already limited resources. An analysis of DIP budgets undertaken with assistance from MSH found that 20 percent of district budgets are consumed by transport, which is too high. The MOH already had a draft Transport Policy at the start of the MSH project that incorporated such aspects as effective planning, guidance on controlling the use of transport, and prioritization of transport, but it was not fully implemented due to lack of national support and lack of resources to finalize the document in areas like supervision mechanisms. MSH supported reimplementation of the transport management process in all eight districts. Major activities were drafting of district transport guidelines, retraining of transport officers, and implementation of routine transport procedures. The project also continued working with the MOH to finalize the National Transport Policy. In the three districts where DHMTs were actively involved in transport management and determined to exert more control, they were actually able to cut costs and improve transport availability. For example, one district implemented the transport guidelines to reduce use of ambulances for administrative purposes and stationed them at health centers instead of the district hospital. The results were a 33 percent reduction in distance traveled each month and a 29 percent decrease in fuel used. In other districts, however, progress was limited due to individual vested interests. As with financial management, improvement in transport management often leads to identification of irregularities, which may be a disincentive. Referral Systems and Communications The project helped to facilitate referral of patients by improving not only transport but also communication between hospitals and health centers, drafting guidelines on transport of emergency cases, and promoting the use of patient referral forms. For instance, MSH invested a significant amount in radios for several facilities. Use of radio communications between the hospital and health centers, the team found, has been useful. In most cases, radios are the only way health centers can communicate with hospitals. Some hospital and health center staff also MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 17 reported that as transport was better managed, they were able to refer emergency cases more quickly. Human Resource Management Human resource management at the district level is an area where the project made little progress; DHMTs were not very interested in addressing their human resource problems. There was one success, however; in one district, the DHO recognized that there was a gap between the skills and the assignments of health staff, and an assessment revealed serious inefficiencies in record keeping. The DHO was able to restore the filing system and update the records. The evaluation team recognizes that human resource issues in Malawi are complex and crosscutting. Local interventions, such as the one described, help to improve some aspects of human resource management, but there are much more serious issues that must be dealt with at the national level. Support to Zonal Offices MSH contributed significant technical and operational support to the rollout of the MOH zonal offices. The project provided not only operational funding for the offices in the Northern and Central East zones but also TA for drafting the Zonal Officers Supervisors Manual, which supports zonal office supervision with checklists and other tools, such as the DHMT needs assessment, the Zonal Health Support Office Reporting Framework, and the Zonal Implementation Plans. MSH also provided assistance in such technical areas as pharmaceutical management, integrated supervision, and the use of HMIS software. In interviews with two zonal officers, the evaluation team found that the offices are functional and working well with the districts. Both officers appreciated the support they have been receiving from MSH and hoped the assistance would continue. We believe that strengthening the zonal offices is a major contribution to strengthening the capacity of the MOH to supervise and support the districts. A major challenge for the future, however, is to ensure MOH funding for zonal operations. Currently, zonal office staff salaries and operational costs are supported by donors (USAID and the German Technical Assistance Agency, GTZ). Support for CHAM For over 40 years the members of the CHAM have been delivering health services in Malawi, primarily in rural areas: An estimated 40 percent of all rural primary care is delivered in the 168 facilities they own and operate (131 health centers, 18 hospitals, 18 community hospitals, and 1 mental rehabilitation hospital). These services have generally been supported financially by CHAM members, the government of Malawi, and nominal fees for services rendered. In 2005– 2006, when the country adopted a policy that all EHP services must be made available free, the districts began to make service-level agreements (SLAs) with CHAM facilities to provide maternal and child health (MCH) services. The SLAs allowed districts to allocate funds to cover the fee revenue that CHAM facilities lost when EHP services became free. As of 2006, 20 district health offices had SLAs covering 45 of 50 CHAM service units. Although these SLAs expired in July 2007, both the MOH and CHAM expect that a revised SLA will soon be introduced to sustain and expand the program. MSH played a significant role in supporting the development and implementation of the SLAs with CHAM through system strengthening activities: training CHAM health coordinators to participate in the DHMTs and in the DIP process, and helping with SLA development. MSH also sponsored a study to establish the basis for the fee schedule for CHAM maternal and neonatal 18 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT services that was incorporated into the SLAs.4 The result has been a significant improvement in access to maternal and neonatal care, particularly where the CHAM hospitals are the dominant providers of hospital-based MCH services. As nominal fees for antenatal care and deliveries were dropped, there was a substantial increase in deliveries by trained health providers within facilities. Results and Impact of District Health Systems Interventions To check on its progress, MSH conducted an end-of-project assessment in 2007. The survey assessed health system performance using the same 47 criteria that were used in assessing district health systems as a baseline in 2004. The 2004 assessment had found major systemic challenges; the majority of districts met only seven of the 47 criteria. The 2007 survey demonstrated that the majority of the districts were complying with 22 criteria (Figure 3 compares the results of the two surveys). The survey results support the findings of the evaluation team; although there is still much room for improvement in many districts, progress has clearly been made. Figure 3: Progress toward Meeting Management Criterion 0 5 10 15 20 25 30 35 1-3 districts met assessed criterion 7-8 districts met assessed criterion Number of Criteria Baseline assessment 2004 End of Project assessment 2007 Measuring the impact on health outcomes of efforts to strengthen health systems is difficult. Because so many other variables affect outcomes, there is usually not a direct link, but the evaluation team found qualitative evidence that in project districts the quality of health services was improved: We examined results from the national health surveys to determine whether the changes had an effect on child mortality and morbidity. A comparison of the infant mortality rates (IMR) recorded in the 2004 Malawi Demographic and Health Survey and the 2006 Malawi Multiple Cluster Indicators Survey is revealing. The surveys provide district-specific IMR for 10 districts, of which five were supported by MSH. The decrease in IMR in the MSH-supported districts is much larger than in the others. Between 2004 and 2006, the decrease in the MSH districts was 26 percent; in the other five districts, the average decline was 9 percent (Figure 4). While it is difficult to prove that the change is attributable to MSH’s efforts alone, the team believes that the systems strengthening efforts, combined with child health interventions in project districts, has made a difference. 4 David H. Collins, Setting Fees For Maternal and Neonatal Services Provided at CHAM Member Facilities, January 2007, (Cambridge, Massachusetts: MSH). MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 19 Figure 4: Percentage of IMR Decrease in MSH-supported Districts and other Districts 2004–2006 0 5 10 15 20 25 30 MSH-supported districts Non-supported districts Combination Percentage of IMR Decrease Sustainability Considerations MSH has been giving priority attention to sustainability issues, particularly in the last year of the project. Staff turnover, which MSH has no control over, is the greatest threat to sustainability. Moreover, many of the key management interventions (such as planning, HMIS, and supervision) are long-term processes that will require additional support to ensure appropriate quality and sustainability. The likelihood of sustaining the gains once the project phases out varies greatly in terms of both specific interventions and project districts. To acquire an objective understanding of the sustainability issues, the team analyzed each activity group using criteria recently developed by MSH, such as: • Is there a policy provision to support the activity? • Is the activity aligned with either the plan of work (POW) or the EHP? • Are specific components of the activity incorporated into the DIPs? Based on these criteria, the team considers the likelihood that the health systems strengthening interventions will be sustained is moderate to high. Almost all activities have policy support and are included in the POW or EHP as well as in the DIPs. One exception has to do with integrated supervision using the cluster supervision model. There is no provision in management structures for mid-level supervisors, and there is also push from vertical programs to maintain a program￾specific coordination system. There is another very important, though subjective, element related to sustainability: the degree to which DHOs and DHMTs have ownership of the activities. In districts where there is a high level of ownership, the likelihood that the gains will be sustained is high. Similarly, within the districts the team found that managers are more willing to maintain some activities than others. Recommendations for Strengthening District Health Systems • The successful model applied by the project to strengthen health care systems in the districts should be continued and rolled out. Although there have been noteworthy achievements in 20 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT the eight project districts, this is a long-term initiative and there is still much room for improvement in specific areas. • The mandated supervisory functions of the new MOH zonal offices have great potential to support district management and would greatly enhance it. Expanded technical support and training on health systems, such as drug management, transportation, quality assessment, and supervision, might be made available through the zonal offices. • MCH services should continue to be developed within CHAM facilities through USAID projects and under the direction of the DHMT. The projects should continue to make available TA and training to CHAM facilities. • Finally, further documentation and dissemination of best practices and lessons learned from this project would be useful. The Dissemination Conference organized in late 2006 to share lessons learned was worthwhile but it is not enough. The project model for strengthening district health systems is unique and powerful—it deserves wider attention not just in Malawi but also internationally. PREVENTION AND MANAGEMENT OF CHILDHOOD ILLNESSES Specific child health programs were integral to this project and central to the operational support, TA, and training provided at both national and local levels. The project aimed to improve child health services by strengthening the health system infrastructure needed to implement and directly support specific major child health programs. Findings and Conclusions Community- based IMCI Originally, the American Red Cross was to partner with the Malawian Red Cross in this project to provide essential community IMCI services within all eight chosen districts. The intent was to 1) improve partnerships between the health facilities and services and the communities they serve, increase the use of health facilities and services, and establish mechanisms for community feedback; 2) increase the number and quality of preventive services through information, education, and communication (IEC) activities; and 3) promote positive family practices critical for child health and nutrition (e.g., breastfeeding, hygiene). Although this intervention was never implemented because the American Red Cross withdrew early on, the evaluation team found that elements of community-based, demand-side programming were embedded within project IMCI programs: community pharmaceutical supply control committees have been established; IEC materials are provided, though sparingly, in health centers; community-based health activities are integrated into all vertical programs; and volunteers from the communities provide outreach and surveillance services throughout the districts. Facility-based IMCI The IMCI approach was adopted in Malawi in 1998 and implemented in four districts in 1999. At first, it had three major components: skill building for health workers, systems improvement, and community-based IMCI. The program was then rolled out nationwide and introduced into each MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 21 district. Nevertheless, IMCI nationwide has deteriorated from the initial phase due to staff shortages and turnover, drug and supply chain issues, and the increased complexity and structural requirements of the WHO IMCI program. In addition, access to IMCI in remote rural areas has been limited by the lack of adequately staffed facilities. MSH partially addressed these problems through its IMCI support and systems strengthening activities within the districts and at the national level. MSH has supported IMCI training and provided logistic and management support for such health care activities as the expanded program of immunization (EPI), mass campaigns, and distribution of child health equipment, based on district-specific needs. Nationally, the project helped to finalize the National IMCI Policy and the five-year Strategic IMCI Plan, to abridge and pilot an IMCI training course scaled down from 11 to 5 days, and to integrate HIV/AIDS and care of newborns into the IMCI guidelines. MSH also enhanced the capacity of staff in IMCI to improve the quality of care and supported supervision using IMCI guidelines. IMCI trainers trained in the districts supervised implementation of a revised training schedule. Oral rehydration therapy (ORT) corners were created in hospitals and health centers to manage diarrhea. Thanks to MSH training of IMCI staff, there was less use of pediatricians from Queen Elizabeth Central Hospital with no apparent reduction in the quality of care. The team concludes that MSH activities significantly strengthened the delivery of IMCI programs both indirectly through general project systems strengthening activities and directly through support of the IMCI program through training, TA, and limited but cost-effective procurement. Based upon client response and the performance of trained staff, training has been successful and the IMCI program has been integrated into the DIPS. Efforts by MSH to decrease the time and coverage of expensive off-site training have helped offset costs. Again, however, staff turnover and shortages significantly reduced the impact of the training and the success of the IMCI package. MSH has played an important role at the national level both in creating policy and training materials and in their printing and dissemination. All health centers visited by the team appear to have the materials and forms they need. Finally, procurement was targeted to low-expense items that are essential to perform IMCI protocols properly and to renovation of facilities. However, problems are now arising from lack of maintenance and repair of equipment. Pediatric Hospital Initiative The Pediatric Hospital Initiative (PHI) grew out of the recognized need for better triage and pediatric emergency care, especially the management of fever in children under five. PHI, which was developed through the Quality Assurance Collaborative to Improve Seriously Ill Children, incorporates the WHO emergency triage assessment and treatment (ETAT) guidelines. MSH supported the national program by printing and distributing the ETAT guidelines and an ETAT manual to participants in ETAT training. In 2004, a baseline assessment of emergency care for children was conducted and the results disseminated. Based on the findings, the PHI program was launched in 2004 in several pilot sites with assistance from MSH. The project assisted in ETAT training for staff and provided ETAT equipment. In 2006 and 2007, the districts integrated the intermittent presumptive treatment (IPT) program for febrile children into PHI. The PHI program is now operational in all eight district hospitals and has been adopted by the MOH and CHAM for childhood emergency care nationwide. The team found that all MSH district hospitals were equipped with essential ETAT equipment. Staff at all hospitals and selected health centers were trained in and now use the care of fever guidelines produced with MSH assistance. All district hospitals now have round-the-clock 22 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT pediatric emergency coverage, which greatly facilitates rapid response to children with fever and other emergency conditions. IPT is now available in all PHI units; fewer children with non-severe malaria are admitted, but malaria death rates on the pediatric wards remain constant because those admitted have more severe illness. There has been a substantial increase in compliance with ETAT guidelines in PHI program. A quality assessment process has been initiated and systems have been strengthened to reduce delays in treatment, improve access, and improve compliance with guidelines. The team concludes that introducing the ETAT guidelines into pediatric emergency services has led to significant improvement in pediatric emergency care, increasing both access and quality. These services, linked to the IPT programs, improved the response to fever and appear to have reduced the morbidity of malaria in the populations served. Because the project has been successful, the MOH plans to roll out the program on a national basis and stabilize funding through district provision of EHP. Community Therapeutic Care (CTC) Nutrition services became a priority for MSH in response to requests from the MOH and the OFDA during the famine of 2005–2006. At that time, over 10 percent of Malawian children were identified as having global acute malnutrition. With support from OFDA, in 2006 a project called Rapid Expansion of Treatment and Rehabilitation of Severely Malnourished Malawian Children (RETR) was launched to implement and mainstream the CTC approach. CTC is a community￾based approach designed to improve case management of acutely malnourished children by health centers and families. It strengthens their nutritional rehabilitation by expanding use of a local food, Chiponde, in therapeutic feeding. The two-year RETR project covered five districts and had four objectives: • Ensure that 60 new outpatient therapeutic programs are able to implement CTC. • Ensure that the CTC program enrolled 65 percent (5,000) of the estimated moderate and severely malnourished children under five in the catchment areas. • Ensure the development of an effective system to refer malnourished children to the various feeding components. • Ensure the sustainability of CTC as a routine district intervention (phase II). MSH participated in the scaling up of CTC from six sites in two districts to 59 in five districts— nearly half of all the facilities in the five districts. MSH worked with the DHMTs in those districts to plan and supervise services; helped get CTC incorporated into the DIPS in all eight districts; trained a network of community volunteers (1,546); and provided essential CTC equipment and supplies besides Chiponde, such as weighing scales, MUAC tapes, and height boards. The project also helped to revitalize the District Targeted Nutrition Forums and enhance coordination between the MOH Nutrition Unit, the DHMTs, and partner organizations, such as Valid International, Concern Worldwide (CWW), and UNICEF. Meanwhile, USAID, with support from the Clinton and Hunter Foundations, helped equip manufacturing plants in Malawi to produce Chiponde. The Clinton Foundation also committed to supply Chiponde for an additional year. From January 2006 to January 2007, 5,557 severe and acutely malnourished children—more than targeted—were enrolled in the program. Of these, 85 percent were cured, 10.1 percent defaulted, and 2.4 percent died. The cure rate also exceeded the 65 percent goal. Moreover, 3,787 health staff and volunteers were trained to provide CTC services, and 407 children were referred from MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 23 nutrition rehabilitation units (NRU) to CTC, which suggests that the referral mechanism was effective. MSH was instrumental in making the CTC program more sustainable as well. Through a workshop it supported in 2006, the MOH Nutrition Unit, DHMTs, and NGOs met and agreed to strengthen annual district planning by incorporating CTC priorities and guidelines into DIP plans. MSH also played a crucial advocacy and advisory role in national CTC Advisory Services offered by CWW, which led to the DHMTs assuming management of the CTC and employing all CTC staff. The CTC program is an important demonstration of how the MOH was able to rapidly and effectively scale up to address a significant child health issue using TA, training, and logistics support from MSH. MSH support earned kudos from both MOH and the other partners. The CTC program met its objectives in the five districts and seems to be well sustained, but there are still questions. For instance, to achieve national impact, there is an urgent need to expand CTC to a majority of health centers in every district. Gaps in funding and the price of locally produced Chiponde might be real obstacles to doing so. The evaluation team also observed an enormous need for harmonizing the nutrition support activities of multiple donors and NGOs. This is a national task well suited for the MOH zonal officers operating through the DHMTs. Referrals and coordination among nutrition units are also problematic. Case management and follow-up is extremely limited once children leave the CTC program. Finally, the causes of malnutrition in Malawi need to be addressed. Recommendations for Prevention and Management of Childhood Illnesses • Despite significant achievements, IMCI at both central and district levels requires additional support—the job is not yet done. Support is needed at the program level and within MOH, preferably through the zonal offices. • Community-based IMCI activities should be considered in all future USAID child health projects. • Improving the performance and management of the PHI programs will require constant effort. The MOH needs support so that it can roll out PHI activities across the country through TA and logistical support for training. • The zonal offices should make periodic quality assessments to assure compliance with PHI guidelines and course corrections. • Systems for nutrition support should be strengthened so it can be expanded significantly to address the treatment of severely malnourished children. That will require (1) rationalization and improvement of referral and tracking systems to assure that there is no backsliding; (2) increasing community-based prevention services; (3) harmonization and coordination of the multiple stake holders and cross-cutting work with agriculture and education to build a stable and safe food supply chain; and 4) increased attention to strengthening NRU systems. MALARIA PREVENTION Findings and Conclusions The scope and direction of the MSH malaria program has shifted dramatically since the project began in 2004. An initial component designed to increase community access to ITNs by 24 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT subsidizing sales was ended when the MOH decided to include ITNs in the EHP, which meant they must be free to all. This eliminated the ability of local social marketing programs to obtain subsidized nets. MSH then made IPT a priority and introduced it into the maternal and child programs already within the MSH scope of work. MSH also increased participation in national activities and its efforts to improve case management of malaria. Community Access to ITNs In the initial activity, Population Services International (PSI) distributed partially subsidized ITNs through MOH and CHAM health facilities, under the direction of the DHMTs. An initial assessment had led to the conclusion that there was a substantial market for subsidized ITNs that could be purchased at a reasonable price. MSH then formulated logistical methodologies to work with community ITN committees to purchase and distribute nets. Procedure manuals and a national ITN Training Manual were written and distributed by health surveillance assistants (HSAs) in the communities. HSAs and other community workers were trained in all eight districts. Sales were conducted through a mass campaign approach. Between 2004 and mid-2006, 19,166 nets were sold through the 178 ITN Committees and in 2005 over half a million nets were redipped. The Malawi ITN program has been considered the most successful in Africa; it increased ITNs in households from 14 percent in 2002 to 60 percent in 2006 against a Roll Back Malaria Goal of 80%. Intermittent Presumptive Treatment (IPT) When significant funds for the prevention and treatment of malaria were made available through the U.S. President’s Malaria Initiative (PMI), MSH shifted major attention to the support of IPT for malaria in pregnancy. IPT is the provision of antimalarial drugs to pregnant women to minimize the effects of malaria on the mother and the fetus. Current recommended practice is two doses of sulfadoxine-pyrimethamine (SP) during pregnancy. MSH embarked upon a program to increase completion of the two-dose SP regime and expand participation. The project conducted problem-solving meetings with community antenatal care (ANC) providers and identified major factors leading to the low IPT coverage: ANC failure to administer SP because of lack of equipment, lack of knowledge among providers, or failure to keep adequate ANC records; poor health provider understanding of the side effects of SP; and public misconceptions about the effects of SP in pregnancy. MSH facilitated adoption of a strategy to introduce directly observed therapy (DOT) for IPT by ANC providers; trained nursing and clinical staff; and drafted job aids. The IPT program benefited substantially from both the systems strengthening activities of MSH and its project-specific support. According to project records and surveys, IPT coverage increased from 53 percent in 2004 to 80 percent in 2006. Including IPT in ANC service has lessened the impact of malaria and presumably reduced morbidity both for women and for their infants. These results, however, are undermined by the 25 percent failure rate for SP that has been documented in Malawi. Occasional stock-outs and turnover of staff further lessened the program’s impact. Malaria Case Management In 2006, after community-level ITN distribution was ended, MSH redirected resources to support case management of children with malaria and inclusion of case management in integrated packages, such as IMCI and the PHI. PHI especially strengthened fever management. The project also trained supervisors, conducted malaria case management training, and trained microscopists from 51 health facilities in malaria diagnosis. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 25 Overall Recommendations for Malaria Prevention • Social marketing is a proven method of expanding ITN access and availability, but it will not be feasible in Malawi under current policies. Future programs should therefore turn to increasing the consistent use of ITNs, given both the widespread availability of nets in Malawi and the inconsistent pattern of their use. • The antimalarial drug used in malaria treatment is scheduled to be changed from SP to artemisinin combination therapies (ACTs) and rolled-out nationally late in 2007. The risk of failure or only partial success can be reduced only through strengthening systems at the district and community level, including logistic support and IEC activities to increase public understanding of the necessity for the drug change. The involvement of ANCs is essential to assure the continuity of the IPT program. • MSH's systems strengthening programs have greatly improved the management of active cases of malaria. TA and training to assure evidence-based treatment and improved quality should be continued through support to the zonal offices. HIV/AIDS INTERVENTIONS Findings/Conclusions While HIV/AIDS incidence has perhaps peaked in Malawi, many Malawians continue to be at high risk. The entire health system is affected; although treatment centers are rapidly increasing, with over 1 percent of the entire population now on antiretroviral therapies (ARVs) the threat to women and children is still high. Less than 10 percent of the pediatric AIDS population is currently being treated. Within its eight districts, MSH's support for of HIV/AIDS prevention and treatment has been targeted to the vulnerable populations of pregnant women and children under five in rural areas. It has three elements: VCT, PMTCT, and creating linkages between TB and HIVAIDS diagnosis and treatment services. Voluntary Counseling and Testing In response to the demand for these services from the MOH and the DHMTs, since 2004 MSH has piloted VCT by providing in each of its districts two full-time counseling and testing service providers and has provided TA to strengthen management of VCT sites. The project secured an HIV rapid testing kit supply chain and then facilitated referral of those testing positive to the nearest ARV sites. VCT service providers met quarterly to discuss management issues, problem solving, dissemination of best practices, and information transfer. Staff from hospital wards, departments, and outpatient clinics were oriented to recruit and refer clients to the VCT sites, and systems were instituted for patient referrals. With assistance from the project, internal HIV referral forms were created and distributed. The Internal Referral Handbook is now being drafted. Other achievements of the VCT intervention: • Eight HIV testing sites served 226,323 clients from October 2004 to March 2007. During the first quarter of support, 6,005 people were tested; in the most recent (January–March 2007), 23,400—nearly four times as many—were tested. • District hospitals were supported to remove blockages to access to VCT by creating a more patient-friendly atmosphere. 26 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT • All eight district hospitals now provide VCT services daily and have introduced on-site testing rather than the laboratory-based testing that had been the norm. • VCT services reached the community through health centers; 16 community volunteers were trained and deployed in eight districts. HSAs were trained in five districts, and 80 providers in the health centers were trained in referrals to improve access to ARV treatment. PMTCT An objective of MSH throughout the project was to scale up PMTCT provision within the ANCs and health centers delivering maternity services. Supportive activities included: • Orienting nurses in ANC and maternity centers through quarterly problem-solving meetings to discuss how to institute routine VCT for pregnant women • Provision of furniture and equipment to roll out PMTCT services in 40 health centers (five per district) to upgrade PMTCT clinics • Training of PMTCT district coordinators in monitoring and supervision • Supporting DHMTs in orienting health care providers and support staff in PMTCT • Where requested, orienting traditional authorities to PMTCT service provision. The PMTCT program has yielded impressive results. Services have been scaled up to 40 health centers. Seven district hospitals integrated VCT services in their ANC departments and two others commenced PMTCT service. Over 100 nurses, 800 other health care providers, and 1,200 community leaders were trained in PMTCT. As a result, 39,922 pregnant women received VCT in eight districts between 2004 and 2007. ANC test rates (number of pregnant women tested against new ANC visits) have gone up from 4 percent in 2004 to 80 percent by 2007 (see Figures 5 and 6 for numbers of pregnant women tested and ANC test rates). All women who tested positive were referred to ARV centers. However, according to MSH reports, the availability of nevirapine syrup and tablets at service delivery points is problematic. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 27 Figure 5: Number of Pregnant Women Tested for HIV in 8 MSH-Supported Districts October 2004–June 2007 243 568 1210 1497 2673 2796 2658 4954 5181 7932 15668 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 Oct-Dec 04 Jan-Mar 05 Apr￾Jun05 Jul￾Sept05 Oct￾Dec05 Jan￾Mar06 Apr￾Jun06 Jul￾Sep06 Oct￾Dec06 Jan￾Mar07 Apr￾Jun07 Number Women tested Figure 6: HIV Testing Coverage of New ANC Visits in MSH￾Supported Districts October 2004–June 2007 4% 7% 10% 13% 31% 27% 31% 49% 56% 72% 82% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Oct-Dec 04 Jan-Mar 05 Apr￾Jun05 Jul￾Sept05 Oct￾Dec05 Jan￾Mar06 Apr￾Jun06 Jul￾Sep06 Oct￾Dec06 Jan￾Mar07 Apr￾Jun07 TB-HIV/AIDS Collaborative Activities The Malawi TB-HIV Technical Working Group asked MSH to support the plan for joint TB and HIV services. The TB program in Malawi had long been recognized as a successful but vertical and self-contained, so cross-cutting and harmonizing efforts were considered of great importance. This plan included routine HIV testing of TB patients, active TB case finding in VCT sites, cotrimoxazole treatment for co-infected patients being treated for TB, and provision of ARV to TB patients. 28 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT At the national level, MSH provided management support and TA to the NTP in formulating strategies to strengthen the TB program, remove blockages to harmonize it with HIV/AIDS activities, and build a single unified M&E plan. At the district level, MSH supported district TB officers, clinicians, and nurses from the TB wards through quarterly problem-solving meetings and helped institute methods to provide VCT for all new TB patients in all district hospitals. As a result, HIV testing improved from 44 percent of newly diagnosed TB patients in 2004 to 66 percent—a 50 percent increase. In real terms, this represents 7,093 TB patients tested for HIV; the great majority of HIV-positive patients would have benefited from CPT. In addition, MSH was able to introduce active TB case findings in seven VCT sites through sputum testing. This established a registrar for cotrimoxazole therapy through referral to TB centers. Recommendations for HIV/AIDS • The national priority of HIV/AIDS treatment and prevention is complemented by significant donor support, in particular from PEPFAR and the Global Fund to Fight AIDS, TB, and Malaria. However, USAID might expand the treatment of children with HIV/AIDS, a group that has been relatively neglected. • USAID can concentrate on systems support for pediatric treatment through support to case management and follow-up and referral programs for children with AIDS identified through the PMTCT program. This should be accompanied by mechanisms to increase access to ARVs for children. • USAID should provide TA to the MOH as it plans a nationwide testing program for HIV/AIDS that may replace voluntary testing. The intent should be to support the HIV/AIDS case identification program; this would require MSH systems strengthening support at the district and community levels. • Continued support is needed for the PMTCT program through systems strengthening and direct programmatic support to continue improving the percentage of pregnant women and infants receiving nevirapine. • There is a need to integrate further the PMTCT program into the ANCs and into new maternal health programs. • Significant improvement in the treatment of TB patients with AIDS is imperative given the potential for the spread of multi-drug-resistant TB in Malawi. USAID should continue linkage of these two vertical programs and assure goals of improved compliance with treatment and reduction in dropout rates. QUALITY ASSURANCE Findings and Conclusions MSH assisted with two quality assurance (QA) interventions through this project: an IP program to improve QA in district hospitals and a policy intervention for the drafting and dissemination of a QA policy and patient’s rights charter. A national QA policy that outlines key practices to be followed at health facilities was developed and disseminated. Infection Prevention MSH supported an IP performance improvement process designed by JHPIEGO in conjunction with the MOH in eight district and two central hospitals. This was part of a larger national scale￾MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 29 up process implemented by the MOH, JHPIEGO, and MSH through the National Quality Assurance Task Force. The IP program was launched in all eight district hospitals. MSH supported the training of staff in the basics of IP and facilitated the formation of district quality improvement support teams (QIST) and IP committees. The project also supported baseline surveys and internal and external assessments for IP, provided TA on IP practices, and printed IEC materials supporting IP. Because of long-standing neglect, all the district hospitals were in significant need of renovations and equipment to bring their facilities up to standard and support quality programs. The evaluation team found that all the hospitals made significant progress in improving IP practices. The participation of each was enthusiastic, morale building, and supported actively by staff (although in one hospital complaints were raised about the slow progress of renovation and the inadequacy of the incinerator). Every hospital increased its assessment scores from the baseline, and one, Salima, actually was accredited. Salima was able to increase its assessment score from 14 percent in 2004 to 93 percent in 2007. The accreditation experience raised Salima’s profile with the MOH, which designated it as a PQI/IP learning hospital. Two more hospitals should be accredited by the end of the year. While the performance and sustainability of the QISTS established in all eight districts are directly related to the hospital director's interest and enthusiasm, most of the hospitals visited were proud of their accomplishments and appeared eager to work on other quality issues (see Figure 7 for IP measured through baseline and follow-on assessments). Figure 7: Comparison of IP Scores in District Hospitals (before and after IP Interventions) 0 20 40 60 80 100 Percentage Baseline Last assessment Balaka Chickwawa Kasungu Mulanje Mzimba Salima Mangochi An additional positive is the involvement of communities in the QA projects: Community leaders and members helped to disseminate IP information, promoted adherence to standards of practice, helped in traffic control for people and animals, and helped with construction by molding or buying bricks for building fences. Several hospitals also experienced increased demand as the facilities became more attractive and hygienic. 30 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT Recommendations for Quality Assurance • The process of quality assessment and improvement is essential for each district hospital. The QISTs should serve as a generator of the quality improvement programs that are essential for evidence-based child and maternal health services. Continued support for these programs through training and TA can best be sustained through the zonal offices with continued donor support. Resources and equipment for quality improvement in both district hospitals and health centers can be provided through programs in child care and maternity. CENTRAL HOSPITAL REFORM INITIATIVE The goal of project activities with Kamuzu and Queen Elizabeth Central Hospitals (KCH and QECH) has been to establish an autonomous governance structure5 and build administrative systems in support of decentralized management. This activity, a carry-over from the earlier PHR-Plus Project, is in line with Government of Malawi strategy. Project activities in the central hospitals were carried out through a subcontractor, HPSA. Its activities fell into two areas: (1) specific TA directly to hospital departments in a number of administrative and management areas, and (2) TA at the national level to create the environment necessary for hospitals to achieve autonomy. Technical Assistance to Hospital Departments In preparation for hospital autonomy, a broad range of TA interventions was carried out at the two hospitals. They were helped to identify individual cost centers; implement planning and budgeting in hospital departments and computerize systems for financial, administrative, and human resources management; and initiate management improvements in pharmacy, transport, and other service areas. Pharmacy Management Improvements were made in both the physical structure and the operations of pharmacies at both hospitals. The system for departments ordering pharmaceuticals from the central pharmacy was improved; in combination with a delineation of specific drugs used by individual departments, the system has improved the movement and the ordering of drugs. With the implementation of the Electronic Pharmaceutical Inventory Control System (EPICS), a joint effort by Baobab Health Partnership and the MSH project, the control of central pharmacy stock at KCH became noteworthy. This stock management system, built on the technological foundation of Baobab’s revolutionary hospital computer system, should be more widely used. Stock management and pharmaceutical distribution and tracking using this system are presenting a more complete picture of the drug situation and beginning to show the impact of drug stock￾outs. One weakness in the system is that it oversimplifies drug stock-out information: Stock-outs of all drugs receive equal weight in the reports no matter how critical a specific drug may be, and where there is “short-stocking” (a partially complete order of a particular drug), the size of the supply deficit is not reported. Due to hospital budgetary constraints and financial and procurement issues at the Central Medical Stores (CMS), essential drugs supplied to the hospitals are well below what they need. Frequent stock-outs at the CMS force the hospitals to repeatedly procure essential drugs from the private sector, causing even deeper erosion of scarce hospital 5 Initial efforts to pass a Hospital Autonomy Bill were blocked by resistance to the term “autonomy” and the fear that this entailed privatization or selling-off of hospitals and the institution of fees for service. To allay fears and make the issue more easily understandable, TA efforts were renamed “Hospital Reform” and the sought-after legal structure became “Trust Hospitals.” In practical terms, trusts are autonomous bodies with a more appealing name. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 31 funds. Structural difficulties in CMS financing, which are partially due to changes in budgetary allocations apparently brought about by decentralization, may further complicate drug procurement. Recommendations: • It is critical to monitor the criticality of specific drugs that are out of stock or in short supply, and the size of any supply deficit in short-stocking. It is hoped that Baobab Health Partnership can address these issues6 if it can procure support for external funding. To better rationalize drug procurement in terms of reductions in morbidity and mortality, hospitals will be given a decision matrix based on the findings of this evaluation (see Appendix H). Central Hospital Registry System An improved method of filing administrative material and handling hospital mail was designed and put in place. The system is very effective and will likely continue past the project. Monitoring Complaints at QECH Nineteen complaint boxes were installed in strategic locations throughout QECH, and a committee was established to review complaints and report to the affected departments. Several significant procedural improvements have resulted. This system is also well designed and should continue past the project. Building up the Referral System One of the biggest problems for central hospitals was the excessive numbers of primary and secondary care patients that bypass district hospitals and health centers and seek care directly at the tertiary facilities. Moreover, patients legitimately referred were often sent to tertiary specialty services without sufficient information on prior diagnosis and care and reason for the referral. The project devised a system to improve the latter problem. Standard referral forms were drafted and medical registrars (medical residents) are routinely sent to surrounding district hospitals to provide specialist support and additional training in patient care, triage, and emergency care. Compliance with referral procedures on the periphery is reported to be good, although improvements are needed in the other direction in relation to information returned to district facilities when a patient is discharged. Visits of medical registrars to district facilities are reported to be very valuable, but there is concern that funds to continue the practice will no longer be available when the project ends. Operating procedures implemented with the assistance of the project in the Pediatric Department at QECH are also highly valued. Cost Centers and Business Planning Shaping a systematic process of business planning and budgeting was done primarily by establishing cost centers and through the annual process of budgeting. Quarterly performance reviews were instituted at QECH, and the excellent understanding of hospital operations that this provides are integrated into the planning process. The system is excellent (and still improving) 6 The Baobab Health Partnership is dedicated to providing computerized hospital and HIV/AIDS systems to Malawi. This private NGO primarily operates on the personal financial resources of the founders (the wife works in the US as a pediatrician and thus provides the bulk of funding) while deploying computer hardware and software systems. Technologically, this system is the most robust and cost-effective health information system that this author (Greg Becker, Hospital Reform Specialist, HMIS consultant, and former faculty member, University of Maryland, Computer Science) has seen. USAID is strongly encouraged to provide all possible support to this organization and include it to the maximum extent possible in future programs. 32 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT and should contribute significantly to hospital improvements. A shortcoming of the process that must be addressed, however, is budget implementation. Recommendations • Given the highly volatile nature of disbursement of actual funds to the hospitals, the plans they make cannot at this point be linked with certainty to funds to be received. The result is that the hospitals must continuously function reactively and make ad hoc decisions on procurement and expenditures. A system of prioritizing procurement that can react swiftly and rationally to funding irregularities and shortfalls is needed. Financial, Human Resources, and Health Information Systems These systems implemented by the project at hospitals form the basis for quarterly performance reviews and data-guided decision-making. Although their operation is significantly hampered by changing personnel in hospital departments, the systems themselves are well developed and implemented. However, one shortcoming that needs to be addressed before the project ends is data and software security. Recommendations • At present, data backups are weak or nonexistent. Moreover, computers are often infected with viruses and very vulnerable to failure. Antivirus software should be provided, and the hospitals should have backup copies of all software. Other systems improvements were made with project assistance in the areas of transportation management, infection control, building and equipment management, accounting, the PHI, and pediatric triage and emergency care. The direct TA provided to QECH and KCH was extensive and well executed, and should contribute to permanent improvements in hospital operations. National TA to Achieve Hospital Trusts The long and difficult effort to achieve trust status for QECH and KCH has been hampered by high turnover of MOH and hospital staff. There has also been a significant problem caused by the conflicting interests of the institutions. It is therefore not surprising that trust status has not yet been achieved. MHS has worked within this complex and changing environment to establish hospital-based system improvements and draft legislation. A core belief was that TA from this project or its successor would continue to build political will and hospital capacity. As this is not to be the case, the core documents must be evaluated on whether they can provide a workable framework without external TA. At the moment, policies and enabling plans such as the Draft National Policy on Hospital Reform and the Discussion Draft Constitution of the Queen Elizabeth Central Hospital Trust are not aligned with Malawi’s resources and current capacities. The structures they outline are too complex and would likely prove unwieldy and costly even in the best of circumstances. The documents need major reworking if they are to prove useful to future autonomy efforts.7 A strategic error in the draft documents and plans was to view autonomy as something to be awarded only after the hospital achieved a superior level of performance. Autonomy is in fact an essential precondition to improved performance. Areas that may need rethinking: 7 Note that the evaluator did not have an opportunity to review the final drafts of these documents. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 33 • The Trust Board structure places fiduciary responsibility on the Secretary of Finance and the Secretary of Health together. • The board is to be composed of members who have full-time positions of great responsibility and, because of their status, have multiple and conflicting roles on other boards and organizations. • The draft reporting and performance requirements are not in line with the human resource and financial capacity of the CEO and QECH as it currently operates. • The structure outlined widens the gap between current hospital obligations and current funding. For the near future, operations will continue to be underfunded as they are now; any realistic options to generate and retain revenues are also likely to bring in far less than what is needed. • The structure outlined exacerbates the critical shortage of trained people. This human resource gap is not likely to improve in the foreseeable future, because the most qualified individuals continue to be taken out of the labor pool by donors and outside organizations. • Significant attention is given to the creation of plans and budgets, and not enough attention is given to dealing with budgetary shortfalls or rational and effective utilization of the limited funds that are allocated. • The way the College of Medicine (COM) is dealt with may institutionalize an adversarial relationship and threaten the continuation of medical education. The COM and QECH are symbiotically interdependent. The COM is realistically the sole potential source of expertise and support for the development and survival of trust status once project TA ends. Building an environment of mutual trust and recognition of mutual dependency should be the goal and tenor of documentation and planning. • Of the nine specific milestones listed in the Draft National Policy on Hospital Reform (Section 5, Implementing Hospital Reform) as necessary for implementation of trust status, the project completed only one. That these requirements could not be achieved with the assistance of project resources indicates a structural design that is far beyond current capacity. • The performance management agreement (PMA) required as a condition of autonomy is far too demanding and complex. A much simpler—and achievable—plan for initial autonomous operation is essential. Recommendations • Autonomy is desirable because it makes improved operations possible. To raise its level of performance, a hospital must begin operations under trust status and seek the gradual improvements that internally managed human and material resources make possible. It is recommended that the two trust documents be simplified. According to HPSA, revision has begun. Annotated versions of the Draft National Policy on Hospital Reform and the Discussion Draft Constitution of the Queen Elizabeth Central Hospital Trust are attached to this report in the hope that specific comments will facilitate rapid rewrite before the project ends. • The only likely path to trust status at QECH (KCH should follow later) is for the COM (and the MOH) to recognize and embrace the symbiotic relationship of the two institutions. All references to the COM in the two documents should be reworked to recognize this essentially 34 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT interdependent relationship. Given the reality of QECH, such a structure is the only viable form of trust that can be achieved. If this approach is taken, QECH is very close to becoming a trust hospital. It is therefore recommended that USAID seek the short-term services of an expert skilled in negotiating hospital autonomy to act as a mediator between the MOH and the COM. National Policy Development As a core activity of the project, draft policies were developed in the areas of biomedical research and national health policy. As in the case of the Draft National Policy on Hospital Reform and the Discussion Draft Constitution of the Queen Elizabeth Central Hospital Trust, the complexity of the drafts reviewed is beyond Malawi’s human and financial capacity. In the case of the national health policy, which should be a brief document providing guiding principles for the health system of Malawi, the long and complex draft was a situational analysis supplemented by a hugely complex assortment of regulations. HPSA agrees that the draft reviewed was overly prescriptive. The latest draft submitted (Version 3; August 2007) is greatly simplified. The Draft Biomedical Ethics and Research Policy, Guidelines, and Procedures were drawn up at the request of the MOH because of substantial concerns about research practices in Malawi health facilities. This document, drafted by a consultant to the project, was presented to the three primary research institutions of Malawi: the National Research Counsel of Malawi, the National Health Sciences Research Committee, and the COM Research Ethics Committee. All three rejected it. A working group was appointed from the three groups to work with the consultant on the next draft. When the consultant presented the second draft, allegedly it did not conform to the recommendations of the working group or reflect the group’s discussions. Representatives of the organizations informed the evaluator that the working group is rewriting the document without the HPSA consultant. Recommendations: • A new draft of the National Health Policy was submitted during the evaluation. This draft (Version3: August 2007) should be reviewed carefully in view of the major problems in the previous document. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 35 36 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT V. FUTURE DIRECTIONS FOR USAID The purpose of this section is to share the team’s perspectives and provide general recommendations to USAID/Malawi for future programming, topic-specific recommendations having already been discussed. The recommendations mainly cover health systems strengthening and child health, but there are also insights for other public health issues. Some of our comments are intended to support and give some flexibility to the BASICS Project, a new activity just being launched to improve child health, but others are beyond the scope of the BASICS Project. The recommendations are based on the following assumptions: • USAID will continue to provide aid in Malawi through specific programs defined in the Mission’s operational program. • USAID will not participate in basket funding of the Malawi SWAp but will remain as a discrete donor; this role gives it the flexibility to respond to changes in situation and to provide a broad range of technical support to the MOH as well as to various USAID projects. Our recommendations fall into two groups. CONTINUED SUPPORT FOR STRENGTHENING HEALTH SYSTEMS MSH’s TA in strengthening district health systems, linked with targeted interventions to improve child health, has been uniquely valuable. The MSH project has earned a good reputation and was able to form strong working relationships with the MOH and other stakeholders. Efforts to strengthen MOH systems at the district, zonal, and national levels have improved the quality of and access to USAID child health programs. There is evidence that the district health strengthening interventions have had impact, and the evaluation team recommends that this assistance model be continued and rolled out. Assistance in planning and budgeting, integrated supervision, HMIS, transport and drugs management, and support to zonal offices has been particularly useful. We therefore recommend that the MOH again be given support in systems strengthening: • Management training in cost-effective modalities, such as on-the-job training, which has proven successful and efficient, should be continued. • Zonal offices should be strengthened to build their greater capacity for supervising and providing TA to districts—but assistance to zonal offices should complement, not replace, support to district management. • TA to the MOH should continue through several channels, including the embedding of positions within the MOH. It should consist of mentoring and empowerment of national professionals and of participation in MOH policy and planning activities. • The MOH should be supported through special projects to improve its planning and program oversight consistent with the SWAp and with USAID expertise. One example might be to invite staff from the USAID Global Health Division to support health systems strengthening efforts. Another approach might be to cosponsor a regional health systems reform conference where neighboring countries can share and build upon successful experiences in health reform. MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT 37 CONTINUED SUPPORT OF USAID PROGRAMS THROUGH SYSTEMS STRENGTHENING • Specific TA assistance should be provided to improve access to and quality of rural health care and coordinate the various programs affecting rural health. Examples are integration of malaria prevention and treatment into ANC, the coinfection treatment of TB and HIVAIDS, and integration of HIV/AIDS and reproductive health interventions. A focus on cross-cutting issues and a renewal of the holistic approach to the delivery of care could support efforts to build quality and improve access. • Finally, the team would like to stress Malawi’s overwhelming staffing and human resource management problems. These are not within the scope of work of USAID and are beyond the control of the MOH, but no project will reach its full potential to change the health situation until the staffing issues are addressed. The team recommends that future goals be adjusted in light of this situation. 38 MANAGEMENT SCIENCES FOR HEALTH–MALAWI FINAL EVALUATION REPORT For more information, please visit http://www.ghtechproject.com/resources/ Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com