September 2006 This publication was produced for review by the United States Agency for International Development. It was prepared by Chemonics International Inc. MALAWI DELIVER End of Project Evaluation The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. MALAWI DELIVER End of Project Evaluation Contract Number: GHS-I-00-03-00025-00 Order Number: GHS-I-07-03-00025-00 ACKNOWLEDGMENTS First and foremost, the evaluation team would like to thank the staff of John Snow Incorporated/DELIVER Malawi for creating an environment conducive to examining progress to date. The entire staff was supportive, and helped facilitate the evaluation. Second, we would like to thank the many stakeholders whom we interviewed in Lilongwe, as well as throughout the field visits. In all cases there was enthusiasm and interest in the progress of the project and in identifying ways to continue and improve activities in the future. Third, we would like to thank the Chemonics home office team — particularly Bill Slocum, Kuhu Maitra, Laura Shahi, and Lara Hensley — for their continued support of the field team. No evaluation is without challenges, and the home office support was particularly valuable in ensuring that field activities went forward with the least number of difficulties. Lastly, but certainly not least, we would like to thank USAID/Malawi, particularly Lilly Banda-Maliro, for her support of the evaluation and the team, and Beatrice Lumande, USAID/RCSA, for her contract support. We have made every effort to capture the comments of stakeholders as they were provided to us. We have interpreted the situation as best we can and given the information at hand. The findings, issues, and recommendations are ours alone, and any errors or omissions in this report are solely the responsibility of the evaluation team. Rene Berger Noreen Teoh Thuguri Mungai October 2006 CONTENTS EXECUTIVE SUMMARY ........................................................................................................................iii CHAPTER ONE SCOPE OF THE DELIVER EVALUATION...................................................................1 Introduction ..............................................................................................................................................1 Background..............................................................................................................................................2 Methodology ............................................................................................................................................4 CHAPTER TWO EVALUATION FINDINGS ............................................................................................6 Issues and Recommendations.................................................................................................................6 Central Hospitals......................................................................................................................................6 District Hospitals ......................................................................................................................................7 Health Centers .......................................................................................................................................10 Medical Stores .......................................................................................................................................13 Evaluation Questions and Answers........................................................................................................15 Conclusions ...........................................................................................................................................22 General Recommendations ...................................................................................................................23 CHAPTER THREE FUTURE DIRECTIONS ..........................................................................................25 Questions and Answers .........................................................................................................................25 ANNEX 1 REFERENCES .....................................................................................................................1-1 ANNEX 2 LIST OF PERSONS CONTACTED......................................................................................2-1 i ACRONYMS ACRONYMS AIDS Acquired Immune Deficiency Syndrome ART Antiretroviral Therapy ARV Antiretroviral BLM Banja la Matsogolo CDC Centers for Disease Control and Prevention CDLMIS Contraceptive Distribution and Logistics Management Information System CHAM Christian Health Association of Malawi CMS Central Medical Store CSH-MH Child Survival Health - Maternal Health DFID Department for International Development (United Kingdom) DHMT District Health Management Team DHO District Health Officer DHTSS Directorate for Health Technical Services and Support DNO District Nursing Officer EHP Essential Health Package e-PICS Electronic Pharmacy Inventory Control System FEFO First-expiry/First-out FP Family Planning FPLM Family Planning Logistics Management FTE Full-Time Equivalent GOM Government of Malawi HIV Human Immunodeficiency Virus HSA Health Surveillance Assistant ID Infectious Disease IDA International Dispensary Association IR Intermediate Result IT Information Technology IUCD Intrauterine contraceptive device IUD Intrauterine device JSI John Snow Incorporated LMIS Logistics Management Information System MERU Monitoring, Evaluation & Research Unit MHCLMS Malawi Health Commodities Logistics Management System MOH Ministry of Health MOU Memorandum of Understanding MSH Management Sciences for Health ii NGO Nongovernmental Organization NSSD National Stock Status Database ORS Oral Rehydration Solution PA Pharmacy Assistant PS Principal Secretary (reports to the Minister of Health) PSI Population Services International PT Pharmacy Technician RH Reproductive Health RHLMIS Reproductive Health Logistics Management Information System RMS Regional Medical Stores SCM Supply Chain Manager (software developed by JSI) SDP Service Delivery Point SIGMED Computerized warehouse software developed by MedICT SO Strategic Objective (USAID) SP Sulfadoxine/pyrimethamine STI Sexually Transmitted Infections SWAp Sector Wide Approach TA Technical Assistance UNFPA United Nations Fund for Population Activities UNICEF United Nations Children’s Fund USAID United States Agency for International Development VCT Voluntary Counseling and Testing WHO World Health Organization EXECUTIVE SUMMARY iii EXECUTIVE SUMMARY USAID/Malawi has supported health commodity logistics for more than 20 years. From 2000 to 2006, health commodity logistical support has been conducted under the global DELIVER project, implemented by John Snow Incorporated. The DELIVER contract is coming to an end, and USAID/Malawi is assessing the project’s contributions to commodity security over the past six years, the remaining gaps in support for logistics and contraceptive security, and the best way for USAID to continue its support into the future. A three-person evaluation team — consisting of one Chemonics International full-time staff person with experience in Malawi, and two Chemonics consultants with experience in pharmacy and commodity logistics — was dispatched to Malawi for three weeks (August 19th to September 9th). The team met with stakeholders in the capital Lilongwe, as well as in central and district hospitals (Mzuzu, Rumphi, Mzimba, Salima, Lilongwe, Mchinji, Balaka, Blantyre, Mulanje) health centers, and Christian Health Association of Malawi (CHAM) facilities in each region of the country. In each region and district, the team met with personnel from the regional medical stores, district health management team, pharmacy staff, medical assistants and nurses, and drug committee members. At the national level, the evaluation team met with stakeholders within the Ministry of Health, and with development partners. The team also met with private sector personnel from Population Services International and Banja La Matsogolo. During the period of the evaluation, there was significant upheaval within the medical stores system, with allegations of corruption among key staff and continued shortages of medical supplies across the system. As a result, many of the pharmacies visited were experiencing significant shortages of key commodities, particularly antibiotics. DELIVER Accomplishments. It is clear that DELIVER has significantly benefited the commodity logistics system (the commodities themselves are essentially out of the control of DELIVER). DELIVER has supported successive versions of a logistics information system. DELIVER has also been instrumental in initiating and institutionalizing the “direct delivery” system, whereby commodities are sent directly from the medical stores to health facilities. The recent installation of the Supply Chain Manager software at the district level has been a clear benefit for pharmacy technicians, enabling them to compile the information needed for monthly requisitions. Through these accomplishments, the DELIVER project has helped USAID/Malawi achieve its Strategic Objective 8. Remaining Challenges. There are still a number of areas that require improvement, such as a transparent system to track commodities from receipt at medical stores to distribution at the health facilities. Opportunities to mismanage drug supplies result directly from the absence of a tracking system; as districts are now poised to manage their own drug budgets, such a system becomes paramount. Insufficient Ministry of Health staffing at the national and district levels has pushed DELIVER into an implementation role, as opposed to a technical assistance role. As a corollary, the absence of true DELIVER counterparts means that technical functions have not been transferred to MOH iv MALAWI DELIVER FINAL EVALUATION staff. These limitations have been recognized by all parties, including the MOH itself, which is committed to improving the human resource situation in the future. Future Procurement Mechanisms. USAID/Malawi remains committed to commodity security, and the completion of the DELIVER contract provides an opportunity to review and revise how support is provided to the Government of Malawi. Looking forward, the mission has two main options: continue support through the follow-on DELIVER contract (yet to be awarded); or devise its own local procurement. There are pros and cons to each option. As the DELIVER follow on will also be a global project, there will be an opportunity to learn lessons from other countries that can benefit Malawi. However, the central project may limit the degree of specificity that USAID/Mali can exercise in designing a project for the Malawi context. Issuing its own solicitation would allow the mission to more specifically target activities to the current context and would allow greater control. The mission must determine the appropriate acquisition or assistance instrument and respective scope of work or program description, and should have a greater level of involvement in its implementation than in the past. Overall Conclusions. Commodity security has been improved in Malawi over the past six years. This can be rightfully attributed to the activities of DELIVER, as well as other inputs. The current commodity crisis is not due to DELIVER; however, the project has an impact on the viability of the overall system. Consolidating actions, functions, and systems in the near and longer term will allow for greater control over the entire commodity system and should lead to improvements in staff morale, transparency and, in the end, client satisfaction. There is a clear opportunity now to impact future systems and support – an opportunity that all the partners involved with commodity security should welcome. CHAPTER ONE EVALUATION SCOPE 1 CHAPTER ONE SCOPE OF THE DELIVER EVALUATION INTRODUCTION The United States Agency for International Development (USAID) has provided logistical support for health commodities, including reproductive health commodities, in Malawi since 1988. Various government assistance contracts have been used, including the Family Planning Logistics Management (FPLM) project, and since 2000, the DELIVER project. As it concludes, USAID/Malawi is evaluating DELIVER’s contribution to achieving its Strategic Objective 8: “Increased use of improved health behaviors and services.” DELIVER is also being analyzed for guidance on the direction on future assistance, if any, for health commodity logistical support in Malawi. This evaluation highlights the achievements brought about by John Snow Incorporated (JSI)/DELIVER over the previous six years, identifying any promising or “best” practices, as well as discussing the challenges faced during implementation. The report also provides recommendations for future assistance in the area of logistics management. The primary audience for this report is USAID/Malawi; however, the findings and recommendations may also interest and benefit the Ministry of Health and Population (MOH), multi- and bilateral donors (“development partners”), and DELIVER project personnel. During the course of the evaluation, the flow of health commodities was tracked from central medical stores to regional medical stores, and from there to central hospitals, district hospitals, and health centers. This included Christian Health Association of Malawi (CHAM) hospitals and clinics in Malawi’s three regions. At each level, discussions were held with available personnel: operators of medical stores, district health management team members, medical assistants, pharmacy technicians, and matrons/nurses. Interviews were also conducted with members of the drug committees (more often known as Community Health Committees). Additional discussions were held in Lilongwe and Blantyre with MOH staff, USAID, development partners, DELIVER staff (current and former), and other stakeholders. At the medical stores, questions focused primarily on the systems in place to fill requests from facilities, ensure timely procurement and appropriate stock levels, and on issues of particular importance and/or future needs. A physical review of the state of the medical stores was also undertaken. At the health facility level, questions focused primarily on the state of the pharmacy, distribution issues within the facility, stock levels and stock-outs, monthly ordering processes and timelines, and other issues and needs, including a review of the training received to date through DELIVER. At the time of this evaluation, the medical stores system was under considerable stress with significant stock-outs and human resource constraints. While medical stores were not the primary focus of the evaluation, they are key to the availability of commodities throughout the system and to staff morale everywhere; medical stores also were a huge factor in the evaluation team’s ability to assess DELIVER’s effectiveness at all service levels. 2 MALAWI DELIVER FINAL EVALUATION BACKGROUND Malawi, like most other countries in sub-Saharan Africa, relies mainly on the public sector for the delivery of health care services to its citizens. The MOH, with the support of donors, has designed a health commodity delivery system aimed at improving service delivery to clients. The current system is designed to deliver health commodities from central medical stores (CMS) to regional medical stores (RMS), district hospitals, and health centers. The following diagram illustrates this movement as well as the flow of related information: Movement of Health Commodities to Clients and Movement of Information Between Levels HTSS SENIOR LOGISTICS OFFICER CENTRAL MEDICAL STORES CHIEF PHARMACIST REGIONAL MEDICAL STORES PHARMACIST-IN-CHARGE DISTRICT PHARMACY PHARMACY TECHNICIAN/ASSISTANT HEALTH CENTRE HEALTH CENTRE-IN￾CHARGE Legend Information Flow Supply flow Sold items Ministry of Health/DONOR CENTRAL/MENTAL HOSPITALS PHARMACIST CLIENTSS NGO/CLINICS /VCT Centres CHAM CLIENTS CLIENTS NGO/CLINICS/ CLIENTS VCT Centres As a major donor to Malawi’s health sector, USAID objectives over the past two country strategic planning periods have focused on improving access to and behaviors associated with good health practices. Result areas have focused on increasing access to family planning commodities as well as increasing the range and quality of health services and products available to Malawians. Primary area of DELIVER support CHAPTER ONE EVALUATION SCOPE 3 USAID/Malawi has supported health commodity logistics, primarily for family planning and reproductive health, for almost 20 years through successive programs — FPLM and DELIVER — managed by JSI. Throughout this time, JSI has had staff on the ground in Malawi, with considerable back-up support provided through its offices in the United States. Both FPLM and DELIVER are called “central projects” in USAID nomenclature, meaning that the contracts themselves are not specific to a given country, and USAID bilateral missions may “buy in” to the package of services provided under the overall contract’s scope of work. A central contract such as DELIVER can benefit a country like Malawi when the experience and lessons learned from multi-country implementation are brought to bear on its own problems. On the other hand, a central project can limit the specificity of a program funded under its particular scope of work. During the implementation periods for both FPLM and DELIVER, donor support for health commodity logistics management in Malawi went through many changes. While historically a core set of commodities has been purchased through the government’s medical stores system, in recent years a number of donor supported vertical programs and associated systems have proliferated. Necessarily, the systems used to track the commodities have changed over time. DELIVER activities shifted away from the medical stores in particular, as other development partners were providing significant support to this aspect of the supply chain. Early USAID support to commodity logistics systems focused primarily on contraceptives (e.g. oral pills, Norplant, IUCDs, etc.) and resulted in the development of the Contraceptive Distribution and Logistics Management Information System (CDLMIS). Based on the success of the CDLMIS in ensuring contraceptive availability, CDLMIS was later updated to include eight drugs used for the management of sexually transmitted infections (STIs) and renamed the Reproductive Health Logistics Management Information System (RHLMIS). In 2003-2004, the system was again updated to track what was then identified as a core set of 80 essential drugs. The system was again renamed, now as the Malawi Health Commodities Logistics Management System (MHCLMS). Each system iteration has been supported in some fashion by USAID. In country, the DELIVER team consists of a resident logistics advisor and two other technical staff who focus on supervision and oversight for project activities across the country. DELIVER also has one administrative assistant who has received training in logistics. All of the in-country staff are Malawian. DELIVER/Washington provides overall technical leadership and supervisory support. DELIVER project activities have undergone significant transformation over the six-year implementation period. While initially expected to focus exclusively on contraceptive logistics, project duties have evolved along the lines of the management systems it sought to support. The DELIVER management and reporting structure has also evolved, starting with their move from the Reproductive Health Unit of the Ministry of Health, to become part of the Directorate for Health Technical Services and Support (DHTSS). This move brought DELIVER closer to the decision makers within the MOH, and also broadened project activities away from an exclusive concentration on contraceptives to a more inclusive focus on the commodities provided under the essential health package (EHP)1 . 1 The essential health package includes over 300 unique drugs and supplies; however, DELIVER has only recently been requested to track 80 of the more commonly used items. 4 MALAWI DELIVER FINAL EVALUATION USAID has been procuring all contraceptives except Depo-Provera and condoms for use in the public sector since the inception of family planning programming in Malawi. However, the mission recently procured an emergency supply of Depo-Provera to help avert a nationwide stock-out. Ideally, contraceptives should be procured as part of the overall procurement process within the Ministry of Health/Medical Stores, and should be included as part of the request for resources through the Sector Wide Approach (SWAp). DELIVER has provided support to the MOH in forecasting contraceptive requirements and providing annual contraceptive procurement tables, and has carried out reviews of consumption and stock status surveys on a periodic basis. These activities have more recently (June 2006) been integrated into a national stock survey covering 80 key commodities from the essential health package. As DELIVER concludes, USAID/Malawi is evaluating project accomplishments to date. The scope of work for the evaluation includes two primary pieces: a retrospective review of accomplishments, challenges, best practices and overall lessons learned; and a prospective review of remaining areas where USAID could provide continued support for commodity logistics in Malawi. METHODOLOGY To evaluate the DELIVER project, the team traced health commodities from their source – medical stores — to delivery points, including central hospitals, district hospitals and health centers. This tracing included both Ministry of Health facilities as well as sites managed by the Christian Hospital Association of Malawi (CHAM). At each step in the process interviews were held with key personnel including the district health management team, pharmacy technicians (where available), and the person in charge of a facility (usually a medical assistant, but sometimes the matron/nurse). Discussions were also held with members of the drug committee from the health centers’ catchment areas. In addition to these site visits, interviews were held with personnel from the Ministry of Health and Population in Lilongwe, USAID, other development partners, and private sector partners. Since DELIVER/Malawi activities are being implemented nationwide, the team attempted to organize a representative sample of sites in each of the three regions — north, south, and central. Visits were made to central hospital and district hospitals, government and CHAM facilities as well as NGOs and central and regional medical stores. The team started in Salima, and then traveled north to Mzuzu, Rumphi, and Mzimba Districts, then south to Balaka, Zomba, Mulanje and Blantyre Districts, before returning to the central region to review Mchinji and Lilongwe Districts. In each region, the regional medical stores were reviewed and personnel were interviewed. Given the short duration of this assignment, and other logistical challenges, some constraints applied. While the team made every effort to speak with the stakeholders who are involved in health commodity logistics at the national level, some may have been missed. The team was unable to meet with the Minister of Health, who has taken a keen interest in commodity security and the current challenges that the medical stores are facing. The team did travel to a number of CHAPTER ONE EVALUATION SCOPE 5 districts in every region; however, they were unable to review each district where DELIVER has carried out activities. Even within the districts that the team visited, not all staff were available to be interviewed. Frequently, district health management team members were difficult to find, and those interviewed may not have been the most aware of the project’s activities. In general, the medical stores system is undergoing significant stress related to scarce human resources and commodity unavailability. Numerous stock-outs were witnessed, primarily in the central and north regions, which may also have an impact on staff morale. Additionally, at the national and regional levels, the logistics package (SIGMED), which is to be used by medical stores, is not functioning. It was envisioned that the Supply Chain Manager (SCM) software implemented by DELIVER and the SIGMED system implemented by the medical stores would interact, but this has not been the case. As a result, there is no electronic record system in operation that can track commodities from the medical stores to district hospitals. Lastly, the evaluation team did not solicit significant feedback from “end users,” or beneficiaries. With the exception of contraceptives, it was deemed to be too great a challenge for patients to determine whether they had received adequate treatment. However, interviews with the drug committees did focus on the availability of contraceptive commodities and this was taken as a proxy for appropriate prescribing patterns of the providers. CHAPTER TWO EVALUATION FINDINGS 6 CHAPTER TWO EVALUATION FINDINGS ISSUES AND RECOMMENDATIONS The team traveled to each region, visiting central, district, and mission (CHAM) hospitals, health centers, and medical stores. Findings — pertinent issues affecting commodity security — and associated recommendations are presented below. CENTRAL HOSPITALS The team visited three of the four central hospitals: Kamuzu (Lilongwe), Mzuzu, Zomba, and the Queen Elizabeth Central Hospital in Blantyre. Two of the four are under the Hospital Autonomy Initiative: an effort to develop the central hospitals into self-sustaining entities, without recurrent funding from the Government of Malawi. The Kamuzu Central Hospital is piloting the e-PICS (Electronic Pharmacy Inventory Control System) developed by the Baobab Health Hospital Reform Program. One of the objectives of the system is to establish an effective tool to operate inventory control and manage pharmaceutical supply costs efficiently and effectively. The hospitals obtain their supplies directly from the medical stores and the reports are sent directly to Ministry headquarters. The e-PICS system is able to generate the Logistics Management Information System (LMIS) reports2 (similar to what SCM provides), which are sent to the MOH. Only Zomba Central Hospital is using the SCM. The pharmacy technicians at Zomba and Mzuzu have been trained in the SCM. The Mzuzu Central Hospital was constructed by the Taiwan Medical Mission, which continues to support the hospital in various technical assistance areas, even though the hospital has already been handed over to the Government of Malawi. An inventory control system built on a unique web-based platform has been developed to monitor the Mzuzu pharmacy stocks. Data from the other hospitals is sent to DELIVER. Issue 1: There are no specific guidelines on how the Kamuzu and Queen Elizabeth Central Hospitals should share information with DELIVER; they order directly from CMS and use their own discretion to decide how and when to send information to DELIVER. It is also not clear if or when these organizations will become autonomous, and how information will flow to CMS for procurement purposes. Issue 2: The Ministry of Health has not developed an information technology (IT) policy to define the roles of different partners, including DELIVER, in the development of IT systems, and to ensure that systems are integrated to improve data quality and reporting. Issue 3: The Mzuzu Central Hospital has not involved other stakeholders, such as DELIVER, as part of the broader LMIS system, which should ideally enable better information sharing and interaction. It is also not clear how and in what form their system could be integrated into other existing systems, and the lessons that could be shared with DELIVER. 2 LMIS reports capture dispensed-to-user data and stock levels at each health center, district or central hospital. CHAPTER TWO EVALUATION FINDINGS 7 Issue 4: The hospitals have not received guidelines on how data should flow from their facilities to DELIVER; it is left to the hospitals and DELIVER to agree on some reporting mechanisms. Recommendation 1: Reporting guidelines should be established for central hospitals, as their data are vital in setting up and maintaining an integrated management information system. Recommendation 2: The MOH should develop an IT policy to guide the development of information systems and support. The policy should reflect the input of all stakeholders. The policy should also be clear regarding who should develop the systems, and where the coordination of IT will take place in the Ministry. DISTRICT HOSPITALS There are 26 districts (plus Likoma Island) in the country. Each has a district hospital that is headed by a district health officer (DHO). District hospitals order supplies from the RMS on a monthly basis. The requirements are calculated by the SCM, based on a three-month quantity reorder level. DELIVER staff undertake supervision once every quarter to collect data and to provide any IT support for the SCM. District Health Management Team (DHMT) The DHO is responsible for the day-to-day management of the hospital and district in general, supported by the DHMT. The members of the DHMT include the DHO, hospital administrator, district nursing officer (DNO), accountant, and the district environmental health officer. The pharmacy technicians are not members of the DHMT, but are members of the hospital drug committee (which also includes the DHO and the DNO), and may be co-opted into the DHMT based on identified need (as opposed to by practice). Issue: The DHOs do not seem to have adopted a proactive approach for keeping abreast of the status of stocks at the pharmacy. Most were not very conversant with DELIVER, although they had a vague idea that there was some form of support directed towards improving drug logistics management. The same case applies to other DHMT members who also had limited knowledge of DELIVER. There is a feeling that DHMTs do not regard the pharmacy as part of the health delivery system, and they tend to overlook its importance. Recommendation: The DELIVER project may have overlooked the need to develop mechanisms to involve the DHMT more in the management of the pharmacy. With the new decentralized focus, the DHMTs will need to be updated periodically on the status of stocks and budget, versus relying on expenditure reports to monitor their budgets. Some orientation into the whole district supply chain management will be necessary as a way of mainstreaming pharmacies into hospital delivery system priorities, as well as making DHMT members aware of the range of information they can receive from the system. This should be a priority action. Pharmacies All district hospitals have pharmacies, each with a designated store room. Some district hospitals have additional stores or lockers for antibiotics or antiretrovirals (ARVs) depending on the set up and available space. They also have refrigerators for cold chain storage. The pharmacies are 8 MALAWI DELIVER FINAL EVALUATION headed by a pharmacy technician (PT) who may be assisted by other PTs or pharmacy assistants3 , depending on human resources available. Dispensing is done by the PTs. Issue: The PTs rely frequently on the pharmacy attendants to assist them, especially with dispensing, though this is outside of the protocols. This is due to their being already overloaded with other administrative issues and reporting duties. The attendants do not receive any form of training — even in the basics of pharmacy technology — and rely only on the knowledge they have acquired on the job. Human Resources. The number of staff varies from one district hospital to another. In districts with more staff there is likely to be better quality reporting, better organized stores, and faster dispensing of drugs. Issue: In most district hospitals visited, it was evident that more PTs are required. Due to the shortage of drugs a lot of work hours are wasted shuttling between districts and the RMS, trying to obtain emergency stocks as monthly orders are often incomplete and delayed. Recommendation: The MOH should assess human resource needs at the pharmacy level with a view to strengthening existing staff levels. The MOH should consider strengthening the capacity of the PTs to be pharmacy managers, while storage and maintenance of stores records could be done by stores personnel hired by the ministry. Whether the stores personnel would combine both pharmacy and stores skills should be left to the MOH to decide. It is evident that the PTs are spending too much time on storage and recording activities and neglecting their core business. There should be a reinforcement of MOH policies regarding prescribing patterns at the district hospital level to ensure that only those individuals who are qualified to prescribe/dispense are carrying out these services. Training. DELIVER has trained all district PTs in both logistics management and supply chain manager software. The training is based on a curriculum developed by DELIVER, and covers all the components of a Health Logistics Management System. Training in SCM was conducted by DELIVER IT staff. Issue: All the PTs interviewed indicated that the training they received was adequate and that they had knowledge and skills to conduct logistics functions without any difficulties, including orienting new staff who had not been trained (though there was little evidence that this was actually done). The trainings were also reinforced by supportive supervision of DELIVER staff. Ordering. District hospitals are responsible for ordering all supplies for all health centers within their jurisdictions, as well as for ordering their own supplies. All orders from the district are sent to the RMS by the 10th of each month. Deliveries are expected to start from the 15th of each month and continue for about two weeks. 3 According to the Malawi College of Health Sciences, the two-year Pharmacy Assistant training program is being phased out. The present training for Pharmacy Technicians is three years. Current Pharmacy Assistant program participants may apply for the Pharmacy Technician program with an additional 1.5 years of training. CHAPTER TWO EVALUATION FINDINGS 9 Issue: The RMS has not been delivering commodities within the stipulated period and there are delays of more than one month (some facilities had not received supplies for two months). Stock￾outs then ensue occur and the number of emergency orders increases, making them more routine. During team visits in August, a number of districts had not received their July commodities, and were experiencing stock-outs. The following examples highlight delays in the receipt of commodities as of September: Mzimba District Hospital – supplies for June not yet received Jenda Health Center – supplies for June not yet received Mchinji District Hospital – supplies for July not yet received Recommendation: The RMS should ensure delivery of commodities within the stipulated time. Delivery schedules should be communicated to the districts and health centers, and followed. Stock Position The district pharmacies and the health centers are expected to maintain inventory levels of three months and to place orders every month. This system works well provided the RMS can regularly deliver the requested commodities. Issue: There are constant stock-outs of essential drugs, hindering the smooth running of pharmacies and hospitals. Some district hospitals in the northern and central regions had stock￾outs of the following: Mzimba District Hospital – no condoms or Ovrette Nkoma Mission Hospital – no Ovrette Mchinji District Hospital – no doxycycline or erythromycin Contraceptives District hospitals provide oral contraceptives (Ovrette and Lo-Femenal), injectable (Depo￾Provera), IUD (Copper T), implant (Norplant), and barrier (condom) methods. Contraceptives were available at all facilities, though few sites had the entire method mix (health centers are not expected to insert implants). Issue: Although Ovrette had been received by medical stores (after a prolonged stock-out) it was still missing in a number of facilities within the central and northern regions even after being delivered to the RMS. Proper communication mechanisms for informing the districts when new stocks arrive are critical. Recommendation: The ongoing reforms at the medical stores should bring on line systems that can integrate not only the central medical stores with the regional medical stores, but also other systems such as Supply Chain Manager and e-PICS. Other forms of communicating availability of stocks will be important in the near term until electronic systems can be implemented. Orders from Clinics DHOs may also serve health clinics or dispensaries which are smaller than health centers, but who order and collect their supplies directly from the district pharmacy. The clinics use the 10 MALAWI DELIVER FINAL EVALUATION LMIS 01C form to order their supplies. There is no provision for the person receiving the supplies to sign for them and confirm receipt. Recommendation: Order forms should be re-written to provide both requisitions and receipts, thereby allowing clinic staff to sign and maintain one copy when receiving supplies. HEALTH CENTERS The MOH has 405 health centers under DHO jurisdiction in the country. In most cases, health centers are managed by medical assistants, who are supported by either one or two nurses. The health centers also have health surveillance assistants (HSA) who work and reside in the community. In a number of the health centers visited, the HSAs are also responsible for the provision of voluntary counseling and testing. Ordering The health centers receive their supplies directly from the RMS through the “direct delivery” system. Health centers are currently sending their monthly reports (LMIS01A) to the district hospital pharmacy, where the information is entered into the SCM, which generates the requirements. Requisition forms are then completed by the PTs and sent to the RMS. A number of the health staff interviewed indicated that they sometimes receive supplies that they did not order (something RMS staff vehemently denied). Some commodities are also received close to their expiration dates. Goods are received by one member of staff and one member of the drugs committee. The members of the drug committee are from the community surrounding the health centers, and have a vested interest in seeing facilities receive the commodities they need to treat patients. Issue: As in the case of DHOs, the health centers do not always receive their supplies on time. The erratic delivery of drugs had been going on for some time, with some health centers experiencing intermittent shortages that can last up to five months. For example, at the Jenda Health Center, the team was told that the facility did not receive drug supplies during the five months from September 2005 to January 2006. Neither did the center receive drugs in May and June 2006. At Kochilira and Tembwe Health Centers in the central region, the supplies for July had not been received by early September. Most health centers reported that they provide the district with their stock status reports by the 5th of the month as requested; some sites implied there were occasional delays. District pharmacy staff (PTs) reported that most facilities do submit their reports on time; the occasional outliers are sites farther away from the hospital. Based on interviews with staff at the health center level, the advent of the direct delivery system has improved the availability of commodities. Although the health centers do not order for themselves, some PTs make it a point to send a copy of the requisition form generated by SCM to the centers. The health centers do not have delivery schedules; they wait for deliveries to arrive. Although the DHO should be servicing emergency orders, this has become increasingly challenging as hospitals themselves have limited supplies. Although the central RMS has delivery schedules, they were not being followed. The schedules called for all the deliveries to be CHAPTER TWO EVALUATION FINDINGS 11 completed by August 28th, but this clearly did not happen based on the evaluation team’s field visits4 . Recommendation: A standardized delivery note for use between medical stores and service delivery points would allow facilities to better manage the commodities they receive. The delivery note that is currently being used in the north and central regions provides most of the information required, and could serve as a national standard. What remains to be included are the initial quantities ordered by the district so the site can ascertain orders vs. receipts. This will allow health centers to understand what commodities have been ordered for them (and what they actually receive) and will enhance transparency across the system. Health centers will require continuous support to ensure the timely submission of their stock status forms. Should there be a review of the entire status reporting and ordering process, it may be useful to assess the degree to which stock status reports, and consequently ordering, might be staggered across the month (as opposed to having all orders stacked up at the RMS on or about the 10th of the month). All issues that relate to the supply of commodities from the medical stores must be addressed within the broader medical stores reforms, and should involve all the stakeholders, such as the end-users, DELIVER, and others. Stock Status Health centers are required to maintain three-month inventory levels, while their review period is monthly. The system is realistic if the stocks are available at the medical stores, but this is not the case currently. Most of the health centers had limited numbers of the essential drugs and other supplies. For example: Mhuju Health Center – Co-trimoxazole out of stock since June Jenda Health Center – no erythromycin, paracetamol, spirit, co-trimoxazole, or Ovrette Phimbi Health Center – no amoxicillin (both capsules and syrup) or paracetamol Mdeka Health Center – no sulfadoxine/pyrimethamine Tembwe Health Center – no antibiotics, only one box of 100 condoms left Kochilira Health Center – no Ovrette, only two boxes of condoms left Recommendation: The direct delivery system needs to be strengthened, especially from the RMS, side by ensuring the following: ƒ Delivery schedules are developed and communicated to all the DHOs/PTs for onward transmission to the health centers ƒ Delivery schedules are honored ƒ Improved allocation of supplies ƒ Criteria used to ration are as objective as possible. 4 In general, all deliveries were to be completed by the 28th of each month, but varied widely, particularly in recent months. 12 MALAWI DELIVER FINAL EVALUATION Training All the medical assistants have been trained in logistics management. The PTs also provide supportive supervision and on-the-job training during their visits to the health centers. They are able to complete the LMIS forms and make the necessary reports. Recommendation: With decentralization in full swing, health center staff must be trained in drug management as part of capacity building; this could be incorporated into other training opportunities to use resources efficiently. Contraceptives The health centers provide oral contraceptives (Ovrette and Lo-Femenal), condoms and Depo￾Provera. Depo-Provera was available in all of the health centers visited, but stock-outs of Ovrette were still occurring. In general, most facilities reported that women by far prefer injectable contraceptives to any other form. This was clearly demonstrated in the reviews of facility records, which showed well over 90 percent of women choosing Depo-Provera over any other option. Though not an area the evaluation team reviewed extensively, it is possible that this bias towards Depo-Provera has resulted in the under-utilization of other commodities, such as Ovrette, which are expiring on the shelves at facilities. This may also impact the rate at which new supplies are ordered. Recommendation: If stocks are to be reviewed prior to making new procurements, it may be worthwhile to review the method mix and ascertain which methods are more commonly used than others, as well as seek opportunities to promote a wider mix of methods for couples. CHAM Hospitals CHAM hospitals attempt to buy drugs and other supplies from medical stores but often the drugs are not available. CHAM hospitals manage their own drug budgets and therefore can procure supplies from private pharmaceutical companies or CHAM pharmacies. CHAM operates a revolving drug fund at the head office with three distribution points in Lilongwe, Blantyre, and Mzuzu. They have developed procedures for monthly procurement. They receive STI drugs; FP commodities, and condoms free from the medical stores, but pay a 5 percent handling fee (this fee is 12.5 percent on non-donated commodities, and charged to both CHAM and MOH sites). Issue: Although CHAM would prefer to procure their supplies from the medical stores due to their wide range of commodities, it is clear that they have become disillusioned by the perpetual lack of drugs, and find it a waste of time and resources to continue to seek supplies from medical stores only to return to their sites with just a handful of items. This forces the facilities to resort to other sources such as direct purchase through the International Dispensary Association (IDA) and other local pharmaceutical companies. By failing to address the current inefficiencies in the system, medical stores are losing opportunities to add CHAM hospitals to their resource base. According to CHAM personnel, the Memorandum of Understanding (MOU) between CHAM and the government is expected to streamline the existing support and also enable the government to allocate funds to procure drugs. CHAPTER TWO EVALUATION FINDINGS 13 Recommendation: CHAM hospitals would make great referral centers for the MOH because they have more personnel and are getting support from their parent churches overseas in the form of specialized equipment and staff. Training While it is important to ensure that all users of government health supplies are trained, only a limited number of CHAM PTs have benefited from training provided by DELIVER, which is partly due to high turnover rates among CHAM staff. Issue: None of the pharmacy staff working at the CHAM hospitals and health centers visited — apart from the PT at the Mulanje Mission Hospital — had been trained. DELIVER training has reached only a limited number of staff. Recommendation: There is a need to involve more CHAM staff in future training programs and to increase the level and regularity of interactions and communications between CHAM and DELIVER staff based in Lilongwe. MEDICAL STORES There is one main central medical store that acts as a transit point for receiving and dispatching commodities. The store has limited storage facilities and goods are issued — “pushed” — out to the three regional medical stores as they are received using the following formula: 20 percent to RMS/North; 35 percent to RMS/Center; 45 percent to RMS/South. The medical store supplies both the government and NGOs, such as CHAM and, in the near future, BLM. There are various categories of staff supporting the medical stores as follows: ƒ Pharmacists ƒ Store personnel ƒ IT personnel ƒ Accountants All professional staff apart from the pharmacists are directly linked to their line departments. For example the store personnel belong to the Central Government Stores, the information technology staff to the Department of Information Systems, and the accountants to the Accountant General. The procurement staff belong to the Department of Public Procurement. Issue: The linking of different personnel working at the medical stores to their line departments means that the MOH, and by inference the medical stores, have little control over where staff should perform their duties or how to retain them, as they can be transferred any time. This interferes with proper management, planning, training, and retention of staff. Recommendation: The MOH should lobby the government to change human resource policy so that the ministry can hire and train its own logistics, procurement, accounting, and IT personnel. This will ensure that the ministry has control over the staff and can budget and plan their training without fear that trained staff will be transferred somewhere else. 14 MALAWI DELIVER FINAL EVALUATION Systems The United Kingdom’s Department for International Development (DFID) has historically supported the medical stores system, including the placement of technical advisors. Through the support of DFID, a warehousing system known as SIGMED was developed by a company called MedICT. The system served the stores until 2005 when it collapsed under unclear circumstances. Following SIGMED’s collapse, the medical stores reverted to manual systems. As a result, it is very difficult to determine the total stocks on hand across the system, or their allocation across the regional medical stores. Issue: Manual systems for collecting and processing information of such magnitude are subject to significant errors and invariably cause delays in reporting. The delays start from the receipt of goods and continue all the way to the dispatch points, where information is processed slowly. The SCM software cannot provide information regarding supplies on hand at the RMS level (but does provide data on stocks on hand at the district hospital and health center levels), creating a gap within the supply chain system. Recommendation: It may not be possible to revive SIGMED, but an urgent need remains to get a system running that can support the warehouse function and provide information to both ministry headquarters and the medical stores, as well as other users within the supply chain. Although the need is critical, it is also important to ensure that previous mistakes are not repeated. Therefore an assessment of the existing systems, current needs, gaps, and integration opportunities is needed. This system (whether a revision to SIGMED or a new approach) must be able to integrate with SCM or capture the functions carried out by SCM, thereby providing one information system capable of tracking commodities from warehouse through distribution. Security and transport systems also require improvement, and should not be overlooked during assessments and in the design of future support. Training Most of the staff have attended the DELIVER training held in-country, and the PTs have attended the DELIVER training in the U.S. Some senior staff have also benefited from UNICEF sponsored study tours to their procurement department in Copenhagen. Recommendation: The ongoing reforms at the medical stores should ensure that improvements to computer systems result in integration between the central medical stores and RMS, and also with other systems such as SCM and e-PICS. Delivery Schedules The medical stores are expected to develop a delivery schedule each month. However, this schedule is not communicated to the DHOs or to the health centers. Commodities are delivered against a delivery note that is signed by one of the staff of the health center, a member of the drugs committee, and the person making the delivery. Issue: In the southern region the RMS do not use a delivery note, but rather the request form from the SCM. The deliverer puts a tick against what they have issued, the person receiving the commodity at the health center makes a cross against the tick, and a member of the drugs CHAPTER TWO EVALUATION FINDINGS 15 committee crosses the tick again as a way of verifying. Some medical stores lack delivery schedules; for example, there was no schedule for the August deliveries at Mzuzu. The absence of a proper delivery schedule means that time management and planning is lacking, leading to delays at all points of the delivery process. Recommendation: Operating systems between the central and regional medical stores need to be standardized. This should also be addressed as part of the broader reforms to improve the warehousing systems. Space Management Issue: Additional warehousing space is needed, but the space currently available should be used properly. Most of the items across all the stores are poorly warehoused: there are no signs or labels to identify the locations and type of product, and many supplies have not been properly stacked. Poor space management puts pressure on the available space. At the RMS in the north and center, some drugs can be found at different locations (including across warehouses) making them difficult to locate, especially given the current lack of a functioning information system. Recommendation: A major exercise to reorganize the medical stores so that space can be better used should be undertaken. This should include installing proper racks or shelves and arranging proper signage and labeling. Expired drugs, items that are no longer in use, damaged and obsolete items should all be disposed of in order to release space. A review of the disposal of expired drugs across the system, including at the district and health center levels, is necessary. Allocation of Stocks The CMS has been using a population-based “push” system to determine the stocks allocated to each region. This is a very simplistic system that does not take into account available consumption and morbidity data which can be used to make adjustments as necessary. Issue: The current system does not use stock reports for allocations. It is a “push” system, based on population criteria of 20 percent in the north, 35 percent in the center, and 45 percent in the south. These criteria are defective and lead to most of the problems relating to overstocks and stock-outs. For example, it was noted that some antibiotics, such as co-trimoxazole, erythromycin, doxycycline, and amoxicillin, were missing at most facilities within the central and northern region, but were available in the southern region. Recommendation: An assessment of the allocation system should be undertaken, including a review of the current system and its impact on commodity availability. The assessment will be useful to show what is currently available and what can be reallocated to other facilities. Once the major issues have been identified, a more rational system based on “pull,” or consumption data, should be developed. EVALUATION QUESTIONS AND ANSWERS The following section answers the specific questions that USAID/Malawi has posed as part of the evaluation scope of work. 16 MALAWI DELIVER FINAL EVALUATION i) What has been the overall impact of DELIVER’s support for contraceptive logistics? Can changes in commodity availability be attributed to DELIVER? Can the project’s activities be sustained without further USAID inputs? Overall, DELIVER has significantly improved the timeliness of inventory/requisitions from the health centers to the district hospital pharmacy, and has improved data availability including monthly updates on consumption and requisition levels for the upcoming month. DELIVER has also improved the ability to obtain automatic calculation of amounts needed by each health center (through the SCM program operated at the district level). The advent of direct delivery from the RMS to the health center, rather than via the district hospital pharmacy, has improved the timeliness and ease of receipt of commodities, and the monthly LMIS reports from the district level can (potentially) serve as useful data for improving estimated needs of each item, consumption of each item, and the identification of discrepancies. There was consensus among interviewees that DELIVER was largely responsible for the gains seen across the logistics systems and improvements in commodity availability (present situation excluded). In general, stock-out rates for condoms and STI drugs have shown improvements.5 There is clearly increased availability of information to determine stock levels and even consumption through the improvements to the LMIS and the presence of SCM. Storage conditions at the medical stores are poor: there is limited space, and this is not well used due to inadequate shelving and improper warehouse management. While this is not within the mandate of DELIVER, this disarray affects the health facilities within the MOH system and cannot be overlooked. None of the pharmacies visited had adequate space. Staff made do with what they had, frequently leading to boxes being piled on top of boxes. While staff have been trained on proper storage procedures, the limited space likely impacted their ability to implement this training. At present, improvements made through DELIVER cannot be sustained without continued support from a logistics support project like DELIVER. This sentiment was echoed by all personnel interviewed, from the PS to facility staff. The routine of submitting requisition forms (e.g. LMIS01A) from health centers to district hospital pharmacies by the 5th of the month, and district hospital pharmacies submitting the entire district’s LMIS to regional medical stores by the 10th of the month, has only been in place for the last two years. Maintaining this schedule, which some facilities are already challenged to do, will require continued supervision. Clearly, the MOH continues to face human resource challenges and there is a high rate of turnover of healthcare worker staff (including PTs and PAs). These factors alone make the DELIVER system unsustainable at present; the factors are also largely outside of the control of DELIVER to change. 5 It should be noted here that both contraceptives and STI drugs have long been handled through vertical programs. Beginning in 2004 this was changed to integrate procurement of these commodities into the basket as part of the SWAp arrangements. In the near term, this change to an integrated system has lead to higher stock out rates, but this is not due to a lack of information being available for determining procurement levels, but rather due to the novelty of the SWAp and confusions at Medical Stores. CHAPTER TWO EVALUATION FINDINGS 17 ii) What role has DELIVER played in system improvements for contraceptive security? What functions have been transferred from DELIVER to the MOH? How has DELIVER improved local capacity to manage contraceptives in Malawi? DELIVER’s support for the various iterations of the logistics system has included a continued focus on the supply of contraceptives. The project’s annual implementation of stock supplies, leading to the development of contraceptive procurement tables, has been delayed for various reasons. However, each successive version of the logistics system retained a contraceptives assessment. At the service delivery level, most staff had been trained to maintain stock status cards and complete monthly stock status reports, including contraceptives. The transfer of logistics capacity to the MOH has met with varying degrees of success. At the national level, there has been limited skills transfer because of the limited human resources available from the ministry. This is true across most activity areas including forecasting, logistics management, training, supervision, and monitoring and evaluation. Supervision has improved with the advent of Zonal teams. However, these team members seem to have a large mandate and limited logistics and pharmaceutical management skills. At the district level, staff have been trained to compile reports (e.g. LMIS01A) from health centers and submit requests to regional medical stores, and some PTs have been trained as trainers. It is not clear how much time these individuals, who likely could carry out a supervisory role, have to actually supervise other facilities as they are frequently the lone “pharmacist” at the facilities where they work. DELIVER/Malawi has opted for a lean staffing pattern. Their current location at ministry headquarters seems logical, though the directorate to which they report should be reviewed. The greatest challenge noted by the DELIVER staff themselves regarding their current ministry location is their ability to be heard by relevant decision makers. DELIVER activities on a day-to￾day basis seem to be adequately carried out by the existing in-country staff. Most trainings and significant oversight appear to be carried out with assistance from the U.S.; there seems to be about one visit per quarter based on a review of trip reports. This seems excessive from a travel/funding point of view, and brings into question the skills, capacity, and autonomy of the in-country staff. iii) What role has DELIVER played in improving the supply chain? Have these improvements been transferred to the MOH? Are there factors which have assisted or hindered this transfer? DELIVER has clearly had a leading role in developing and updating logistics management systems. The current use of SCM (version 3) highlights this progress, with the previous systems (e.g. RHLMIS) being integrated into the current version. The most significant challenge in this regard is the integration of SCM with software at the regional medical stores. Ideally there should be a seamless data flow from the medical stores to the district level and back; this is not currently possible as medical stores systems are not functional. SCM itself, while not proprietary, may be limited in its ability to reach significant scale (due to limitations with the Microsoft Access database format used to house data). Activities carried out through the Hospital Autonomy Initiative, as well as through bilateral donors such as the Government of Taiwan, are also implementing electronic information systems. At a 18 MALAWI DELIVER FINAL EVALUATION minimum some effort to reconcile these efforts to agree on common standards allowing for data integration is critical and overdue. The existing system requires monthly updating with data being returned from the health center. Late receipt of these data hinders the collation process at the district level and subsequently results in late ordering from medical stores. While health center systems are paper based, they still require a physical stocktaking followed by collation and submission of the document (LMIS01A) to the district. The current process of moving from paper to electronic systems at a district level is logical; however, it naturally results in some backlogs as data must be entered manually and some districts have many health centers. Limiting factors in sustaining these efforts stem from the availability of equipment, both hardware and software, and human resources. Because not all staff have been trained to use SCM — and there seems to be limited skills transfer within a facility from trained to untrained staff — any loss in these personnel will bring the system to a halt. DELIVER assistance included significant inputs from JSI headquarters in Arlington, Virginia. At some points in the project lifespan, there appeared to be almost quarterly visits from the U.S. It is not clear if these interventions were due to expected program inputs, a concern over in-country management capacity, or some other reason. However, the expenses incurred to the project for this level of support are not insignificant, and it is not clear whether the activities could have been carried out by in-country staff. There also appears to have been a large number of backstops to the project over its six year lifespan. While turnover is inevitable, these changes undoubtedly resulted in additional technical assistance visits in order for headquarters staff to understand in-country activities. iv) How has DELIVER improved coordination among stakeholders? What are the strengths and weaknesses of these coordination efforts? Has this led to increased funding levels for contraceptives? While DELIVER has little authority or mandate to coordinate activities regarding logistics for the ministry, they have been able to bring interested parties together to discuss the status of commodities within the country. Unfortunately, there is still very limited consumption level data within the country to be able to assess real needs and gaps. While many stakeholders see the value in such information, it is costly to collect and to date no one has been willing to cover this cost. The most recent quantification exercise undertaken by DELIVER in June 2006 was the closest attempt to date to retrieve consumption data at a national level, and this effort left significant gaps. DELIVER has managed to raise the necessary alarm bells — albeit not always before the outages have occurred — to relevant stakeholders to avert what could have been significant and long-term stock-outs (e.g. the recent emergency procurement of Depo-Provera by USAID). However, to raise an alarm, there must be a direct intervention at the district level to retrieve data from facilities about shortages: there is no other way for information collected at the district level to reach the central level. This has been a DELIVER function, but ideally should become a routine procedure, either by sending files via the internet, or through an integrated system. This is a limitation the ministry noted: there is simply no system which provides MOH staff in CHAPTER TWO EVALUATION FINDINGS 19 Lilongwe with up-to-date information on stocks on hand, consumption, and procurement requirements. Overall, development partners and other stakeholders appreciate the work that is carried out by DELIVER. The project has been able to develop good working relationships with donors. In fact some donors recognize the challenges the project must face with their location inside the ministry. Partners have appreciated the efforts that the project has made in disseminating information, and only ask for more frequent reporting. v) What areas require further support in order to institutionalize supply chain functions? What DELIVER functions can be turned over to the MOH? What risks and/or benefits might be seen in this process? Currently, the human resources are not available within the ministry to take over DELIVER activities and expect them to be continued. Ministry officials themselves note that they have limited staff and while DELIVER should have counterparts, the ministry frequently operates in a gap filling mode. Ministry officials did express an interest in the information supplied by the project and asked that it be made available more widely and more frequently. Until such time as the ministry can guarantee that counterparts will be available, it will be very challenging for any logistics activities to be handed over to the government. USAID may need to extend the efforts they currently provide to ensure that counterparts become available. Alternatively, staff could be requested, if they are not already, through the SWAp. Regardless of the mechanism, personnel must be identified, and there should be reasonable assurance that they will remain in their positions for further capacity building and information transfer to be effected. Additionally, the “supply side” of the equation must be strengthened so that the system has in it the commodities that staff are supposed to track. There is a critical need for a clearly defined set of procurement policies, backed up with real data, tracking, and reporting systems at medical stores. DELIVER staff note: “No Product, No Program.” Given the current disarray within the medical stores system, and with the country facing potentially significant and deep stock-outs, there may not be a program to speak of soon. This does not mean that functions will stop completely, but without commodities, staff and patients alike will be disillusioned, and the morale of facility staff will likely suffer. vi) What experiences from the private sector could be used to support changes in the public sector in order to improve logistics? In an attempt to understand private sector logistics system approaches that could be used as best practices, two NGOs were visited. Population Services International (PSI), which is involved in social marketing of condoms, does not procure commodities per se; rather, they relay their requirements to the donors who procure on their behalf. PSI staff manage the packing and distribution. Banja la Matsogolo (BLM); a reproductive health NGO running 29 clinics across the country, conducts an annual forecasting exercise followed by procurements every six months. This is augmented by monthly purchases as necessary. 20 MALAWI DELIVER FINAL EVALUATION While there are few supply-related activities that might be directly adopted by the MOH, the annual forecasting and semi-annual procurement process may be worth examining. For the MOH, it is likely that more frequent procurements may be required, but a detailed review of the appropriate timing and scheduling of delivery would enhance this process. Across the CHAM facilities, there is an effort to provide semi-annual training/refresher courses on drug management and the rational use of drugs. Institutionalizing some time frame for refresher training within the MOH system would benefit existing and new staff. Additionally, and related to reforms which have been proposed but not fully implemented, CHAM facilities have their own directly managed drug budgets, with accountability for expenditures. For the MOH there will likely be challenges in implementing this system due in no small part to the assumptions made in the allocation of funding at the outset. Ideally, like consumption data in general, these estimated funding levels will be revised based on the experience at the district. vii) What challenges remain for planning and coordination regarding contraceptive security? How can these functions be institutionalized within the MOH? There are many actors with a keen interest in commodity security within Malawi. Many of the development partners — both those who currently contribute to the SWAp and those who are outside of the basket — have expressed concern over the current state of medical stores and the impending national stock-outs. Particularly, DFID, USAID, UNICEF, and the SWAp Secretariat are actively engaged in commodity security issues and have raised concerns over the crisis currently facing the country. There is an opportunity in the current awareness around commodities to ensure that all relevant stakeholders have access to the information which projects like DELIVER can produce. To date, dissemination of project activities has been a challenge due to various factors. In the current setting, however, there is a chance for DELIVER to shine by providing information and expertise to ongoing efforts. Ideally, this will result in a greater call for information from partners. In the past, the flow of information has not always been clear, and while DELIVER seems to have made efforts to disseminate its findings, they do not seem to have always been widely received and/or incorporated. Perhaps in the current situation there will be a greater call for such information, allowing DELIVER to take a more proactive stance in dissemination. One effort that seems to be gaining ground is the National Stock Status Database (NSSD). This system — which as currently envisioned seems to be an outgrowth of the SCM but with data aggregated from all districts — should allow personnel from the PS on down to access stock status (and financial status) information and have a better understanding of where the challenges lie. Until there are greater numbers of staff within the MOH, it will be difficult to institutionalize the functions that DELIVER is currently performing. The PS has shown an interest in increasing the number of staff allocated to logistics services, a necessary step before DELIVER can hand over functions to the MOH with an expectation that they will continue. CHAPTER TWO EVALUATION FINDINGS 21 viii) What challenges remain to achieving commodity security and improved logistics? How can MOH leadership in these areas be strengthened? The primary weaknesses in health commodity logistics are lack of human resources and poor information management. There are not enough staff to carry out all the necessary functions at the medical stores, district facilities, or health centers. There are also no systems to allow tracking of commodities, and therefore no empirical understanding of what should be procured, what could be reallocated, and where the consumption might be greatest. Until such resources and systems are in place, commodity security will continue to be at risk. The news is not all bad, however, and there appears to be a current opportunity to institutionalize required staffing — at least at the national level — to improve commodity security. This is an opportunity not to be lost. The upcoming stakeholder meetings are the venue to coordinate the inputs of the various development partners, and identify the remaining gaps. These gaps may in￾turn present new project areas for the DELIVER follow on project. Within the existing construct of DELIVER, the emphasis has clearly been at the district level. Here, the focus has been almost solely on the PTs. There is an urgent need to bring in the entire DHMT and enlighten them as to what information is available. Ideally, this would result in information feeding up to the national level, as DHOs begin to sound alarms on commodity shortages. This bottom up management may, in fact, be superior to the top down “push” coming from MOH headquarters. This is not to say that there cannot be improvements at Headquarters, but through the NSSD and other efforts, greater awareness of the issues will come forth and lead to meaningful responses. DELIVER is a commodities logistics management system that can be transferable, and not limited only to health commodities. To continue to remain as a health commodities logistics management system, especially in dealing with pharmaceuticals, the following factors need to be considered: ƒ Pharmacy technicians at the district level need to be trained and supervised to be “managers”; currently, their work is focused on logistics. ƒ Although guidelines are provided for the storage of essential medicines and other health commodities6,7, improvements still need to be made in the storage of health commodities.8 Most of the health commodities supplied are pharmaceuticals. To enhance the pharmacy technician’s role as “manager” of these items, his/her education and on-the-job training needs to be enhanced to include more of the therapeutic (“Selection” and “Use”) aspects of the Drug Management Cycle.9 6 John Snow, Inc./ DELIVER in collaboration with World Health Organization. Guidelines for the Storage of Essential Medicines and Other Health Commodities. 2003. Arlington, VA.: John Snow, Inc./DELIVER, for the U.S. Agency for International Development. 7 John Snow, Inc./ DELIVER. No Product? No Program. Logistics for Health. Guidelines for Proper Storage of Health Commodities. (Poster). Arlington, VA.: John Snow, Inc./DELIVER, for the U.S. Agency for International Development. 8 At least two regional medical stores visited had many cartons stacked on top of each other, way above the maximum of 8 feet specified in DELIVER’s Guidelines for Proper Storage of Health Commodities. 9 MSH and WHO. Managing Drug Supply. The Drug Management Cycle includes Selection, Procurement, Distribution and Use in the cycle. DELIVER’s system emphasizes the Procurement and Distribution aspects of the cycle, which is applicable to supply 22 MALAWI DELIVER FINAL EVALUATION CONCLUSIONS The support provided through the DELIVER project was linked to USAID/Malawi’s Strategic Objective (SO) 8, “Increased use of improved health behaviors and services” Specifically, the activities implemented over the course of the project were designed to contribute to the following intermediate results (IRs): 8.1: Behavior change enabled 8.2: Quality of services strengthened 8.3: Access to services increased 8.4: Health sector capacity strengthened Over its six years of implementation, DELIVER has supported the mission’s IRs. The development and rollout of the SCM is linked to IRs 8.1, 8.3, and 8.4; the coordination meetings which DELIVER has supported relate to IR 8.4. Taken together, the development, training, rollout, and follow up of the SCM system have improved information availability, provided an avenue to disseminate activities, and instituted a means for capturing data. The dissemination of this information has been used to support donor coordination and may have lead to some improvements in the institutionalization of support for commodities and the supply chain in general. While DELIVER’s objectives and results have changed over the course of six years, they have continued to be formulated to support USAID/Malawi’s SO and IRs. This does not mean there is not more to do to institutionalize the logistics systems in Malawi; however, improvements have clearly been made since the inception of the DELIVER project. The Government of Malawi, through the MOH, has also benefited from the DELIVER project’s activities. There is clear recognition that the achievements to date would not be possible without DELIVER, and an equally clear concern that the job is not finished and further USAID support is warranted (and requested!). While the present situation with the medical stores has made a complete assessment of commodity security more challenging, all stakeholders mentioned improvements in overall commodity availability. Here, too, there have been challenges, including the move from more discrete project funding to the SWAp. However, overall efficiencies have been seen and further improvements are expected. DELIVER has trained a number of staff, particularly at the district level. Pharmacy technicians, medical assistants, and nurses have all received training on the SCM and associated logistics forms, including the LMIS01A. Information flow has improved, with most sites submitting their stock status reports by the required time, and data as a whole is largely available across the country to determine where commodities may be in excess, and where shortages can be found. chain management of most commodities. The Selection and Use aspects of the Drug Management Cycle emphasizes the application of clinical knowledge to enhance the holistic and effective management of drug supply. CHAPTER TWO EVALUATION FINDINGS 23 Here it can also be noted that the development and rollout of the SCM has led to improvements in data availability. Gaps in utilization still remain and further work is required, particularly with the DHMTs. But if implemented correctly, staff will realize there is a wealth of data available to them on a monthly basis which can be used to make real, evidence-based, management decisions. GENERAL RECOMMENDATIONS These recommendations are primarily targeted towards the work of DELIVER II unless otherwise stated. Information systems 1. Begin or increase collaboration with MOH’s Monitoring, Evaluation & Research Unit (MERU) and the MOH’s senior logistics officer. Also, collaborate with DHTSS in integrating LMIS data with the NSSD. The intention is to create a uniform information system in the near future, as emphasized by the PS. Additionally, hold regular meetings for the various parties involved in information collection, dissemination, and use, including systems which will integrate across platforms (e.g. from the RMS to the district and SCM). 2. As outlined by the PS (September 8th, 2006 debrief meeting at MOH), participate in creating a uniform information system, beginning with a meeting with all parties involved. Besides DELIVER, parties include: USAID, UNICEF, CMS, Glocoms, and the MOH senior logistics officer. 3. Identify locally available IT staff and ensure their familiarity with existing software (SCM), and involve such staff in future design activities across the logistics system. Local ownership and knowledge should be a key focus. This recommendation includes existing staff within the MOH to familiarize them with the existing and planned systems. This will cut down the need for expensive technical assistance from outside the country, and ideally lead to a more internalized and owned approach. Storage and organization Invest in general warehouse equipment (warehouse racks/shelves, proper receiving and shipping counters, clear aisle and other signage/labeling, forklifts, fire fighting equipment, locked delivery containers, improved security systems, etc.) and storage space/buildings as appropriate. This recommendation is a priority especially at the CMS and RMS levels. Although it has not been the mandate of the DELIVER staff to be involved at this level in the past, it may now be appropriate for the DELIVER II staff and their counterpart(s) (e.g. the senior logistics officer) to be involved. This is to ensure that the products will be made available from the medical stores level to, ultimately, the beneficiaries at the hospitals and rural health centers. Beginning at the medical store level on down to health centers, supervision, guidance and inspection of proper ways to manage the pharmacy and store rooms are required. For example, corrosive /flammable items such as hydrochloric acid should be separated from other items; 24 MALAWI DELIVER FINAL EVALUATION internals from externals; refrigeration for storing items at proper temperature; appropriate room temperature year-round (especially in health centers, where there may be no electricity). Pharmacy Management 1. The formal education and on-the-job training of pharmacy technicians needs to be enhanced to include the therapeutic aspects of managing drug supplies. This should include identifying other donors who may already be, or interested in, supporting similar training efforts to ensure no duplication, or the need for extended absences of key staff. 2. As treatment patterns for various diseases, including malaria and HIV, common in Malawi, continue to become more complex, there is a need to ensure that both the clinicians diagnosing, and the pharmacists dispensing are aware of the complexity of the regimens patients are to receive. With this in mind, a review of the Malawi Standard Treatment Guidelines is overdue.10 This review should address who is qualified to dispense from the pharmacy, and who is qualified to prescribe.11 Adherence to these guidelines must be part of the monitoring and inspection of these locations. Follow up should include a review of the dispensing environment which must be clean and organized for activities to be performed accurately and efficiently. 3. DELIVER staff should ensure that pharmacy technicians at district and central hospitals are active members of the DHMTs as well as active members of the hospital’s Drugs and Therapeutics Committee (if and when they do exist). 4. DELIVER staff are encouraged to regularly stress to the DHMT the importance of pharmacy contributions to improving the supply of necessary drugs and related materials for hospitals and health centers. Pharmacy technicians should also be trained to be more visible and vocal to DHO and DHMT (something which may be counter to local culture, but will be beneficial in the long run), to address any issues that impede the operation of the pharmacy, i.e., supplies out of stock, human resource issues, etc. This increased communications at the DHMT level will allow the DHOs to better forecast the districts’ needs, and advocate on each district’s behalf. Information should not stop at the DHMT, but should also include the recently developed zonal coordinators. While the current system of having every health center report to the district by the 5th and the district submitting requisition forms to the RMS by the 10th is logical, and provides consistent data countrywide, it is not clear whether this is the most efficient from the RMS point of view. Following the submissions on the 10th, an RMS has two weeks to pack and deliver each facility’s requirements. This leaves another few weeks in every month where presumably the medical stores focus internally on their own management. However, the question arose whether it would not be more efficient for RMS to receive requisitions on a rolling basis across the month – with different districts submitting requisitions at staggered times. This may pose a challenge to facilities, and may impact the timing of data availability, but could potentially lead to efficiencies and fewer delays from the RMS to the health facility. At a minimum, some level of discussion regarding the timing of receipt and return of requisitions would be beneficial. 10 The last update to the Standard Treatment Guidelines was in 1998. 11 During field visits, at a District Hospital, a pharmacy attendant was observed to be dispensing and counseling patients even though there is a well trained PT on staff. CHAPTER THREE FUTURE DIRECTIONS 25 CHAPTER THREE FUTURE DIRECTIONS QUESTIONS AND ANSWERS i) Overall, what technical areas in logistics, procurement, and health commodities supply chain strengthening and contraceptive security should receive greater or lesser attention by USAID/Malawi in the coming three years? (Forecasting; Procurement of commodities; Logistics Management Information System; Establishing and Strengthening computerized systems; Drug/Commodity Policy; Training and capacity building) Clearly the current situation of medical stores has opened a window of opportunity to improve management of logistics and information systems all along the supply chain. Developing and/or ensuring that there is one integrated system along this chain would be a significant accomplishment. Forecasting methods must improve and expand to include a larger number of commodities, and therefore should be an extended mandate for DELIVER’s successor project. But forecasting inputs will have only a short-term impact if capacity is not built and institutionalized, simultaneously. USAID should work closely with the MOH to ensure that the FTE slots are available to recruit staff, should turnover continue to be an issue. One gap area also noted is the linkage between the clinician and the pharmacist/PT. Ideally, the two should work together collaboratively, with the clinician diagnosing the problem and determining the appropriate course of treatment and the pharmacist/PT reinforcing the course of treatment through drug dispensing. This is an area that could easily be strengthened through on the job training and a reinforcement of position descriptions. As the scope of DELIVER’s work has widened from RH and STI’s to the EHP, the mix of funds available to carry out the activities should also be broadened. Within the U.S. Government, this could include funding from different accounts within USAID (e.g. CSH – MH, ID (TB/Malaria), and HIV), but could conceivably also benefit from integration with the systems activities planned through the CDC. ii) What are the priorities to support continuing institutionalization of supply chain management functions in the Ministry of Health? To what extent is it appropriate to turn over functions of the DELIVER/Malawi office to the department of Health and Technical Support Services of the Ministry? What would be the benefits and risks? What steps can be taken to further this process and minimize the risks? With the current shortage of human resources and potential personnel shifts within the MOH/DHTSS, there is a lack of capacity to manage DELIVER functions internally. However, comments by the PS during the DELIVER debrief opened the door for expanded MOH staffing, and USAID should follow up on this opportunity. As a first step, staff positions could be added to the DHTSS (even if in the long run, the functions of logistics are better carried out through another section of the ministry). Positions can be “solidified” by ensuring they are budgeted, are advertised, and that the new staff have well identified counterparts. The director of the SWAp also noted that new positions could be requested through the Annual Implementation Plan, tying 26 MALAWI DELIVER FINAL EVALUATION positions that USAID supports into the overall staffing levels proposed by the MOH. Implementation planning also provides another forum to address staffing needs and begin to establish FTEs within the MOH. In the longer term, the dialogue should continue over where the logistics function should rest. The PS believes the function belongs along side the pharmacy. This may be the case, though some degree of autonomy may be beneficial. It may also be that in the near term, logistics would be significantly strengthened by inserting FTEs into the RMS structure. Given the potential to use other development partner resources to insert pharmacists into the RMS structure, linking them with logistics support could make a winning team for medical stores management. The positioning, level, and number of staff can be addressed in future program expectations and plans to be articulated by USAID. Ideally, these would be codified in a memorandum whereby both USAID and the MOH would agree to a proposed level of support. iii) What are the areas where the supply management experiences of the private sector should be applied to further improve logistics management in Malawi? USAID supports other commodity distribution efforts in Malawi, particularly those of Population Services International (PSI), whose current focus is on the social marketing of condoms, ORS, water purification, and bed nets. While these products are provided to the project, PSI is responsible for the onward sale to the user. PSI maintains four warehouses across the country from which they distribute products to retailers. As USAID and other donors manage procurements for PSI based on rough estimates of need, there are few lessons which can be transferred over to the MOH and DELIVER. While PSI enjoys good success, the process is based more on a “push” type of system to the retailers, rather than a true “pull” from consumers. Banja la Matsogolo (BLM) is another private sector company in Malawi that provides RH and STI services, including HIV testing. They currently operate a chain of more than 35 facilities (including clinics and outreaches) across the country and intend to expand further. They maintain their own budget to procure commodities and perform an annual quantification effort followed up by semi-annual procurements. BLM is interested in moving to a system like CHAM, that is, they would have an MOU with the MOH allowing them to procure commodities from the medical stores. Though this would be ideal, it is not advisable in the current environment. In addition to direct donor funding, BLM currently enjoys flexibility to procure from multiple sources. While they do not have the size of an MOH to leverage the best prices, their volume is not insignificant. In fact the supply practices of BLM and CHAM may be suited to the districts, given current efforts to decentralize and allow districts greater autonomy over their budgets. Under this scenario the district’s first choice in procurement would be the medical stores, but if they do not have the commodities required, there would be flexibility to procure from other sources (a real benefit to the patient). iv) What technical assistance needs are likely as the Ministry of Health transitions to a sector wide approach for procurement of health commodities? What steps can be taken to address choke points in health commodity supply within the Ministry? CHAPTER THREE FUTURE DIRECTIONS 27 The current situation has shown that better systems are needed to track existing commodities, to forecast future needs, and to structure procurements to ensure an uninterrupted supply of commodities. DELIVER or its follow-on will need to provide assistance, both through the existing data in SCM, and through expanded quantification exercises, to ensure that the MOH or medical stores (whichever is going to actually be responsible for procuring commodities), has the information they need. It is possible that USAID will need to procure emergency commodities in the near future, as the move to the SWAp leaves the country dangerously short of some key commodities. The challenge for USAID will be to determine where the resources can come from, and how to procure the required commodities in the most expeditious manner. v) What are remaining weaknesses in coordination and planning for contraceptive security in Malawi? What steps can be taken to strengthen and institutionalize Government of Malawi leadership and responsibility in these areas? Both the sector wide approach and the medical stores system have weaknesses in coordination and planning for contraceptive security. Assuming that the current LMIS (e.g. SCM) continues to function, information will be available on consumption patterns that can be used to forecast upcoming requirements and schedule procurements (lead times are also known within the country). What remains to plan and coordinate is funding to carry out the procurements and the system to distribute commodities. Outside of reverting to a vertical program, it will be difficult to ensure that these systems do not suffer some sort of breakdown in the near or longer term. The MOH has shown concern about the shortage of a number of commodities, including contraceptives; it remains to be seen whether the concern translates into significant action. This is likely an area which will require input from both USAID and DELIVER: USAID pushing to ensure that commodities remain available, and DELIVER providing the information necessary to quantify and forecast need, and to schedule procurements. At the same time the MOH must identify a “champion” to put contraceptive security on top of the health agenda. Apart from ensuring that commodities are available, the MOH should also address the critical issue of counterparts to the DELIVER project staff and increased capacity in health logistics personnel. vi) What can be done to further increase the understanding and appreciation of program managers, policy makers, civil society, and others in Malawi of the importance of sound health commodities supply management and of contraceptive security? There are a number of ongoing activities to coordinate logistics and emergency procurement to minimize impending stock-outs. These meetings provide an opportunity to convey and underscore the importance of logistics information and services. DELIVER should use the meetings as platform to explain the scope of current activities, and to liaise with other partners to identify additional needs and opportunities. 28 MALAWI DELIVER FINAL EVALUATION Whether fortunate or unfortunate, many of the medical stores’ current problems are being detailed for the public in newspapers. Clearly there is a heightened public awareness of some of the problems, as evidenced in op/ed articles, et cetera. DELIVER should also be encouraged to seek out forums to disseminate information on its activities and educate other stakeholders regarding commodity logistics, U.S. government inputs to the system, and achievements after the many years of continued support. ANNEX 1 REFERENCES 1-1 ANNEX 1 REFERENCES (Including bibliographical documentation, meetings, interviews, and focus group discussions) JSI/DELIVER work plan 2000-2002 JSI/DELIVER work plan 2002-2004 JSI/DELIVER work plan 2004-2006 JSI/DELIVER Country Strategic and Evaluation Plan 2002-2004 JSI/DELIVER Malawi Logistics System Assessment and Stock Status Report 2004 JSI/DELIVER Malawi Logistics System Assessment and Stock Status Report 2006 JSI/DELIVER Technical Assistance Record (various from 2001-2006) JSI/DELIVER Quarterly Report (various from 2002-2006) John Snow, Inc./ DELIVER in collaboration with World Health Organization. Guidelines for the Storage of Essential Medicines and Other Health Commodities. 2003. Arlington, VA.: John Snow, Inc./DELIVER, for the U.S. Agency for International Development. John Snow, Inc./ DELIVER. No Product? No Program. Logistics for Health. Guidelines for Proper Storage of Health Commodities. (Poster). Arlington, VA.: John Snow, Inc./DELIVER, for the U.S. Agency for International Development. GLOCOMS Inception Report 2006 O&M IT/S and Management Services Limited – Assessment of Drug and Medical Supplies Leakage from Medical Stores and Public Health Facilities (Confidential Report) Government of Malawi – A Joint Programme of Work for the Sector Wide Approach 2004-2010 MSH/WHO. 1997. Managing Drug Supply. 2nd ed. West Hartford: Kumarian Press. NEWS MEDIA ARTICLES RELATED TO MEDICAL STORE PROBLEMS “Police impound truckloads of government drugs” The Malawi Nation, March 22, 2006 http://www.nationmalawi.com/articles.asp?articleID=15783 “Foreign firm takes over medical stores” The Daily Times (Malawi), May 18, 2006 www.dailytimes.bppmw.com/article.asp?ArticleID=1777 “Govt moving in right direction on drugs but “… Comments The Daily Times (Malawi), May 18, 2006 www.dailytimes.bppmw.com/article.asp?ArticleID=1783 “Extend the swoop on illegal drug traders” Comments 1-2 MALAWI DELIVER FINAL EVALUATION The Daily Times (Malawi), May 26, 2006 www.dailytimes.bppmw.com/article.asp?ArticleID=1877 “Drug shortage hits referral hospitals” The Malawi Nation, August 5, 2006 http://www.nationmalawi.com/articles.asp?articleID=18085 “Heads roll at medical stores” The Malawi Nation, August 14, 2006 http://www.nationmalawi.com/articles.asp?articleID=18160 “Drug theft consultants claim threats” The Malawi Nation, September 5, 2006 http://www.nationmalawi.com/articles.asp?articleID=18455 “Drug shortages worry Ngaunje” The Malawi Nation, September 20, 2006 http://www.nationmalawi.com/articles.asp?articleID=18679 ANNEX 2 LIST OF PERSONS CONTACTED 2-1 ANNEX 2 LIST OF PERSONS CONTACTED U.S Agency for International Development Curt Reintsma, Mission Director Alisa Cameron, Health Team Leader Lilly Banda–Maliro, Reproductive Health Specialist Centers for Disease Control and Prevention Dr. Austin Demby, Country Director Ministry of Health and Population Dr. W.O.O. Sangala, Principal Secretary Dr. Ann Phoya, Director SWAp Lillian Ng’oma, Director HTSS Trish Araru, Health Planning Technical Officer, SWAp Secretariat Sam Chirwa, Senior Logistics Officer Chris Moyo Paul Dielemans, Sexual Reproductive Health Programme Development Officer Len van der Hoeven Office of the President and Cabinet Hon. Richard Msowoya, MP JSI/DELIVER Veronica Chirwa, Resident Logistics Advisor John Zingeni, Logistics Advisor Ruth Nyirongo, Administrative Assistant Central Medical Stores/Glocoms Charles Abondo, Glocoms Project Manager Malawi College of Health Sciences John Kandaya Management Sciences for Health/Health Partners Southern Africa Rudi Thetard, Chief of Party Cynthia Kamtengeni, Drugs Management Specialist Rodion Kraus, Hospital Autonomy Team Leader Population Services International Chikondi Khangamwa, Product Manager HIV Davidson, Logistics Manager 2-2 MALAWI DELIVER FINAL EVALUATION Banja La Matsogolo Linely Vinyo, Technical Advisor, Health Services United Nations Fund for Population Activities Dorothy Lazaro, Programme Officer United Kingdom, Department for International Development Joanna Nicholls, Health Adviser Cynthia Rowe, Governance Adviser World Health Organization Dr. Libambala World Bank Alfred Chirwa, Population and Health Specialist UNICEF Samayog Karmacharya, Supply & Logistics Officer Henry Neufville, Assistant Logistics Officer Christian Health Association of Malawi (CHAM) Sister Nympa Que, Pharmacist Baobab Health Partnership Gerry Douglas Judy Wang Mike McKay Oliver Gadabu Site Visits Regional Medical Stores (North, South, and Central) Salima District Salima District Hospital Mchoka Health Center Mchinji District Mchinji District Hospital Tembwe Health Center Kochilira Rural Hospital Mzuzu District Mzimba District Hospital Jenda Health Centre ANNEX 2 LIST OF PERSONS CONTACTED 2-3 Edingeni Health Centre Rumphi District Rumphi District Hospital Ekwendeni Mission Hospital Bolero Health Center Mhuju Health Center Balaka District Balaka District Hospital Phimbi Health Center Ulongwe Health Center Zomba District Zomba Central Hospital Blantyre District Blantyre District Health Office Mdeka Health Centre Mulanje District Mulanje Mission Chilomoni District Chilomoni Health Center Lilongwe District Nkhoma Mission Hospital