RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION July 2009 This publication was produced for review by the United States Agency for International Development. It was prepared by Steve Hawkins, Dr. Tsige Gebre-Mariam, and Erwin Lassooy through the Global Health Technical Assistance Project. ; RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 09-001-162) is available in printed and online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources.aspx. Documents are also available through the Development Experience Clearing House (www.dec.org). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION i ACRONYMS 3TC Lamivudine AA Addis Ababa ABC Abacavir ADR Adverse drug reaction ADR-KAP ADR-knowledge, attitude and practice ADT ARV Dispensing Tool AMR Antimicrobial resistance APR Annual Performance Review ART Antiretroviral therapy ARV Antiretroviral ATC Anatomical Therapeutic Chemical BPR Business Process Reengineering CAs Cooperating agencies CD4 CD4 (also called cytotoxic T-cells) lymphocyte count CDC Centers for Disease Control and Prevention CDR Central Data Repository CHAI Clinton HIV/AIDS Initiative CHBC Community and home-based care commodities CHBCK Community and home-based care kit COP08 Country Operating Plan for Fiscal Year 2008 CS Commodity security CSMB Corn-soya-milk blend CTX Co-trimoxazole d4T Stavudine DACA Drug Administration and Control Authority DBS Dried blood spot DG Director General DIC Drug Information Committee DNA-PCR Polymerase Chain Reaction test for DNA DTCs Drug Therapeutic Committees EAN European Article Number EFV Efavirenz EHNRI Ethiopian Health and Nutrition Research Institute EOI Expression of interest EPA Ethiopian Pharmacists Association FDC Fixed dose combination FEFO First expiry, first out FHAPCO Federal HIV/AIDS Prevention and Control Office FMOH Federal Ministry of Health ii RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION FO Field Office FTC Emtricitabine FTEs Full Time Equivalent (staff) GFATM Global Fund to Fight AIDS, Tuberculosis, and Malaria GH Tech Global Health Technical Assistance Project GOE Government of Ethiopia GPRS General Packet Radio Service HACTS HIV/AIDS Automated Commodities Tracking System HAPCO HIV/AIDS Prevention and Control Office HC Health center HCMIS Health center management information system HCSP HIV & AIDS Care and Support Program HCSS Health Commodity Supply System Master Plan HIV/AIDS Human immunodeficiency virus / Acquired immune deficiency syndrome HMIS Health management information system HO Home office HQ Headquarters HSDP III Health Sector Development Program Phase III ICAP International Center for AIDS Care and Treatment Programs (Columbia University) IDA International Dispensary Association IEC Information, education, and communications INRUD International Network for Rational Use of Drugs IP Infection prevention IT Information technology I-TECH International Training & Education Center on HIV IV Intravenous JHU Johns Hopkins University KHB Shanghai Kehua Bio-engineering Co., Ltd KPI Key progress indicator LMIS Logistics management information system LOE Length of Engagement LOP Life of Project MAARD Modified Acquisition & Assistance Request Document M&E Monitoring and evaluation MOH Ministry of Health MOU Memorandum of Understanding MSH Management Sciences for Health NGO Nongovernmental organization NRTI Nucleotide Reverse Transcriptase Inhibitors NVP Nevirapine OIs Opportunistic infections RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION iii PDA Personal digital assistant PEPFAR President‘s Emergency Plan for AIDS Relief PFSA Pharmaceutical Fund and Supply Agency PLMP Pharmaceutical Logistics Master Plan PLWHA People living with HIV/AIDS PMI President‘s Malaria Initiative PMP Performance Monitoring Plan PMTCT Prevention of mother-to-child transmission PSCM Partnership for Supply Chain Management PSLD Pharmaceutical Supply and Logistics Department RDU Rational drug use RHB Regional health bureau RPM Plus Rational Pharmaceutical Management Plus RTK Rapid test kit RUTF Ready-to-use therapeutic foods SAPR Semi-Annual Performance Review SCMS Supply Chain Management System SDP Service Deliver Point SNNPR Southern Nations, Nationalities and Peoples Region SOP Standard operating procedure SOW Scope of work SPS Strengthening Pharmaceutical Systems TA Technical assistance TB Tuberculosis TDF Tenofovir TLC Thin layer chromatography (a type of laboratory test) TOT Training of trainers TPM Team Planning Meeting TWG Technical Working Group UA Unitaid UNFPA United Nations Population Fund UNICEF United Nations Children‘s Fund USAID United States Agency for International Development USG United States Government USP United States Pharmacopeia VCT Voluntary counseling and testing WHM Warehouse manager WHO World Health Organization WIB Warehouse in a Box WMS Warehouse management system ZDV Zidovudine iv RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION v CONTENTS ACRONYMS ...................................................................................................................................... i EXECUTIVE SUMMARY ................................................................................................................ vii I. INTRODUCTION ..................................................................................................................... 1 CONDUCT OF THE EVALUATION ........................................................................................ 1 STRUCTURE OF THIS REPORT ........................................................................................... 1 II. COMPONENTS OF THE LOGISTICS CYCLE ....................................................................... 3 QUANTIFICATION AND FORECASTING .............................................................................. 3 PROCUREMENT .................................................................................................................... 6 STORAGE AND INVENTORY CONTROL ............................................................................ 11 DISTRIBUTION AND TRANSPORT ..................................................................................... 15 RATIONAL DRUG USE......................................................................................................... 17 LOGISTICS MANAGEMENT INFORMATION SYSTEM ...................................................... 22 III. QUESTIONS AND RESPONSES ......................................................................................... 29 TASK 1: ASSESS PROGRESS TO DATE IN ACHIEVING PLANNED RESULTS .............. 29 TASK 2: ASSESS COLLABORATION MODALITIES IN SUPPORT OF IMPROVED SUPPLY CHAIN MANAGEMENT ......................................................................................... 33 TASK 3: IDENTIFY ADDITIONAL APPROACHES OR ACTIVITIES TO ACHIEVE OBJECTIVES ........................................................................................................................ 35 APPENDIX A. SCOPE OF WORK ................................................................................................ 39 PROJECT IDENTIFICATION DATA ..................................................................................... 39 APPENDIX B. LIST OF DOCUMENTS REVIEWED ..................................................................... 53 RPM+/SPS DOCUMENTS .................................................................................................... 53 SCMS DOCUMENTS ............................................................................................................ 57 APPENDIX C. PERSONS CONTACTED ...................................................................................... 59 vi RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION vii EXECUTIVE SUMMARY During a four-week period in March 2009, a three-person technical team evaluated the support provided by United States Agency for International Development (USAID) to the HIV/AIDS supply chain in Ethiopia. The evaluation focused primarily on the activities of two projects supported by USAID: Rational Pharmaceutical Management Plus (RPM Plus)/Strengthening Pharmaceutical Systems (SPS) and Supply Chain Management System (SCMS). The evaluation had three overarching objectives: Compare project accomplishments and progress to scopes of work (SOWs) (50%), Evaluate collaboration and cooperation between the two projects and with other stakeholders in the sector (10%), and Suggest improvements for the future (40%). The team found an astonishing level of progress in the supply chain in Ethiopia in a very short period of time. In only a few years, the antiretroviral therapy (ART) patient load in Ethiopia has grown from a few thousand patients to nearly 150,000 at the time of this report. During that time, there have been no reports of substantial antiretroviral (ARV) stockouts. At the same time, the level of ARV expiry in the supply chain is relatively small. There have also been significant improvements in the availability of other commodities needed to support the growth of the HIV/AIDS sector, such as test kits, laboratory supplies, and drugs for opportunistic infections. Both projects extensively document their plans and progress. This documentation illustrates the high proportion of the projects‘ activities that are on schedule, or nearly so. The monitoring systems of both projects are designed to flag prospective problems while it is still possible to take corrective action. Coordination between the two projects and among the three projects in the supply-chain sector (these two and the USAID/DELIVER project—funded with USAID Population monies) has been somewhat less than ideal over the last year or so. This was because overlapping scopes of work and different approaches sometimes came into conflict. However, it appears that recent clarifications by USAID have reduced the confusion and improved this situation. In general, the evaluation team found that other stakeholders in the sector were quite satisfied with the level of support, coordination, and collaboration they received from both projects. In a few instances, there was some confusion as to different roles provided by the two projects (which can be perceived as having significant overlap), but also a feeling that simple contact with either of the projects would quickly provide access to the correct information source. In sum, the historic activities of these two projects have contributed significantly to the current outstanding success and growth in the HIV/AIDS program in Ethiopia. The project teams, management, and program funders have every right to be proud of their accomplishments. Looking to the future, these successes have resulted in a program that is large, fast-moving, and continuing to grow at a time when many of its fundamental underpinnings are relatively new, fragile, and still in a state of rapid transition. The evaluation team‘s visits revealed that there are several areas of concern about the logistics system‘s continued smooth operation and growth. While the system is working, it is more fragile than it is healthy, for such a large undertaking. In order to address these areas of fragility, while not undoing any of the good work that has been done, the evaluation team posed seven specific challenges in the road ahead: Transition to a full supply system: The transition of the logistics system from rationed supply to full supply is both recent and only partially complete, i.e. only a few commodities viii RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION have been in reliable full supply, and these only in recent years. It will take a great deal of reinforcement and some time for operational staff at the various facility levels to fully comprehend the changes implicit in full supply operations and the complexity involved in continuing to operate full supply and rationed supply systems side-by-side. Completing this transition, while continuing to provide support for a rapidly growing system, is likely to be a challenge. Transition to pull system operations: Similarly, the transition of the logistics system from ―informed push‖ to a pull system1 is both new and partial. Only in recent months have responsive ordering systems been put in place for the facilities, and then only for selected HIV/AIDS commodities. Operation of a pull system involves dramatic shifts in the locus of power and control, not simply changes in filling out forms and using different formulae. It will take time and reinforcement for this transition to be fully realized and for the new operational principles to be absorbed and internalized. Potential impact of changes in prescribing practices: Federal level policymakers are now in the process of changing the guidelines for prescribing practice for patients identified as HIV positive. At present, the guidelines indicate that patients with a CD4 level under 200 should be started on ART, while patients with a cytotoxic T-cells lymphocyte count (CD4) level under 350 should be placed on Co-trimoxazole (CTX). At the facility level, there are as many or more patients identified as ―pre-ART‖ as there are active ART patients. The current proposal is to change the recommendations so that patients with a CD4 level under 350 would be started on ART, rather than CTX. If this is implemented quickly, the ART patient load could roughly double in short order. It is not clear that the system is sufficiently robust to absorb such a quick change in terms of supply or distribution. Support of laboratory infrastructure and systems: While the supply chain for laboratory commodities has improved in recent months and it is now functioning in a fairly reliable and responsive manner, it was clear that laboratories‘ supply management practices and, in fact, their overall operations, are not as robust as, for example, that of the dispensaries. Much reinforcement of supply management procedures will be required in the coming months and several components of the basic operations, including power reliability and maintenance, will need to be addressed if the laboratories are to be able to support the continued growth of the HIV/AIDS program and the rest of the medical needs of the country. Potential system overcrowding: It was clear that although the vast majority of the stores were functioning at acceptable performance levels, they were operating close to full storage capacity. At the central level, significant volumes of orders for new commodities have been placed and will be delivered in the coming months. The evaluation team is concerned that these new volumes, on top of the full loads already being carried by the lower-level stores, may simply be too much for the lower-level facilities and for the transport system. Pace of transition of supply-chain responsibilities to the Pharmaceutical Fund and Supply Agency (PFSA): In the plans for the future of the health commodity supply chain in Ethiopia, PFSA figures quite prominently. For efficiency reasons, it has prospective roles in procurement, supply management, distribution, some data collection, and much of the information processing associated with managing the supply chain. In the long run, this could be a smooth running system. However, the evaluation team has concerns about the size of the 1 A ―push‖ system determines quantities to be distributed from information available at the center, or top, of the supply chain, typically targets of some sort. A ―pull‖ system determines quantities to be distributed from supply requests submitted by the prospective recipients, typically based on stock levels and product uptake. The ―informed push‖ system used in Ethiopia uses some data collected from the sites to determine distribution quantities, but is also significantly target-driven and rationed. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION ix demands that are about to be placed on PFSA, and the potential for collapse if demand is applied too quickly. Extensive and intensive capacity development assistance should be made available to PFSA in the coming months to ensure that it can handle the task load, grow its capacity, and support the system in the way it needs to continue growing. Sustainability: The activities that these two projects have been contributing to over their time in Ethiopia have contributed significantly to the success and growth of the HIV/AIDS program. Many of these activities have also improved infrastructure, systems, and equipment, increasing human capacity to support the country‘s health program. Many of the activities, however, are in the form of direct support for commodities, labor, or operational expenses. These activities contribute to the impact of the program, but are not sustainable without the continuing extensive external resource flows of recent years. If USAID is moving into a time of reduced funding, these activities will have to be seriously examined and other support found, or they will have to be curtailed. x RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 1 I. INTRODUCTION CONDUCT OF THE EVALUATION At the request of USAID/Ethiopia, a three-person technical team consisting of Steve Hawkins, Team Leader; Dr. Tsige Gebre-Mariam, Senior Pharmaceutical Technical Specialist; and Erwin Lassooy, Senior Logistics Specialist, was fielded under the auspices of the USAID/The Global Health Technical Assistance Project (GH Tech). The team was in Ethiopia from March 2 to April 1, 2009. Team logistics were supported by Kuleni Berhanu. The team‘s broad scope of work (SOW) was to conduct an evaluation of the activities and progress of the two major USAID projects—RPM Plus (succeeded by SPS in 2008), and SCMS—that had been supporting the HIV/AIDS supply chain in Ethiopia since 2004 and 2006, respectively. The complete SOW is attached to this report as Annex A and will not be repeated here. The team reviewed extensive documentation provided by USAID and by the two projects (see illustrative list in Annex B), interviewed knowledgeable individuals with experience of the two projects‘ activities in the supply chain (see list in Annex C), and traveled within Ethiopia to see sites throughout the supply chain. The team visited Ethiopia‘s four most populous regions, the capital city, and five of the nine regional logistics hubs. At the conclusion of the evaluation, the team conducted briefings with USAID and with local partners. The presentation materials for these briefings are also attached to this report as Annex D. STRUCTURE OF THIS REPORT This report is divided into two major sections. The first section, subtitled ―Components of the Logistics Cycle,‖ analyses the two project‘s contributions and the challenges within Ethiopia‘s HIV/AIDS supply chain using the components of the logistics cycle (see diagram below) as its organizational framework. This first section contains sections on Quantification, Procurement, Storage and Distribution (as two sections), Use, and Information. Figure 1: The Logistics Cycle Quantification Forecasting Use Procure￾ment Storage/ Distribution Information The second section, titled ―Questions and Responses,‖ provides a list of 30 specific questions that were posed by USAID in requesting the evaluation and responses from the team to each question. 2 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 3 II. COMPONENTS OF THE LOGISTICS CYCLE QUANTIFICATION AND FORECASTING Introduction Accurate quantification and forecasting are essential for all health commodities but are of vital importance for HIV/AIDS-related commodities because uninterrupted access for patients must be ensured. Therefore, routine and comprehensive quantification is a critical step in support of effective program implementation and expansion. Comprehensive and effective quantification typically requires multi-sector and multi-programmatic coordination between all stakeholders that rely on the availability of a reliable supply chain. This provides a wealth of information in support of (1) determining required funding needs and identifying potential funding gaps, (2) estimating the procurement effort necessary and (3) determining warehousing and distribution requirements. Securing HIV/AIDS -related commodities and sustaining the continuity of rapidly growing programs such as the one in Ethiopia require strong supply and resource management as well as continuous coordination and harmonization between all stakeholders. Determining pharmaceutical requirements is one of the more challenging tasks faced by health professionals and supply specialists. To add to the complexity, the most appropriate quantification method used in a setting such as Ethiopia (a resource-limited environment with a program in scale-up mode and with a lack of consumption data) is the morbidity method. Even when based on clearly set targets such as the expected scale-up of the existing number of patients receiving pre-ART treatment combined with real-time data on expected treatment regimens, treatment failure, etc., the morbidity method is basically a highly sophisticated and complex guessing game with a significant margin of error. Underestimating can result in drug shortages, interruptions in treatment, and expensive emergency procurements. Conversely, overestimating can tie up already limited resources and storage space and increase the risk of expired stock, wastage, theft, and diversion. To reduce the impact of errors, the quantification exercise needs to be repeated periodically, adding new information as it becomes available. Background The need for a comprehensive, harmonized, and coordinated quantification exercise in Ethiopia was underscored by the fact that the Federal Ministry of Health (FMOH) never performed a comprehensive quantification of HIV/AIDS commodities before SCMS began the practice. There was no formalized national-level coordinated system for quantification and supply planning—and as a result there was little or no information to support efficient allocation of financial and human resources in the Ministry and analysis of the commodity and financial gaps. In early 2007 the SCMS project was requested to provide forecasting and quantification of commodities to support the National HIV/AIDS program and, by doing so, to strengthen forecasting/quantification capacity within the Ministry and develop a number of tools and methodologies. The first quantification exercise was carried out in March 2007 and was soon followed by a second one in October 2007. The most recent exercise was finalized in December 2008. The data gathered by RPM Plus since 2005 has proven to be instrumental in determining a quantification baseline for ARVs. Approach and Methodology In response to the initial request by the FMOH and the Federal HIV/AIDS Prevention and Control Office (FHAPCO), SCMS provided technical and financial support in organizing national quantification workshops. The initial workshop was attended by a broad range of stakeholders such as FMOH, FHAPCO, Ethiopian Health and Nutrition Research Institute (EHNRI), HIV/AIDS implementing partners, international and other donor and technical support 4 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION organizations, and representatives from the Regions and other health programs. The methodology adopted for this and the following quantification exercises involved the following comprehensive approach: 1. The definition of scope was agreed through discussions with various stakeholders, including FHAPCO, the Pharmaceutical Supply and Logistics Department (PSLD), the Clinton HIV/AIDS Initiative (CHAI), the United Nations Children‘s Fund (UNICEF), USAID and the Centers for Disease Control and Prevention (CDC). These discussions led to the formation of a technical committee to develop the set of assumptions required for the quantification exercise. 2. The identification of data elements was undertaken by the quantification team and discussed with the technical committee prior to submission to the various expert panels for validation. 3. Agreement was reached on the assumptions used by the technical committees of identified experts in the clinical or commodity areas. The expert committees deliberated on the assumptions, data elements, and methodologies to be applied and arrived at consensus. 4. The results of the deliberations of the expert committees were presented to a plenary session for validation of the assumptions to be used in the quantification exercise. 5. Finally, the quantification team developed forecasts and supply plans based on the validated assumptions and other data that was obtained following the plenary sessions. The same overall methodology was used for all three workshops. However, over the last 1½ years the access, the quality and quantity of consumption and other program data has improved considerably, the number of participating stakeholders and products reviewed has grown. This has resulted in an exponential growth in the required HIV/AIDS-related commodities and budgets, which as of December 2008 were estimated to exceed $3 billion for a six year period (2009–2014). The data from these exercises has been used in two successful Global Fund to Fight AIDS, Tuberculosis, and Malaria (GFATM) proposals totalling over $800 million, as well as for supply planning and procurement. The continuous change of indicators, such as the number of commodities, product mix, and the treatment schemes being considered, as well as improved access to real-time program data, justifies continuous monitoring of procurement and distribution plans. This requirement is reflected in the projects‘ work plans. Progress Against the Work Plan SCMS has submitted a detailed work plan and measures its own progress on a regular basis. The work plan contains the following quantification-related objectives and activities: 1. Facilitate annual Quantification Exercise for HIV/AIDS commodities in coordination with HIV/AIDS Prevention and Control Office (HAPCO), PFSA, and EHNRI. – Coordinate timing and technical content (Completed) – Data collection, compilation and analysis: HIV/AIDS Automated Commodities Tracking System (HACTS) 2. Conduct quarterly review of supply plans for HIV/AIDS commodities and agree on procurement adjustments. – Technical Working Group (TWG)) for key commodity areas (Ongoing) – Commodity tracking mechanism (Completed/Ongoing) RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 5 – Monitor and update pipeline status (Ongoing) – Generate and disseminate reports (Ongoing) 3. Monitor and update pipeline status of HIV/AIDS commodities on a monthly basis (Ongoing). 4. Develop and implement a system for national forecasting and supply planning for HIV/AIDS-related commodities. – Information system for forecasting and supply planning (Completed/Ongoing) – TWG for the development of the technical area (pending identification of counterparts) – Work plans, roles and responsibilities for public sector capacity building (Outstanding, pending above) – Technical assistance and support (Ongoing) 5. Actively support HIV/AIDS Supply Management Advisory Committee of HAPCO (Ongoing). 6. Provide technical assistance and training to PFSA to integrate Quantification and Supply Planning Systems and pipeline monitoring systems for HIV/AIDS commodities. – Identification of technical counterpart (Completed) – Develop and endorse technical proposal for capacity building (After completion of Business Process Reengineering (BPR) by PFSA) 7. In collaboration with the Laboratory Team, support EHNRI in strengthening Forecasting, Supply Planning and Pipeline Monitoring for all lab commodity needs. – Identification of a technical counterpart (Completed) – Technical assistance (Ongoing) – Development of technical proposal for capacity building (after completion of BPR) 8. In coordination with commodity security (CS) activities, provide support to regional health bureaus (RHBs) and partners on quantification tools/systems, supply planning, and pipeline monitoring (to be started next quarter after the Regional CS coordinators are on board). 9. Document and share monthly pipeline monitoring and quarterly supply planning analysis and transfer reporting system to counterparts (Ongoing). A Performance Monitoring Plan (PMP) is in place and contains Key Progress Indicators (KPIs), clearly defined targets, and responsibilities. The following quantification-related KPIs are listed: 1 The number of pipeline reviews conducted and reported in a year, 2 Annual national quantification reports available, 3 The number of supply plan reviews/reports in a year, 4 The number and type of technical assistance (TA) provided to the TWGs, and 5 The number and type of TA provided to PFSA, EHRNI and HAPCO to build capacity. 6 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Observations and Recommendations Based on the documents reviewed and information gathered during stakeholder interviews, the quantification exercises have been successful in reaching the objectives. The evaluation team believes that efforts should be made to continue active support of these quantification exercises with continuous monitoring and adjusting of planned activities. This is reflected in the current work plans. A similar exercise for essential drugs and medical supplies might also be considered in order to further increase awareness and impact and to take advantage of synergies. The results are clearly target-driven, based on FHAPCO‘s Proposal for Strengthening of Health Sector Harmonization and Alignment in Ethiopia (the Road Map). They reflect the assumptions that the development of supporting systems and required infrastructure will be realized within the same timeframe. Based on the observations made by the team, this seems rather optimistic. Because this is an ambitious plan, which does not necessarily take the increased supply-chain challenges into account, the quantification data should be moderated based on realistic uptake figures when translated into procurement plans and delivery schedules. One crucial element is not reflected in the current quantification figures. If the current World Health Organization (WHO) CD4 count guidelines are adopted, it is estimated that the number of patients eligible for receiving ART treatment will roughly double from nearly 150,000 to over 280,000. If this is indeed implemented, it seems advisable to recalculate the current quantification data and consequent financial (and capacity) gap. The current supporting systems and supply￾chain infrastructure might not be sufficient to absorb the sudden influx of patients and products. PROCUREMENT Introduction Both RPM Plus/SPS and SCMS are or have been actively involved in the procurement of HIV/AIDS related commodities. Under SPS, the procurement and supply-chain responsibilities formerly handled by RPM Plus/SPS were shifted to SCMS and the current SPS involvement in procurement focuses primarily on data gathering and quantification. RPM Plus primarily utilized centrally-managed international mechanisms (UNICEF, International Dietary Association (IDA) , etc.) for the procurement of HIV/AIDS commodities. SCMS is currently active in both operational procurement of commodities and building capacity within the Ministry of Health (MOH)/PFSA. The SCMS procurement activities are supported by a team of six staff members dealing with the full range of HIV/AIDS commodities. The procurement budget over the last three years is an estimated $ 90 million (USAID/CDC, HAPCO/GFATM, and EHNRI/GFATM) and stands to increase as other donors are considering use of SCMS as their procurement agent. Background Based on coordinated procurement plans developed by FHAPCO and PSLD (now replaced by PFSA) with support from RPM Plus and SCMS, the procurement of HIV/AIDS related commodities has been handled through three major procurement mechanisms. PFSA (with actual procurement through UNICEF), CHAI and SCMS are currently involved in operational procurement. For all practical purposes, the rough division of operational procurement activities is based on the typical limitations of each of the procuring entities. As these limitations are mainly donor-related, the division is primarily based on the source of funding. Because CHAI‘s procurement activities are relatively small, this mechanism is not discussed in this report. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 7 The division is as follows: COMMODITY PRIMARILY FUNDED BY PROCUREMENT IS DONE BY: PFSA* CHAI/UA** SCMS*** ARVs: 1st line Adult GF X X ARVs: 2nd line Adult and all Pediatric CHAI X X Lab supplies and RTKs GF/ PEPFAR X X PMTCT Equipment PEPFAR X X Infection Prevention (IP) PEPFAR X RUTF PEPFAR/GF/CHAI X X OI/STI FMOH/GF/PEPFAR X X Condoms PEPFAR/GF X CHBCK PEPFAR X * ARVs are procured through UNICEF ** The UNITAID mechanism is currently being phased out and it has not yet been determined which institution will handle future procurement of these commodities. *** SCMS has procured almost all commodities at some time when a stopgap measure was required. Procurement by PFSA The Pharmaceutical Logistics Master Plan (PLMP) was first introduced in May 2006 as part of the Health Commodity Supply System Master Plan (HCSS), which was conceived in July 2005 as part of the strategic Health Sector Development Program Phase III (HSDP III). The PLMP aims for a harmonized and efficient system for procurement and distribution of essential health commodities for all levels of Ethiopia‘s public health sector. Although the PLMP was officially launched in October 2006, problems with its management, transition of the Central Medical Stores (PHARMID) from parastatal status to the public sector and related BPR activities delayed the actual implementation of the Master Plan. Finalization of the BPR process resulted in the recent transition of all MOH procurement and distribution activities from the PSLD and PHARMID to PFSA. In support of strengthening MOH procurement capacity, the MOH requested the Partnership for Supply Chain Management (PSCM) take the role as lead partner for the transformation of PHARMID to PFSA. FMOH requested that SCMS execute a mission with the objective to evaluate ―… current procurement capacity; to support PHARMID with the development of procurement tools as basis for the implementation of procurement practices acceptable both nationally and internationally; and to formulate the longer term options for technical assistance to actually achieve implementation.‖ Despite this formal request, it has proved difficult to provide the requested support because PHARMID/PFSA repeatedly refused to be subjected to an in-depth procurement assessment pending the outcome of the BPR process. To date it is still unclear how the substantial procurement responsibilities are managed by PFSA, how the procurement department is structured, and which guidelines are being followed. As a result of an extensive assessment supported by SCMS, clear and fully harmonized procurement guidelines and manuals have been developed, but it is again unclear if these will be adopted by PFSA. 8 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION During an interview with the team, PFSA‘s Deputy Director General (DG) indicated that he expects PFSA to be fully operational and geared up to assume its full procurement responsibilities within two years after its inauguration in September 2007 (i.e., six months from this writing). Although some progress is being reported, information gathered through informal interviews indicates that this timetable is overly optimistic. In fact, due to capacity constraints, many of the MOH procurements are currently being managed through an international contract or UNICEF and it remains questionable whether PFSA will be able to take on its full responsibility anytime soon. However, if PFSA succeeds in (1) implementing harmonized procurement guidelines, (2) adopting international best procurement practices as regards transparency which are acceptable to most if not all donors, (3) streamlining its operations, and (4) reducing the average procurement cycle time, it should be able to execute procurements in a timely, transparent, and competitive manner. In many cases procurement of HIV/AIDS commodities through PFSA may prove to be much more cost-effective because it will be less restricted by donor guidelines (although commodity restrictions will remain in those cases where GFATM or the President‘s Emergency Plan for AIDS Relief (PEPFAR) funding is involved). Also, the volume discounts that would result from combined procurement of HIV/AIDS-related drugs for opportunistic infections (OIs) or for other health needs could provide substantial cost savings. Procurement by SCMS Although global procurement is one of SCMS‘ strategic core functions at the central level, operational procurement at the local level has not been considered a core activity until recently. However, significant cuts in core funds for the U.S.-based procurement unit have forced SCMS to streamline headquarters (HQ) operations and to begin implementation of decentralized procurement activities. In support of this strategy and developments in the local market, SCMS has been gearing up a local procurement department, which currently consists of six Full Time Equivalents (FTEs): Two Quantification and Supply Planning Officers, Three Procurement Officers, and One Shipping and Clearance Coordinator. To date, SCMS has handled procurements for USAID/CDC, HAPCO (GFATM) and EHNRI (GFATM) valued at $90 million and handled more than 400 line items ranging from ARVs and rapid test kits (RTKs) to OI drugs and Community and Home Based Care kits (CHBCK). SCMS has proven to be capable of handling complex procurements within the restrictions associated with USAID funding and continues to improve this service. To achieve this, the following goals have been set: Continue support of procurement of required HIV/AIDS related commodities and supplies, Continue support of EHNRI in procuring essential laboratory supplies, Move away from ―emergency‖ response mode to a more systematic (information-based) approach to meeting growing demand (based on supply plans)—linking the distribution system with procurement, Close coordination with MOH, HAPCO, EHNRI, CHAI, CDC and PEPFAR partners on supply planning and pipeline monitoring, Ensuring timely movement of shipments, Implementing local procurement, RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 9 Establishing a pre-qualification process for local vendors, PSCM registration as a local organization, and Transferring Procurement Intelligence to PFSA. Progress against Work plan and performance monitoring SCMS has submitted a detailed work plan and measures its own progress on a regular basis. The work plan contains the following procurement-related objectives and activities: ACTIVITY TASK 1 Adequately staff Procurement Unit in Field Office to manage procurements of drug and nondrug supplies from local and international sources; closely oversee and monitor procurements of drugs supplies by HQ. Staffing includes Procurement Manager (possible expatriate position), two staff for Quantification/Supply Planning/Client Management, three Procurement focused staff, one Shipping and Clearing officer. Finalize local procurement (in-country procurement) proposal; identify required resources, training needs; system needs/changes. Establish approval limits (value and product). Develop work plan for implementation. Recruit Procurement Manager, as appropriate. Finalize appropriate standard operating procedures (SOPs), work instructions, reporting requirements and reviews. 2 Provide training for field office unit in appropriate United States Government (USG) procurement regulations, including SOPs for managing quality, competitive bidding, vendor management, client management, payment processing. Devise a training schedule; Identify key training needs and training mechanisms. Conduct appropriate training. 3 Conduct vendor pre-qualification assessments and implement mechanism for monitoring performance/quality of (local) suppliers. Expression of Interest (EOI) already completed, hence identify remaining activities required for pre-qualification. Identify TA to assist with Pre-qualification process, including devising time line for managing pre-qualification activities, developing work instructions for how pre-qualified vendors are re-qualified and new ones added. Develop work instructions on how pre-qualified vendors are accessed. Establish and use list of pre-qualified vendors. 4 Execute procurement of required commodities, including local procurement of commodities. Transition procurement management from home office (HO) to field office (FO). Develop communication linkages with HO for order management, including clear SOPs to triage products for international (or HO-procured) vs. local procurement. 10 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION ACTIVITY TASK Develop and implement client management tools (reports on procurement status, deliveries, order values, etc.). 5 Facilitate registration of PSCM to facilitate payments to local vendors only. Obtain required documentation. Coordinate process with lawyer. Finalize registration. Open bank account. Develop SOP or work instruction for payment processing and conduct training on payment processing. 6 Support EHNRI, HAPCO and MOH with procurements of HIV/AIDS commodities using Global Fund and other funds. Determine how to handle these types of procurements with new cost structure being implemented at HO. Define Memorandums of Understanding (MOUs) with Mission and GFATM on procurements (cost coverage, etc). 7 In Coordination with PFSA activities, provide TA in developing procurement capacity for Global Fund funded procurements. Provide TA on HIV/AIDS commodities procurement as appropriate. A PMP is in place and contains KPIs and clearly defined targets and responsibilities. The following procurement related KPIs are listed: 1 Availability of a Procurement Plan for key ARVs, Lab and OIs 2 Number of orders placed based on supply plan review 3 Availability of an in-country procurement plan and SOPs 4 Number of field staff trained on procurement 5 Availability of pre-qualified list of local vendors 6 The number and type of TA provided to PFSA on procurement 7 Number of procurement status reports submitted to clients. Observations and Recommendations The current state and speed of development of procurement management capacity at PFSA are unclear. However, if PFSA succeeds in implementing harmonized guidelines, adopting international best procurement practices, streamlining their operations, and reducing the procurement cycle time, it should be able to execute procurements in a timely, transparent, and competitive manner. In some cases, procurement of HIV/AIDS commodities through PFSA may prove to be much more cost-effective because it will be less restricted by donor guidelines. There is no formal health commodity procurement training available in Ethiopia. Despite earlier resistance, the team encourages increased efforts to provide PFSA with technical assistance. Bearing Management Sciences for Health (MSH) training expertise in mind, USAID could consider developing an in-house ―Managing Drug Supplies‖ training. The SCMS procurement department has proved to be capable of handling complex procurements within the restrictions associated with utilizing USAID funding and continues to improve this RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 11 service. The team encourages SCMS to share its experiences with PFSA. Secondment of staff trained by the SCMS procurement unit might also be considered. STORAGE AND INVENTORY CONTROL Introduction Drugs and other health commodities require appropriate storage conditions as these influence their shelf-lives, safety, and efficacy. Because storage conditions of drugs and dosage forms can significantly influence their physico-chemical properties, due care should be taken to preserve drugs so that they remain physically, chemically, and microbiologically stable. Proper storage of drugs requires well-designed warehouses with adequate storage capacity. Other considerations include distribution sites, the time it takes for delivery, alternative transport mechanisms (when regularly scheduled transport is not available or becomes inoperative), and seasonal closing of transport corridors. Staffing patterns of the warehouses, stock card recording, procedures for product receipt, stock rotation, issuance, and documents required are also factors that should be given due attention. Commodity arrangement in the stores is also an important aspect of storage. When commodities are arranged systematically, tracing a specific commodity is easy, simplifying inventory taking and minimizing expiry. In view of the above, the availability and management of adequate storage facilities in the regional hubs, warehouses, and stores of health facilities was assessed. Storage PFSA has nine hubs— Central hub (Addis Ababa), Adama, Bahir Dar, Dessie, Dire Dawa, Hawassa, Jimma, Mekele, and Nekemte—supporting existing warehouses that are inadequate to store volumes of products currently handled. The warehouses in Kaliti (Addis Ababa), Adama, and Nekemte are rented by SCMS. The plan is to expand the existing five warehouses and to build six new primary warehouses and seven additional secondary warehouses, for a total of 23 hubs (not all of which will be supported by SCMS). Each hub will serve roughly 10 million people, within a 160-kilometer radius. According to an official at PFSA, funds have been secured for construction of the hubs. SCMS has procured racks and constructed cold rooms at the Central Gullele hub, which has increased its cold room capacity by 50%. Although there is a shortage of space at the central hub, the warehouses are kept neat and goods are arranged in an orderly manner. Still, these warehouses require more systematized arrangement. The Kaliti hub is suffocated with condoms (30 million, some with a shelf-life of 10 months) detergents, and home-based care commodities. Currently, PFSA‘s warehouses are manually operated with most items stored in shelves and on floor pallets. Items are moved without mechanical assistance. For the primary storage hubs, it is recommended that the warehouses be mechanized, with modern storage and material handling systems such as tiers of pallet racking and mechanical handling equipment. Due to the inclusion of OI drugs, community and home-based care commodities (CHBCs), and other commodities in the PFSA system, space shortage in warehouses is common. Therefore, immediate solutions should be sought, either by implementing the warehouse in a box (WIB) concept on the short term or through effective distribution to facilities. Stock arrangement in PFSA warehouses is generally poor. Stocks are placed using a fluid location system where individual items are stored wherever space is available, with item tracking 12 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION dependent on the memories of warehouse staff. There is also a chronic shortage of racks and inadequate shelf labeling. To improve the warehouses, SCMS has seconded 24 staff (logistics experts at hub and sub-hub levels as well as distribution officers) to PFSA. SCMS is also assisting PFSA in warehouse and distribution system design. The rented Adama warehouse is new and is not yet equipped with facilities like racks and cold rooms. It has refrigerators to store lab reagents and a deep-freezer for producing ice packs to transport the reagents and other commodities to health facilities. The Jimma hub is well-organized, but frequent power breakdowns are affecting the cold-chain system. The deep freeze was reading 12 degrees Centigrade at the time of the visit. At the Bahir Dar hub, enormous quantities of commodities are stored in a haphazard manner, except for products being stored in separate stores categories as ―foreign product store‖ and ―local product store.‖. With the overflow stock everywhere on floor pallets or directly on the floor (there are no shelves or racks), it is difficult to trace a commodity, let alone to track expiry dates of drugs. Hence, the team has observed that expired drugs were stored together with medical supplies. The cold room is not functioning properly. The warehouse has no back-up generator and with frequent power-outages, reagents may deteriorate quickly. At the Hawassa PFSA hub, items are arranged in order, but there is a lack of shelves and pallets. Power interruptions are common. One of the cold rooms is not working, and the temperature indicator is not reliable. By keeping a simple thermometer in the cold room, it would be possible to check its temperature. A large quantity of Cell-dyn diluent (39 packs of 20 liters) with an expiry date of May 2009 was received a month before the evaluation and is unlikely to be used in time to avoid significant expiry. In the Southern Nations, Nationalities and Peoples Region (SNNPR) regional hub warehouse, half of the space is packed with ready-to-use therapeutic food (RUTF), which limited the store‘s management flexibility and its overall space availability. ART Stores The storekeeper manages the stock using stock cards, generally manual, but computerized in a few places. ART pharmacies supply their stock from the store of the health facility. Where the store is located at a different place, ART pharmacies have mini-stores or lockable cabinets to maintain a small supply of ARVs to support regular dispensing needs. Agaro Health Center, near Jimma, is one of the exemplary health centers (HCs) visited by the team. The store, renovated by RPM Plus, and stocked with ARV drugs, is well organized by a young and dedicated druggist. Two of the three RTKs used in Ethiopia, KHB and Stat Pak, are available, but are not supplied in a manner that is proportional to their actual usage. This results in frequent shortages of Stat Pak. Patients requiring a third rapid test (the tie-breaker) have to be sent to another site, because the health center has no stock of Uni-Gold. Up-to-date bin cards and stock cards are maintained. Charts such as eligibility criteria to initiate ART and statistics on patients on ART and Pre-ART care are posted on the walls of the clinic. The other exemplary Health Center visited was the Kolfe HC in Addis Ababa. It is well organized and stocked with all three RTKs, ARVs, OIs, anti-tuberculosis (TB) drugs, and other essential drugs, which are systematically arranged by pharmacological categories in the stores. The ART pharmacy has a computerized version of the ARV Dispensing Tool (ADT), an RPM Plus￾designed software program, which is used for recording and aggregating dispensing and patient￾related information. The health center has a shortage of vehicles and uses its ambulance for transporting procured commodities. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 13 The ALERT Hospital in Addis Ababa handles one of the highest numbers of HIV patients on ART in Ethiopia (6,601 adult and 500 pediatric patients). The chief pharmacist stated that: ―Had it not been for the renovation of the stores by RPM Plus, they wouldn‘t have handled HIV patients in addition to the high number of patients with TB and leprosy, to whom they provide specialized service.‖ The hospital has a separate ARV store (with a four-month stock) and mini￾store (with a one-month stock) and dispensary for ARVs. Stock cards and bin cards are well synchronized. The whole system is computerized and ALERT uses consumption data for its supply of pharmaceuticals. It is one of the health facilities that have implemented the ―pull system‖ and has also become the first hospital to introduce its own hospital formulary. RPM Plus has done a commendable job in renovating, restoring, or upgrading the hospital and health center pharmacy units providing ART services, enabling them to meet minimum requirements for storage and dispensing of ART drugs and other commodities. Major work includes the provision of separate rooms for ART counseling and dispensing, mini-stores for ARVs, partitioned spaces to separate functions; a strengthened drug storage and security system, waiting rooms for HIV patients, counseling and dispensing booths to minimize stigma, data recording, and archiving. This remarkable work has resulted in a comfortable and professional environment for both patients and health care providers. Achievements Over the last five years, RPM Plus/SPS has renovated 158 health facilities (68 hospital pharmacies, 86 HC Pharmacies, and four RHB stores), covering almost all regions. The renovation activity of MSH/RPM Plus has raised the standard of pharmacies in the country and created a good working environment for patients and professionals alike. Inventory Management The use of stock cards (paper-based or computerized) in stock management is mandatory. This is so because stock cards easily display the amount of drugs and commodities on hand. Stock cards also simplify planning and budgeting and assist in the control and utilization of drugs and commodities. Taking regular inventory of drugs helps to more accurately estimate drug and commodity demand and to minimize expiry. At PFSA hubs, stock control is based on first expiry, first out (FEFO) principles. Inventory control is based on a computerized system and commodities are tracked based on the five stores where they are stocked. PFSA receives reports from hubs on hard copies, which are entered into the computer manually. SCMS is currently designing an electronic version of the bin card to track commodities easily and to overcome problems associated with a single commodity supplied across different factory batches of products. At the hub level, a manual system of inventory control (stock cards and bin cards) is used. Invoices are also processed manually. To speed up the process, a computerized inventory control system should be implemented for efficient inventory control and distribution. At ART sites, inventory control systems were improved after the implementation of the newly designed ARV Logistic Management Information Systems (LMIS) by SCMS. The system, which is designed based on the Forced-Ordering Maximum-Minimum Inventory Control principle, enables ART sites to assess their stock status every two months, preventing wastage and promoting ARV orders based on consumption. Each ART store keeps stock for a maximum of four months. The pharmacies/stores are expected to have a minimum of two-week‘s stock and are obliged to call the hubs for emergency delivery. This has greatly reduced gaps in distribution, stockouts, and emergency orders. Inventory control in all ART facilities visited is adequate. Stockouts have been non-existent or minimal for ARVs. Use of bin and stock cards is adequate. 14 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM Plus/SPS has trained regional pharmacists and hospital staff in a computerized ARV drug use and stock management tool. The sites visited use the ARV dispensing tool. RPM Plus/SPS has prepared several forms, such as a stock card and bin card, ordering and receiving forms, a stock movement card, an expiry date tracking chart, ARV and patient information sheets, an ARV drugs dispensing register for adult and for pediatric patients, a monthly ARV drugs dispensing and consumption summary for adults, an ARV drugs dispensing register for post-exposure prophylaxis, an ARV drugs dispensing register for emergency supply, a patient tracking chart, and pharmacy monthly ARV drugs activity reports for adult and pediatric patients. Major Achievements RPM Plus provided ARV supplies to all ART sites in Ethiopia without any interruption from 2005 to 2007, with SCMS assuming this function during 2008. Over the past three years, there was no stockout of ARVs in the sites and no zero balance on bin or stock cards at any facility the evaluators visited. SPS–generated data is still being used for quantification exercises, supply planning, and distribution of ARVs, with SCMS managing these activities. This data enabled facilities to prepare patient uptake and drug consumption, and to stock-on-hand reports monthly. To sustain these activities, RPM Plus/SPS conducted mentoring and supportive supervision using checklists, and continues to support effective reporting using the ADT. SPS has also introduced OI recording and reporting tools. SCMS is developing an HIV/AIDS Automated Commodities Tracking System (HACTS), which would be used to track HIV/AIDS commodities and other related activities. The system can be used for re-supply, quantification, and supply planning. The tool is being designed to generate aggregate data, support analysis, and provide flexible report generation. It will have a capacity to automatically retrieve previous reporting data and ultimately should reduce or eliminate errors inherent in manual processes. Challenges/constraints Although the number of ART sites is rapidly increasing, logistical support has remained at the same level, with no corresponding increase in staff and other resources at hubs. Some storekeepers are frequently unavailable in their stores due to a lack of motivation. A regular schedule of their availability does not exist. Inadequate storage facilities/management, capacity, temperature monitoring— especially for the cold chain. The PFSA hubs are constrained by shortages of vehicles, portable cold-chains, cold rooms, racks and pallets. In addition, staff turnover is high. Recommendations Strengthen regional and central hub warehouse operations in warehouse and inventory management of HIV/AIDS commodities, including resources for cold chain management and material handling. Improve stores management practices to continue the reduction of space occupied by expired commodities and to increase stock turnover rates in those cases where good information and transport can support more frequent, smaller deliveries. Undertake physical expansion of stores in cases where space is inadequate, in poor condition, or poorly designed. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 15 DISTRIBUTION AND TRANSPORT Distribution Commodity distribution is a continuous process of receiving drugs from the supplier and transporting them safely to the desired health care facilities. Distribution is a complex and highly varied function. It includes storage, inventory control, wastage minimization, and transport. Wastage of pharmaceuticals is the outcome of overcrowding, poor warehouse organization, and inadequate control of moisture, temperature and other physical conditions. The PFSA replaced the parastatal organization, PHARMID, in late 2007. It was established to handle all procurement, storage, and distribution of drugs and related activities. Antiretroviral drugs (ARVs) flow through the PFSA system. Products are directly provided from PFSA hubs to ART sites. This has greatly shortened the pipeline and ensured continuous and uninterrupted availability of ARV drugs at ART sites. The central hub at PFSA distributes commodities to nine hubs in the country, including the one in Addis Ababa. SCMS sends a distribution request for required stock to the PFSA warehouse and PFSA distributes ARV drugs and other HIV commodities. From regional hubs, PFSA also distributes to sites where large volumes of commodities are required, such as hospitals. To support the system, SCMS distributes commodities to prevention of mother-to-child transmission (PMTCT) sites and to ART sites that require low volumes of commodities, as well as to remote ART sites where there are no roads for large trucks. The Addis hub is virtual: PFSA directly delivers commodities to health facilities in Addis. PFSA, from either its central hub or one of the nine regional hubs, delivers ARVs, RTKs, and OI drugs to public health facilities (hospitals and health centers) directly. It does not take a profit, adding only a small percentage service charge to cover distribution costs. Health posts receive their commodities through the health center with which they are associated, or the health office of the woreda (district) in which they are located. According to some PFSA officials, existing pharmacy stores at the RHBs will be maintained until PFSA is fully operational and then merged into the PFSA system once it is proven. PFSA has now started distributing test kits and reagents and is planning to handle the procurement of medical equipment and lab reagents. SCMS has 13 logistics associates and 12 distribution officers who work with PFSA in the supply and distribution of ARVs and HIV commodities. The major objective of SCMS is to support the implementation of an integrated supply-chain management system for HIV/AIDS-related commodities, supporting PFSA and its role in the management and distribution of HIV/AIDS program-related commodities. OI drugs Drugs for OIs were distributed through the RHBs until September 2008. This caused an unbalanced distribution of the few available products among the different health facilities needing these drugs. In addition, wastage and expiry were more common at facilities due to the holding of the stock at the RHB, Zone and Woreda stores. In October 2008, SCMS took over the responsibility for OI drugs on an interim basis until PFSA is fully functional and started to distribute the available OI drugs directly to facilities. This increased availability and has reduced wastage. The currently available OI drugs are about half of those mentioned in FHAPCO‘s OI drug list. This has limited the availability of needed antibiotics, antivirals and IV fluids. Under the Government of Ethiopia (GOE)-USG MOU, the GOE is responsible for OI drug procurement using GFATM monies, but PEPFAR has stepped in to temporarily support needed procurements given MOH problems in the procurement arena. 16 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RTKs KHB (primary), Stat Pak (secondary) and Uni-gold (tie-breaker) are provided from central PFSA to RHB warehouses. The RHBs receive RTKs based on the number of testing sites and the number of tests that each zone and hospital is expected to perform per the Road Map universal access targets. Zones and hospitals collect the RTKs from the RHB warehouse, and the zones in turn distribute the commodities to Woreda pharmacy stores where distribution is made to health centers and testing sites. This five-level distribution network increases the wastage in each stage of the pipeline. In addition, high wastage rates are seen at the facility level where some sites receive an excess quantity, while others receive a short supply. Such a lengthy process is the result of centrally-fixed targets based on assumptions for provision of universal access. RUTF and Corn-Soya-Milk Blend (CSMB) are prescribed for HIV/AIDS patients on ART, HIV￾positive pregnant women, and orphans and vulnerable children as time-limited nutrition support. The distribution of these nutritional supplements is handled by PFSA. UNICEF also procures and distributes these supplements during times of food shortage for the general population, pregnant HIV-positive women, and orphans and vulnerable children. CHAI procures these products for pediatric patients but will end its support in December 2010. CHBC: Previously, CHBCs were distributed through PEPFAR‘s HIV/AIDS Care and Support Project implemented by MSH. However, starting in March 2009, SCMS is supporting the distribution through the PFSA system. These commodities are extremely bulky and require special storage conditions and transportation. SOPs for distribution and use of these products have been finalized to provide guidance and to clearly identify the role and responsibility of those involved with distribution at the site level. UNICEF distributes vaccines, health post medical equipment, mosquito bed nets, and the antimalarial Coartem to health posts and health centers. UNICEF is currently distributing 9,000 Health Post Kits at a rate of about 500 kits per month. A kit consists of a delivery bed, a sterilizer, some consumables, and either a refrigerator or a cold box, depending on the capability of the health post. Approximately two-thirds of the over 9,000 kits delivered to the central level have been distributed. UNICEF has been achieving a success rate of nearly 90% in locating staffed health posts. They have distributed over 5,000 Health Post Kits, using their own trucks, vans, pick-ups, and sometimes donkeys, camels, and porters to reach remote areas. Another roughly 3,000 kits are located in the Addis Ababa warehouse and waiting for distribution. In addition to these kits for health posts, another approximately 7,000 are already on order and kits for some 2,000 health centers are also in the queue. Storage space for these very large numbers of kits is a major concern. Transportation Currently, PFSA has 22 heavy trucks (10 from PSLD and FMOH and 12 inherited from PHARMID) and is scheduled to secure 29 more trucks (of which four are heavy trucks) through SCMS. SCMS has helped relieve the chronic shortage of transportation on an interim basis by renting trucks from the private sector. PFSA also hopes that UNICEF will hand over its 10 trucks, which would bring the total number of vehicles to 61. The shortage of transportation is clearly manifested in health centers and limits the voluntary counseling and testing (VCT) services they are able to provide. Early diagnosis in infants is difficult as there are few labs equipped with Polymerase Chain Reaction test for DNA (DNA￾PCR) equipment in the country, and testing is not possible at health centers. Dried Blood Spot (DBS) testing requires transport of samples to the nearest hospital, while transferring whole blood samples for CD4 cell count testing to hospitals remains challenging. Thus, to improve services, some means should be designed to judiciously use the vehicles of the various vertical programs to transport samples to the nearest hospitals. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 17 Although EHNRI is currently charged with the responsibility for providing diagnostic services, other partners, including HIV & AIDS Care and Support Program (HCSP), International Center for AIDS Care and Treatment Programs (ICAP), Johns Hopkins University (JHU), and International Training & Education Center on HIV (I–TECH) are supporting the process. Concerns PFSA envisages distribution to all hospitals (currently around 140) and health centers (currently around 690, but scheduled to be increased to over 3,000 according to HSDP III). The plan depends on maintenance of a revolving drug fund (requiring a staggering US$160 million to capitalize) to be solicited from partners and donors. The absence of a detailed action plan from PFSA is worrisome to partners. There is a shortage of transport at all levels (Central PFSA, RHB and facility) for timely delivery. Severe strains on the limited delivery and storage capacity exist as heavy demands are placed by the delivery of equipment to Health Posts (3,000 RTKs already in stock, 7,000 on order), and 1,000 RTKs for Health Centers (with procurement of another 1,000 on the way). In addition, 130 container-loads of pharmaceuticals are on order for delivery within the next few months. Recommendations : The transition from a push system to a pull system for products in full supply, operation of both systems simultaneously, and transition from rationed supply to full supply are still in their beginning stages. These are enormous technical and operational challenges, which will require continued training and reinforcement. The long pipeline for distribution of RTKs and some other HIV/AIDS commodities (five levels) should be shortened to avoid the high rate of wastage seen at each stage of the pipeline. An inventory control system should be implemented for RTK consumption at the facility level, with RTK distribution based on consumption data. Tracing commodities must be strengthened. A phased road map should be prepared for PFSA to implement its plan. It is unrealistic to expect that it will be able to assume its new responsibilities overnight. During the transition to PFSA‘s full responsibilities, its capacity should be enhanced and supplemented by SCMS and other technical support partners, such as CHAI, UNICEF, UNFPA, the World Bank, and WHO. RATIONAL DRUG USE Background ART began in 2003 and free ART was scaled up in Ethiopia in 2005. At that time there were only a few sites throughout the country and roughly 8,000 patients on ART. By December 2008 the number of ART sites had increased to over 400, including private sites, and about 850 total sites were providing a range of services including either ART, VCT, or PMTCT. Today, the total number of patients ever started on ART stands at around 180,000 with over 140,000 currently on therapy. An estimated one million Ethiopians are currently living with HIV, of whom about 260,000 require ART. There is an ambitious national plan to increase the provision of universal 18 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION free ART to 400,000 patients at about 2,000 sites by 2010. SCMS and SPS are expected to play a vital role in building this capacity. With the support of RPM Plus and SCMS over recent years, uptake of ARVs increased from 20,000 patients in 2006 to about 140,000 at the beginning of 2009. As of January 2009 the number of patients on ARVs was 133,046 adults and 6,794 infants and children. According to the ICAP Country Director, due to the chronic shortage of ARVs that prevailed in early 2006, patients were dispensed ARVs on a rationed basis for three days only. This is no longer the case, underscoring the remarkable contributions of RPM Plus/SPS and SCMS to the uninterrupted provision of ARVs for a large number of Ethiopians. MSH started in Ethiopia in 2004 as a registered nongovernmental organization (NGO), initially to provide PMTCT services. Later, in 2006, it began supporting the logistics of ART, namely procurement, storage (with PHARMID) and distribution throughout Ethiopia. RPM Plus handled both logistics and promotion of rational drug use, until it transitioned into SPS to focus on pharmaceutical management and rational drug use (RDU). SPS‘s focus areas include promoting RDU, antimicrobial resistance (AMR) containment, adverse drug reactions (ADR) surveillance and reporting, rational dispensing, promoting ARV adherence, and enhancing good pharmaceutical governance and ethical practices among professionals (together with the Drug Administration and Control Authority [DACA] and the Ethiopian Pharmacists Association [EPA]). SPS also focuses on pre-service training of ARV dispensing and ethical practices to pharmacy students, training of quality assurance and quality lab management, promoting public-private-partnerships, establishing/strengthening drug therapeutic committees (DTCs) in hospitals, and strengthening drug information systems. RDU involves prescribing and dispensing the right drug (based on established treatment guidelines) in adequate doses for sufficient duration. It must be appropriate to the clinical needs of the patient and in compliance with the dosage regimen. According to FHAPCO, the first ARV guidelines were issued in 2003. These were revised in 2005 to facilitate a rapid scale-up of ART. Within two years, patients on treatment (in 117 hospitals and 108 health centers) increased from 900 to 62,000. Stavudine- and zidovudine-based regimens were used as first-line regimens. As a result, most patients are currently on these regimens. In 2008, the third guideline for treatment of adult and adolescent patients attempted to address the rapidly increasing knowledge in ARV treatment science. The latest guidelines outline clinical staging events to guide decision-making on switching ARVs. Tenofovir (TDF) became an accepted first-line drug in treatment of naïve patients, which provided the possibility of a first￾line regimen of TDF/Emtricitiabine (FTC)/Efavirenz (EFV) as a single pill per day triple fixed dose combination (FDC). First-line ARV regimens: According to the 2008 ARV guidelines, the first-line regimens are categorized as preferred and alternative ARVs. The preferred regimens are: TDF+FTC+EFV (triple FDC, ATRIPLA); Zidovudine (ZDV)+Lamivudine (3TC)+EFV (Combivir+EFV); or ZDV+3TC+Nevirapine (NVP) (triple fixed dose combination (FDC), preferred for women of child-bearing age without reliable contraception). The alternative ARVs include Stavudine (d4t)/3TC/EFV (double FDC d4t/3TC+ EFV at one pill per day); TDF/3TC/NVP; d4T/3TC/NVP (triple FDC); Abacavir (ABC)/3TC/EFV; ABC/3TC/NVP; and ABC/3TC/ZDV. FDCs, either triple or double combinations, are much preferred to loose compounds because they ensure patient convenience and promote much greater adherence to ART. The other nucleotide reverse transcriptase inhibitors (NRTIs), d4T and ABC, are used as alternatives to TDF or ZDV based on the patient‘s condition. Currently, the preferred first-line FDC preparation (ATRIPLA) is not available in the public sector. Instead, a few facilities have started to use TDF+3TC+EFV RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 19 as first-line regimen where patients take four pills, once per day. Today, about 80-85% of all patients are on FDCs. Adult loose ARV drugs: When FDCs were introduced, significant quantities of loose drugs were available at facilities and at PFSA hubs. In consultation with health care providers, attempts were made to consume most of the single drugs prior to switching to FDCs. SCMS played a vital role in the redistribution of the loose drugs, preventing significant quantities from expiring. Health facilities were advised to consume their extra stock of NVP 200 mg tablets with the dual d4T and 3TC combination and ZDV and 3TC combination. By so doing, all d4T 30 mg capsules, most 3TC 150 mg tablets, and much of the NVP 200 mg tablet stock were consumed. Second-line ARV regimens: In the event of first-line treatment failure, second-line regimens are indicated. The selection depends on the first-line regimen that the patient had been taking. Currently, all second-line regimens are available only as single or loose drugs since there are only a small percentage of patients on second-line regimens and no FDCs are purchased. Thus far, in Ethiopia less than 2% of the patients are on the second-line regimen, indicating good prescribing practices, high adherence, and hence less antiviral resistance. According to the observations of the evaluation team, patients with CD4 counts ≤ 200 are eligible for ART. If patients with CD4 counts ≤ 350 are to be eligible for ART as is being proposed, the number of patients would almost double. This calls for doubling ARVs and other HIV commodities, creating an enormous burden on health facilities to provide clinical services and the logistics supply chain to provide the necessary commodities. All the necessary precautions and preparations should be made before embarking on this scheme. Pediatric ARVs: Because awareness of appropriate pediatric ART was low, CHAI started providing pediatric first-line ARVs in ALERT Hospital in 2006. In 2007, CHAI took the full responsibility of procuring pediatric ARVs and all hospitals and health centers in Addis Ababa started the services, with JHU supporting clinical service. CHAI also introduced the fixed-dose combinations for children, initially d4t-based and later ZDV-based. The introduction of pediatric FDCs containing d4T, 3TC, and NVP improved the accuracy of dosing and adherence. It also simplified the pediatric supply chain. The tablets are available in two different formulations: ―FDC Baby‖ (d4T 6 mg/3TC 30 mg/NVP 50 mg) and ―FDC Junior‖ (d4T 12 mg/3TC 60 mg/NVP 100 mg). Health facilities are also provided with dual pediatric FDCs, containing only d4T and 3TC. This is for use during the two-week lead-in period, when the child needs only a single dose of NVP, and for those patients on d4T-based regimens, which do not contain NVP (i.e. d4T/3TC/EFV). The availability of FDCs also supported the expansion of the pediatric treatment program from hospitals to health centers. PMTCT services: Sites with ART services provide the combined ARV prophylaxis regimen while those with no ART services provide NVP-based prophylaxis. Adherence: ART is a complex health care intervention, which requires near 100% adherence and a life-long commitment for sustained benefit. Because of this requirement, patients on ARV, families, and the community at large have roles to play alongside health workers in ensuring effective and sustained treatment. Failure to comply with treatment results in more rapid emergence of drug-resistant viruses that are difficult and expensive to manage. RPM Plus, together with International Network for Rational Use of Drugs (INRUD) developed adherence indicators and carried out an ART adherence base line survey in March 2006. In August 2007 it conducted an ART adherence determinant study, and led a national dissemination workshop in June 2008. Adherence among patients was estimated at about 93%. Rational drug use: To promote RDU, SPS has prepared information, education, and communications (IEC) materials on proper use of drugs, and a brochure on AMR. In addition, it 20 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION has provided training of trainers (TOT) for RHBs, DACA, and health center staff. The health care facilities visited by the evaluation team are following good prescribing and dispensing practices, in line with the principles of RDU. Good dispensing practice: After installing ADT, SPS collects, collates, and analyzes monthly reports from the pharmacies dispensing ARV drugs. The management information system currently supports 450 health facilities, of which 108 are hospitals, 320 are health centers, and 22 are Private and NGO health facilities. Approximately 164 ART sites are computerized. The evaluation team observed that there are good dispensing practices in all the sites it visited. Adverse drug reactions: In the pharmacovigilance/ADR sphere, RPM Plus/SPS has supported DACA in converting ADR reports and data into valuable information to help decision making, revising ADR guidelines, preparation of ADR posters in three languages, and producing brochures for health care providers. Furthermore, it has carried out ADR-knowledge, attitude and practice (ADR-KAP) studies and is working with DACA on post-marketing drug-quality surveillance work. Such continuous engagement with the regulatory body, the health care providers, and the public at large has undoubtedly contributed to improved adherence of patients to ART. Antimicrobial resistance: RPM Plus/SPS has made some attempts to address AMR. It supported the establishment of a National Advisory Committee in 2006-07 and, based on the recommendation of the Committee and DACA, it completed a baseline survey on ―Antimicrobials use, resistance and containment‖ in August-September 2008. On the basis of these findings, it is currently embarking on interventions together with stakeholders like DACA, health facilities, health bureaus, and academic and research institutes. DTCs: RPM Plus/SPS has supported the establishment and revitalization of functional DTCs in hospitals that are designated to ensure the safe and effective use of medicines. Areas being addressed include evidence-based standard treatment guidelines, essential drug lists based on treatments of choice, independent drug information provision, effective ADR monitoring systems, problem-based training in RDU, continuing medical education, supervision, audit and feedback, public education about drugs, initiating segregation and disposal of expired/damaged items, and appropriate and enforced drug regulation. SPS is also assisting hospitals to administer and manage their formularies. To date, 92 hospitals have prepared a plan of action for DTC activities; 34 hospitals have prepared draft drug lists/formularies; and nine hospital DTCs have conducted drug list/formulary development workshops. In this regard, exemplary work has been done in ALERT Hospital, which has become the first hospital to launch its own formulary. Public-private initiative: To strengthen private health facilities in the scaling-up of ART, SPS has started working with the private health sector. Thus far, it has conducted trainings for private sector pharmacists on ―good community pharmacy practice,‖ facilitated a process of familiarization with pharmaceutical ethics, supported the Ethiopian government‘s initiative of involving the private sector in ART service provision, undertaken a baseline assessment of pharmacy practice of private community pharmacies in Addis Ababa, and facilitated comprehensive basic ART training for druggists from the private sector. Training and manuals: RPM Plus/SPS has produced guidelines on ―Management of opportunistic infections and ARV treatment in adolescents and adults in Ethiopia‖ and ―Standard operating procedures for ARV drug management at health facilities,‖ which are used as reference materials. To strengthen the overall pharmaceutical system of the country, RPM Plus/SPS is closely working with DACA in jointly developed thematic areas, namely, strengthening the national drug regulatory system, promoting rational drug use and quality assurance of drugs, and improving pharmaceutical services. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 21 Observations in hospitals: The hospitals visited (ALERT, Zewditu, Adama, Bishoftu, Jimma University Referral Hospital, Felege Hiwot Hospital, and Hawassa Referral Hospital) not only provide pre-ART, ART, and PMTCT services to a large number of patients, but also serve numerous other ART and PMTCT sites with total catchments of several million people. The services provided are commendable, as are the staff, prescribers, counselors and dispensers. The doctors strictly follow the national guidelines for prescribing ARVs. Patients on ART are resupplied with ARVs on a monthly, bimonthly, or quarterly basis, depending on their individual condition. The support that the clinical labs and the ART pharmacies are getting from PEPFAR‘s CDC-supported University Partners, SPS, and SCMS has enabled them to provide professional and quality services. Observations in Health Centers: Kolfe, Agaro, Bahir Dar, and Bushulo Health Centers were visited. The evaluation team was impressed by the professionalism of the staff at these health centers and the quality of services provided. Not only do they promote rational drug use, but basing their practice on well-founded rationale and patient conditions, they make judicious selections of first-line drugs for patients. A good example of this is the choice made between EFV-based regimens and NVP-based regimens. All patients at the health centers are kept on first￾line, and second-line patients are referred to hospitals. Another example is the dispensing of ARVs at the Bushulo Catholic Clinic in Hawassa. Even where computerized ADT is not available at this clinic, dispensing of ARVs is well-managed using the ARV Dispensing Register for Adults prepared by RPM Plus/SPS. Drug administration and control: The evaluation team paid a visit to the Jimma Regional DACA, one of the five regional DACA offices. As a result of the support it receives from SPS and the United States Pharmacopeia (USP) Drug Quality and Information Program, this regional office, in addition to promoting rational drug use, DTCs and Drug Information Committees (DICs), is conducting drug regulatory activity in four regions. The office is provided with a mini-lab, which performs post-market surveillance. The simple physical and chemical tests that the office performs — such as visual inspection (color, shape, presence of empty capsules), disintegration of tablets, and Thin Layer Chromatography (TLC) — are quite effective. The evaluation team believes that this is a case in which the HIV/AIDS program in Ethiopia is also strengthening national drug regulatory activities. Overall observations The capacity enhancement (technical, resource, and infrastructure) of the HIV/AIDS program in Ethiopia provided by RPM Plus/SPS and SCMS is highly commendable. It has enabled health care facilities to provide professional and quality services to patients. As Ethiopia moves to ART universal access by 2010, the logistic and technical support of SCMS and SPS is indispensable. Since FY08, SCMS has been working to initiate and develop the robust operation of a ―pull‖ system for ARVs and lab commodities, based on a direct linkage between the facility and PFSA. This undertaking is vital for timely responses to the ART scale -up. The availability of Zidolam (NVP-based fixed combination: d4T 30 mg+3TC 150 mg + NVP 200 mg) has lead to less use of the double-combination Combivir® (3TC 150 mg+ ZDV 300 mg with loose NVP). Some drug expiries will result. As OI drugs were in short-supply and an inventory control system to track these had not yet been deployed, they were procured in bulk and supplied without assessing the real demands of the health facilities. 22 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION In some facilities there are shortages of OI drugs like CTX suspension, anthelmintics, IV fluids and fluconazole tablets, while excess quantities of fluconazole injection are sometimes found. In some facilities, there are large quantities of Combivir® and Didanosine (Videx) 250 mg due to expire shortly. There are some expired second-line drugs, due to the limited numbers of patients on second￾line regimens. Expired ARV drugs (from December 2008) were kept together with active drugs at one site (Felege Hiwot Hospital in Bahir Dar). Recommendations Reduce the long list of commodities (approaching 600 items) and streamline it into vital, essential and non-essential categories based on studies specific to regions and sites. As patients are now taking FDCs, make sure they are made available on a sustained basis. Pediatric ZDV-based FDCs should be made available for use with treatment-naïve patients and for those who have been taking the syrup preparation. OI drugs should be supplied on the basis of request, not on quota, assuming these are available in full supply. LOGISTICS MANAGEMENT INFORMATION SYSTEM Introduction Information technology (IT) is valuable in managing the supply chain of pharmaceuticals. Some IT applications have an operational and short-term focus, automating activities such as dispensing, inventory management, distribution, demand planning, and purchasing. Such applications help optimize the number of deliveries and manage the daily distribution of pharmaceuticals. Other, more sophisticated applications are designed to support strategic and long-term supply chain decisions and permit administrators to evaluate different supply-chain network scenarios that balance cost reduction with improving service levels. A typical Logistics Management Information System (LMIS) captures ―logistics‖ data such as quantities requisitioned and issued, receipt information, inventory levels, distribution data, and lead times from multiple sources and aggregates this data. The data are then analyzed and presented to support short, medium, and long-term supply-chain decisions. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 23 Background Currently, there are many systems and data capturing methods being used in Ethiopia. Some of the most relevant of these are shown in the following table: System/ Software Primary use Level in Supply Chain Complexity Support Partner QUANTIMED Quantifying requirements for pharmaceutical products (medicines, supplies, etc.) and calculating their estimated cost for short-course treatments or long-term treatment for chronic conditions. Central Medium MSH/SPS PIPELINE Planning of optimal procurement and delivery schedules based on supply chain (pipeline) information such as rate of consumption, pending orders and inventory levels. Central Medium SCMS HACTS Reporting generation/decision support system. Central Data Repository (CDR) system. Central/ PFSA Medium SCMS Warehouse/ Distribution System Interim ART/Lab reporting solution for warehouse reporting. Central/ PFSA Low SCMS RX Solution Comprehensive and fully integrated hub, store and dispensary tool. Designed for large hospitals with multiple dispensaries. Hospital and larger Health Centers High MSH/SPS Health Center Management Information System (HCMIS) Facility level commodity management tool that tracks/manages inventory consumption. Multi dispensary management. Hospital and Health Centers Medium DELIVER (JSI) ADT/EDT Dispensary level patient tracking and dispensary tool. Dispensary Low MSH/SPS Until recently, most of the middle and lower- level systems (e.g., ADT and HCMIS) were implemented independently of each other by their respective partners. Integration, multi-product interfacing, or even data sharing seemed to be hampered by different corporate philosophies, misunderstandings, and lack of clarity regarding partner roles. However, as the need for an integrated LMIS providing data to and from different levels and from different sources grew, USAID encouraged the partners to work on a joint solution, facilitated partner meetings, and clarified their roles in this effort. Current situation Despite the generally good working relationship between the partners, there had been some misunderstandings about systems and capabilities. However, these misunderstandings have been addressed and effective collaboration between the USG partners re-established. All partners recognized the need for integration of the systems and agreed that there is, in fact, no overlap in 24 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION functionality between the existing tools. During intensive sessions a timeline was set, communication data protocols have been defined, and a division of labor has been agreed upon, with guidance from USAID. It was agreed that the LMIS should at a minimum consist of the following components supported by the respective tools: Component System Partner Patient Servicing/Dispensing ADT/EDT SPS Commodity Inventory HCMIS DELIVER Warehousing and Distribution WHM PFSA (SCMS - 2010) CDR HACTS SCMS The current tools remain virtually unchanged and will continue to be implemented and maintained by the respective ―owners,‖ with PFSA and health facilities gradually assuming management of the entire system. Interface software will be developed to ensure data input into the CDR software (HACTS) from which a wide range of reports can be generated. Hospital/Health Centre Storage PFSA Central/Hub Laboratory (ART) Pharmacies WMS & HACTS HCMIS ADT/EDT Form Forms HCMIS ADT/EDT PDA RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 25 Progress against work plan and performance monitoring SCMS has submitted a detailed work plan for Country Operating Plan for Fiscal Year 2008 (COP08), set targets, and measured its own progress on a regular basis. The work plan contains the following LMIS-related objectives and activities: ACTIVITY TASK 1 Develop, pilot, and begin to use Personal Digital Assistant (PDA)-enabled commodity data capture system, using Distribution and Lab staff, as well as appropriate health facility staff. Agree on system and resource requirement and begin development of Alpha-version. Develop and finalize Beta version, including user acceptance testing. Conduct and complete pilot. Roll out Phase I of PDA system, including completing final version of system, user training. Evaluate system and begin Phase II. 2 Provide equipment, computer hardware and software as required for implementation of HACTS. Procure and provide PDAs, internet connectivity for facilities, General Packet Radio Service (GPRS), as needed. 3 Develop and begin to use CDR for HACTS. Define comprehensive Input-Output requirements (i.e., central/regional data sources) from CDR for all programs/product categories (including OIs, DNA PCR, CHBC, etc). Define user access protocols. Design the CDR system to capture commodity data requirements not captured by the PDA system. Continually test CDR system. Study and begin to design linkages with existing data sources as possible, e.g., Quantimed, Pipeline, PFSA Warehouse Management System (WMS). Begin linkage between HACTS, HCMIS, and MOH HMIS for logistics and monitoring purpose. Provide SCMS staff with training on CDR system administration. Conduct user orientation for regional staff to access, enter data and use CDR system—central, regional. 4 Provide technical assistance and system service support throughout implementation process to organizational stakeholders (PFSA, EHNRI, HAPCO) as well as users. Closely link with central stakeholders (EHNRI, HAPCO), PFSA (central and regional warehouses), and regional health bureaus to ensure that adequate information inputs and outputs are provided through the HACTS. Actively reach out to and link with stakeholders to inform and develop system linkages with HACTS; conduct technical assessments and stakeholder workshops as appropriate. 26 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION ACTIVITY TASK 5 Implement and use an ―interim‖ data aggregation and reporting system (Access -based) to support large volume of facility and PFSA hub data needed for logistics management, e.g., LMIS report for ARVs, lab supplies, RTKs, OIs, PMTCT, and IP among the major commodities. Refine database and tool in Access. Conduct two-day on-job-training to appropriate logistics staff. Begin use of database to capture, report data, and guide decision-making. 6 Ensure HACTS provides timely reports to all stakeholders and partners, as needed. Define reports for all commodities (outputs) Generate reports, validating missing data as appropriate. Define and report appropriate Monitoring & Evaluation (M&E) indicators (e.g., reporting rates). 7 Work with Distribution and Quantification Units to design, capture, report and use information on ―new‖ category products, e.g., RUTF, CHBC, IP, PMTCT supplies. Link with Distribution and Quantification Teams to develop and implement appropriate TA and work plan. 8 In collaboration with EHNRI and Lab Team, provide technical assistance and support strengthening of commodity information system for RTKs, to ensure adequate availability of product at sites. Link with Lab Team to develop adequate strategy and synchronize work plans (above). 9 Continue to collaborate closely with USG logistics partners (USAID/DELIVER and SPS) to ensure technical linkages and synergies with interventions to strengthen logistics management information systems in the country; jointly design and implement solutions as appropriate. Actively reach out to and link with USG partners to develop and implement join work plan in this area. 10 Continued support for the positions of One (1) HACTS Advisor, One (1) HACTS MIS Officer, and One (1) Lab LMIS Officer. To measure progress, a PMP is in place, which contains KPIs and clearly defined targets and responsibilities. The following LMIS– related KPIs are listed/measured: 1 The quantity and months of stock status for major drugs at the national level 2 LMIS reporting rate from the service delivery points (SDPs) for ARVs and Labs 3 Availability of PDA to enable commodity data capturing system 4 Availability of CDR system for commodity tracking 5 The number and type of TA provided to the stakeholders for use and interface with HACTS 6 Number of meetings, presentations for coordination with USG logistics partners (USAID/DELIVER, SPS) RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 27 Observations and Recommendations The Evaluation Team supports the approach of an LMIS system consisting of individual interfaced applications as opposed to a fully-integrated system. The latter tend to be very complex, unstable, and require high maintenance budgets. The Evaluation Team also appreciates the fact that all USG partners agree that 70% of the data collection will be done by a paper-based system. Even at locations that have been using a computerized ADT for some time, a paper trail was still maintained. Bearing in mind the unstable power supply in Ethiopia (some health centers go without power for days) and the continuous threat of viruses and equipment breakdowns, this seems justified. However, in the larger facilities/hospitals with a high number of ART patients and a growing number of products, it will prove increasingly difficult to maintain such a manual system while the external ―technical‖ circumstances remain unchanged. Furthermore, it is important to note that this form of LMIS, in which a CDR tool is used as its core data engine, is not suited for use as a transactional or operations management system unless the data input frequency is increased considerably (currently one-to-two months). It is therefore recommended to increase efforts to introduce and implement a WMS at PFSA and replace the interim tool currently being used. Although none of the Evaluation Team members has practical experience with RX Solutions as a patient management and dispensing tool, the information provided indicates that it is a fully integrated and high maintenance system, which is specifically designed to work in a stable multi￾workstation network environment. Based on observations made in several target hospitals, the team doubts that the required technical environment in terms of uninterrupted power supply can be guaranteed. The Evaluation Team has no insight into the technical limitations of the proposed ADT/EDT/DELIVER (HCMIS) combination for health facility dispensaries and stores; however, in light of standardization/staff rotation considerations as well as system complexity/feasibility, this option might be further considered. Given the increased number of products and envisioned inclusion of other health commodities (essential drugs and medical supplies), the use of PDAs as a data collection tool seems cumbersome. If the pilot shows that it benefits end users, the Evaluation Team supports the implementation. We must emphasize that the pilot should be performed with a representative number of products and not only with the current ARVs and OIs. During the joint presentation of the new LMIS, the lack of a recognized central and harmonized coding system was presented as an issue. The Evaluation Team recommends a national exercise to agree upon and implement a common system. It is common practice to use both a classification system and an identification system. A widely- used classification system for medicines is the Anatomical Therapeutic Chemical (ATC) system, which is recommended by WHO (www.whocc.no). The identification system may be simply a numbering system. It is recommended that an eight-digit system be used, which would allow room for one or two control digits. In addition, the system should be compatible with the European Article Number (EAN) bar code system. Until a national system is agreed upon, an alternative primary key must be agreed upon between the partners. The accompanying software should be made flexible enough to allow the codes to be changed once a harmonized coding system is in place. 28 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 29 III. QUESTIONS AND RESPONSES TASK 1: ASSESS PROGRESS TO DATE IN ACHIEVING PLANNED RESULTS 1. What has been the two projects‘ progress to date in achieving planned results and performance indicators? The various work plans, presentations, and progress reports provided to the evaluation team indicate that both projects have achieved a high degree of progress against their workplans. Almost all workplan tasks are either completed, nearly completed, or ongoing as scheduled. The few exceptions are tasks that involve coordination with host country counterparts, many of whom have either changed or been unavailable due to the extensive BPR now being concluded by the FMOH. The expectation is that, with the completion of the review process, these activities will either be replaced due to changes in government counterpart organizations, plans or structures, or will be able to get back on track once the counterpart organization has clarified the way forward and people are able to resume their normal duties without the interruption of the reengineering process. 2. What have been some of the projects‘ important lessons learned to date? Some of the important lessons learned by the RPM Plus project include: (1) the importance of improved physical infrastructure at the facility level in supporting the appropriate handling and distribution of health commodities; (2) the value of undertaking capacity development in improving the systems supporting the handling, distribution, and dispensing of health commodities— and the value of forms, charts and other physical tools as constant reminders of the proper operation of these systems; and (3) the value of accurate, timely, and consistent information generated at the facility level to support management and decision-making at all levels of the system. Some of the important lessons learned by the SCMS project include: (1) the importance of quantification exercises in building understanding and consensus around the needs of the supply chain in supporting national-level health programs; (2) the need for constant and reinforced capacity development at all levels in improving the storage, handling, and distribution of health commodities supporting national-level programs; and (3) the value of accurate, timely, and consistent information in supporting the management of complex and broad-based logistics systems. 3. What contributions have the two projects made in increasing the availability of HIV/AIDS commodities in Ethiopia? The various work plans, presentations, and progress reports provided to the evaluation team indicate that both projects have contributed in significant ways to the overall availability of HIV/AIDS commodities in Ethiopia‘s HIV/AIDS program in recent years. Among the achievements that are most frequently cited, we include: Establishment and operation of an ARV supply chain with no reported large-scale stockouts (both projects), Management of significant increases in total commodity flow with relatively modest cases of expiry (both projects), 30 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Significant improvements in the consistent availability of laboratory supplies and the reduction of turn-around times for test results (although this is still an area identified as open to additional improvement) (SCMS), and Significant improvements in the consistent availability of Rapid Test Kits throughout Ethiopia (this is also still an area identified as open to additional improvement) (SCMS). 4. Are SCMS and RPM Plus/SPS workplan activities (short and long-term) compatible with the Pharmaceutical Logistics Master Plan (PLMP)? Are the two projects‘ comparative strengths being leveraged in the implementation of the PLMP? The two projects‘ workplan activities are compatible with the PLMP. In the 2004–2006 period, when RPM Plus was first supporting activities within the HIV/AIDS supply chain in Ethiopia, some of their support (facility renovation, procurement, direct distribution) were probably not their greatest strength. Nonetheless, the data indicate that they performed these tasks well and contributed greatly to the significant growth of the AIDS treatment program over those years. Since 2006, when SCMS began supporting activities in the supply chain, there has been a planned transition of the procurement and distribution support functions from RPM Plus over to SCMS and of the renovation support to other mechanisms (still to be identified). These transitions have occurred smoothly, have been consistent with SCMS‘s areas of strength, and have resulted in RPM Plus (now SPS) being able to resume its focus of activities in its areas of natural strength (pharmaceutical management). The PLMP itself has been significantly overtaken by the BPR exercise undertaken by the FMOH. Many of the entities identified as focal points within the PLMP have been abolished, merged, or otherwise changed. Despite these challenging and ongoing changes, the evaluation team found that the activities of the two projects were consistent with their respective areas of strength and consistent with the strategic directions being taken by the stakeholders in developing the HIV/AIDS supply chain in Ethiopia. 5. Are Logistics Management Information System (LMIS) development goals being met? Are additional monitoring processes needed to ensure that this important and complex project activity stays on track? If so, please suggest new monitoring systems or implementation strategies based on assessment results. Improvements in information availability and use are recognized as having a vital role in the continuing growth and improvement of supply-chain operations. Specialized information systems are being developed to support the necessary functions undertaken at various points in the supply chain. Recent decisions have helped clarify areas of responsibility among the various projects supported by USAID. Efforts are also underway to ensure that appropriate information and reports from each system can be communicated smoothly to the other systems that require it. There are still technical and bureaucratic challenges to be overcome in this area. Continuing communication among the two projects evaluated here and other key participants in the sector (such as the USAID/DELIVER project) will be essential. 6. How are the two projects strengthening the overall national supply and pharmaceutical management systems? Both projects are spending significant time and resources supporting the development of management systems, operational systems, and information systems to strengthen supply and RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 31 pharmaceutical management in Ethiopia. Both projects are also supporting the extensive capacity development that will be needed to ensure that the investments in systems‘ development are integrated into the Ethiopian system. RPM Plus, in its early years, and SCMS now, are both supporting needed infrastructural improvements through the renovation of warehouses, stores, and other spaces, as well as through the provision of needed rolling stock and equipment such as vehicles, cold chain equipment, computers, and power supply and stabilization apparatus. One area of concern is in the level of support being provided through the direct provision of staff to assist in key supply-chain functions. While these staff may be critical to achieving the continuing levels of growth and service needed in the sector, there is little indication that the government will be able to continue supporting these staff if project funds are withdrawn, despite the inclusion of salaries for seconded data clerks in a recent Global Fund grant at USAID‘s suggestion. Both projects must increasingly focus on ensuring that the skills, capabilities and work being undertaken by these staff are transferred to permanent counterparts. USAID support is essential to the success of this effort. 7. What monitoring and evaluation systems are in place to assess the two projects‘ impact, including for sustainability of project results? Do the systems identify implementation problems so that corrective action can be taken? Both projects have extensive M&E systems and Performance Monitoring Plans in place and report regularly to their managers at USAID and their counterparts within the government. Both projects have also specifically designed portions of their systems to provide advance warning of supply-chain problems such as anticipated shortages or expiry. In many cases,these warnings have allowed them to initiate corrective actions to obviate or minimize these problems. Moreover, both projects are working to incorporate advance planning behaviors into the system management and capacity development components of their activities so they become part of a sustainable program. 8. Please also identify key barriers/constraints that have moderated projects results. While the supply chain and these projects have faced no shortage of technical and institutional challenges over the last few years, it would be hard to say that these challenges have moderated project results. The astonishing achievements of the HIV/AIDS program overall, and the role of the supply chain in supporting these achievements, continue to amaze the evaluation team. If these achievements represent ―moderated‖ success, the evaluation team has a hard time imagining what ―high achievement‖ might look like. One bureaucratic hurdle that was mentioned during nearly every conversation with the evaluation team was the extent of the BPR process and its impact on the availability of government counterparts. One significant impact of the BPR on the activities of these projects was the lack of formal coordination mechanisms. Many of the formal interagency coordination mechanisms that had been in place have suspended meetings throughout the duration of the BPR process. In a sector that is being actively supported by such a breadth of donors and projects, one cannot help but be concerned that this lack of coordination could significantly impact results. It may be that the impact of this lack of coordination is small, or it may just not have been felt to this point. The BPR process is drawing to a close and the new structures and responsibilities are being finalized and announced. As has been suggested by USAID and other donors, one assumes that formal coordination and collaboration mechanisms will be reestablished once this process is complete. 32 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 9. How have the projects contributed to reaching PEPFAR‘s 2-7-10 goals? The various work plans, presentations, and progress reports provided to the evaluation team by the two projects indicate that both projects have contributed in significant ways to the overall achievements in Ethiopia‘s HIV/AIDS program goals in recent years. Some of the achievements that are most frequently cited include: An increase in the total number of ART patients to levels approaching 150,000 in early 2009 (both projects). Significant improvements in the consistent availability of prevention commodities, such as Rapid Test Kits (RTK) throughout Ethiopia—although this is still an area identified as open to additional improvement (SCMS project). Test kit availability allowed the MOH to implement its highly successful ―Millennium AIDS Campaign‖ that included HIV/AIDS testing for 4.6 million individuals during 2008. 10. What impact have the two projects had to date in terms of building institutional capacity and providing needed commodities and other inputs to support HIV services including ART? The various work plans, presentations, and progress reports provided to the evaluation team indicate that both projects have had significant impact on both the strengthening of institutional capacity and the supplying of needed commodities and other inputs. Some frequently-cited achievements include: Institutional Capacity: – Significant improvements in the physical infrastructure for storage and dispensing at over 200 sites throughout Ethiopia (RPM Plus). – Management and support of a successful effort to destroy expired product in a transparent, organized, and systematic way to clear significant space throughout the supply chain (RPM Plus). – Significant improvements in the organization and management of stores at all levels of the system (both projects, also with support of the USAID/ DELIVER project in some cases). Commodities and Inputs: – Establishment and operation of an ARV supply chain with no reported large-scale stockouts (both projects) while the HIV/AIDS program was growing dramatically. – Management of significant increases in total commodity flow with relatively modest cases of expiry (both projects). – Significant improvements in the consistent availability of laboratory supplies and the reduction of turn-around times for test results—although this is still an area identified as open to additional improvement (SCMS). – Significant improvements in the consistent availability of Rapid Test Kits throughout Ethiopia—also still an area identified as open to additional improvement (SCMS). RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 33 11. Are these projects effective models for technical services? If so, how, and if not, why not? Based on the outstanding results achieved by the HIV/AIDS program in Ethiopia with the support of these projects, it would be hard to argue that they are not effective models. Both projects have been supplying a mix of direct support, systems support, and capacity development that has resulted in significant measurable growth of services, commodity flows, and systems. Program managers might be interested in changing the balance of direct support in relation to systems and capacity support in the future in order to put more emphasis on sustainability, but these shifts may cause some reduction in immediate direct impact. 12. Have TA, procurement and capacity building services been provided in the most cost￾efficient ways possible? Both of these projects were the result of competitive award processes undertaken by USAID/Washington. These award processes consider both the technical merit of the proposed approach and the cost competitiveness or reasonableness of the costs presented. Within this broad framework, the evaluation team observed no obvious gross inefficiencies in the provision of services. Whether these structures represent ―the most cost-efficient ways possible‖ is not for us to say. Their selection is a reflection of the approach that USAID has determined to be the best combination of efficiency and effectiveness. 13. Have the two projects provided quality, state-of-the-art services, and utilized state- of-the-art TA approaches for low-resource settings? The evaluation team was consistently impressed with the professionalism, knowledge, and dedication of the staff at the two projects. The approaches being undertaken are consistent with, and at least as good as, any observed in programs within our experience. The staff of both projects maintains contact with resources supporting larger program management and operations in Washington. These resources also help maintain consistency of state-of-the-art approaches across individual implementation programs. 14. Have appropriate quality-assurance mechanisms been utilized, both for the internal activities of the two projects, and to promote optimum services in GOE institutions supported? Both projects have internal and external review and evaluation mechanisms in place that help ensure the quality of the projects‘ services. Both projects also have mechanisms in place as part of their capacity-building functions that assess the performance of the counterparts with whom they are working and provide for corrective interventions when that performance is not up to expectations. 15. What are the contributions of the two projects to implementation of the PLMP? Despite the fact that many of the detailed elements of the PLMP have been overcome by events, the two projects continue to provide support that is consistent with the broad directions and strategy of the PLMP. For SCMS, this can be seen in the extensive support being provided to PFSA (the successor to PHARMID, a key player in the PLMP). SCMS has also offered procurement, warehousing, and distribution support, as well as in support for the nascent hub system and laboratory support to EHNRI. For SPS, this can be seen in the extensive support that it provides to DACA and its work on prescribing practices and AMR. TASK 2: ASSESS COLLABORATION MODALITIES IN SUPPORT OF IMPROVED SUPPLY CHAIN MANAGEMENT 1. How well are these projects coordinating with each other, as well as with the other supply￾chain focused activities in Ethiopia, including USAID partners such as DELIVER, plus non- 34 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION PEPFAR stakeholders such as the Clinton HIV/AIDS Initiative (CHAI), UNICEF, the World Bank, and GOE institutions? In general, the evaluation team found that stakeholders in the sector were quite satisfied with the level of support, coordination, and collaboration they received from both projects. Stakeholders indicated that they had a clear understanding of their points of contact within the projects and the role that the project was supporting. In a few instances there was some confusion as to different roles provided by the two projects (which can be perceived as having significant overlap), but even then stakeholders felt that simple contact with either of the projects would quickly put them in touch with the correct information source. Coordination between the two HIV projects and among the three USAID projects in the supply-chain sector (these two and the USAID/DELIVER project) has been somewhat less than ideal over the last year or so, as overlapping scopes of work and different approaches came into conflict. It appears that recent clarifications by USAID have improved coordination. 2. To what extent have SCMS and RPM Plus coordinated and actively collaborated with other cooperating agencies (CAs) and institutions to finalize design and begin implementation of the Master Plan? While these two projects have coordinated and collaborated with others to improve the operation of the supply chain in Ethiopia (the overall supply chain, not just the AIDS sector), these efforts cannot be said to center around the Master Plan. Many elements of the Master Plan are reflected in these efforts, but the Master Plan has, to some extent, been set aside because so many of the institutions and staff reflected in its details are no longer in place. 3. What division of roles and responsibilities among USG partners could best serve efforts to develop an effective supply chain management system under the PLMP? The current division of roles and responsibilities is technically solid in that SPS focuses primarily on the ―Use‖ portion of the logistics cycle while SCMS and DELIVER focus on other components. This is consistent with SPS‘ historic strength in the prescribing, policy, and regulatory aspects of the logistics cycle as well as the other organization‘s strengths in the actual implementation elements of operating a logistics system in full supply. 4. How should LMIS development be handled under the PLMP, given the three USG partners with various models and software packages (SCMS, SPS, DELIVER)? The current division of LMIS responsibilities among the projects is consistent with their strengths. DELIVER has developed and is now rolling out a stores management system that is very strong at the facility level and was well received by the initial recipients contacted. The systems being put in place by SCMS will provide good support at the hub and central warehouse level. The systems supported by SPS provide useful patient data and prescribing guidance for first-level providers. Any data that are needed in common among the various systems can be easily shared across platforms using agreed-upon data interchange protocols. Other stakeholders in the sector, such as groups working on TB, Malaria, or other programs, should be brought into the efforts to coordinate among platforms and ensure the portability of necessary data. 5. How can coordination of TA partners among themselves and with the MOH be optimized? Many of the established coordination mechanisms between these projects and their Ministry counterparts have been put on hold during the BPR process. These mechanisms worked RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 35 adequately when they were meeting regularly and it is assumed that they will re-form and become viable again as the BPR process is concluded and the structure of the new institutions becomes clear and official. TASK 3: IDENTIFY ADDITIONAL APPROACHES OR ACTIVITIES TO ACHIEVE OBJECTIVES 1. What are the key project initiatives, activities, and approaches that warrant continued investment in the future (consider each of the workplan components), particularly in a scenario with potential funding decreases? Current project activities are contributing to high levels of program achievement in the AIDS and health commodity sectors. An increasing emphasis on those activities that not only contribute to program impact but also support activity sustainability would be a good strategic direction, particularly in a time of decreasing funding. This focus would tend to favor activities that develop infrastructure, systems, and human capacity over those that provide direct program inputs such as commodities, consumables, supplies, and salary support. 2. Are there initiatives, activities, and approaches that should be scaled back, reformulated or eliminated altogether? The activities of the projects that involve direct salary support for individuals, while having a positive effect on program impact, are unsustainable. The salary support should be transferred to the government as quickly as possible. Other forms of direct financial support should begin to include at least some form of matching requirement to increase the sense of ownership of the host country program. At present, the level of support is so extensive that program implementers are losing sight of their own role in supporting and shaping the program. Direct support for commodities should also be minimized. 3. Are there other promising initiatives, activities, and approaches not addressed by the projects that should be considered for future investment to maximize impact? The supply chain in Ethiopia is rapidly moving into a phase where management of time, communications, information. and transport will be as important to the overall effectiveness of the program as the management of space. Efforts to incorporate these elements more explicitly into program management and capacity development activities should be emphasized to take the program to the next level of efficiency and sophistication. In addition, the HIV/AIDS supply chain is now dramatically affecting, and being affected by, the supply chains and commodities for the rest of the health system. Efforts that manage the system in more comprehensive ways should also be emphasized. 4. Are there best or better practices, or significant products and tools from RPM Plus/SPS and SCMS, worthy of consideration for possible scale-up, dissemination, and/or replication? Many of the best practices from the projects here are already being shared with other programs through the project‘s home offices and internal ―best practices‖ networks. The information systems and stores management systems are best practices that are being adopted and adapted from other programs being managed by the same central mechanism. 5. Are these project models good examples of capacity-building mechanisms for the FMOH, PFSA, PSLD (if it endures), DACA, RHBs, and health facilities? 36 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION The capacity-building mechanisms being used by these projects in Ethiopia reflect state-of￾the-art practices used by similar projects throughout the developing world. The direct support of staff is a technique that may have been needed to get the level of program impact that has been achieved over the last few years, but is not a sustainable technique and should be phased out as the programs move to increase their sustainability. 6. What are the realistic prospects that project activities will increase sustainability in terms of installed capacity, if not in terms of financial independence, which seems unlikely? What can promote successful assumption of supply chain and pharmaceutical management functions by GOE partners? To the extent that these projects are supporting the creation of infrastructure, systems, and (most importantly) human capacity, they are supporting sustainability. At this point, the level of support is so extensive that the evaluation team saw strong evidence of undesirable levels of dependence. Efforts should be introduced to require key support and investments to parallel GOE investments (even if only a small matching portion is possible) on the part of the recipient program. This would increase the sense of GOE responsibility and decrease the sense of total donor responsibility. From a technical perspective, the supply chain‘s operation as a ―push‖ system has severely constrained the feeling of ownership at the lower levels. Shifting to a ―pull‖ system, and continually reinforcing the principles of operation of such a system, will help to increase a sense of ownership and responsibility. This shift may also increase job satisfaction, which could contribute to retention and thereby sustainability. This must, of course, be balanced against the fact that many products will remain in less than full supply, necessitating a mixed push-pull system. The practice of using trained pharmacists for many of the logistics management positions should also be reexamined. While pharmacists bring detailed drug knowledge to the setting, this level of knowledge is not needed for many of the actual stores management and procurement functions that these staff perform. The mismatch between these individuals‘ skills, training, technical interests and the actual work they are undertaking on a day-to-day basis contributes to job dissatisfaction and high turnover, which obviously detracts from sustainability. 7. What indicators might be adopted to measure program impact and process/outputs in the areas of capacity building, supply chain program strengthening and overall logistics system function, improved pharmaceutical management, and donor/implementation coordination? The significant M&E programs now in place within both programs contain ample and adequate measures of these characteristics, including such fundamental measures as number and duration of stockouts. 8. What are the assessment team‘s expectations regarding the project‘s future progress? While the achievements of these projects in supporting the HIV/AIDS supply chain over the last few years are outstanding, there are several factors that caused the evaluation team to express concerns about future success. Most of these concerns focused around the need for continuing capacity building and reinforcement of the system‘s new operating principles at the facility level. The cautions raised concern several aspects of a system that are currently functioning well, but appear to be ―fragile.‖ These fragile components must be monitored and strengthened in order for the system to perform at its historic high levels of achievement under a continuing growth scenario. The evaluation team identified seven specific areas where caution is imperative: RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 37 Transition to full supply system: The transition of the logistics system from rationed supply to full supply is both recent and only partially complete, i.e. only a few commodities have been in reliable full supply and only in recent years. It will take a great deal of reinforcement for operational staff at the various facility levels to fully comprehend the changes implicit in full supply operations and the complexities involved in continuing to operate full supply and rationed supply systems side-by-side simultaneously. Completing this transition. while continuing to provide support for a rapidly-growing system, is likely to be a challenge. Transition to pull system operations: Similarly, the transition of the logistics system from ―informed push‖ to a ―pull‖ system is both new and partial. Only in recent months have responsive ordering systems been put in place for facilities and only for selected HIV/AIDS commodities. Operation of a pull system involves dramatic shifts in power and control, not simply changes in the filling out of forms and employing different formulae. It will take time and reinforcement for this impact to be fully realized and for the new operational principles to be absorbed and internalized. Potential impact of changes in prescribing practices: Federal-level policy makers are now in the process of changing the guidelines for prescribing practice for patients identified as HIV positive. At present, the guidelines indicate that patients with a CD4 level under 200 should be started on ART while patients with a CD4 level under 350 should be placed on Co-trimoxazole (CTX) prophylaxis. At the facility level, there are as many or more patients identified as ―pre-ART‖ as there are active ART patients. The current proposal is to change the recommendations so that patients with a CD4 level under 350 would be started on ART, rather than CTX. If this is implemented quickly, the ART patient load could roughly double in short order. It is not clear that the system is sufficiently robust to absorb such a quick change. Weakness of lab logistics: While the supply chain for laboratory commodities has improved in recent months and it is now functioning in a fairly reliable and responsive manner, it was clear that laboratory supply management practices and overall operations are not as robust as, for example, that of the dispensaries. Much reinforcement of supply management procedures will be required in the coming months and several components of the basic operations, including power reliability and maintenance, must be addressed if the laboratories are to be able to support the continued growth of the HIV/AIDS program and the rest of the medical needs of the country. Potential system overcrowding: In the visits of the evaluation team, it was clear that although the vast majority of the stores were functioning at acceptable performance levels, they were also operating at close to full storage capacity. At the central level, it is also clear that significant volumes of orders for new commodities have been placed and will be delivered in the coming months. The evaluation team is concerned that these new volumes, on top of the full loads already being carried by the lower level stores, may overwhelm the lower level facilities and the transport system. Pace of transition of supply-chain responsibilities to PFSA: In the plans for the future of the health commodity supply chain in Ethiopia, PFSA figures quite prominently. For efficiency reasons, it has prospective roles in procurement, supply management, distribution, some data collection, and much of the information processing associated with managing the supply chain. In the long run, there are indications that this could be a smooth-running system. The evaluation team, however, has concerns about the size of the demands that are about to be placed on PFSA and the potential for its fragile systems to 38 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION collapse if too much demand is applied too quickly. Extensive and intensive capacity development assistance should be made available to it in the coming months to ensure that it can handle the task load, grow its capacity, and support the system in the way it needs to continue growing. Sustainability: The activities of these two projects in Ethiopia have contributed significantly to the success and growth of the HIV/AIDS program. They have also contributed improved infrastructure, systems, equipment, and increased human capacity that will continue to support the health program in the country for years. Many of the activities, however, are in the form of direct support for commodities, labor, or operational expenses. These activities contribute to the impact of the program, but are not sustainable without the continuing extensive external resource flows of recent years. If we are moving into a time of reduced funding, these activities will have to be seriously examined and other support found, or they will have to be curtailed. 9. Given possible decreased funding levels in coming years, how can support for commodity procurement, supply-chain system strengthening, and pharmaceutical management be balanced to further development and HIV/AIDS program goals? As stated above, increased emphasis must be placed on activities that not only contribute to program impact but also have a substantial sustainability. Activities such as (1) human capacity development, (2) systems development, (3) infrastructure, and (4) durable equipment have these characteristics. Policy interventions that support staff retention and knowledge retention are also desirable. 10. How can the two projects maximize their impact in terms of short-term support, as well as longer-term capacity building? The types of activities discussed above strike a good balance between short-term impact and longer-term contribution to capacity and development. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 39 APPENDIX A. SCOPE OF WORK USAID/Ethiopia President’s Emergency Program for AIDS Relief (PEPFAR) Scope of Work (SOW) for the External Assessment of The Rational Pharmaceutical Management Plus Program (RPM+) and The Supply Chain Management Systems Program (SCMS) PROJECT IDENTIFICATION DATA Project 1. Project Title: Rational Pharmaceutical Management Plus Program (RPM+)/follow-on project Strengthening Pharmaceutical Systems (SPS) has begun and funding is now channeled through that mechanism. 2. Project Number: 936-3104.02 3. Project Dates: September, 2000–March 31, 2009 (includes six month no-cost extension) 4. Project Funding: Life of Project (LOP) in Ethiopia $36,386,000 (includes $2,950,000 for SPS under COP08) 5. Implementing Organization: Management Sciences for Health (MSH) 6. Cognizant Technical Officer (CTO): Mr. James Browder Project 1. Project Title: Supply Chain Management Systems Program (SCMS) 2. Project Number: 936-3090.56 3. Project Dates: September 2005–September 2010 4. Project Funding: LOP in Ethiopia $122,574,673 ($54,121,302 under COP08) 5. Implementing Organization: Partnership for Supply Chain Management (PSCMS; prime in Ethiopia: MSH) 6. Cognizant Technical Officer (CTO): Mr. James Browder I. Identification of the Task The USAID/Ethiopia (USAID/E) PEPFAR office requests technical assistance from the Global Health Technical Assistance Project (GH Tech) to design and implement an independent evaluation of two programs: an end of project assessment of the Rational Pharmaceutical Management Plus Program (RPM Plus) and a mid-term assessment of the Supply Chain Management Systems Program (SCMS). Both projects focus on procurement and logistics management of commodities related to HIV services, including ART and PMTCT. Additionally, RPM Plus focuses on pharmaceutical management, including proper dispensing and handling of pharmaceuticals, promoting rational use of drugs (RDU), slowing the rate of development of antimicrobial resistance (AMR), and increasing the capacity of Ethiopia‘s drug regulatory agency, the Drug Administration and Control Authority (DACA), as well as the Ethiopian Pharmacists Association (EPA) and the country‘s pharmacy schools. Private sector approaches are also included under RPM Plus. 40 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION The assessment will draw from and build on feedback from USAID/E, key informant interviews with in-country partners conducted by external consultants to determine the strengths, weaknesses, and future direction of services provided by the SCMS and RPM Plus/SPS Projects in Ethiopia. The SCMS Project‘s overall purpose is to establish, operate, and support a safe, secure, reliable, and sustainable supply chain management system to procure pharmaceuticals and other products needed to provide care and treatment of persons with HIV/AIDS and related infections. The RPM Plus/SPS project‘s overall purpose is to improve the availability of health commodities of assured quality for population, health, and nutrition priority interventions, and promote their appropriate use, providing technical leadership in pharmaceutical management to global initiatives such as PEPFAR. USAID/E implements activities under these global projects through the use of field support funds. As part of its overall evaluation of all HAPN projects, USAID/E is commissioning this mid-term assessment to examine the following: The two projects‘ progress toward achieving results against stated objectives as expressed in its work plan. Identify strengths and weaknesses within the projects‘ portfolios. Determine areas and activities that may warrant continued investment, as well as other key initiatives and approaches not covered by the projects, but which would likely contribute to improving the availability of HIV/AIDS commodities and other essential health commodities. Analyze current implementing partner coordination modalities and provide recommendations as to optimum technical areas of responsibilities in the implementation of the national Pharmaceutical Logistics Master Plan (PLMP), with particular emphasis on the three USAID￾funded implementing partners supporting the PLMP (of which SCMS and RPM Plus/SPS are two). Given the huge gaps in funding to fully implement the PLMP, provide recommendations as to optimum U.S. government (USG) roles in supporting HIV commodities and system strengthening/capacity building, in the short, medium and long term, given funding levels which cannot cover all areas (of commodities or systems strengthening). The USAID/E PEPFAR office requests that the in-country components of this assessment be fielded by on or about (o/a) January 12, 2009, and that the assessment report be completed by February 20, 2009, in order that the findings, conclusions, and recommendations can be used to enhance project implementation and inform future project directions. II. Background From 2004 through 2006, an estimated 288,000 Ethiopians died from HIV/AIDS-related causes. The 2008 Federal estimate2 of national HIV prevalence is 2.2%: 7.7% in urban areas but much lower in rural areas at 0.9% (FHAPCO, 2007). As of 2008, more than one million (1,037,267) persons are estimated to be living with HIV and about 289,734 people living with HIV/AIDS (PLWHA) are in need of ART (FHAPCO, 2008). In response to this situation, the Government of Ethiopia (GOE) has launched an ambitious push to provide universal access to primary health care services, including a full range of HIV services, by the end of 2010. Given that the country is the eighth poorest in the world and heavily dependent on donor financing for health programs including HIV, reaching this goal will be a challenge. Notwithstanding, the GOE is moving forward with multiple redesigns of basic support 2 Federal HIV/AIDS Prevention and Control Office (FHAPCO), Single Point Estimates of HIV and OVC Indicators, April 5, 2007. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 41 systems, including the health management information system (HMIS) and the supply chain. A well-designed framework for the national logistics roll-out, the PLMP was developed in 2006, but a long governmental reengineering process, Business Process Reengineering (BPR), among other factors, has delayed final design, and thus implementation of the PLMP. Structures originally developed to support the PLMP, such as an Implementation Support Unit (IST) with expatriate technical advisors, including some from the USG, were dismantled, and currently there is no mechanism for the several donor and international technical support agencies which support the PLMP to channel or coordinate their input. The GOE has obtained substantial funding for the PLMP from the Global Fund To Fight AIDS, Tuberculosis and Malaria (GFATM) and the Global Alliance for Vaccines and Immunization (GAVI), but deployment of these resources has been slow, and reprogramming of some resources has also occurred. Notwithstanding these resource inflows, huge gaps in funding for the PLMP exist, particularly in the area of capitalization for an envisioned Revolving Drug Fund (RDP), and in the building of stores for hospitals and health centers, which currently lack space to handle the hoped-for throughput under the new system. SCMS has helped design the distribution network of regional hub warehouses, as well as rent temporary warehouses until construction occurs in some areas. SCMS has also purchased 25 vehicles estimated as needed to begin more effective distribution under the new system. At present, the distribution system, other than a system put in place by the USG with RPM Plus (originally in 2006) and now SCMS providing antiretroviral (ARV) drugs and a few other products, is not effective, and stockouts of many products are frequent. Additionally, MOH procurement of HIV commodities using GFATM funds has met great difficulties, despite SCMS, RPM Plus, and World Bank technical assistance (TA). These are among the many challenges which must be overcome if the PLMP is to become a reality. III. Overview of the Rational Pharmaceutical Management Plus Program (RPM+) The RPM Plus Project has provided strong support to pharmaceutical management efforts in Ethiopia since the beginning of PEPFAR in 2004, and before that under the Presidential PMTCT Initiative as well. The centrally-funded activity has procured and distributed first Nevirapine for PMTCT efforts, and then the full range of first and second line ARVs for Ethiopia‘s treatment programs. Procurement of second-line and pediatric ARVs continued through 2006, when the Clinton HIV/AIDS Initiative (CHAI) took responsibility for procuring those commodities. In 2006, PEPFAR/Ethiopia decided to begin utilization of the SCMS, specialized in logistics, with some of RPM Plus‘ activities gradually transitioning to SCMS. These included responsibility for national-level procurement and procurement TA, as well as distribution to regional levels. RPM Plus maintained its role in distributing ARVs to site level, and through the Antiretroviral Dispensing Tool (ADT) it implemented previously, it determined the amounts needed at each site. Additionally, RPM Plus vehicles actually delivered the ARVs to facilities. These important roles have gradually been shifted to SCMS, and under COP09 SCMS will assume distribution to site level, as well as site-level support for supply chain management. These shifts have responded to budget limitations as PEPFAR‘s COP09 budget was reduced from COP08 levels, and to a desire to simplify logistics management TA responsibilities, which are shared at USAID/Ethiopia with USAID/DELIVER, supported with Population funds. 42 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Notwithstanding these reductions in scope, RPM Plus, now entering the follow-up program Strengthening Pharmaceutical Systems (SPS) with the same local team, maintains an important role in the USAID/E portion of the PEPFAR/Ethiopia portfolio. The project, with its expert personnel specialized in pharmaceutical management, provides USG support to strengthening Ethiopia‘s FDA equivalent agency, the DACA, through technical, staff and material support. SPS has an MOU with the agency. Staff support for the country‘s Pharmaceutical Supply and Logistics Department (PSLD) may end as a nationwide BPR effort appears likely to dissolve the unit and merge it into the Pharmaceutical Fund and Supply Agency (PFSA), the country‘s central medical stores supported by SCMS. Other important SPS roles maintained under COP09 include the promotion of rational drug use (RUD), the critical efforts to slow antimicrobial resistance (AMR), the establishment and support of Drug Therapeutic Committees (DTCs) and Drug Information Committees (DICs), as well as support to the country‘s professional association for pharmacists, the Ethiopian Pharmacist Association (EPA), and university pharmacy schools and private sector pharmacies supporting HIV services such as ART. RPM Plus has additionally relinquished its role supporting drug quality assurance, in support of DACA, to the United States Pharmacopeia (USP). This year the project transitions out of supply chain management, and continued close collaboration with SCMS and other PEPFAR implementing partners will be critical to ensure success of that important effort. IV. Overview of the Supply Chain Management Systems Program (SCMS) SCMS began operations in Ethiopia in late 2006, and with very substantial funding and a strong local team, including some staff from USAID/DELIVER and MSH/RPM Plus, particularly the latter. Rapid scale-up was essential, as the Ministry of Health‘s Millennium AIDS Campaign (MAC) began almost simultaneously with SCMS, and SCMS was called upon to make very large procurements of laboratory reagents and supplies, as well as rapid test kits (RTKs), at a time when its own systems were in their infancy. The project successfully passed through this difficult period, and with USAID support, began procuring commodities funded by GFATM, providing the procurement services, customs clearance and paying 5% of the total product value for distribution costs. In March 2007, the project led the first of three national HIV commodity-quantification exercises to date, which were highly successful in defining total needs and costs for all types of HIV commodities needed to reach the country‘s ambitious universal access goals. Information from these exercises has been used in two subsequent successful GFATM proposals which have brought pledges for over $800 million in aid through 2014, including almost $600 million in commodities. Equally importantly, the project has helped define the funding gap to reach the national goal. SCMS has a second, very important focus: capacity building in support the country‘s PLMP, which seems increasingly likely to focus on PFSA, from national to site level. SCMS has assisted Ethiopia in designing a potentially effective distribution system, based on regional hub warehouses, and has also quantified, and now procured, the necessary vehicles to begin distribution. A major bottleneck has been the lack of definition by the Ethiopian government of final roles and responsibilities in the system, which has delayed final system design and implementation. In the face of this situation, the USG has asked SCMS to work with SPS and other PEPFAR partners to develop an interim distribution system for all HIV commodities, RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 43 modeled on the successful ARV system, until such time as the PLMP is functional. That system is currently under design, and will require negotiation with the FMOH if it is to be implemented. As shifting responsibilities took ARV procurement away from the USG, SCMS has changed its procurement focus, purchasing large amounts of opportunistic infection (OI) drugs, infection prevention (IP) materials, PMTCT equipment and supplies, ready-to-use therapeutic food (RUTF), in addition to lab reagents and supplies already procured by the project. These purchases, using up most of a large pipeline, have filled critical gaps in the country‘s HIV commodity needs which were not covered by the GOE. With the pipeline dwindling, it will likely not be possible for SCMS to continue this role in 2009. SCMS and RPM Plus maintain co-located offices in Addis Ababa and has additional technical staff located in regions throughout Ethiopia—with a collective total of approximately 61 staff members for SCMS and 268 for RPM Plus, including 195 placed by RPM Plus/SPS with GOE organizations such as DACA (15) and individual health facilities (180). The Project works with diverse partners in Ethiopia, including the GOE (PFSA, PSLD), multilateral/donor agencies (i.e. UNFPA, UNICEF, Clinton Foundation), and other CAs (Pathfinder International, Crown Agents, Carter Center, SCMS, RPM+). Indicators The high-level indicator in SCMS/Ethiopia‘s Work Plan with the objective to ―Enhance and develop the capacity of logistics systems in Ethiopia to support the National HIV/AIDS program‖ is ―Number of stock-outs of ARV commodities resulting in vital program interruptions.‖ This indicator is used for maturing systems, but design of the PLMP has not begun and existing systems function poorly. Historically, USAID has struggled to define indicators for government logistics systems and this may be a constraint for the assessment team in terms of qualitative assessment. V. Purpose of the Assessment USAID/E requires a team of three independent consultants to conduct an end-of-project assessment of the RPM Plus Program and a mid-term project assessment of the SCMS Program. GH Tech staff for this assessment must be limited to GH Tech staff or sub-contractors that are not currently implementing HIV/AIDS related programs in Ethiopia. This assessment will collect information about RPM Plus/SPS and SCMS implementation, efficiency, progress and challenges, as well as coordination with other technical partners supporting HIV/AIDS programs in Ethiopia, with a particular emphasis on those supporting the implementation of the Pharmaceutical Logistics Master Plan (PLMP). While RPM Plus is ending, the continuation project SPS receives USAID/E funding (through the President‘s Malaria Initiative (PMI) as well as PEPFAR), so the ―end-of-project‖ RPM Plus activity in reality looks at a continuing project as well, and should be seen in that context. This is an external assessment to assess both program process and impact, document lessons learned, and to inform follow-on activities. The external assessment team will be expected to develop a detailed scope of work and plan of action prior to the assessment. Given the complexity and diversity of the two projects‘ portfolios, much of the assessment should be qualitative in nature—relying on key informant interviews with a range of partners. This assessment will: TASK 1: Assess Progress to Date in Achieving Planned Results (Estimated level of effort – 50%): Review RPM Plus and SCMS‘ technical and programmatic strengths, weaknesses, successes and constraints, identifying contributing factors. Based on the assessment findings, 44 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION the team will present results and impact achieved to date, as well as contributions that the two projects have made to meet the PEPFAR 2-7-10 targets; document lessons learned; and make recommendations towards achieving planned results in the next two years of implementation. TASK 2: Assess Collaboration Modalities in Support of Improved Supply Chain Management (Estimated level of effort – 10%): Assess the relationships among SCMS, RPM Plus/SPS and various partner institutions, other cooperating agencies, other donors, HAPCO, the Ministry of Health and its components related to logistics, supply chain, and pharmaceutical management, (including how the two projects cooperate currently and how that should shift given new mandates under COP09). TASK 3: Assess Management Structures and Their Role in Achieving Planned Results (Estimated level of effort – 10%): Evaluate the two projects‘ staffing and management structure. TASK 4: Identify Additional Approaches or Activities to Achieve Objectives (Estimated level of effort – 30%): Identify project activities and approaches that warrant continued investment, as well as new areas and approaches not covered by the projects, but which would likely contribute to improving the availability of HIV/AIDS commodities and strengthening the overall public sector supply system. Illustrative questions to assist in the assessment are provided below. The assessment team is expected to refine, prioritize, and finalize assessment questions in discussion with USAID at the start of the assessment. TASK 1: Assess Progress to Date in Achieving Planned Results (Estimated level of effort – 50%) 1. What has been the two projects‘ progress to date in achieving planned results and performance indicators? 2. What have been the projects‘ most important lessons learned to date? 3. What contributions have the two projects‘ made in increasing the availability of HIV/AIDS commodities in Ethiopia? 4. Are SCMS and RPM Plus/SPS workplan activities (short and long-term) compatible with the PLMP? Are the two projects‘ comparative strengths being leveraged optimally in the implementation of the PLM P? 5. Are Logistics Management Information System (LMIS) development goals being met? Are additional monitoring processes needed to ensure this important and complex project activity stays on track? If so, please suggest new monitoring systems or implementation strategies based on assessment results. 6. How are the two projects strengthening the overall national supply and pharmaceutical management systems? 7. What monitoring and evaluation systems are in place to assess the two projects‘ impact, including for sustainability of project results? Do the systems identify implementation problems so that corrective action can be taken? 8. Please also identify key barriers/constraints that have moderated projects results. 9. How have the projects contributed to reaching PEPFAR‘s 2-7-10 goals? RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 45 10. What impact have the two projects had to date in terms of building institutional capacity and providing needed commodities and other inputs to support HIV services, including ART? 11. Are these projects effective models for technical services? If so, how, and if not, why not? 12. Have TA, procurement, and capacity building services been provided in the most cost￾efficient ways possible? 13. Have the two projects provided quality, state-of-the-art services and utilized state-of-the-art TA approaches for low-resource settings? 14. Have appropriate quality assurance mechanisms been utilized, both for the internal activities of the two projects, and to promote optimum services in GOE institutions supported? 15. What are the contributions of the two projects to implementation of the PLMP? TASK 2: Assess Collaboration Modalities in Support of Improved Supply Chain Management (Estimated level of effort – 10%) 1. How well are these projects coordinating with each other, as well as the other supply chain￾focused activities in Ethiopia, including particularly USAID partners such as DELIVER, plus non-PEPFAR stakeholders such as the Clinton HIV/AIDS Initiative (CHAI), UNICEF, the World Bank, GOE institutions, and others as appropriate? 2. To what extent have SCMS and RPM Plus coordinated and actively collaborated with other CAs and institutions to finalize design and begin implementation of the Master Plan? 3. What division of roles and responsibilities among USG partners could best serve efforts to develop an effective supply chain management system under the PLMP? 4. How should LMIS development be handled under the PLMP, given three USG partners with various models and software packages (SCMS, SPS, DELIVER)? 5. How can coordination of TA partners among themselves, and with the MOH, be optimized? TASK 3: Identify Additional Approaches or Activities to Achieve Objectives (Estimated level of effort – 40%) 1. What are the key project initiatives, activities, and approaches that warrant continued investment in the future (consider each of the workplan components), particularly in a scenario with potential funding decreases? 2. Are there initiatives, activities, and approaches that should be scaled back, reformulated, or eliminated altogether? 3. Are there other promising initiatives, activities, and approaches not addressed by the projects that should be considered for future investment to maximize impact? 4. Are there best or better practices, or significant products and tools from RPM Plus/SPS and SCMS, worthy of consideration for possible scale-up, dissemination, and/or replication? 5. Are these project models the best form of capacity-building mechanisms for the FMOH, PFSA, PSLD (if it endures), DACA, RHBs, and health facilities? 6. What are the realistic prospects that project activities will maximize prospects for sustainability in terms of installed capacity, if not in terms of financial independence, which 46 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION seems unlikely? What can promote successful assumption of supply chain and pharmaceutical management functions by GOE partners? 7. What indicators might be adopted to best measure program impact and process/outputs in the areas of capacity building, supply chain program strengthening and overall logistics system function, improved pharmaceutical management, and donor/implementation coordination? 8. What are the assessment team‘s expectations regarding the project‘s future progress? 9. Given possible decreased funding levels in coming years, how can support for commodity procurement, supply chain system strengthening, and pharmaceutical management best be balanced to further development and HIV/AIDS program goals. 10. How can the two projects maximize their impact in terms of short-term support, as well as capacity building with longer-term implications? VI. Methodology of the Assessment The assessment will be carried out in Ethiopia by a team of external consultants over a three-to￾four-week period. The methodology of data collection will include: key informant interviews, focus group discussions, field observations, review of reports, and other project documents. Pre-Assessment Briefing During the Team Planning Meeting (TPM) period (below), the assessment team will hold a preliminary meeting with the management team of USAID to review the scope of the assessment, agree on the key research questions, and finalize the schedule. The outcome of this meeting will be a detailed work plan for the assessment, including milestones and deliverables with due dates clearly established. In addition to formal briefing and debriefing meetings, the assessment team may contact the USAID management team as necessary to provide updates on its progress and obtain additional guidance on logistics, additional data, and information sources. Team Planning Meeting A three-day TPM will be held in country before the assessment begins. This meeting will allow USAID to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will: – clarify team members‘ roles and responsibilities, – establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion, – review and develop final evaluation questions, – develop data collection methods, instruments, tools, and guidelines, – review and clarify any logistical and administrative procedures for the assignment, – develop a preliminary draft outline of the team‘s report, and – assign drafting responsibilities for the final report. Document Review USAID, SCMS and RPM Plus/SPS will provide the assessment team with a package of briefing materials relevant to the assessment. This documentation will include strategy/concept papers, RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 47 project quarterly reports, work plans, and any management reviews or other documents that are developed and reviewed as part of the continuous monitoring of the project. Field Visits The assessment team is expected to conduct site visits to areas in which the two projects implement substantial activities, such as Oromia, Amhara, Tigray, Southern Nations, Nationalities and Peoples Region (SNNPR), and Dire Dawa. Key Informant Interviews, Meetings and Focus Group Discussions (as indicated) The assessment team will conduct qualitative, in-depth interviews and meetings with key stakeholders and partners (a preliminary list of stakeholders and partners is attached in Annex 1, but the assessment team should add to this list as necessary). Key informants should include, but not be limited to: USAID/Ethiopia staff, particularly the SCMS/RPM Plus/SPS and DELIVER Project management units. Government of Ethiopia representatives from the FMOH, including the national and regional HIV/AIDS Prevention and Control Organization (HAPCO), the Pharmaceutical Supply and Logistics Department (PSLD), the Drug Administration and Control Administration (DACA), and Pharmaceutical Fund and Supply Agency (PFSA) at national and regional levels; as well as the Ethiopian Pharmacy Association (EPA), Regional Health Bureaus (RHBs), Woreda (District) Health Offices, and health facility staff. SCMS and RPM Plus/SPS staff (representatives of both the Addis Ababa office and regional offices). In-country partners, including CHAI, UNICEF, and the World Bank. Other Cooperating Agencies, including DELIVER, and other PEPFAR partners as indicated (e.g., CDC, MSH/HIV-AIDS Care and Support Project, Abt Associates etc.). VII. Information Sources Consultants will be provided the following background documents in preparation of the assignment: SCMS Contract and RPM+/SPS Cooperative Agreement, including all major modifications (these will be obtained from USAID/Washington, as these are centrally funded projects), SCMS and RPM+/SPS PEPFAR Semi-Annual Report submissions, SCMS and RPM+/SPS Quarterly Reports, SCMS and RPM+/SPS Work Plans, USAID trip reports summarizing past field visits/reviews of SCMS/RPM+/SPS, COP ‗06, ‗07, ‗08, ‗09/Annual Performance Review (APR)/Semi-Annual Performance Review (SAPR), MOUs with DACA and PFSA (draft), Pharmaceutical Logistics Master Plan (PLMP), latest PLMP Partner Accountability Matrix and PFSA BPR (draft), and 48 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Other relevant technical reports/concept papers/documents. VIII. Tasks to be accomplished Below is a list of the specific tasks to be accomplished by the consultant team, with an estimated level of effort for each task. (See Annex 2: Planning Calendar, for the exact schedule). Team Leader preparation 1 day Team reviews background documents/develops assessment methodology 3 days Travel for international consultants 2 days Team Leader and team members final planning in country 2 days Meetings and interviews with key stakeholders in Addis 3 days Site visits in Addis 2 days Conduct field visits and interviews 9 days Conduct follow-up stakeholder interviews in Addis 1 day Full Team Synthesis/Analysis of Findings and Draft Report 5 days Conduct debriefings for USAID/E and Partners (separately) 1 day Finalize and submit draft report to USAID/E in-country 2 days Travel for international consultants 2 day Finalize Report – Team Leader incorporates Mission comments and submits report electronically to CTO (TL: 5; TM: 3) 5/3 days Total level of effort (LOE) – 38 days of LOE for Team Leader and up to 35 days for Team Members, including four travel days each for the international consultants. A six-day work week is authorized for work in Ethiopia. IX. Team Composition and Participation This will be four-person team. USAID/E seeks four consultants: a senior expatriate Team Leader experienced in evaluating USAID HIV/AIDS programs with in-depth knowledge and experience in supply chain management, a senior expatriate Team Member with experience in logistics information management systems, a local or expatriate consultant with expertise in pharmaceutical management, and a local Logistics Assistant. USAID/E staff may also join the assessment team during the site visits. SCMS and RPM+ partner agencies as well as MOH counterparts may accompany the team on site visits as appropriate, but will not be present during interviews with stakeholders or beneficiaries. 1. The Team Leader will be an international consultant with extensive PEPFAR or HIV/AIDS program implementation and evaluation experience, with particular focus on HIV supply chain management. S/he will agree to fulfill his/her responsibilities in over five weeks, spending three and one half weeks in-country, and will play a central role in guiding the assessment process. The consultant can hold conference calls with core team members and USAID/E representatives before and after the visit to Ethiopia, as needed. The consultant will also be required to participate in an in-brief USAID/E on arrival, a debrief with USAID/E and SCMS/RPM+/SPS on assessment findings, and to produce a draft report within one week of departing Ethiopia followed by a final report for USAID/E. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 49 The Team Leader will: Finalize and negotiate with client for the team work plan for the assignment; Establish assignment roles, responsibilities, and tasks for each team member; Ensure that the logistics arrangements in the field are complete; Facilitate the TPM or work with a facilitator to set the agenda and other elements of the TPM; Take the lead on preparing, coordinating team member input, submitting, revising, and finalizing the assignment report; Manage the process of report writing; Manage team-coordination meetings in the field; Coordinate the workflow and tasks and ensure that team members are working to schedule; and Ensure that team field logistics are arranged (e.g., administrative/clerical support is engaged, ensuring that payment is made for services, car/driver hire or other travel and transport is arranged, etc.). 2. The second Senior Expatriate Team Member will be a consultant with extensive implementation and evaluation experience in logistics management information systems, preferably with experience in Sub-Saharan Africa. Knowledge of HIV/AIDS programming and PEPFAR is essential. The consultant will be responsible for writing some sections of the report to be determined during the TPM in consultation with the team leader. The consultant will assist the team leader in the development of any qualitative and quantitative instruments to be used during site visits as well as the analysis of any data collected. 3. The Senior Local Team Member will be a consultant with extensive implementation and evaluation experience in pharmaceutical management in Sub-Saharan Africa, and preferably with experience in supply chain management. Knowledge of HIV/AIDS programming and PEPFAR is essential. The consultant will be responsible for writing the pharmaceutical management sections of the report and possibly others to be determined during the TPM in consultation with the team leader. The consultant will assist the team leader in the development of any qualitative and quantitative instruments to be used during site visits as well as the analysis of any data collected. 4. The Logistics Assistant will be a local consultant, preferably fluent in Amharic, with a demonstrated: ability to be resourceful and to successfully execute complex logistical coordination; to multi-task, to work well in stressful environments and perform tasks independently with minimal supervision; and to work collaboratively with a range of professional counterparts at all levels. The Logistics Assistant will be responsible for logistics, coordination and administrative support, and ensuring all aspects of the evaluation are carried out seamlessly. He/She will assist the Team Leader and the implementing agencies in facilitating meetings, coordinating logistics, and organizing site visits. The Logistics Assistant will collect and disseminate background documentation to the evaluation team. 50 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Selection Criteria for Evaluation Team Senior Team Leader (Maximum 100%) distributed as follows: 1. Education: (25%) An advanced degree (M.D., R.N., M.P.H., Ph.D., M.A., M.S. or M.B.A.) from a reputable accredited institution in medicine, public health, or any of the social sciences pertinent to work with supply chain management. 2. Work Experience: (35%) Minimum 10 years of progressively responsible experience with recognized organization(s) in the design, implementation and evaluation of HIV/AIDS programs with demonstrated technical expertise and skills in supply chain management. 3. Skills and Abilities: (40%) Demonstration of strong analytical, managerial, and writing skills with great attention to detail is very critical for the evaluation work. Exceptional leadership in coordinating, assigning the team with the appropriate responsibilities, communication, and interpersonal skills is absolutely critical. In addition, the team leader must be able to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGOs counterparts. Must be fluent in English and have proven ability to communicate clearly, concisely and effectively both orally and in writing. Must be able to produce a succinct quality document that gives direction and facilitates improvement of SCMS and RPM+/SPS programs in country. Senior Expatriate Team Member (Maximum 100%) distributed as follows: 1. Education: (25%) M.D., R.N., M.P.H., Ph.D., M.A., M.S. or M.B.A. from a reputable accredited institution in medicine, public health, or any of the social sciences pertinent to working in HIV/AIDS programs in the area of supply chain management, with special emphasis on logistics management information systems (LMIS). 2. Work Experience (35%) Minimum eight years of progressively responsible experience with recognized organization(s) in the design, implementation and evaluation of logistics programs with demonstrated technical expertise and skills in HIV/AIDS, preferably in Sub-Saharan African countries. 3. Skills and Abilities (40%) Demonstration of strong analytical, managerial and writing skills. Able to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGO counterparts. Must be fluent in English and have proven ability to communicate clearly, concisely, and effectively both orally and in writing. Senior Local Team Member (Maximum 100%) distributed as follows: 1. Education: (25%) M.D., R.N., M.P.H., Ph.D., M.A., M.S., M.B.A., or B.A. from a reputable accredited institution in medicine, public health, or any of the social sciences pertinent to working in HIV/AIDS programs, with special emphasis on pharmaceutical management. 2. Work Experience: (35%) Minimum six years of progressively responsible experience with recognized organization(s) in the design, implementation and evaluation of pharmaceutical management programs with demonstrated technical expertise and skills in HIV/AIDS in Sub￾Saharan African countries; supply chain management experience useful. 3. Skills and Abilities: (40%) Demonstration of strong analytical, managerial, and writing skills. Able to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGO RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 51 counterparts. Must be fluent in English and have proven ability to communicate clearly, concisely, and effectively both orally and in writing. Mid-Level Logistics Assistant (Maximum 100%) distributed as follows: 1. Education: (25%) M.A., M.S., M.B.A. or B.A. Four years of work experience may be substituted for the degree. 2. Work Experience: (35%) Minimum 6 years of progressively responsible experience within GOE, USG and implementing partners, and/or NGO work settings handling complex logistics, such as coordinating business travel and meetings. 3. Skills and Abilities: (40%) Must have a demonstrated: ability to be resourceful and to successfully execute complex logistical coordination; ability to multi-task, work well in stressful environments, and perform tasks independently with minimal supervision; ability to work collaboratively with a range of professional counterparts at all levels, including those from host country governmental and non-governmental organizations, U.S. Government agencies, and other donors; capacity for effective time management and flexibility. Must be able to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGO counterparts. Must be fluent in English and preferably Amharic, and have proven ability to communicate clearly, concisely, and effectively both orally and in writing. X. Schedule and Logistics The in-country phase of the evaluation will be conducted over a period of up to 24 days (including Sundays) with a desired start date TBD based on consultant availability. The Evaluation Logistics Assistant, in collaboration with the USAID/E Evaluation Coordinator and SCMS and RPM+/SPS, will arrange all of the partner meetings, site visits, and debriefings in advance. Meeting space will be provided at USAID/E, but the agency cannot provide access to fax and email. All associated travel and per diem costs for non-USAID staff will be covered by GH Tech under the contract with USAID/E (See Attachment 2: Planning Calendar for the illustrative schedule). XI. Period of Performance Work is to be carried out over a period of approximately five weeks (three and one-half weeks in￾country), beginning in-country on or about (TBD, based on consultant availability). XII. Financial Plan A budget plan agreement between the USAID/E PEPFAR and GH Tech will be reached and USAID/E will process a Modified Acquisition & Assistance Request Document (MAARD) to transfer funding for the assessment activity into the GH Tech Indefinite Quantity Contract (IQC). XIII. Deliverables Finalized no later than in-country TPM: Team Leader will develop an assessment methodology and field visit and interview schedule in consultation with the USAID/E CTO, USAID/E Evaluation Coordinator, SCMS, and RPM+/SPS. Three days after Team Leader arrival: Team meeting and in-briefing with USAID/E. USAID/E HIV/AIDS technical staff to review and comment on assessment methods. Prior to departure: Team makes a presentation to USAID and/or USG PEPFAR staff, a separate presentation to SCMS and RPM+/SPS, and Team Leader submits a draft report. The report will 52 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION be appropriately edited/formatted after the final draft is approved by the Mission, (See separate MS Word file for GH Tech Evaluation Report Guidelines), to USAID/E CTO—two hard copies and one electronic copy on CD ROM or flash drive. After departure: Team Leader submits final edited content to USAID/E within one week of receiving comments from USAID/E. The report (not including attachments) will be no longer than 30 pages with an Executive Summary, Introduction, Methodology, Findings, Lessons Learned, Conclusions, and Recommendations in English in the exact format specified by the USAID/E Evaluation Coordinator. The Team Leader submits a final report that incorporates the team responses to Mission comments and suggestions. The draft final report should be completed within five days after USAID provides its feedback on the draft report incorporating the comments received from the review of the draft and sent to the Mission. The final report (excluding executive summary and annexes) should be no more than 30 pages. After the final but unedited draft report has been reviewed by USAID, GH Tech will have the documents edited and formatted, and will provide the final report to USAID/ES for distribution (10 hard copies and a CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. If there is procurement-sensitive information in the report, it will be removed from the public report and prepared as a separate Memo marked for Internal USAID Distribution Only— Procurement Sensitive. RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 53 APPENDIX B. LIST OF DOCUMENTS REVIEWED RPM+/SPS DOCUMENTS Sheraton Presentation1 07 Sep 04.pdf Pharmacy Practice of Private Community Pharmacies in AA Asse.pdf Pharmacy capacity 15 hospitals 3 Sept draft.pdf Pharmaceutical mgt study TB-HIV-Ethiopia-Phase two.pdf National Drug Quality control Consultancy report-James Binka.pdf Layloff_Ethiopia_Feb2004_Assessment.pdf Daniel_Witt_Mekonnen_Ethiopia_Jan2004_Assessment.pdf Daniel_Witt_Ethiopia_Sep2003_Assessment.pdf ADR-Draft report on the assessment of health care providers.doc DACA-RPM DPOA.doc DACA training.doc DACA RPM+ Joint Project Sept-Oct Progress Report.xls DTC progress report Jan-Sept 08 oct.doc Summary Report Ethics B D.doc SUMMARY REPORT Ethics Mekelle.doc SUMMARY REPORT Ethics Jimma.doc SUMMARY REPORT Ethics Diredawa.doc USER GUIDE- DIP reporting tools.doc User guide - Daily Event Registration Form.doc Standard Operating Procedures for Supportive Supervision.doc TA event registration - MSH.doc Reports of RPM (revised).doc RPMPlus Bulletin-Ethiopia-final.doc PMP for RPM Plus-Ethiopia.pdf Proceeding Adama Review Meeting.zip Monthly report submission.xls Performance Indicator Tracking Table.doc MERP RPM Ethiopiajuly_06.doc Meeting (program update-quarterly) guide (rev).doc Field visit reporting form.doc Meeting (Management) report template.doc DIP-Higher level indicators.doc Event registration forms.xls DIP progress reporting format (final-revised).xls DIP progress tracking sheet (RPMAs).xls Checklist for DTC Monitoring (rev-rev1).doc Checklist for Monitoring MSH (rev-rev).doc DIP budget monitoring tool.doc 54 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION Treatment TWG Minutes Oct.2005 No. 19.doc Treatment TWG Minutes No. 20.doc Treatment TWG Minutes No. 14.doc Staff Meeting Minutes November 26 08 doc'.doc Report of the Quarterly Review Meeting of MSH May 3-4.07.doc Minutes 2nd Staff Meeting Dec 30 2008.doc Minute of MSH SPS Management Team Meeting (August 21, 08).doc Minute Mgt Team Meeting (May 20, 08).doc July 14 2008 meeting.doc Draft Minutes November 26 08 doc'.doc Action from May minutes.doc Dec 08-Patient Uptake final.xls Special Brief Bullet Points 041904.doc Ethiopia assessment summary v1 09 12 03.doc 15. ART in Addis Ababa.ppt 14. OVERVIEW OF RENOVATION ACTIVITIES No 2.ppt 12. RDU Adherence 02 May 2008.ppt 11. ADR DEPARTMENT.ppt 9. Yaregal-Contributions of RPM plus for PMTCT.ppt 10. AMR presentation DK.ppt 8. PPP update May 2008 NEW SPS TEMPLATE.ppt 6. DTC report May 02-08 (rev).ppt 7. Report of Training activities.ppt douglass.ppt 5. EPA-MSH.ppt-Re.ppt 4. MSH TA to FMOH Edm.ppt 4. MSH PSLD Daniel.ppt 3. Final QA Presentation.ppt 2. DACA-RPM Plus.Joint.Project.Presentation Final.Hailu.ppt 1. Antenane- MER update.ppt December-Issues and challenges of the month.ppt Concerns of the Month.ppt Challenges of the Month.ppt RPM Plus Notable achievements (2005-2008).ppt Notable achievements, 2008.doc May - progress report.ppt June - progress report.ppt Jan.-March progress (rev).ppt Jan.-June progress-rev.ppt April - progress report.ppt An update on RPM plus's MIS August 4, 2008.ppt 12. MIS-slides.ppt 11. FI-slides.ppt RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 55 11. FI- slides.ppt 10. Teketel - slides.ppt 9-Redahegn slidesl.ppt 8-EAST AMHARA REGIONE.ppt 7. Tewodros-slides.ppt 6. Abebaw slides.ppt 5. Addisu slides.ppt 4. Training-slides.ppt 4. Training-slides.ppt 3. DTC-slides.ppt 2. RMU 4-6August Report Presentation.ppt 1. Summary of Review Meeting - April 14-16, 2008.ppt 5. DACA-MSH.ppt 4. SPS- Core activity.ppt 3a. SPS framework.xls 3. SPS Planning.ppt 0. Elias Recap of Day One.ppt 4. Mis team.ppt 3. ARVs & Regimen Update-Hailu.ppt 2. RDUDispensingCounseling.ppt 1. Pharmaceutical care EPA July26-2008ver02.ppt UPDATE ON MIS.ppt SPS presentation Neg01.ppt ARV Drugs and ART-Negussu ver03.ppt 2nd quarter DTC review report.ppt Transition from RPM to SPS, DR. Nigussu.ppt Tell The World.ppt Planning meeting Summary of review meeting August at Adama, Elias.ppt MSH-SPS Planning Edmealem.ppt Hailu Power Template.ppt Base-Line Survey on DSM Getachew.ppt Assessment Report for QM Edgegayehu.ppt Summary of ADR reportsTenaw28Jan2008.doc Draft report on the assessment of health care providers.doc ADR SummaryEthiopia Feb2008.pdf Literature Review on Human and Veterinary Infections and AMR.pdf Literature Review on Human and Veterinary Infections and (1).pdf Course Content on AMR Review HCPs Training Draft Final2.pdf 7. TOOL 7 Prescription ReviewTenawComment latest2.pdf 6. GeneralMedicalInpatient Report2.pdf 5. Report TOOL 5 Surgical ProphylaxisTenawComment latest2T.pdf 56 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 4. Clients Exit Interview Final Draft.pdf 3. Dispensing Practices AsssementAMR Draft ReportTenaw.pdf 2. PrescribingPractices AsssementAMR Draft Report.pdf 1. Health Facility AMR Assessment Report.pdf TBHIVPhaseTwoReport DraftR Nov2006.pdf TBHIVPhase one ReportDraft Nov 2006.pdf TB-HIV-Ethiopia-Findings.pdf TB_HIV_collaboration_ Summary Report.pdf Update on MIS-Oct09-2005.doc Transfer and Use of Equipments Furniture.doc TOT Manual.doc SOP for Request Transfer of Equipments.pdf PMTCT Products_management_SOP.doc Ethiopia ARTguide.pdf Staff list.xls Training-DTC Summary Report2 (2) Adama 14 Jan 2008.doc Training - DTC Minilik hospital.doc Summary report RDUT to Antenane.doc TOT Manual.doc Summary report on DTC Jimma.doc SOP Training.doc Summary of ADR Guideline review workshop.doc Summary Report of 3rd Year deployment and Pre-service ART DT.doc Orientation on Fixed Dose Combinations Report.pdf Progress report on GCPP.doc Jimma University ART report.doc GCPP third round Bahir Dar summary1.doc GCPP second round summary to be sent.doc GCPP third round Bahir Dar summary.doc GCPP Fourth round Jimma summary.doc DTC-Summary report.doc GCPP 3rd round summary (Dire dawa).doc DIC second round summary Hawassa to be sent.doc Workplan FY06.Draft Nov05.doc DIP (08-09).doc DIP (07-08)-revised.doc COP08 Work Plan.doc COP07 Workplan Final (January 2009 version).doc RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 57 SCMS DOCUMENTS HCSS_MP_Vol2-and Costing.zip HCSS MP Vol 1 Final.doc HCSS MP Summary final.doc Pharmaceuticals Supply core process.doc SCMS-ET Achievements - Dec 08.doc Workplan detailed 6 months-2 years PFSA procurement.doc Volume II of II Procurement manual outline_February08.doc Volume I of II PLMP Procurement Implementation.doc Technical Assistance for PSA procurement.doc STTA-Completed - Planned.xls SCMS Ethiopia-Review of Pharmid Distribution Capacity_A4.pdf SCMS Ethiopia Lab Logistics Design Report_A4.pdf Management of ARV Drugs SOP Manual.pdf SCMS Ethiopia ACTS Assessment Report - Final.pdf Ethiopia Technical report -Francois Burger-Recommendations.doc Ethiopia Technical Report - 10-07.doc Ethiopia HACTS Design Report Final.pdf Ethiopia ARV Supply Chain Trainer's Manual.pdf Ethiopia ARV Supply Chain Participant's Manual.pdf ET Quantification Report-October 2007.pdf ET Quantification Report-June2007.pdf ARV Supply Chain Design.pdf SCMS ARV Drugs Annual Report.FY07.doc SCMS ARV Drugs Annual Report FY08.DOC Ethiopia PEPFAR-USAID Report-November 2008.doc Ethiopia PEPFAR-USAID Report-January 2009.doc Ethiopia PEPFAR-USAID Report-December 2008.doc SCMS ARV drugs-January2008.doc SCMS ARV drugs-January2007.final.doc SCMS-SAR-April -September 2008.doc SCMS SAR-October 06 - March 07.doc SCMS SAR- October 07-March 08.doc SCMSWorkplan 2006_v.092707.doc SCMS-COP07 Activity Sheets.pdf SCMS-Ethiopia Workplan FY09-12-09-08.doc Public COP08-SCMSpluspartners.pdf ET SCMS COP08 Annexes A-C Workplan and PMP_02-2009.xls SCMS-Ethiopia Workplan FY09-12-09-08.doc Public COP08-SCMSpluspartners.pdf ET SCMS COP08 Annexes A-C Workplan and PMP_02-2009.xls 58 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 59 APPENDIX C. PERSONS CONTACTED List of Persons Interviewed Name Position Institution James Browder HIV/AIDS Officer USAID/Ethiopia Sophia Brewer Evaluation Coordinator for Health and HIV/AIDS Programs USAID/Ethiopia Rolf Bohlin Chief Supply Officer/Logistics UNICEF Abel Kuiper Logistics Specialist/ Health UNICEF Jan Debyser Logistics Specialist/ Health Center and Health Post Kit Distribution UNICEF Yigeremu Abebe (Dr.) Country Director Clinton Foundation John S. Kim Deputy Country Director Clinton Foundation Erin Koehler Manager, Procurement and Supply Chain Management Clinton Foundation Marisa Traniello Program Manager, Procurement and Supply Chain Management Clinton Foundation Assefa Gashaw Deputy Program Manager, Procurement and Supply Chain Clinton Foundation Stefano Ellero Consultant World Bank Andrew Piller Country Director DKT Ethiopia Negussu Mekonnen (Dr.) Chief of Party MSH/SPS Laike Gebreselassie Deputy Director, Technical Operations MSH/SPS Gail Amare Director of Office Operations MSH/SPS Workineh Getahun PMA and DTC Coordinator MSH/SPS Gultineh Kebede MIS Team leader MSH/SPS Antenane Kotra M&E Specialist MSH/SPS Hailu Tadeg Joint Projects Coordinator MSH/SPS Tenaw Andulaem RDU Team Leader MSH/SPS Yosef Wakwoya PMA Training MSH/SPS Edgegayehu Hailu PMA PPI MSH/SPS Yedilken Kebede PMA Training MSH/SPS Getachew Ayalew Facility Improvement Coordinator MSH/SPS Tesfaye Erega Regional Pharmaceutical Management Associate, Central Oromia MSH/SPS Yilema Desta (Dr.) Regional Pharmaceutical Management Associate, Addis Ababa MSH/SPS 60 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION List of Persons Interviewed Name Position Institution Mihret Tamir Regional Pharmaceutical Management Associate, Addis Ababa MSH/SPS Dinkneh Bikila Regional Pharmaceutical Management Associate, Western Oromia MSH/SPS Tesfaye Godana Regional Pharmaceutical Management Associate, Eastern Amhara MSH/SPS Mulugeta Asfaw Regional Pharmaceutical Management Associate, Southern Regions MSH/SPS Bannet Ndyanabangi (Dr.) Chief of Party, HIV Care and Support Project MSH Hany Abdallah Resident Advisor SCMS Mike Healy PFSA Technical Advisor SCMS John Vivalo Deputy, Procurement and Quantification SCMS Gashaw Shiferaw HACTS Coordinator SCMS Alemayehu Nigatu Lab Advisor SCMS Dessalegne Tesfaye Commodity Security Advisor SCMS Feseha Tassew M&E Advisor (FHAPCO) SCMS Shimelis Endailalu Distribution Advisor SCMS Andualem Mohammed Quantification and Supply Planning Advisor SCMS Abyu Faris Regional Logistics Associate, Central Oromia SCMS Huda Mohammed Distribution Officer, Adama Hub SCMS Gamachis Galache Regional Logistics Associate, Western Oromia SCMS Fikreselassie Alemu Regional Logistics Associate, Southern Regions SCMS Demisachew Lemma Distribution Officer, Gulele Hub SCMS Mathewos Felke Project Engineer, Hubs SCMS Tariku Mohammed Regional Logistics Associate, Amhara Region SCMS Belete Argaw Regional Lab Associate SCMS Jeff Sanderson Country Director DELIVER/JSI Ayalkebet Shiferaw Health Management Information Officer DELIVER/JSI Solomon Zewdie (Dr.) Country Director JHU Ayele Zewde Director for Clinical Services ICAP Kathleen Poer Country Director Abt Associates Gashaw Mengistu Coordinator AIDS Resource Center Alfred De’Laney Team Leader, Senior Procurement Specialist PBS Project Dawit Dikasso Deputy Director DACA Mengistab Woldearegay Deputy Director DACA RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION 61 List of Persons Interviewed Name Position Institution Daniel Yami Head, Jimma DACA Branch Office DACA Haymanot Assefa President EPA Melese Alemu Executive Manager EPA Tesfaye Seifu Project Coordinator EPA Wondwossen Ayele Deputy Director General PFSA Asnake Gemetchu Adama Hub Manager PFSA Kassahun Muluneh Head, Kalite Warehouse PFSA Mulugeta Getaneh Bahir Dar Hub PFSA Wondimnew Abate Bahir Dar Hub PFSA Kebede Nigatu Manager, Jimma Hub PFSA Gebre-Selassie Tegegn Manager, Hawassa Hub PFSA Fetiya Lalu ART Store Head, Hawassa Hub PFSA Berhanu Feiysa Head, Human Resource Directorate FMOH Tsehaynesh Messele (Dr.) Director General EHNRI Gudeta Tibesso Head, Regional Capacity Building Directorate EHNRI Meskele Lera Deputy Director FHAPCO Getaneh Derseh Yismaw Head, Amhara Regional HAPCO Amhara State Asrat Genet (Dr.) Head, Amhara Regional Health Bureau Amhara State Negash Tesfu Care and Support Officer Amhara RHB Kassa Hailu (Dr.) Deputy Director, Oromia Regional Health Bureau Oromia HB Zenebetch Yadete (Dr.) Head, Oromia Regional HAPCO Oromia HB Getachew Aga Head, Oromia Regional Lab Oromia HB Henok Wogderes Clinical Case Team Leader Addis Ababa Regional Lab Demke Ashagre Technician, Regional Lab SNNP RHB Yoseph Getachew Manager, Regional Warehouse SNNP RHB Yehun Telila Head, RHB Pharmacy SNNP RHB Fekade Mamo Distribution Head, RHB Warehouse SNNP RHB Head, RHB Warehouse Head, RHB Warehouse Amhara RHB Alefe Mersa Amhara Regional Data Manager Amhara RHB Dagem Assefa (Dr.) Medical Director Adama Hospital Yonas Kemau Store Head Adama Hospital 62 RPM+/SPS AND SCMS IN ETHIOPIA: AN EVALUATION List of Persons Interviewed Name Position Organization Hailu Aberra (Dr.) Medical Director Bishoftu Hospital Gezahegn Endale Chief Pharmacist Bishoftu Hospital Hanna Godana ART Pharmacist Bishoftu Hospital Munira Adem Store Head, Pharmacist Bishoftu Hospital Getahun Paulos Head, Hospital Pharmacy Jimma Hospital Rahel Yisak ART Pharmacist Jimma Hospital Fedial Shifa Store Head Jimma Hospital Ayalew Adnew Chief Pharmacist ALERT Hospital Haymanot Kifle ART Pharmacist ALERT Hospital Yonas Asheber ART Store Head, Pharmacist ALERT Hospital Kidist Mulugeta Pharmacy Store Head ALERT Hospital Wondafrash Teklemariam Store Head, Druggist Zewditu Hospital Yehun Geta Store Head, Druggist Felege Hiwot Hospital Mohammed Hussein Deputy Head Agaro Health Center Senait Adnew Store Head, Druggist Agaro Health Center Ali Mohammed VCT Agaro Health Center Azeb Kinfe Head Pharmacy Unit, Druggist Kolfe Health Center Muluken Yilak Head Bahir Dar Health Center Chala Gudina Head Pharmacy Unit, Druggist Bushulo Health Center For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com