EVALUATION USAID/Ethiopia: Private Health Sector Program Mid-term Evaluation NOVEMBER 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Michael Thomas, Michael Dejene, Fikreab Kebede, and Carina Stover through the GH Tech Bridge Project. Cover Photo by Carina Stover EVALUATION USAID/Ethiopia: Private Health Sector Program Mid-term Evaluation NOVEMBER 2012 Global Health Technical Assistance Bridge II Project (GH Tech) USAID Contract No. AID￾OAA-C-12-00027 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-02-010) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge II Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00027. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION i CONTENTS ACRONYMS ....................................................................................................................................... iii EXECUTIVE SUMMARY ................................................................................................................... v I. INTRODUCTION ..................................................................................................................... 1 Purpose of the Evaluation ......................................................................................................... 1 Scope of Work and Methodology .......................................................................................... 2 II. BACKGROUND ........................................................................................................................ 5 USAID/Ethiopia Support to Private Health Care Providers ............................................. 5 III. FINDINGS AND DISCUSSION ............................................................................................. 7 PHSP’s Predecessor: The PSP-E Final Evaluation ................................................................ 7 The Policy Enviroment Supports Private Sector Partnership ........................................... 8 Sustainability of Public Sector Support for Private Sector Health Care ......................12 Increased Access to Services Through the Private Sector .............................................16 Sustaining Quality Assurance in the Private Sector Services ..........................................23 Demand for Quality Private Health Sector Services ........................................................25 IV. MANAGEMENT AND PERSONNEL .................................................................................. 29 Monitoring and Evaluation ......................................................................................................29 PHSP and USAID Staff Management of PHSP ....................................................................30 Annual Workplans ...................................................................................................................30 Promotion of Women on PHSP Workforce .....................................................................30 Recommendations for Management and Personnel .........................................................31 V. LESSONS LEARNED ............................................................................................................... 33 Supply of Drugs .........................................................................................................................33 Training Programs ....................................................................................................................33 Referrals from Private Sector Clinics to Public Hospitals ...............................................33 Marketing Innovations .............................................................................................................33 VI. RECOMMENDATIONS ......................................................................................................... 35 Policy ...........................................................................................................................................35 Sustainability ..............................................................................................................................35 Increasing Access ......................................................................................................................35 Improving Quality .....................................................................................................................36 Demand Creation .....................................................................................................................36 Management and Personnel ...................................................................................................37 ii USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION ANNEXES Annex A. Scope Of Work .............................................................................................................. 39 Annex B. Persons Interviewed and Sites Visited ....................................................................... 51 Annex C. References ....................................................................................................................... 55 Annex D. Interview Questions ...................................................................................................... 65 Annex E. National and Regional Indicators by Result Through Year Three ....................... 69 FIGURES Figure 1: PHSP Sites Geographical Coverage ............................................................................... 6 Figure 2: Systematic Program Implementation Approach ....................................................... 19 Figure 3: Messages about Malaria Prevention and Treatment ................................................ 27 TABLES Table 1: Number and Location of Persons Interviewed ............................................................ 3 Table 2: Services Available by Region as of the End of Year 3 ............................................... 17 Table 3: Number of Private Hospitals and Clinics with TB/HCT Services .......................... 20 Table 4: Number of Private Hospitals and Clinics with Standard ART Services ................ 20 Table 5: Number of Malaria Patients Diagnosed and Treated in Private Clinics................ 22 Table 6: Number of Private Hospitals and Clinics with Standard TB-CT and FP/STI Services ................................................................................................................... 22 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION iii ACRONYMS Abt Abt Associates Inc. ACT Artemisinin Combination Therapy AIDS Acquired Immune Deficiency Syndrome AOTR Agreement Officer Technical Representative ART Anti-retroviral therapy ARV Anti-retroviral CME Continuing medical education CT Counseling and testing DACA Drug Administration and Control Authority DOTS Directly observed treatment short-course DQA Data quality assessment EHNRI Ethiopian Health and Nutrition Research Institute EMLA Ethiopian Medical Laboratory Association EQA External quality assessment ESA Ethiopian Standards Agency FHAPCO Federal HIV/AIDS Prevention and Control Office FMHACA Food, Medicine, and Health Care Administration and Control Authority FMOH Federal Ministry of Health FP Family planning GH Tech Global Health Technical Assistance Project GoE Government of Ethiopia HCT HIV counseling and testing HIV Human Immunodeficiency Virus HPHEIA Higher Public Health Education Institutions Association IPC Interpersonal communication IQC Internal quality control IUD Intrauterine device MAPPP-E Medical Association of Physicians in Private Practice-Ethiopia MARP Most at-risk populations MOU Memorandum of understanding NGO Non-governmental organization PEPFAR President’s Emergency Plan For AIDS Relief PFSA Pharmaceutical Funds and Supply Agency PHSP Private Health Sector Program PMTCT Prevention of mother-to-child transmission iv USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PPM Public-private mix PPP Public private partnership PSP-E Private Sector Program-Ethiopia QA Quality assurance RHB Regional Health Bureau SBMR Standards-based management and recognition SNNP Southern Nations and Nationalities Peoples STIs Sexually transmitted infections TA Technical assistance TB Tuberculosis USAID United States Agency for International Development WHO World Health Organization USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION v EXECUTIVE SUMMARY Ethiopia is Africa’s second most populous country and predominantly rural. There is high incidence of communicable diseases including tuberculosis (TB), HIV/AIDS and malaria. While the government is committed to universal access to health care, there are severe restrictions, including limited availability of trained health care personnel, poorly developed health systems and unreliable supply chains. To address these needs, the Government of Ethiopia (GoE) is looking to its private health care sector as an untapped resource. However, to use all available resources, Ethiopia has had to address policies that would enable the private sector to be fully utilized. This meant giving the private sector access to drug supplies, equipment and training programs. To support the GoE in formulating and strengthening policies to include the private health care sector, USAID has provided assistance through two successive programs: the Private Sector Project-Ethiopia (2004–2010) and the current Private Heath Sector Program (PHSP). The objective has been to effectively partner with private health providers to deliver public health services while improving the quality and affordability of these services. The expected results of PSHP are to:  Ensure a supportive and sustainable policy environment for the private health sector.  Enhance both geographic and financial access to packages of essential health services through the private sector.  Sustain improvements in the quality of these services.  Increase demand for quality services by informed, proactive consumer groups. To support these expected results, PHSP established a central office in Addis Ababa and three regional offices (Bahir Dar, Dire Dawa, Southern Nations and Nationalities Peoples [SNNP]) and programs in eight regions. Within all the regions, the implementation strategy was to provide an integrated package of services as follows:  Diagnosis and treatment of TB through Public-private mix, directly observed treatment short￾course (PPM-DOTS).  Comprehensive HIV care (HCT, prevention of mother-to-child-transmission [PMTCT], and antiretroviral therapy [ART]).  Family planning (FP) and sexually transmitted infection (STI) services.  Diagnosis and treatment of malaria. Also there were four cross-cutting programs: policy, laboratories, pharmacy and quality management. Initial challenges included health policies that were not inclusive of the private health sector; guidelines that were outdated and often delayed; and decisions that were often inconsistent due to a lack of ownership in the public sector. Exacerbating these problems was resistance at some public sector levels to include the private sector in meeting the health care needs of Ethiopian families. Despite these challenges, PHSP has had a number of successes including: vi USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION  New guidelines and subsequent increased access to PPM-DOTs.  A strengthened regulatory system that includes the private sector.  Initial progress in encouraging and supporting local associations to take control of the future of the private health care.  Improved regulation and accreditation of private laboratories.  Development of a five-year national TB control program for the private sector and broad acceptance, especially at the regional level.  Increased regional demand for PHSP support beyond the original scope of the project.  A greatly improved referral program for diagnosis and treatment of HIV/AIDS cases.  Development of productive relationships with the public health sector at both the federal and regional levels. A continuing problem with respect to provision of public health services—such as TB, HIV, FP and malaria treatment—within the private health sector clinics is the absence of a standard policy and guidelines on the supply of essential commodities to private facilities providing public health services. Lack of consistent supply of commodities to the private facilities is part of the reason PHSP was unable to support the roll out of integrated HIV, TB, FP and STI services. Additionally, ongoing programs such as TB services experienced either disruptions of drug supplies or receipt of drugs close to expiring. Drug and other essential supplies are the responsibility of the Pharmaceutical Funds and Supply Agency (PFSA) and PHSP continues to monitor drug flow to the private clinics. Working with the PFSA, the PHSP has intervened at times to remind those at the regional level that private clinics and hospitals are to be included in the drug supply for programs on TB-DOTs, HIV counseling and testing (HCT), malaria, FP and STIs. Overall, the program has experienced significant delays in the integration of services in each of the regions. The most serious of these delays has been within the FP component of the project. This delay was caused in part by the sudden departure of FP technical staff within PHSP, resulting in a delay of nearly two years in the reintroduction of FP to targeted private sector facilities. While some Regional Health Bureaus (RHBs) have requested that additional sites in their region be included, the gap in establishment of planned services in existing clinics suggests that geographic expansion should be delayed. No plan for sustainability (or exit plan within PHSP’s Systematic Program Implementation Approach) has been articulated after PHSP is phased out. Some elements of sustainability such as capacity building by transfer of staff to the Federal Ministry of Health (FMOH) and joint supportive supervisions at regional level have been well established. However, other areas such as support for training of private sector health care personnel, the provision of needed drugs, and empowering the FMOH and the RHBs in ownership are not clear. Both the FMOH and the RHBs value the contribution of PHSP, but there is no person or unit in either that has a private health sector focus. In discussions with the Director of Resource Mobilization at the FMOH, this problem as identified as a gap. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION vii PHSP is well positioned to influence the development of policies and strategies concerning the private health sector. Key recommendations include the following:  As a first step, PHSP should articulate a long-term sustainability strategy to support the private sector. This strategy should detail all the components of private health sector service delivery and how each aspect of sustainability can be achieved, through which means and by whom.  The PHSP should organize national and regional forums in Ethiopia to further enable a sustained public-private health sector partnership. Creation of a comprehensive and accessible information base on private sector dynamics in Ethiopia, both from the demand and supply side, would support this effort.  In cooperation with FMHACA, PHSP should assist the RHB in organizing regional workshops to provide feedback on the approved licensing and accreditation standards.  Support for national and regional associations of private sector providers and professionals working in the private sector should be continued and expanded to all the regions where PHSP is working. PHSP should also complete additional needs assessments to determine how these associations can make meaningful contributions to the health needs of the country—and how they can be sustainable through membership fees, taxation of private health facilities and other potential revenue sources.  Encourage the establishment of private sector focal persons and teams within the RHBs.  In collaboration with the RHBs, PHSP should prepare guidelines to support and work closely with private sector clinics to ensure that they are prepared to take over responsibility for all essential health services, beginning with TB-DOTS, so that access to these services continues.  Disruption of the flow of drug supplies to private sector hospitals and clinics continues to be a problem. During selected training courses, effort should be made to assist clinics to predict their drug needs (e.g., ARVs, FP, TB and malaria drugs) and effectively request and secure drugs well in advance of any potential stock-outs.  Given the current status of introducing the full range of standard services in the existing PHSP supported clinical network, no geographic expansion with PHSP support is recommended.  Focus on integration of all essential health services. viii USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 1 I. INTRODUCTION PURPOSE OF THE EVALUATION Evaluators were expected to perform the following asks: 1. Assess the program’s process toward achieving set objectives and anticipated results. Specific questions include: – What has been PHSP’s progress to date in terms of achieving planned results and performance indicators (as provided in the program’s performance monitoring plan)? – What strategies did the program adopt in order to achieve the four major results? – What are the main reasons for exceeding or not meeting expected results? What were the major policy challenges (consider GoE, USAID and PEPFAR policies) and opportunities with respect to achieving program objectives and targets? – How well has the partner monitored and evaluated the outputs and outcomes of the program and the extent to which the results are achieved? How can the monitoring and evaluation (M&E) system be improved? – How well has PHSP incorporated lessons learned from the predecessor Private Sector Program (PSP) into the current program? – Have there been any management (consider both PHSP and USAID) or staffing issues or challenges during the program and, if so, how have they been identified, communicated, addressed or resolved? – What are the key lessons learned from the PHSP? What have been the strengths, weaknesses and best practices with respect to PHSP implementation, M&E, capacity building and the program’s relationships with the GoE and other stakeholders? – What arrangements have been made to ensure sustainability of the program’s results and impacts? 2. Make actionable-recommendations for the direction and management of PHSP in its remaining two years of implementation. Specific questions include: – What do government officials (regional and federal levels) and other stakeholders perceive as priorities and opportunities for the private sector? – Are there missed opportunities, gaps and/or potentially effective private health sector models and approaches that PHSP or a future private sector program should consider? The purpose of this evaluation is to obtain an independent assessment of the performance of the Private Health Sector Program (PHSP) and to learn from the program’s accomplishments and challenges to date in order to guide USAID/Ethiopia and PHSP staff with regard to the direction and management of the program in its final two years. 2 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION – What, if any, modifications should be made to program targets in the remaining two years of implementation? – What are the opportunities and challenges to improve on and/or initiate new and innovative strategies in order to achieve key results? – What, if any, modifications should be made to program design, management, and staffing in the remaining two years of implementation? SCOPE OF WORK AND METHODOLOGY The Global Health Technical Assistance Project (GH Tech) conducted this mid-term evaluation of PHSP from October to December 2012 at the request of USAID/Ethiopia (see Annex A for the full Scope of Work). In total, four weeks were spent in the field, conducting interviews in Ethiopia. Team Composition The evaluation team was composed of two international and two Ethiopian health consultants as well as a local logistics assistant. GH Tech provided the evaluation team. Basic Approach The following is the sequence of key steps in conducting the evaluation.  Background literature review.  Team planning meetings to outline the report and evaluation methodology and to define team responsibilities.  Meetings with USAID/Ethiopia project management team.  Meetings with Abt’s Private Health Sector Program team in Addis Ababa and Washington DC.  Interviews with the Federal Ministry of Health (FMOH), Regional Health Bureaus (RHB), Medical Association of Physicians in Private Practice-Ethiopia (MAPPP-E), Food, Medicine and Health Care Administration and Control (FMHACA), Pharmaceutical Fund and Supply Agency (PFSA) and other federal agencies.  Interviews with other donors and development programs supporting the private health sector.  Field visits to clinical sites.  Report writing.  Findings and recommendations presentation to USAID/Ethiopia and Abt.  Drafting of the report for review by USAID/Ethiopia.  Finalization of report. The team met with officials and regional coordinators of PHSP both at the beginning of the assignment and after compiling the fieldwork. The team synthesized the collected data during a two-day discussion where themes and sub-themes emerged under the four programmatic areas USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 3 and conclusions and recommendations were formulated. By the end of the fieldwork, the evaluation team debriefed with both USAID/Ethiopia and PHSP. The assessment was carried out in Addis Ababa, Amhara, Dire Dawa, Harari, Oromia, SNNPR and Tigray regions, which covers all PHSP operational regions. RHB staff and heads of private health facilities and regional laboratories, representatives of associations of private sector providers and officials of the Food, Medicine and Health Care Administration (FMHACA) acted as primary informants for the evaluation as they are key stakeholders (see Annex B for a list of persons interviewed and sites visited). Interviews were carried out using a checklist of questions for each type of interviewee (see Annex C for interview questions). Table 1: Number and Location of Persons Interviewed Addis Regional Facility level Total Interviewed USAID 2 2 Abt/PHSP 12 15 27 MOH 1 15 15 EHNRI 1 1 FMHACA 1 1 FHAPCO 1 1 PFSA 1 1 Private Providers: Hospital 21 21 Private Providers: Higher-level Clinic 19 19 Private Providers: Lower-level Clinic 8 8 Regional Laboratory 1 2 3 NGOs 5 2 6 Medical Associations 3 1 4 Clients 2 2 Total Interviewed 27 35 50 112 Document Review and Data Analysis The team conducted a literature review prior to and throughout the course of the assessment. Key documents include a variety of background reports on the health care landscape in Ethiopia as well as documents from Abt’s PHSP and USAID. Constraints and Gaps to the Evaluation Process  Many critical documents, such as the original Cooperative Agreement and any follow-on modification or the draft Year 4 workplan, were not available to the evaluation team until well into the field visit stage of research. 4 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION  The lack of clarity on which result indicators are being measured by Abt and used for monitoring purposes to USAID made it difficult for the evaluation team to assess the PHSP using quantitative analyses.  Many of the successes of PHSP are not quantifiable. This includes the countless hours building relationships as well as changing attitudes toward the private sector by the public sector at both the federal and regional levels.  Due to the unavailability of key persons, critical interviews with officials at the FMH were not completed until mid-December. Thus, the team did not incorporate this information until the end of the time for submission of the second draft of the report in mid-December.  The evaluation team was unable to evaluate the financial accountability of PHSP. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 5 II. BACKGROUND Ethiopia is Africa’s second most populous country and predominantly rural (82%). There is a high prevalence of communicable diseases including tuberculosis (TB), malaria and HIV/AIDS. Uptake of important programs such as prevention of mother-to-child transmission (PMTCT) has been slow. These challenges are exacerbated by high population growth, which puts tremendous pressure on all social services, especially those in the health sector. In response, the GoE has expanded primary health care services to reach the rural population and enlisted the private health sector to help meet this need. The private health sector is growing and currently utilized by a sizeable portion of the population. At the same time, there are significant challenges to expanding private health sector services in the treatment of communicable diseases such as TB, malaria and HIV/AIDS. For private health care professionals in private practice, the development and implementation of training programs in the prevention and diagnosis of these diseases require significant resources and time. Importantly, the limited availability of drugs at private facilities presents significant barriers to fully utilizing the private health sector. And while the GoE says it is committed to engaging the private sector in order to increase access to health services—including issuing major health strategy documents that mention the importance of working with the private health sector—an enabling environment has not been fully realized. In effect, there are few clear policies or guidelines that foster real private sector involvement and ensure a private public partnership (PPP) in health care delivery. USAID/ETHIOPIA SUPPORT TO PRIVATE HEALTH CARE PROVIDERS In response to these challenges, USAID/Ethiopia is working with Abt Associates Inc. (Abt) through the Private Health Sector Program (PHSP), a follow-on to the Private Sector Program– Ethiopia (PSP-E). PHSP, funded by the President’ Emergency Plan for AIDS Relief (PEPFAR) through the United State Agency for International Development (USAID) and implemented by Abt, works to increase demand and provision of high-quality public health services in the private sector by building sustainable PPPs. The goal of PHSP is to enable the Federal Ministry of Health (FMOH) and Regional Health Bureaus (RHBs) to effectively partner with private health providers to deliver public health services, while improving the quality and affordability of these services. The program is designed to: 3. Facilitate a supportive policy environment for the private health sector. 4. Enhance both geographic and financial access to packages of essential health services through the private sector. 5. Build toward sustainable improvements in the quality of these essential health services. 6. Increase demand for quality services by informed and proactive consumer populations. The PHSP mandate is to work toward strengthening regional stewardship of private health facilities. PHSP’s engagement with the private sector has supported the expansion of high impact public health interventions, namely TB, comprehensive HIV care (CHC), FP and malaria services in five regions (Amhara, Harari, Oromia, Southern Nations, Nationalities and Peoples [SNNP] and Tigray) and two city administrations (Addis Ababa and Dire Dawa) through facility-based and mobile services offered by private sector providers (see Figure 1). Before USAID began its 6 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION support for the private health sector, the availability of HIV and TB services at private clinics was either very limited or non-existent. Figure 1: PHSP Sites Geographical Coverage PHSP continues to scale up key clinical programs to ensure that all targets stated in the cooperative agreement with USAID are reached. As a future activity, the PHSP should begin identifying, consolidating and handing over processes of mature programs to the RHB. This strategy includes preparing the regions to take over programs deemed mature and continue to strengthen systems that will allow a smooth and sustainable transition without sacrificing access and quality of care. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 7 III. FINDINGS AND DISCUSSION PHSP is presently following four expected results as outlined in PHSP Associate Award No, 663- A-0-09-00434-00, along with additional discussion on sustainability.  Supportive (and sustainable) policy environment for the private health sector.  Enhancement of both geographic and financial access to packages of essential health services through the private sector.  Sustained improvements in the quality of these services.  Increased demand for quality services by informed, proactive consumer populations. PHSP’S PREDECESSOR: THE PSP-E FINAL EVALUATION In September 2004, USAID/Ethiopia issued a four-year task order to Abt to support private health sector services in Ethiopia; this project eventually became the Private Sector Project–Ethiopia (PSP-E). The PSP-E task order was extended through September 2010 with a final evaluation completed in July 2009. The PSP-E made a number of recommendations and this report highlights those that are relevant to PHSP. It should be noted that the PSP-E final evaluation did not address two of the three strategies of PSP-E: promotion of social franchising and targeting of social marketing of HIV/AIDS prevention. Addressing these two strategies may have led to further recommendations in these areas. Program Management  The PHSP followed the same implementation strategy as the PSP-E though they added an exit strategy as that of the PSP-E as the final tier of their 14-step Systematic Program Implementation Approach (discussed later in this report). Further discussion on the challenges facing development of this exit strategy is discussed under the Sustainability section of this report.  Both projects emphasized supportive supervision and close mentoring of the private sector providers to achieve desired results.  The PHSP strengthened support to Ethiopian subcontracting mechanisms though this support was selective (e.g., MAPPP-E) and not a major focus of the project.  The PSP-E recommended continued advocacy for an improved private sector policy environment and this was continued by PHSP as an important element.  The recommendation to recruit senior level technical assistance (TA) was not followed after the start-up period of PHSP—apart from short-term visits from Abt headquarters and other technical advisors. The presence of these advisors may have positively affected PHSP’s impact, particularly in the FP component of the project.  It was strongly suggested that the PHSP develop and implement a comprehensive plan for M&E. While there is a strong M&E team within PHSP, results indicators and reporting was not as accessible and transparent as expected. 8 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION  The recommendation to develop a strategy to promote an enabling environment among workplace management is not a priority for the PHSP.  Some modest effort has been made by the PHSP to develop strategies to increase support for preventive and clinical services through affordable health insurance and other schemes. However, this is not a major focus of the project.  Several recommendations were made to address PPM-DOTS and TB/HIV; these are discussed in the Access Result section of this report.  It was recommended that future private support include assistance in the development of implementation guidelines to facilitate initiation of private sector program activities. The implementation guideline prepared by PSP-E for TB-DOTS enabled PHSP to roll out the program without delay. However, PHSP is still working on guidelines for the provision of FP services in the private sector.  As recommended in the PSP-E evaluation, PHSP continued support for Mobile Counseling and Testing (MCT) and outreach with a particular focus on most-at-risk populations (MARPs). As part of this effort, special attention was given to more precisely defining MARPs in the Ethiopian context. The PHSP may have missed an important opportunity to continue reaching out to this population as this is a particular need not met by the public sector. For example, by extending coverage into identified “hot spots” and underserved areas with high demand, as recommended by the PSP-E, PHSP may have enabled private providers to offer services in these locations.  It was suggested that a MCT-focused consortium should be developed among the sub￾contractors, but there is no evidence of project activity in this area.  Of particular note in the PSP-E evaluation report is summary item No. 5 (PSP-E final evaluation p. xvi): “While the PSP-E has made considerable progress in working to establish meaningful and effective linkages with the public sector, the capacity of the GoE to develop and support mechanisms, policies and guidelines to enable Ethiopia to develop and fully profit from the potential contribution of the private sector remains a challenge that should be addressed now and in the future.” This evaluation team strongly recommends that this continue to be a major focus during the remaining two years of the program while also recognizing the considerable effort expended by the PHSP in this area. THE POLICY ENVIROMENT SUPPORTS PRIVATE SECTOR PARTNERSHIP The goal of PHSP is to assist the FMOH and RHBs in their efforts for wider inclusion of the private sector in public health services through PPPs. The primary strategy has been to use advocacy and consensus-building discussions on policy directions and guidelines, as well as to provide TA to the FMOH and other policymakers. A major obstacle to this work is that by its very nature, the public sector is not very interested in the private sector. And at times, there is clear antagonism between the two sectors. This has made the work of the PHSP particularly difficult. At the same time, they have achieved a number of successes and built positive relationships at the federal and regional levels. This is an intangible indicator of success, but one that cannot be underestimated. Still, the Director of the Resource Mobilization Directorate at the FMOH acknowledged the important role of the private sector in addressing the health care USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 9 needs of Ethiopian families. He further acknowledged the role of the PHSP with considerable appreciation. Specifically, PHSP’s advocacy work to demonstrate the value of working with the private sector and how it is done has resulted in increased support for private health sector services within the FMOH and the RHBs. Significant contributions to the strengthening of federal and regional stewardship of the private health sector have been made in the regulation and accreditation areas including more than 10 health care standards for different levels of care. This means that the capacity and willingness to do this work have been significantly enhanced. Policy Framework for Private Sector Participation PHSP continues to offer TA to the FMOH in its effort to develop a national framework for public-private collaboration. In collaboration with the FMOH, a WHO consultant developed a framework in 2010. However, it was deemed unworkable by the FMOH and ultimately rejected. As a result, the development of a framework was on indefinite hold awaiting a request for assistance from the FMOH. The PHSP recently hired a consultant to help develop the framework; it is expected that this activity will be completed in the first quarter of 2013. Additionally, PHSP offered to work with the FMOH to adapt existing program implementation guidelines for ART, FP, malaria and STIs. The TB guidelines were completed in the previous project and are being implemented in the current program. A guideline for ARTs was completed in 2005 and covered existing hospitals in the private sector. In discussions with the Pharmaceutical Funds and Supply Agency (PFSA), it was found that the regulatory requirement that clinics have pharmacists on site was the primary reason for not including higher clinics in the distribution of ARTs. According to Abt, there are two problems with this regulation: employing a pharmacist is expensive and there is a shortage of pharmacists in Ethiopia. The PHSP is currently exploring an alternative approach, utilizing pharmacists in existing and nearby pharmacies to dispense the drugs. PHSP has discussed with FMHACA on how to expand access to antiretroviral (ARV) medications to the private higher clinics; to date, the recommendation to allow a nurse to dispense ARVs at higher clinics has not been resolved. PHSP has made concerted effort to facilitate the provision of ARTs in private clinics. In the meantime, PHSP gives technical support to hospitals providing ART and higher clinics providing pre-ART care as part of comprehensive HIV care and support. To increase access to standard malaria care services, PHSP has worked with both the FMOH and the RHBs to create a supportive policy environment that involves the private health sector. Working with the RHB in the Tigray region, PHSP developed a PPP guideline on malaria care and malaria drugs are now being introduced in some private health facilities. Following this activity, malaria drugs also have been introduced in private clinics in the Oromia region. On the policy side, there are currently no FP implementation guidelines for the private sector. PHSP’s FP efforts have been slowed by personnel changes within the project. In part, this was the result of pay differentials between the project and non-governmental organizations (NGOs), such as Marie Stopes. Second, delays in the supply of FP commodities have been an issue. However, given the commercial availability of FP products, this should not be seen as a major obstacle to program implementation. On the positive side, PHSP has helped to form national 10 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION and regional technical working groups to advocate greater involvement of the private health sector in delivering quality family planning services. New Training Modalities for the Private Sector Together with the FMOH, FHAPCO and RHBs, the PHSP explored strategies for shortening private sector capacity building in order to reduce time away from the facility. The duration of training courses was decreased without compromising content. For example, the number of days for the Directly Observed Treatment Short-Course (TB-DOTS) was decreased from nine to six. In addition, they assessed standardizing the pre-service curriculum in private higher education institutions to minimize the need for in-service training. Assessment results are still in draft and once completed will be considered in future PHSP workplans. However, it should be noted that in discussions with Abt, there is still some concern that training programs (e.g., FP training) are still too long and time away from the clinic is a deterrent to participation from private sector personnel. PHSP’s support for Private Higher Learning Institutions PHSP has conducted an assessment of private higher learning institutions (HLIs); dissemination of assessment results are still awaiting USAID approval and should be carried out in collaboration of Federal Ministry of Education. Further support for private HLIs will depend on the outcome and recommendations forwarded during the dissemination workshop. Improving Access to Credit for Private Providers Program With funds guaranteed by USAID, PHSP was to collaborate with lending agencies participating in the Development Credit Authority (DCA) in order to identify new opportunities for financing private sector facilities. The DCA was established in a direct relationship between USAID and the banks. According to USAID/Ethiopia, PHSP was initially tasked with providing TA (e.g., business skills and financial management training) to potential borrowers and has continued to do this. Considerable resources were expended to organize this program, with PHSP ready to launch the program a year ago. These activities included conducting a series of training of the trainers (TOT) to deliver programs on business skills and financial management for professionals in the private sector and private banks. Unfortunately, the banks that had agreed to participate withdrew at the last minute saying that their financial commitments elsewhere prevented them from proceeding. According to the PHSP team, USAID shifted the responsibility of the program to another USAID-funded program, Strengthening Health Outcomes through the Private Sector (SHOPS). PHSP will continue to provide logistical support as needed or requested by SHOPS and may still have an important role to play in referring private providers to banks and improving their business skills. It was also reported that PHSP has conducted a TOT and a series of trainings on business skills and financial management for those in the private sector and private banks. Abyssina and Nib Banks were selected to provide credit for the private providers, and they signed an memorandum of understanding (MOU) with USAID. USAID was supposed to provide USD $10 million of collateral to the banks that they will lend to the private sector. The program was launched by USAID on November 2010. Licensing, Accreditation and Standards as Tools to Promote Quality PHSP has contributed to establishing standards of licensure and accreditation for the private sector through a broad participatory process—with attention to building strong systems and USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 11 organizational capacity. As a first step, PHSP helped design and pilot clinical and management quality standards to support selected private sector clinics operating within public health clinics. These clinics offer private sector services at the primary care level. FMHACA has asked for further support in the form of a medical doctor to help with implementation of the national licensing and accreditation strategy/plan. In addition, PHSP supported the FMHACA in developing 39 regulatory standards that have been approved by the Ethiopian Standards Agency. While standards have been approved, they have yet to be printed and formally disseminated. Next steps are to hold workshops with the various levels of service providers in the public and private sectors to obtain feedback. The plan is to make changes to the standards as needed through an amendment process rather than rewriting the standards themselves. A critical issue under discussion is how long to allow existing facilities to operate outside of these standards before they lose their licenses. For the private sector, key concerns are floor space and human resource requirements, which are both limited and difficult to obtain. It is planned that all new facilities must comply with the standards from the start in order to become licensed. Strengthening Capacity among Private Sector Representative Bodies Through small grants, PHSP was to assist both the Higher Public Health Education Institutions Association (HPHEIA) and the Medical Association of Physicians in Private Practice Ethiopia (MAPPP-E). Though negotiations with HPHEIA have recently been reopened, early discussions failed to reach agreement on programs to strengthen the private sector. For MAPPP-E, small grants have enabled them to strengthen their organizations by seeking other funding sources (e.g., through the provision of small grants or contracts with agents for pharmaceutical companies in Ethiopia). An important goal has been to provide TA for strategic and financial planning to support independence and sustainability. Additionally, PHSP engaged MAPPP-E to deliver programs covering topics such as the strategic planning process, fundraising, financial planning and business management. A list of recent programs delivered by MAPPP-E under the sponsorship of other organizations includes programs on contraceptive updates, long acting contraceptives and management of STIs. MAPPP-E is one of the few non-governmental groups available to provide continuing medical education (CME) programs to the private sector. They have provided training using FMOH training materials and guidelines. The structure, though not the full capability, is in place to provide medical education programs should the public sector choose not to include the private sector once PHSP is phased out. In addition to these activities, the MAPPP-E advocates at the federal level through such activities as developing the FMOH national guidelines and, at the request of FMHACA, drafting a code of ethics and scope of practice. Recommendations for Policy  PHSP is an expert in understanding issues that affect the private health sector. Periodic briefings and/or briefing materials should be made available to USAID and other interested groups. These materials should clearly outline the contributions of the private sector to the overall provision of health services in Ethiopia so that they, in turn, can effectively advocate for the private sector when speaking with the FMOH and other partners.  PHSP is well positioned to influence the development of policies and strategies concerning the private health sector. To this end, it is recommended that they organize and strengthen national and regional forums in Ethiopia designed to further an enabling environment for 12 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION sustaining a PPP in the health sector. Creating a comprehensive database on private health sector dynamics in Ethiopia—both from the demand and supply side—would support this effort.  FMHACA has requested a medical doctor be provided to them and paid for by PHSP. Placement of this doctor by PHSP would provide a key contact within FMHACA, as well as a PHSP-supported advocate for the private health sector. This could result in better communication between PHSP and FMHACA and in better representation of the private sector in issues such as the dissemination and review of the new facility licensing standards. The PHSP should be closely involved in the hiring of this person and ensure that, administratively, they report to PHSP.  In cooperation with FMHACA, PHSP should assist the RHB in their facilitation of regional workshops to provide feedback on the approved licensing and accreditation standards.  Some TA should be provided to MAPPP-E to develop linkages to private sector companies. For example, connecting to pharmaceutical company representatives that could provide speakers/content for medical education programs related to their products (such as FP methods) marketed in Ethiopia. Additionally, they need help in preparing their strategic plan and resource strategy in order to identify other sources of funds to support their activities. SUSTAINABILITY OF PUBLIC SECTOR SUPPORT FOR PRIVATE SECTOR HEALTH CARE To sustain quality health services provided by the Ethiopian private sector requires at least the following:  Political will to accept, acknowledge and support the contribution of the private sector to provide health services to the public.  Policy framework that provides regulation and licensing of private health care providers and the facilities they work in.  Support for the provision of supplies that are needed in the treatment areas of HIV/AIDS, TB and malaria.  Financial resources from private, public or development partners.  Human resource availability.  A PPP framework that ensures supportive supervision and mentoring of private providers.  Human resource knowledge, attitudes and practices regarding promotion and provision of health services.  Infrastructure.  Knowledge of and demand for health services in the private sector by potential and existing clients.  Replicable systems in place for expanding support to existing and potential private providers. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 13 Contributions of PHSP to Sustainability to Date Building on the work of PSP-E, the PHSP has continued to develop relationships with government and private health care professionals at both the federal and regional levels. A major challenge to increasing access to health care in Ethiopia has been the lack of urgency among public policymakers and decisionmakers to work with the private sector. However, at the implementation level in the field, members of the PHSP team have made a significant contribution to the opening of doors to facilitate referrals to public hospitals for treatment. And though supplies of drugs continue to be an obstacle to program expansion in the private sector, significant gains have been made in securing these supplies to the private sector. Importantly, PHSP has shown that the private sector is a valuable ally in providing health care, which has resulted in public sector support in providing much needed commodities, access to training and supportive supervision. One benefit of PHSP’s work has been to sensitize professionals in the public sector to open discussions with a broad spectrum of health care professionals. The critical question is how this effort can continue once USAID’s financial and personnel support is withdrawn—and through which realistic mechanisms? It is urgent to find local existing or potential mechanisms that allow for sustainability of the project. It is also important to recognize the tendency of the public sector to support their programs first, potentially disadvantaging private sector initiatives. The public sector controls drug supplies that are essential to treat diseases like HIV/AIDS, TB and malaria. And it is essential that the government provide oversight to ensure quality of these treatment programs. Given the length of time that Abt has been working in Ethiopia (through PSP-E and PHSP among others) to support private health care delivery, it is well positioned to develop a strategy for sustainability to ensure long-term quality private health care that will complement the services provided in the public sector. Financial Sustainability There is resistance from the government to continue its current level of support to the private sector, especially in training or mentoring. This is primarily a result of a lack of ability to do so. In part, this is because the government must focus on completing, equipping, staffing and launching a large number of new public health facilities throughout the country. However, relative to capacity, PHSP has strengthened the RHB’s ability and willingness to take full responsibility for the private sector in their training programs and mentoring services; although this is in the short term only and as long as there is financial support. There is a strengthened capacity to implement these programs, although budgeting for future training and mentoring of “Initially when I tried to obtain ARTs from the government, I was met by hostility. The PHSP intervened, explained that we were entitled to receive them and ever since we have had no problems.”—Hospital Administrator, Mekele Countless times the evaluation team heard positive comments about the contribution of PHSP from private providers and government officials. An RHB director said that PHSP is “helping build a bridge between the public and the private sector.” Another interviewee in Bahir Dar said that most governmental officials know that the public sector needs the private sector to help meet the increasing health care needs of Ethiopian families. 14 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION those in the private sector will require funding at a level that is well beyond what is currently allocated. Finding banks and systems to supply loans for private health sector providers to expand their services and for associations to provide trainings should continue to be explored. This may take additional advocacy work in order to persuade banks of the potential returns on lending to private health care entities. Participation in government training programs without their provision of per diems may also be considered. PHSP may consider linking with other programs in Ethiopia (e.g., USAID’s SHOPS or the Health Financing Project or other development partners) to explore cost sharing and ways to access more financing to strengthen the private sector. PHSP should continue to expand on and explore these potential financial growth initiatives. Ultimately, the goal should be to make the Ethiopian private health sector financially self-sustaining. The Continuum from Participation to Ownership, and the Integration Participation in the PPP appears to be well established, but full ownership of the partnership by the public sector is still to come. PHSP has played a significant role in brokering PPPs. However, a major gap still exists in terms of ownership at both the federal or regional levels. There is no “private sector” focal person or team within the FMOH or the RHBs who will continue to foster private sector participation in meeting the health needs of Ethiopians. While there is support for the private sector at the highest levels by the FMOH and the RHBs, anecdotal information suggests that much work is still to be done to ensure public-private cooperation at all levels. Two particularly critical areas are in the supply of drugs through public sector distribution channels and the referrals from private sector clinics to public hospitals. Building Local Capacity to Ensure Continued Access to Services Both the PHSP and PSP-E have been a catalyst in establishing programs such as TB DOTs and HIV/AIDS detection and treatment in private sector clinics. Once PHSP support for these programs is phased out, based on discussions with selected clinics in PHSP network, it is likely that the diagnostic portion of these programs will continue as part of their clinical offerings. However, treatment for TB, HIV/AIDS and malaria will most likely continue to be dependent on government resources given the reliance on imported free commodities associated with these services. In terms of FP, a broad range of contraceptives is available in the marketplace and linkage of private clinical services to commercial pharmacies may be an alternative to continuing dependency on government FP supplies. Training in IUD and implant insertion and removal will need to continue; in many countries, companies selling these products offer training programs associated with their products’ proper use. Sustaining Laboratory Quality Assurance The importance of providing long-term quality laboratory services is critical, and the private sector must develop the capacity to ensure this. Until such time as the private sector is capable of providing this service, the public sector will be required to continue to monitor the labs. As a future activity, PHSP may wish to explore a payment by private labs to maintain their certification. The cost of CD4 count machines—which can determine the stage of HIV infection, guide drug choices and indicate a patient’s response to treatment and disease progression—is expensive, ranging from USD $30,000 to $150,000 each. There are also additional costs, including USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 15 necessary reagents for each cell count, a technician to operate the machine and shipping and handling costs. Ultimately, this means that private providers in Ethiopia are unable to afford the machines. (This does not include other essential equipment and supplies, such as microscopes, refrigerators and chemicals.) Therefore, privately owned laboratories that can provide a broad range of services to the private sector are extremely limited in Ethiopia, both in number and distribution throughout the country. Discussions with the Regional Laboratory Capacity Building Directorate: Ethiopian Health and Nutrition Institute (EHNRI) revealed that the current structure of the institute and the available funding will not allow them to build the capacity of the private sector. Thus, the only viable option is for the public sector to continue serving the private sector and collaborating with existing and new private laboratories as much as possible. In some cases, it may be possible to establish a reciprocal relationship between public and private laboratories to provide specific services. EHNRI points out that establishing a referral linkage between public and private laboratories is an area in which a PPP must be created and nurtured. In general, personnel at EHNRI were appreciative of the work of the PHSP and acknowledged their dedication and help. Training and Mentoring In its proposed Year 4 workplan submitted to USAID/Ethiopia, PHSP is set to focus on ways to ensure the sustainability of the training and mentoring programs adapted to scale-up ART. Plans for sustainable programming, particularly as they also relate to TB, FP, STIs, malaria, and PMTCT, were explored with the PHSP team. As yet, there is no plan in place to achieve sustainability beyond awareness that clinics that initiated these programs under PHSP will likely continue them without donor support. This in and of itself is a notable achievement. Drug and Commodity Supplies The Deputy Director General of the PFSA cited the strategic plan for ART, TB and malaria drugs and indicated that based on their targets, there is sufficient drugs in country to support the plan. For example, there are an estimated 9.8 million people to be tested in 2012 based on regional plans. This target has yet to be reached, so there are sufficient supplies to meet this objective. Importantly, the PFSA directly delivers drug supplies to clinic sites based on an approved plan; and must have approval from the RHBs to deliver drugs to a specific private care facility in their region. Still, there are many complaints about the lack of pharmacists in the private higher clinics. The perception is that the supply of ART, TB and anti-malaria drugs continues to be a problem for the private health care sector as there are few alternative “The private sector is a key partner to the government in the provision of health services to the public. PHSP is working with the private sector and trying their best to link the public and private sectors in areas like TB, HIV diagnostics. We give due recognition for their role and as the result we involve them in many of the trainings and capacity building activities EHNRI is conducting at the national level. In this regard we can say EHNRI is building the capacity of PHSP staff so that they can share/transfer their knowledge and skill to the private sector.”—Mr. Gonfa Ayana, Director of the Regional Laboratory Capacity Building Directorate: Ethiopian Health and Nutrition Institute (EHNRI) 16 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION sources, affordable or otherwise, for these drugs in the commercial sector. And in the face of shortages, perceived or real, it is likely that the public sector will always be served first. Because of the long lead times between ordering and arrival of products, both public and private sector providers must have a solid understanding of how to forecast drug and supply needs well in advance to ensure they have no stock-outs for essential health care services. At times, when a private clinic is out of such things as test kits, they may “borrow” from other clinics with the promise of returning that product once they are supplied. Given that there are commercial alternatives for FP supplies and treatment of STIs, these areas are less affected by the shortage of supplies in the public sector. DKT is a major supplier of contraceptives in Ethiopia and has a country-wide presence. For pills, injectables and condoms, links to local pharmacies should be encouraged rather than dependence on government supplies. Recommendations for Sustainability  As a first step, PHSP should articulate a long-term sustainability strategy to support the private sector. The strategy should detail all the components of private health sector service delivery and how each aspect of sustainability can be achieved, through which means and by whom.  Support for national and regional associations of private sector providers and professionals working in the private sector should be continued and expanded to all the regions where PHSP is working. PHSP should also complete additional needs assessments made to discern ways to strengthen the meaningful contribution to the health needs of the country by these associations and how they might be made sustainable through membership fees, taxation of private health facilities, and other potential revenue sources.  Encourage the establishment of private sector focal persons and teams within the RHBs.  Assess the willingness of private health services to pay a fee for training, mentoring, and supervision programs involving their staff, access to laboratory quality assurance services and other services now provided to them free from the public sector.  In collaboration with the RHBs, PHSP prepare guidelines to support and work closely with private sector health facilities to ensure that they are prepared to take over responsibility for all essential health services, beginning with TB-DOTS, so that access to these services continues.  Encourage links between the private clinics and local pharmacies as an alternative to government supplied family planning products. INCREASED ACCESS TO SERVICES THROUGH THE PRIVATE SECTOR Performance targets and results related to service access through the private sector are measured and monitored at the national and regional levels, though most emphasis is on the national level per the Associate Award for PHSP (see Annex E for a list of national and regional indicators based on performance monitoring plans [PMP]). Over- and under-achievements are discussed in this report only in cases of extremes. (See Table 2 for a summary of the services available by region.) USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 17 The PMP is a USAID approved matrix of indicators and results. The regional numbers are derived from PHSP annual targets and allocated to the sites served under the program. While Abt thought that this was a fair regional allocation, they report that in actuality it negatively affected PHSP’s achievement number because of the variability of resources needed across the different sites. Across the regions, there were differences in their willingness to make decisions. While some regions moved aggressively to implement programs and requested more sites for inclusion, others were slow to implement and sometimes delayed decisions to start. Regions selected their own sites and, at times, decided on the number of sites to be assessed for readiness. Thus, the regions made the final decision as to which sites were eligible for program implementation. It should also be noted that according to the Associate Agreement, the annual plan is nationally based as opposed to regionally based (e.g., total of 159 TB sites, 105 FP sites, etc.). Accordingly, PHSP allocated the targets among the regions as evenly as possible. Also, some activities have surrogate planned indicators (e.g., number of trainings) and the number of new sites approximates the number of trainings since the private facility cannot initiate programs without at least two trained professionals. PHSP also counted trainings conducted in the regions and used the number of new sites as the planned performance indicator. Table 2: Services Available by Region as of the End of Year 3 Services Region or City Administration Addis Ababa Amhara Dire Dawa Harari Oromia SNNP Tigray Grand Total ART only 3 1 1 5 ART, PMTCT 1 1 FP only 1 1 1 1 3 4 11 FP, HCT 3 3 FP, malaria 1 1 FP, PMTCT 1 2 1 4 FP, PMTCT, malaria 1 1 HCT only 30 30 Malaria only 2 11 13 26 PMTCT only 7 2 1 5 6 6 27 PMTCT, HCT 1 1 PPM DOTS 19 31 3 1 37 6 97 18 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Services Region or City Administration Addis Ababa Amhara Dire Dawa Harari Oromia SNNP Tigray Grand Total PPM DOTS integrated with one or more other programs (ART, PMTCT, FP, Malaria) 14 34 8 3 41 21 14 135 Grand Total 79 73 13 6 84 48 39 342 Mobile Counseling and Testing to Reach Most At-Risk Populations During the first two years of PHSP, MCT, primarily for HIV, was used in selected regions to meet the needs of most at risk populations (MARPs). The percentage of MARP clients receiving counseling and testing services exceeded the set performance targets. More than 40% of clients were women, essentially meeting expectations. PHSP extended the program for a third year, partially to verify impact. Though relatively expensive, the MCT program had the flexibility to target emerging towns with a high HIV prevalence. An added benefit to the program was that the program was outsourced and transferred skills to local firms and NGOs. The MCT program finished as planned based on the cooperative agreement with USAID/Ethiopia. Access for Women Building on the PSP-E, PHSP continued to reach women who are at a higher risk of contracting HIV (e.g., female sex workers, local brew sellers and petty traders). As part of the Mobile CT program, some effort was made during the early stages of PHSP to link private services to women in general, as well as women at higher risk of contracting HIV. Since the end of that activity, there has been little done to reach women at high risk. Some early accomplishments include women gaining greater access to services as part of a work-based service provision. More could be done to identify times (e.g., during school hours or while walking between home and work) and locations (e.g., private facilities near a market place, work place or other areas frequented by women) so that women and MARPs can more conveniently access a facility. Activities, such as the distribution of coupons or the use of different communication materials designed to create greater awareness of services for women, were never implemented. Expansion Strategy for Facility-Based Services For each of the areas in which PHSP operates, the basic program implementation approach is shown in Figure 2. Progress in the areas of TB, HIV/AIDS, malaria, FP and STIs is well established for some (e.g., TB, HIV/AIDS), but not so well for others (particularly FP and STIs). While there has been some progress in strengthening the supply chains within treatment areas, the availability of drugs, especially ARTs and ACTs, remains an obstacle to full service delivery. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 19 Figure 2: Systematic Program Implementation Approach Building on the success of PSP-E, PHSP has utilized the PPP model to enhance the FMOH and RHBs’ effort in engaging the private health sector in the national TB program (See Figure 2: Systematic Program Implementation Approach). As part of the effort, PHSP supported the integration of additional services. In order to build on established relationships, the PHSP team supported new services in the best performing PPM-DOTS sites. While PHSP supports the integration of additional services in all clinical sites with a focus on FP and malaria, this objective has been only partially realized. Based on an analysis of PHSP-supported clinics, the following tables suggest that much is yet to be accomplished regarding the capacity building needs and the system strengthening requirements for ensuring quality and consistency in provision of TB-DOTS, HCT, ART, FP, malaria and STI services. Until these programs are better established across all planned regions, it seems unlikely that geographic expansion is practicable at this time. The status of each of these follows. TB/HCT Overall, the number of clinics with standard TB/HCT services and supported by PHSP met national goals (see Table 3). However, results were mixed regionally with the majority of TB/HCT services primarily accounted for in three administrative/regional areas. 20 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Table 3: Number of Private Hospitals and Clinics with TB/HCT Services Planned/ Actual Year 1 Planned Year 1 Actual Year 2 Planned Year 2 Actual Year 3 Planned Year 3 Actual National 96 103 187 186 229 232 Addis 24 27 29 29 33 33 Amhara 34 36 61 59 65 65 Dire Dawa 0 0 9 9 10 11 Harari 0 0 2 3 4 4 Oromia 35 37 63 65 79 78 SNNP 3 3 14 14 24 27 Tigray 0 0 9 10 14 14 ART The ART roll-out has been delayed due to implementation guidelines that restrict distribution of ART only to private hospitals with a pharmacist on staff (see Table 4). Thus, at the end of Year 3, 21 private hospitals (16 in Addis and Amhara combined) have been approved to receive drug supplies from PFSA. Though private clinics have received training in the diagnosis of HIV/AIDS, they cannot administer drug treatment and must refer HIV-positive patients to a public facility for treatment. Initially, there were reports that referrals were not well received at public facilities that offer ART treatment. However, PHSP has been effective in establishing relationships between private and public facilities to increase acceptance of these referrals. In terms of HIV-positive patients enrolled for pre-ART/chronic care at private clinics, the program fell significantly below its planned performance in both Year 2 (planned: 2,831; actual: 1,222) and Year 3 (planned: 5,264; actual: 2,051). Note that this is primarily a result of directives given to the PHSP to stop service in clinics without a pharmacist, as they are not supplied with ARTs. Table 4: Number of Private Hospitals and Clinics with Standard ART Services Planned/ Actual Year 1 Planned Year 1 Actual Year 2 Planned Year 2 Actual Year 3 Planned Year 3 Actual National 0 0 21 0 31 21 Addis 0 0 21 0 6 6 Amhara 0 0 0 0 10 10 Dire Dawa 0 0 0 0 4 1 Harari 0 0 0 0 1 2 Oromia 0 0 0 0 0 0 SNNP 0 0 0 0 7 0 Tigray 0 0 0 0 3 2 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 21 PMTCT The Director of Resource Mobilization at the FMOH said that current coverage for PMTCT programs is far below expectations. Hence, the private sector needs to play a bigger role and has requested USAID support. PHSP initiated PMTCT services in the private sector with objectives to:  Strengthen PHSP’s community PMTCT model pilot by incorporating a pilot tracking system.  Ensure access to affordable, safe and effective PMTCT services in private facilities.  Build demand for quality PMTCT service in private facilities. The PMTCT program was initially rolled out at the end of Year 2 with the training of health care providers in Addis Ababa. Training has been expanded to include all regions and city administrations in which PHSP works; overall, 179 health care providers have been trained. The first year of the PMTCT program included the training of private sector physicians in PMTCT. Thirteen private clinics in Addis Ababa piloted a community PMTCT care model developed by PHSP. In Year Three, PHSP supported the expansion of the customary PMTCT services to facilities in all regions where it operated and increased the total number of facilities with PMTCT services to 87. Training has expanded to include all regions and city administrations in which PHSP works and overall 87 health care providers have been trained. A total of 7,960 mothers were tested and 212 tested positive. Those who tested positive were enrolled in the initiative. It was reported that the community PMTCT care model could not be fully materialized in Addis Ababa because of the difficulty to establish effective linkage with the Urban Health Extension Program (UHEP). The gap in the UHEP model, which failed to link the urban private health facilities with the community-based services managed by the UHEWs, was reported to be one of the major obstacles to effective implementation. Hence, during the first quarter of Year Four, PHSP proposed to complement the community PMTCT care model with an SMS-based Patient Tracking System that can enable active communication between providers and the UHEWs. During the coming Year Four, PHSP has planned to initiate pre-ART and HIV exposed infant (HEI) care in all 87 private health facilities that currently provide PMTCT services. If ratified by the government, the plan will extend the implementation of Option B+ PMTCT program in all 87 clinics. In its Year Four plan, PHSP has clearly outlined what is needed to initiate the Option B+ PMTCT program, including a detailed proposal to start in 60% of PMTCT clinics and cover the remaining clinics its final Year 5. The possible policy shift by the government to allow Option B+ to be implemented in private higher clinics may provide an opening for them to also provide ARVs to other PLHIV in need of service. However, it should be noted that supply issues are likely to be an issue in this program and that women diagnosed with HIV/AIDS and treated with Option B+ in the private clinics still require lifelong ART. Accordingly, the FMOH should be clear as to what it will supply to the private sector higher clinics. Support for this program will include training, mentoring and logistics in private higher clinics both during and after the project phase out. The PHSP submitted a request to USAID for additional funds to support this effort. However, even with funds available, how will PHSP start a new program in its fourth year when it should be consolidating existing efforts in the clinical network it supports? 22 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Malaria Support for malaria services in the private sector started in Year 3 of the project. While no guidelines for treating malaria has been approved, PHSP has sought support at the regional level. Consensus has been reached with four RHBs to initiate malaria services with cases diagnosed and treated in private clinics happening in Amhara and the SNNPR. Private clinics in Oromia and Tigray have received training, but as yet, have not received drugs for treatment (see Table 5). Table 5: Number of Malaria Patients Diagnosed and Treated in Private Clinics Planned/ Actual Year 3 Planned Year 3 Actual National 15,000 33,924 Addis 0 0 Amhara 3,300 11,335 Dire Dawa 0 0 Harari 0 0 Oromia No data available No data available SNNP 11,700 21,959 Tigray No data available No data available Family Planning/STIs Support for FP and STI services was to begin in Year 1. Delays in implementation occurred primarily because of staff changes at PHSP and low demand for FP services from the RHBs. Further, there was no alternative plan in place to accomplish project objectives in the face of the staff challenges. No services were initiated in Year One, and the objectives in FP fall short (see Table 6). Further, STI services have yet to be initiated though PHSP has worked with TransAction Project to map potential areas in which each might work. Table 6: Number of Private Hospitals and Clinics with Standard TB-CT and FP/STI Services Planned/ Actual Year 1 Planned Year 1 Actual Year 2 Planned Year 2 Actual Year 3 Planned Year 3 Actual National 0 0 50 30 114 122 Addis 0 0 10 0 13 13 Amhara 0 0 0 0 22 22 Dire Dawa 0 0 5 8 11 9 Harari 0 0 0 4 4 4 Oromia 0 0 20 0 26 26 SNNP 0 0 15 18 20 20 Tigray 0 0 0 0 18 18 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 23 Strengthening the Supply Chain Like many countries, ensuring a continual supply of drugs is essential to maintaining quality care in both the public and private sectors. The issue of supplying ARTs to private higher clinics has already been addressed in the Policy Framework for Private Sector Participation section. Further, supplies often are delivered with short or expired expiration dates (e.g., anti-TB drugs, lab reagents). Many clinics have developed their own coping mechanisms to address shortages, networking with other private or public health clinics to obtain some of these drugs. As part of the solution, PHSP continues its focus on linking private facilities to reliable public and private sector suppliers for the necessary drugs. Recommendations for Improving Access  In collaboration with the RHBs, PHSP should prepare guidelines to support and work closely with private sector health facilities so that they are prepared to take responsibility for all essential health services, beginning with TB-DOTS, so that access to these services continues.  Effort should be made to identify opportunities to provide services to women, keeping in mind their unique needs, schedules and other socio-economic hurdles such as commercial sex work.  Disruption of the flow of drug supplies to private sector hospitals and clinics continues to be a problem. During selected training courses, effort should be made to assist clinics in forecasting their drug needs (e.g., ARVs, FP, TB and malaria drugs) and how to effectively request and receive them well in advance of any potential stock-outs. Networking systems should also be put in place to ensure that excess supplies are made accessible to other facilities with potential stock-outs.  Given the current status of introducing the full range of standard services in the existing PHSP supported clinical network, no geographic expansion of the program is recommended. At this stage of the project, focus should be on the existing network.  There should be focus on integrating STI services with FP services in private health clinics where the services does not exist.  Relative to introduction of Option B+, in cooperation with USAID, a roadmap should be defined in such a way so that the PHSP is able to ensure a sustainable program is in place during their remaining two years.  USAID/Ethiopia with the PHSP should address the indicator results and determine what adjustments may be required to achieve the planned results or whether the targets themselves should be adjusted. This should be done in a timely manner so as to maximize resource allocation within the project. SUSTAINING QUALITY ASSURANCE IN THE PRIVATE SECTOR SERVICES PHSP’s Approach to Quality Assurance PHSP’s approach to Quality Assurance (QA) is to strengthen the technical and managerial quality of project work and to ensure that project interventions and deliverables satisfy or exceed quality standards required by the client and the company. The project’s home office- 24 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION based portfolio manager has primary responsibility for application of the QA system—in collaboration with the PHSP COP—to ensure that technical expertise is made available at critical junctures in planning, implementation and production of key program deliverables. The QA system also extends to financial management and contractual compliance, and is incorporated into Abt’s routine management processes such as quarterly contract reviews with the home office, annual performance reviews and work planning, budget analysis and realignment as required, and Abt’s annual site risk assessment exercise. Quality Assurance and Standards-Based Management and Recognition (SBMR) PHSP was not defined by SBMR, however they did take advantage of many of its principles such as a preventive and integrated approach, an emphasis on quality, regulations and accountability, and, to a certain extent, decentralization of management functions and increased decision making power at the local level. PHSP should be encouraged to review JHPIEGO’s SBMR Guide (see http://www.jhpiego.org/files/SBMR%20FieldGuide.pdf) for additional guidance and inspiration over the next two years. Training Strategy and Approach At the outset, PHSP conducted a facility assessment to select suitable private health facilities including clinics and hospitals. The assessment’s objective was to identify their needs for training and equipment. Based on these assessments and in collaboration with the RHBs and the FMOH, modifications to the training programs were made in both length and content of the course. There is a heavy reliance on trainings provided through the RHBs. It is not certain that this training will continue to be made available to private sector personnel in the future. Recognizing the importance of continuing medical education in maintaining quality of care includes developing courses that meet the needs of health care workers at all levels. To maintain quality, the PHSP should consider “update” courses that last one or two days and art targeted toward specific issues of particular interest to clinicians. Equipping Sites with Simple, Minimal Equipment All visited private health facilities reported receiving different types of furniture and equipment from PHSP, including examination tables and lights, screens, weighing scales, insertion and removal kits for IUDs and implants, and tables and chairs. Clinic personnel and/or owners expressed appreciation for this support, which was clearly seen as a reminder of the value of quality standards and served to open dialogue between PHSP, the RHBs and the private facilities in assessing ongoing quality assurance in their facilities. Improving Interpersonal Communication PHSP provided skill building in interpersonal communication (IPC) at all levels to enable decision-makers and service providers to better communicate their needs and intentions in a more positive and effective manner to each other and to clients. “Health providers are crucial to informing the decision-making and health-seeking behavior of individuals, especially for disease prevention and healthy living. However, skill-building for interpersonal communications (IPC) is largely missing from the current health provider education in Ethiopia.”—The PHSP Cooperative Agreement, page 36 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 25 Supportive Supervision Utilizing a team approach, PHSP and the local health office or RHB conducted supportive supervision, which includes regular M&E activities and program performance. Note that while PHSP continues to build capacity to offer supportive supervision, it is unclear that this can be sustained once PHSP phases out as it requires the RHB’s financial and technical support as well as resources currently provided by PHSP. Currently, a staff member from the regional project office, along with a member from the Woreda and the RHB, visits clinics almost every three months. Laboratory Quality Assurance Representatives of the RHBs expressed the commitment of their respective bureaus to continue providing the test kits, drugs, reagents and other supplies to private providers currently supported by PHSP. The regional laboratories also indicated that they have the technical capability to continue regular TB External Quality Assurance (EQA) for private sector providers in the absence of PHSP, though they do not have the required financial support to independently take over project activities such as supportive supervision and trainings for private sector providers. A particular concern of the RHBs was their resource capacity to support increasing demand both from the public and private sectors. Several RHBs support the establishment of regional private sector associations to ensure that the private health sector is able to participate in a policy dialogue with the government. Initially this will require some capacity building, with the objective of sustaining some PHSP project activities. Recommendations for Improving Quality  PHSP may consider phasing out some elements of the provision of basic equipment and service provision tools to the private health facilities. Prior to the phase out, the PHSP should conduct a needs assessment of a sample of clinics to determine what is needed to provide services such as TB-DOTS, malaria, HCT, FP and treatment for STIs. In the interim, PHSP should also ensure that providers and those who provide ongoing supervision and monitoring are aware of how the equipment and tools are used and maintained.  Given that IPC skills for health providers is still a relatively new concept in most medical training institutions around the world, PHSP should continue to advocate and assist where possible the inclusion of IPC in the development of any training curriculum and service delivery tools developed in the future. DEMAND FOR QUALITY PRIVATE HEALTH SECTOR SERVICES Several strategies exist for creating and expanding demand for essential health services in the private sector. Most strategies include the following components:  Increase overall demand for products with public health benefits.  Direct those with some ability to pay toward private sources of supply.  Encourage more effective health seeking behavior in the private sector to elevate the quality and perceived quality of the services.  Encourage healthy lifestyle practices (e.g., safe sex to prevent transmission of HIV/AIDS and STIs, and use of bed nets to prevent malaria). 26 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Critical conditions to consider:  Behavior change activities must be sustained if they are going to continue to be effective.  Different groups of people require specific types of outreach (such as remote populations who have no access to television).  Cultural, social and regulatory conditions must always be taken into consideration when developing behavior change and social mobilization strategies and approaches. Demand Creation in the Ethiopian Context Ethiopia has its own particular constraints and opportunities that allow for the promotion of positive health seeking behavior by the private sector. In its favor, traditionally, rural Ethiopian women are largely self-governing and have frequent “meetings” to discuss matters of concern in their local communities. This is much less available in urban areas. However, because of economic reasons and general disenfranchisement of the private sector by the public sector, many people do not view private facilities as a viable option for seeking health care. Some private providers, because of their own lack of knowledge of health promotion or a history of minimizing advertising of their services due to fear of recrimination, have not utilized many of the demand creation tools available to them. PHSP Approach to Demand Creation To create demand for quality private health sector services, the “PHSP team will use evidence￾based strategies that encourage clients to seek quality health services in the private sector, and empower them as consumers to be more proactive and mindful about managing their health. While implementing these behavior change activities, Abt will also expand local capacity”1 through:  Qualitative research.  Segmentation and prioritization.  Behavior change communication (BCC).  Community-level mobilization. Overall, PHSP’s approach to creating demand has been passive. However, there has been considerable demand created indirectly through the following: Public/Private Linkages PHSP has contributed to stimulating demand for private health services by linking the public and private health care systems, thus increasing the quality and range of essential services available in private facilities. Consumers are becoming more aware of this improvement. Integration of Services to Create Demand Private health delivery sites provide a limited but diverse range of information about their services by posting information and discussing other services available in the same facility. One approach is to remind women immediately postpartum to return to the 1 Associate Award No. 663-A-00-09-00434-00, USAID Private Health Sector Program, pg 36 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 27 clinic in 45 days for immunization of their newborn. During this visit, the women are given FP counseling and provided FP services if needed. Informational Tools Some facilities, took advantage of informational posters and other materials, although most were not provided by PHSP. These approaches are promoted in the trainings and supportive supervision provided by PHSP and the RHBs. It should be noted that all the informational tools produced by PHSP were largely word dependent, meaning literacy is required to understand the message (see Figure 5). Further, the desktop service delivery guide for FP providers, which that is to sit on a desktop in order to remind providers of important points, is text heavy and hard to follow while also consulting with a patient. It is not clear why it was developed in that form. Figure 3: Messages about Malaria Prevention and Treatment Male Involvement Service providers did not seem to see an advantage to supporting male involvement in service messages. For instance, all providers interviewed said that they had never given information or counseling to men on FP. It is not clear if this is included in the in-service training or supportive counseling guidelines that PHSP supports. Advertising Services There was little evidence of service advertising in and for the facilities; when it was present, the impetus came from the owners and administrators of the clinics or hospitals. Some clinics and hospitals displayed bulletin boards outside, in the entrance or on the back of prescriptions as a list of services provided and in some cases the price. PHSP indicated that they plan to promote 28 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION the creation and posting of these bulletin boards in the facilities where they work during Year Four. Social Mobilization PHSP used different social mobilizing strategies, such as involving community elders and coffee ceremonies, to gather community members for HCT services provided by the mobile team. Several NGOs also provided information about which services are provided in certain private health facilities. Recommendations for Demand Creation  Given that many Ethiopian private health service providers do not have specific training and experience in advertising and demand creation, PHSP should focus on limited methods of advertising for facility-based services that can be easily implemented without much expense to the facility owner. Adding the value of advertising services provided by the facility via bulletin boards outside the facility or on referral slips or prescriptions would be a simple, inexpensive and valuable suggestion.  Develop demand creation tools that do not require literacy, which is imperative given the low level of literacy, particularly among women.  A training and supervision curriculum and follow-up should include looking for opportunities to promote services within and outside the facilities. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 29 IV. MANAGEMENT AND PERSONNEL MONITORING AND EVALUATION The Performance Monitoring Plan (October 2009–September 2014) and the performance indicators set for the project are the basis for the M&E activities carried out by PHSP. The Quality Management Unit (QMU) at PHSP’s head office and PHSP’s regional offices jointly coordinate M&E activities. The QMU staff is made up of nine professionals including a director, deputy director, M&E manger, three M&E analysts and two data clerks. On a quarterly basis, program officers collect data across all program areas from all 342 private health facilities supported by the project. Data are collected using formats that are consistent with the National Health Management Information System. Smart phone technology is used to collect and transfer facility-specific data from the center to the regions. Data reported in hardcopy from the same facilities are also collected and used for data quality checks. PHSP sends this program-specific data and reports (PPM-DOTS, FP, HCT, ART and malaria) to USAID/Ethiopia. Feedback is often program specific and sent by the specific program unit at USAID. At times, the PHSP also uses the M&E data to adjust targets based on previous performance. The Strategic Management Unit at USAID closely works with PHSP to carry out data quality audits on a regular basis on selected private facilities. The AOTR provides feedback on the annual plan. However, his involvement on the M&E activities is very limited. Staff from the QMU and the other program advisors from the PHSP country head office provide M&E support to the regional program officers. PHSP has prepared a data quality assurance manual to assist and guide the regular data collection made by the program officers. As part of the M&E activities of the project, PHSP staff, in collaboration with representatives from Woreda, Zonal and RHBs, carries out integrated supportive supervision on quarterly basis. Review meetings are conducted in all regions on annual basis in the presence of all stakeholders, including private health facility owners and government partners. However, PHSP reports that USAID rarely attends these meetings. The annual program reports prepared by PHSP are shared with the RHBs. Areas for Improvement Together USAID and PHSP M&E may need to review the monitoring plan and make adjustments that will help PHSP set their plans to respond to existing trends in compliance with USAID requirements. It is possible that additional assistance from Abt headquarters may be required. PHSP should organize a high-level visit to each region in which they work as part of an advocacy initiative to ensure the full engagement and policy report at both the regional and federal levels. The result of this effort is expected to be a common understanding of the current situation in the regions with respect to PHSP programs and productive discussions on the way forward to build on strengths and address gaps. 30 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PHSP requests more comprehensive feedback on the quarterly and annual reports from USAID/Ethiopia. PHSP AND USAID STAFF MANAGEMENT OF PHSP A number of staffing issues have adversely affected PHSP’s ability to complete tasks on schedule. Some of these are to be expected in any long-term project, while others require special attention to correct. Some points are as follows:  Abt reports that salaries for PHSP are not as competitive as other similar NGOs in country thus leading to high turnover of staff. This issue was conveyed to USAID in a timely fashion although it took some time for USAID to revise salary scales based on FSN salaries, which eventually resulted in an overall salary increase of 37% in January 2011. By this time, many PHSP staff had found outside employment.  One example of staff loss at PHSP was when another NGO offered higher salaries for FP experts and the entire PHSP FP team quit. While this is unavoidable, Abt should have seen that this would slow an important component of PHSP and provided alternative TA until this staffing issue could be resolved.  High attrition of program staff is expected in the final years of a program. PHSP should be prepared and obtain additional support from headquarters when needed. ANNUAL WORKPLANS Abt produces annual workplans for approval by USAID/Ethiopia. These workplans are general overviews of what has been accomplished in the previous year and propose what will be accomplished in the following year. The PHSP mid-term evaluation team was not given previous year workplans. Moreover, the team received the current draft workplan for Year Four only after considerable probing as to how the team could obtain a copy. The team understands that the workplan for Year Four has now been approved by USAID. PROMOTION OF WOMEN ON PHSP WORKFORCE There is an absence of women on the PHSP senior level management team, particularly in technical and administrative positions. Currently, only two women hold technical or management positions in the Addis office. While Abt went to great effort to advertise their positions, encouraging women to apply, and even engaging a hiring firm to help them locate suitable female candidates for their vacancies, the PHSP team has not been successful in achieving gender balance in their hiring. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 31 RECOMMENDATIONS FOR MANAGEMENT AND PERSONNEL  Though late in PHSP, Abt should make every effort to fill any current or future vacancies with appropriate female candidates.  USAID and PHSP should have routine meetings to discuss issues relating to the management of the program. If possible, select members of the senior health office staff to be present in order to be aware any current issues.  Abt should give priority to finalizing the Year Four workplan and ensure that the Year Five workplan is submitted in a timely manner as prescribed by USAID/Ethiopia.  USAID/Ethiopia may want to review their process of review and feedback to PHSP, relative to workplans.  A high attrition of program staff is expected in the last years of a program. PHSP should be prepared and obtain additional support from headquarters when needed. 32 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 33 V. LESSONS LEARNED Acceptance of the PPP appears to be well established, but full ownership of the partnership by the public sector is still to come. The PHSP is to be commended for building trust and relationships with various elements of the public sector, understanding that this is a long-term effort. This is a major strength of the PHSP. As an example, when changes occur within RHBs, new relationships must be forged and special effort must be made to ensure that there is continuity of support for the previously-approved programs under the RHB’s jurisdiction. PHSP has played a significant role in brokering PPPs. However, a major gap still exists in terms of “ownership” at both the federal and regional levels in that there is no “private sector” focal person or team within the FMOH or the RHBs who will continue to foster private sector participation in meeting the health needs of Ethiopians. Three particularly critical areas are in the:  Supply of drugs through public sector distribution channels.  Heavy dependence on the GoE for training, including medical updates.  Referrals from private sector clinics to public hospitals. SUPPLY OF DRUGS Partnering agencies and future programs should address the supply of ARTs, with the objective of securing explicit agreement on the modality of ART implementation, particularly in higher clinics. With an explicit agreement, better allocation of project resources can be expected. However, short of USAID intervening to try to secure for the private sector an explicit percentage of the drugs provided by the Global Fund, efforts to fully utilize private sector resources in addressing issues associated with HIV/AIDS treatment will always be limited. TRAINING PROGRAMS In Ethiopia, government training activities are not associated with cutting edge training programs. This will remain the case as long as the FMOH controls the curriculum, the teachers and the length of medical education programs. Medical schools should be enlisted to provide professors or teachers for short courses, with a private association such as MAPPP-E managing it. REFERRALS FROM PRIVATE SECTOR CLINICS TO PUBLIC HOSPITALS At the outset of the project, it was assumed that referrals from private sector facilities would be welcomed in public hospitals. Given the very natural attitudes that public health care groups have to private sector ones, any future program dependent on referrals should hold personal meetings with hospital administrators and ensure that hospital professionals at all levels understand the agreement to welcome referrals. MARKETING INNOVATIONS Branding requires considerable effort to establish the belief that there is something of value in the advertised service, which of course should be high quality. Currently, the private sector health programs with whom PHSP works have different levels of expertise in those areas that 34 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION might be associated with a brand. It may be too early in the development of private sector capabilities to begin to think of branding. Also of note, private sector facilities may not be mature enough themselves to understand the value of a brand name associated with their clinic or hospital. For example, there were no prominent displays of training certificates. Similar comments can be made about franchising. It is too early in the development of the private sector to assume that owners would pay a fee to be part of a franchise unless they discerned a real benefit. Importantly, there is little buy in to the idea of franchising within the PHSP. Implementing such a broad program requires total commitment. Should there be a follow-on project to the PHSP, branding and franchising may be considered, but with the understanding that branding requires a significant budget and commitment—as well as discussions on how to ensure sustainability. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 35 VI. RECOMMENDATIONS POLICY  PHSP is an expert in understanding issues affecting the private health sector. Periodical briefing and/or briefing materials should be made available to USAID and other interested groups to clearly outline the contributions of the private sector to the overall provision of health services in Ethiopia. They, in turn, can effectively advocate for the private sector when speaking with the FMOH and other partners.  PHSP is well positioned to influence the development of policies and strategies concerning the private health sector. To this end, it is recommended that they organize national and regional forums designed to further an enabling environment for a sustainable public-private health sector partnership.  FMHACA has requested a medical doctor be provided to them and paid for by PHSP. This could provide a key contact within FMHACA and a PHSP-supported advocate for the private health sector. This may result in better communication between PHSP and FMHACA and result in better representation of the private sector in issues such as the dissemination and review of the new facility licensing standards.  In cooperation with FMHACA, PHSP should assist the RHB in their facilitation of regional workshops to provide feedback on the approved licensing and accreditation standards. SUSTAINABILITY  PHSP should first articulate a sustainability strategy for support to the private sector. This strategy should detail all the components of private health sector service delivery and how each aspect of sustainability can be achieved, through which means and by whom.  Support for national and regional associations of private sector providers and professionals working in the private sector should be continued and expanded to all the regions where PHSP is working. PHSP should also complete additional needs assessments to determine how these associations can make meaningful contributions to the health needs of the country—and how they can be sustainable through membership fees, taxation of private health facilities and other potential revenue sources. TA should be part of this support.  Encourage the assignment of private sector focal persons and creation of teams within the RHBs.  Assess the willingness of private health services to pay a fee for training and supervision programs for their staff and access to laboratory services and other services, which they now receive free of charge from the public sector. INCREASING ACCESS  In collaboration with the RHBs, PHSP should prepare guidelines to support and work closely with private sector clinics to ensure that they are prepared to take over responsibility for all essential health services, beginning with TB-DOTS, so that access to these services continues. 36 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION  Effort should be made in the remaining years of PHSP to identify opportunities to provide private services to women, keeping in mind their particular needs and schedules.  Disruption of the flow of drug supplies to private sector hospitals and clinics continues to be a problem. During selected training courses, effort should be made to assist clinics in forecasting their drug needs (e.g., ARVs, FP, TB and malaria drugs) and how to effectively request and receive them well in advance of any potential stock-outs.  USAID should enter directly into talks with the FMOH, concerning the allocation of drugs (e.g., ARTs) provided through the Global Fund, to obtain explicit agreement on higher clinics dispensing supplies critical to the programs of the PHSP.  Given the current status of introducing the full range of standard services in the existing PHSP-supported clinical network, no geographic expansion with PHSP support is recommended.  Focus on introduction of STI services integrated with PPM DOTs and CT services. IMPROVING QUALITY  In the short term, PHSP should continue providing basic equipment and service provision tools to private health facilities on an as needed basis so they can provide basic essential services like TB-DOTS, malaria, HCT, FP and STI treatment. PHSP should also ensure that providers and those who provide ongoing supervision and monitoring are aware of how these equipment and tools are used and maintained. However, PHSP should conduct a needs assessment of equipment to assess the importance of continuing this program in Year Five.  Given that IPC skills for health providers is still a relatively new concept in most medical training institutions in Ethiopia, PHSP should continue to promote the inclusion of IPC in the development of any training curriculum and service delivery tools developed in the future. DEMAND CREATION  PHSP should focus on limited methods of advertising for facility-based services that can be easily implemented without much expense to the facility owner and with limited training. Advertising services provided via bulletin boards outside a facility or on referral slips or prescriptions would be a simple and inexpensive option.  Demand creation tools should not require literacy given the low level of literacy, particularly among women.  Training and supervision curriculum and follow-up should include looking for opportunities to promote services within and outside the facilities.  Some consideration should be given to conducting seminars through MAPPP-E to help private clinics and hospitals increase their patient flow. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 37 MANAGEMENT AND PERSONNEL  Though late in PHSP, Abt should make every effort to fill vacancies with appropriate female candidates.  Abt should give priority to finalizing the Year Four workplan and making sure that the Year Five workplan is submitted in a timely manner as prescribed by USAID/Ethiopia. 38 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 39 ANNEX A. SCOPE OF WORK GLOBAL HEALTH TECHNICAL ASSISTANCE BRIDGE II PROJECT GH Tech Contract No. AID-OAA-C-12-00027 SCOPE OF WORK USAID/Ethiopia: Private Health Sector Project Mid-Term Evaluation Final (9-19-12) I. PURPOSE OF THE EVALUATION The purpose of this evaluation is to obtain an independent assessment of the performance of the Private Health Sector Program (PHSP) and to learn from the program’s accomplishments and challenges to date in order to guide USAID/Ethiopia and PHSP staff with regard to the direction and management of the program in its final two years and to inform future private health sector program design and implementation. II. PERFORMANCE PERIOD In-country field work will take place over the course of two to three weeks commencing in early October 2012. III. FUNDING SOURCE The evaluation will be funded with mission HIV/AIDS earmarked funds. IV. BACKGROUND Country Context As Africa’s second most-populous country, Ethiopia has a large, predominantly rural (82%), and impoverished population with poor access to safe water, housing, sanitation, food, and health services. These factors result in a high incidence of communicable diseases including tuberculosis (TB), malaria, respiratory infections, diarrheal diseases and nutritional deficiencies. Although the fertility rate has decreased to 4.8 (DHS 2011) the many years of very high fertility rates and low contraceptive prevalence contribute to an annual population growth rate of 2.6%. This high population growth puts tremendous pressure on all social services including education and health, arable land and also employment opportunities. The proportion of women using modern family planning (FP) methods is increasing; however the unmet demand for FP by women of childbearing age is 25%. In 2000, only 6% of women were using modern FP methods as compared to 27% in 2011 (DHS 2011). Despite this increase, high fertility and lack of access to quality services as well as cultural preferences result in high rates of maternal and neonatal mortality. Ethiopia has one of the highest rates of maternal deaths at 676 per 100,000 live births (DHS 2011) in the world. An estimated 19,000 women die from childbirth-related causes every year. Since 2005, under-five mortality has decreased by 28% from 123 to 88 deaths per 1,000 live births; neonatal deaths account for 42% of the under-five 40 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION mortality (DHS 2011). Diarrhea, pneumonia, and problems around delivery are the leading causes of child mortality, with malnutrition a major underlying factor. The 2011 DHS reports that the national HIV/AIDS prevalence is 1.5%, with higher average prevalence in urban areas at 4.2% including prevalence as high as 6.0 % in some areas such as Addis Ababa. Uptake of prevention of mother-to-child transmission (PMTCT) services is low with high loss-to-follow up of HIV positive mothers. For instance, only 12,000 of 20,000 mothers identified as HIV positive received prophylaxis in 2010/11. Ethiopia also has the 7th highest TB burden in the world with 314,000 cases per year, approximately 20% of whom are also HIV-infected, and 3,000 multi-drug resistant TB (MDR-TB) cases per year (WHO 2008). Underlying these serious health problems in Ethiopia are poorly developed health systems. While much progress has been made in the last 15 years in many areas of the health sector, including increasing physical infrastructure and the training and deploying of a new cadre of Health Extension Workers, the delivery of health services is seriously undermined by several deep-seated systemic problems. These include a severe health workforce crisis, including high attrition of certain cadres of health workers and poor systems to deploy and support health workers; supply chain challenges that keep necessary equipment, supplies and commodities from reaching their destination; weak financing systems that provide health centers and hospitals with only a fraction of their annual operational budgets; lack of data and poor use of data for decision-making; low skills and lack of resources needed for program monitoring and evaluation; inadequate infrastructure and equipment; untapped potential in the private health sector; and weak management skills at Woreda, Zonal and regional levels. Ethiopia’s Private Health Sector The private health sector in Ethiopia comprises both for-profit (formal and informal) and not￾for-profit entities ranging from individual practitioners to large institutions. The formal private health sector in Ethiopia encompasses a broad range of actors including: private hospitals, specialty clinics, higher clinics, medium clinics, and lower clinics; private laboratories; private radiology services; private pharmacies, drugstores, and rural drug vendors; manufacturers of pharmaceutical health commodities and technologies, importers, and distributors/wholesalers; private health professional training institutions; private health insurance providers; numerous NGOs, civil society organizations, faith-based organizations, and private foundations providing health services; and various cadres of health care professionals in private practice. The informal private health sector consists mainly of traditional healers, traditional birth attendants, and vendors of herbal or alternative medicine. Currently there is no comprehensive national data on the size and distribution of private health sector actors in Ethiopia. While such data on the private sector is scant, surveys and evaluations point to significant growth and high utilization of the sector. (Health Systems Assessment Ethiopia, Health Systems 20/20 Project, July 2012). According to the 2008 National Health Accounts study, total per capita health expenditure from all sources, including the government, is only $16 compared to $34 per person recommended by the World Health Organization to provide a minimum package of essential health services. Out of the $16 per capita expenditure, the contribution of the GoE is estimated to be 22%, while the contribution from external assistance, out of pocket expenditures, and the private/NGO sectors are 39%, 37%, and 2-3% respectively. The private sector was thus the second-largest source of financing for health (39%) when both household and employer contributions are combined. Although government funding increased by 71% between 2004/05 and 2007/08, household and private employer contributions increased by 176% and 112% USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 41 respectively. All private sector financing agents (households, private employers, not-for-profit institutions, and others) together managed 44% of national health expenditures in 2007/08. The private health sector thus is an important source of health care for many people in Ethiopia, especially those in rapidly-growing urban areas and including the poor. While the GoE articulates a commitment to engaging the private sector in order to increase access to health services, and while major health strategy documents mention the importance of working with the private sector, an enabling environment, including clear policies and guidelines to foster real private sector involvement, is lacking. For instance, there is no national policy on engaging private health facilities to deliver subsidized public health services or for including private sector providers as part of the national health insurance scheme and there continues to be little attention paid to health professionals working in the private sector. As a result, the private sector in Ethiopia remains underutilized and there are concerns regarding the quality of services the sector provides. Engaging the private sector to serve those who can and will pay for private health services and ensuring the quality of those services frees up public sector resources to serve those with the greatest need and expands access to critical primary health care services. Private Health Sector Project (PHSP) PHSP is a five-year program (2009–2014) operating in five regions in the country with the aim improving quality and accessibility of public health services in the private health sector. The current program is follow-on to the predecessor Private Sector Program (2004–2009). USAID/Ethiopia’s HIV/AIDS, Population and Nutrition Office is organized into four teams: 1) the Health, Population and Nutrition (HPN) Team, 2) the HIV/AIDS (President’s Emergency Plan for AIDS Relief [PEPFAR]) Team, 3) the Malaria (President’s Malaria Initiative [PMI]) Team; and 4) the Health Systems Strengthening (HSS) Team. PHSP receives the majority of its funding from PEPFAR with some population and reproductive health funding from the HPN Team. The program, however, encompasses activities in all areas, including malaria and HSS. As PEPFAR has moved from an emergency mode to more sustainable program interventions, there has been focus on building the capacity of government institutions and strengthening local organizations and the private health sector in order to broaden the resource base for PEPFAR supported activities and to ensure sustainability. Thus, PHSP is one of the flagship programs at USAID and a key instrument for ensuring sustainability of United States Government- supported programs. Abt implements PHSP, which works to increase demand and provision of high-quality public health services in the private sector by building sustainable public-private partnerships. The goal of PHSP is to enable the Federal Ministry of Health (FMOH) and Regional Health Bureaus (RHBs) to effectively partner with private health providers to deliver public health services, while improving the quality and affordability of these services for Ethiopians. The program is designed to achieve the following results: 7. Supportive policy environment for the private health sector; 8. Enhancement of both geographic and financial access to packages of essential health services through the private sector; 9. Sustained improvements in the quality of these services; and 10. Increased demand for quality services by informed, proactive consumer populations. 42 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Specifically, PHSP’s engagement with the private sector helped bring about the expansion of high impact public health services, HIV/AIDS, TB, FP, and malaria services at private health facilities . Before the start of USAID private Sector Program the availability of HIV , TB services at private clinics were very limited and outside of the capital nonexistent. The program currently supports 13 private health facilities to provide PMTCT services including HIV testing, pre-anti-retroviral therapy (ART) and ARV prophylaxis. It supports 11 private facilities to provide ART and 222 private and 10 workplace clinics to provide TB-Directly Observed Treatment Short Course (DOTS). The program also assists 215 private and workplace health facilities and mobile sites to provide HIV counseling and testing services. The program has recently commenced family planning service delivery, supporting 29 private facilities, and is in the initial stages of supporting malaria service delivery in private hospitals. PHSP also works with 215 public and private laboratories to strengthen their laboratory quality assurance and works towards accreditation of private labs. Additionally, PHSP enhances government oversight, strengthens public-private referral mechanisms, develops interventions to increase financing for the private health sector, and improves client education. For instance, the program recently worked with the Ethiopian regulatory agency, the Food, Medicines Healthcare Administration and Control Agency (FMHACA), to finalize draft regulatory standards for service delivery and health facility accreditation. V. STATEMENT OF WORK The purpose of this independent external evaluation of PHSP is to assess program performance to date and learn from the program’s accomplishments and challenges in order to guide USAID/Ethiopia and PHSP staff with regard to the direction and management of the program in its final two years and to inform future private health sector program design and implementation. The evaluators are expected to perform the following tasks: 11. Assess the program’s progress towards achieving set objectives and anticipated results. Specific questions include: – What has been PHSP’s progress to date in terms of achieving planned results and performance indicators (as provided in the program’s Performance Monitoring Plan)? – What strategies did the program adopt in order to achieve the four major results? – What are the main reasons for exceeding or not meeting expected results? What are the major policy challenges (consider GoE, USAID and PEPFAR policies) and opportunities with respect to achieving program objectives and targets? – How well has the partner monitored and evaluated the outputs and outcomes of the program and the extent to which the results are achieved? How can the M&E system be improved? – How well has PHSP incorporated lessons learned from the predecessor Private Sector Program (PSP) into the current program? – Have there been any management (consider both PHSP and USAID) or staffing issues or challenges during the program and, if so, how have they been identified, communicated, addressed or resolved? USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 43 – What are the key lessons learned from PHSP? What have been the strengths, weaknesses and best practices with respect to PSHP implementation, monitoring and evaluation (M&E), capacity building and the program’s relationships with the GoE and other stakeholders? – What arrangements have been made to ensure sustainability of the program’s results and impacts? 12. Make actionable-recommendations for the direction and management of PHSP in its remaining two years of implementation. – What do government officials (regional and federal levels) and other stakeholders perceive as priorities and opportunities for the private health sector? – Are there missed opportunities, gaps and/or potentially effective private health sector models and approaches that PHSP or a future private sector program should consider? – What, if any modifications should be made to program targets in the remaining two years of implementation? – What are the opportunities and challenges to improve on and/or initiate new and innovative strategies in order to achieve key results? – What, if any, modifications should be made to program design, management, and staffing in the remaining two years of implementation? VI. METHODOLOGY The evaluation team is responsible for developing an appropriate methodology that responds to the evaluation tasks and answers the evaluation questions above. USAID/Ethiopia expects that both quantitative and qualitative methodologies will be employed, including team planning discussions and meeting(s), a desk review, key informant interviews and site visits. The exact number of interviews and site visits will be finalized in collaboration with the evaluation team prior to the visit. The following are anticipated elements of the methodology: Document and literature review: The evaluation team will conduct a comprehensive literature review of pertinent documents including studies and assessments regarding the private sector in Ethiopia, the GoE strategies and plans, and USAID and program documents, including but not limited to:  Ethiopian government policies, strategies and plans, including but not limited to the Ethiopian Growth and Transformation Plan, the Health Sector Development Plan IV, and the FHAPCO Strategic Plan for Intensifying the Multi-sectoral HIV and AIDS Response in Ethiopia.  Ethiopia Global Health Initiative Strategy.  Ethiopia PEPFAR Partnership Framework.  PHSP Cooperative Agreement and sub-contracts.  PHSP performance reports, performance monitoring plan, assessments and evaluations. 44 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Team Planning Meeting: The in-country work will commence with a team planning meeting. During this meeting, the evaluation team will meet with USAID/Ethiopia staff to be briefed on the assignment including its purpose, expectations, and agenda of the assignment. The team will clarify team members’ roles and responsibilities; review and finalize the assignment timeline; review and finalize data collection methods, instruments, tools, guidelines and analysis methods; review and clarify any logistical and administrative procedures for the assignment; establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; develop a preliminary draft format for the team’s report; and assign responsibilities for report writing. Interviews and site visits: Including but not limited to the following:  PHSP management and technical staff, both in Addis Ababa and field sites.  Federal MOH and Regional Health Bureau staff.  Food, Medicines Healthcare Administration and Control Agency (FMHACA).  Federal HIV/AIDS Prevention and Control Agency (FHAPCO).  Pharmaceutical Fund and Supply Agency (PFSA).  Select private providers.  Key informants at USAID.  Select donors and other stakeholders (e.g., the World Bank, professional associations, insurance companies, medical colleges). USAID/Ethiopia will provide a contact list of key informants and a draft schedule of site visits to the consultants prior to arrival in-country. VII. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORT (LOE) The five-person evaluation team should be comprised of two international (expatriate) consultants and two local Ethiopian consultants in addition to a local logistics coordinator. USAID/Ethiopia may identify a private health sector specialist from USAID/Washington or another mission to participate in the evaluation. No consulting firm costs would be associated with this sixth potential team member. Profile of Evaluation Team: 13. The team leader will be an international consultant with more than 10 years of experience in international health (Africa experience required) and experience leading at least two external performance evaluations. Specific experience designing, managing and/or evaluating private sector health programs required. Strong writing, evaluation methods and analytical skills required (a writing sample will be requested). Experience with USAID programs and PEPFAR desired. The team leader will be responsible for team coordination and performance, and for ensuring the timeliness and quality of deliverables. The consultant will hold conference calls with the other team members and USAID/Ethiopia representatives as necessary before and after the visit to Ethiopia in order to develop the evaluation methodology and take the lead in developing the evaluation report. The Team Leader is USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 45 expected to present preliminary findings of the evaluation to USAID/Ethiopia and PHSP staff prior to departure from the country. 14. Two private sector health specialists are requested (local Ethiopians), each with at least five years of experience in private health sector engagement, strategies and policy (specific areas may include social marketing, private provider networks and franchising, corporate social responsibility, public-private-partnerships, workplace programs and health financing). The consultants will have experience in program evaluation and knowledge in conducting surveys, key informant interviews and focus groups. Strong English language and writing skills required. 15. One international private sector health specialist, with at least five years of experience in private health sector engagement, strategies and policy (specific areas may include social marketing, private provider networks and franchising, corporate social responsibility, public￾private-partnerships, workplace programs and health financing). The consultants will have experience in program evaluation and knowledge in conducting surveys, key informant interviews and focus groups. Strong English language and writing skills required. 16. A local logistics coordinator will assist in the evaluation preparations and implementation, including making hotel reservations for team members; scheduling key informant interviews and focus group discussions; and organizing field visits and all associated in-country travel. The logistics coordinator should be available to start planning for the evaluation prior to the external team’s arrival in-country. Required qualifications include: minimum three years experience handling complex logistics, such as coordinating business travel and meetings; demonstrated ability to be resourceful and to successfully execute complex logistical coordination; ability to multi-task and work well in stressful environments and perform tasks independently with minimal supervision; ability to interact effectively with a broad range of internal and external partners, including international organizations, host country government officials, and NGOs; and proven ability to communicate clearly, concisely and effectively both orally and in writing. Estimated Level of Effort (LOE): The desired start date for the in-country work is no later than October 2012 and includes two to three weeks spent in-country. A six-day work week will be approved when the consultants are working in country. Weekend travel may be necessary. Below is a list of the specific tasks to be accomplished by the consultant team, with an estimated LOE and proposed timing for each task. 46 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Illustrative Activities and Estimated Level of Effort Activity Person(s) Responsible Total LOE Period of Performance (illustrative depending on start date) Mission sends background documents to GH Tech Bridge II Activity Manager USAID/Ethiopia Review documents and initial drafting of evaluation protocol and instruments. Team planning call with USAID/Ethiopia. Logistics coordinator prepares for survey All 5 Travel to country International Consultants 2 In-briefing with USAID, team planning meeting to finalize protocol, and survey tools and methodology for data collection, clarify team roles, meet with key stakeholders to finalize tools, organize logistics for field work Evaluation team Key USAID personnel 2 Conduct interviews and field work (including travel days) Evaluation team 11 Preliminary data analysis and synthesis; drafting report and presentation materials with additional follow-up meetings as needed in Addis. Debriefing of mission staff with draft findings and recommendations Evaluation team 4 Team departs country International consultants 2 Draft report writing/finalizing (Team leader=5; private health sector specialists=4) Evaluation team 5/4 Draft report submitted to Mission Evaluation team Mission sends technical feedback/comments on draft to GH Tech Bridge II Activity Manager to forward Team Leader USAID/Ethiopia Draft report revisions (Team leader=5; private health sector specialists=3) Evaluation team 5/3 Revised report submitted to Mission Evaluation team Mission approves report and sends to GH Tech Bridge II Activity Manager. USAID/Ethiopia Consulting firm provides final edited report (30 days) Total LOE = Team Leader (1) 36 Total LOE = Private Health Sector— International (1) 33 Total LOE = Private Health Sector—Local (2) 29 Total LOE = Local Logistics Consultant (1) 25 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 47 VIII. LOGISTICS USAID/Ethiopia will provide overall direction to the team and key documents and background materials. USAID/Ethiopia is unable to provide workspace for the evaluation team at the Mission except for initial in-briefing, debriefing and meetings with USAID staff. External meeting space and printing and photocopying services may be provided by the consulting firm, the implementing partner, or through local hotels and printing businesses. USAID will schedule the internal meetings and assist the logistics coordinator with scheduling meetings with key stakeholders. Abt (the implementing partner) will be responsible for assisting the logistics coordinator in scheduling site visits and for providing internal program documents. The consulting firm will provide all logistical arrangements such as flight reservations and tickets (business class is not authorized), country cable clearance, in-country travel funds, airport pick￾up, lodging and daily transportation, as appropriate. The logistics coordinator will arrange transport to field sites (number and location to be determined), assist in setting up appointments, and arranging lodging as necessary. The consulting firm will pay for these elements. IX. DELIVERABLES AND PRODUCTS Based on the above stated purpose and key tasks, the evaluation team will submit the following deliverables:  Evaluation protocol: Including detailed approach/methodology, revised evaluation questions, data collection tools, and plans for analysis and dissemination of findings. The team leader will submit the draft evaluation protocol to USAID/Ethiopia and to the consulting firm before arrival in-country. USAID/Ethiopia will then review the proposed workplan/methodology and data collection tools and submit comments to the team during the in-country team planning meeting. The evaluation team will revise the workplan/methodology and data collection tools and send the final version to USAID/Ethiopia and to the consulting firm. The final evaluation framework must be finalized and approved prior to the initiation of the interviews and site visits, at the completion of the team planning meeting.  In-briefing to USAID mission staff during the team planning meeting.  Interim briefings including status reports: The team leader will provide bi-weekly status reports on evaluation plan implementation to USAID/Ethiopia and to the consulting firm.  Debriefing to mission (and copy of the slides, handouts) to USAID mission staff before departure from country. Draft findings and recommendations must be included in the debriefing.  Draft report in English no longer than 35 pages, excluding coversheets and appendix. The report shall follow the general format indicated below: 1. Coversheet indicating the tile of the assessment 2. Table of Contents 3. Acknowledgments 4. Acronyms 48 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 5. Executive Summary (3-5 pages) 6. Background 7. Scope and Methodology 8. Findings and Discussions 9. Lessons Learned 10. Recommendations 11. References 12. Appendix (includes, but not limited to, SOW, data collection instruments, sources identified/people contacted or interviewed, disclosure of Conflict of Interest signed by the Evaluation Team Members, etc.) The findings and recommendations should address the evaluation tasks and questions. All findings and recommendations should be linked to data and information gathered and referenced in the evaluation report. The Team Leader will submit the first draft report to USAID/Ethiopia and to the consulting firm five working days after arrival back in the team lead’s home country. The Mission will provide consolidated, written comments to the evaluation team and to the evaluation contractor within 10 working days of receiving the draft report. Final draft report will address the Mission’s comments. The team leader will submit the final unedited report to USAID/Ethiopia within five working days after the team receives consolidated comments from USAID/Ethiopia. Final content approval: USAID/Ethiopia will have five business days to review the revised report and provide their approval of the final content. USAID/Ethiopia will highlight if there is any procurement sensitive information in the report so that information can later be removed from the final publishable report. The report needs to follow the standardized report format and meet the quality requirements provided by the mission before final approval will be given. Final publishable report: The consulting firm will provide the edited and formatted final document within 30 days of receiving final content approval from USAID. Procurement sensitive information will be removed from the final report and incorporated into an internal USAID Memo. The remaining report will then be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the consulting firm program Website, if applicable. X. RELATIONSHIPS AND RESPONSIBILITIES The consulting firm will coordinate and manage the evaluation team and will undertake the following specific responsibilities throughout the assignment:  Recruit and hire the evaluation team.  Make logistical arrangements and associated payments for the consultants, including travel and transportation, country travel clearance, lodging, and communications.  Edit, format and ensure overall quality of the final report from the consultants. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 49 USAID/Ethiopia will provide overall technical leadership and direction for the evaluation team throughout the assignment and will provide assistance with the following tasks: Before In-Country Work  SOW. Respond to queries about the SOW and/or the assignment at large.  Consultant Conflict of Interest (COI). To avoid conflicts of interest or the appearance of a COI, review previous employers listed on the CVs for proposed consultants and provide additional information regarding potential COI with the program contractors evaluated/assessed and information regarding their affiliates.  Documents. Identify and prioritize background materials for the consultants and provide them to the consulting firm, preferably in electronic form, at least one week prior to the inception of the assignment.  Local Consultants. Assist with identification of potential local consultants, including contact information.  Site Visit Preparations. In collaboration with the implementing partner, provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line items costs.  Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in￾country travel (i.e., car rental companies and other means of transportation). During In-Country Work  Mission point of contact. Throughout the in-country work, ensure constant availability of the point of contact person and provide technical leadership and direction for the team’s work.  Meeting space. Provide guidance on the team’s selection of a meeting space for interviews and/or focus group discussions. USAID will NOT provide meeting space in the Embassy for the consultant team.  Meeting arrangements. Assist the team in arranging and coordinating meetings with stakeholders.  Facilitate contact with implementing partners. Introduce the evaluation team to implementing partners and other stakeholders, and where applicable and appropriate prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. After In-Country Work  Timely reviews. Provide timely review of draft/final reports and approval of deliverables. XI. MISSION CONTACT PERSONS Faris Hussein, MD, MPH Health System Advisor /Private health sector USAID Ethiopia Fhussein@usaid.gov 50 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Tel 251-11-1- 306861 Fax 251-11-1-242438 XII. COST ESTIMATE The consulting firm hired for the assignment will provide a cost estimate for the activity. XIII. REFERENCES Official Websites: http://www.usaid.gov// http://www.pepfar.gov// USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 51 ANNEX B. PERSONS INTERVIEWED AND SITES VISITED No. Name Organization Position Addis Ababa City Administration 1 Ato Abdeljelil Reshad Resource Mobilization Directorate/FMOH Director 2 Samson Tekeste Addis Ababa RHB Family Health Team Leader 3 Hanna Kumssa Addis Ababa RHB Disease P&C Team Leader 4 Abraham Tesfaye Bika Addis Ababa Regional Laboratory Health Research and Laboratory Focal Person 5 Abebe Shibru Marie Stopes International Program Director 6 Dr. Birhanu Sendek Director, Quality & Clinical Services 7 Dr. Yosef Burka Trans Action Chief of Party Addis Ababa -PHSP- Head Office 1 Fidel Lakew PHSP-Addis Ababa DCOP Fin & Admin 2 Mesfin Eshetu PHSP-Addis Ababa Quality Management Director 3 Meheret Elias PHSP-Addis Ababa FP & Malaria Manager 4 Menges Hagos PHSP-Addis Ababa Pharmacy coordinator 5 Mesele Damte PHSP-Addis Ababa Malaria Program Coordinator 6 Leulseged Takele PHSP-Addis Ababa Laboratory Advisor 7 Yehualeshet Belew PHSP-Addis Ababa TB/HCT Program Manger 8 Yonas Yilma PHSP-Addis Ababa Senior Regional Manger 9 Anteneh Tsefaye PHSP-Addis Ababa ART/ PMTCT Program Manger 10 Petros Mitiku PHSP-Addis Ababa Private Health Sector Advisor for FMHACA 11 Afework Gelleta PHSP-Addis Ababa Program Director 12 Derebe Tadesse PHSP-Addis Ababa Assistance Quality Manager 13 Nichcolas Welch USAID M&E Advisor 14 Faris Hussein USAID Private Health Sector Adviser Amhara Region 1 Ali Gebeyehu Amhara Regional Health Bureau Deputy Head-Operations 2 Zebideru Zewdie Amhara Regional Health Bureau Deputy Head- Programs 52 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION No. Name Organization Position 3 Tsegaye Alebel PHSP-Bahir Dar Regional Coordinator 4 Yonas Yilma PHSP-Bahir Dar Senior Regional Program Manager 5 Habtamu Aderaw PHSP-Bahir Dar Malaria Program Officer 6 Melkie Assefa PHSP-Bahir Dar Program Officer 7 Almau Manalebh PHSP-Bahir Dar Laboratory Program Quality Officer 8 Hilawe Agizachew PHSP-Bahir Dar Clinical Mentor 9 Getachew Abebe Aflagat Hospital-Bahir Dar Laboratory Expert 10 Etsubdink Tegene Aflagat Hospital-Bahir Dar Head of VCT Room 11 Belaynesh Addisu Aflagat Hospital-Bahir Dar Head of ART and TB 12 Abebe Daganew Aflagat Hospital-Bahir Dar CEO 13 Dr. Yohannes Abate St. John Higher Medical Center Physicians 14 Hiruth Abreham St. John Higher Medical Center Nurse 15 Birtukan Adamu St. John Higher Medical Center Laboratory Technician Amhara Region 1 Daniel Mengeste Selam Medium Clinic - Gonder Nurse 2 Gashaw Fentie Selam Medium Clinic Laboratory Technologist 3 Wubshet Girma Selam Medium Clinic Nurse 4 Dr. Seife Chekol Selam Medium Clinic General Practitioner 5 Abdurazik Zeinu Nour Higher Clinic Lab Technician & Manager 6 Worku Kebede Nour Higher Clinic LAB Technician 7 Nurainy Mostofa Nour Higher Clinic Clinical Nurse Dire Dawa City Administration 1 Amsal Ashagre Dire Dawa Health Bureau Health Promotion Case Team Coordinator 2 Teklu Molie Dire Dawa Health Bureau TB-Leprosy and TB/HIV Coordinator 3 Mustefa Mohammed Dire Dawa Health Bureau Health Promotion & Disease Prevention Process Head 4 Tigist Getachew ART general Hospital-Dire Dawa Family Planning Focal Person 5 Roman Mohammed ART general Hospital-Dire Dawa TB/HIV Focal Person USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 53 No. Name Organization Position 6 Sayo Osman ART general Hospital-Dire Dawa ART & VCT Focal Person 7 Radiel Geuta ART general Hospital Medical Director 8 Hayat Seid ART general Hospital Clinical Nurse 9 Yohannes Belay Yemariamwerk Clinic-Dire Dawa TB Focal Person Harari Region 1 Shemshidin Omer Harari RHB HIV/AIDS Program Coordinator 2 Nejaha Abdosh Harari RHB Health Promotion Officer 3 Afendi Busha Harari RHB Deputy Head 4 Asefa Tufa Harari Regional Laboratory Laboratory Coordinator 5 Nagash Yehualshet PHSP-Harar and Dire Dawa Coordination office Harer & DD Regional Program Coordinator 6 Zelalam Mengistu PHSP-Harar and Dire Dawa Coordination office Program Officer 7 Tsiyon Alemayemu Yimage Medical Center - Harar Clinical nurse 8 Meron Kassahun Yimage Medical Center - Harar Clinical nurse 9 Alemnesh Dabessa Harar General Hospital Head Nurse 10 Selamawit Girma Harar General Hospital Matron 11 Sebri Abdurheman Imag Medical Center Owner Oromia Region 1 Dr. Kassa Hailu Oromia RHB TB/HIV Technical Advisor 2 Tsegaye Tesfaye Oromia RHB TB/HIV Program Expert 3 Dr. Zelalem Habtamu Oromia RHB Health Care Delivery Manager 4 Mekonene Mengesha Hora Higher Clinic TB Focal Person 5 Dr. Tegene Ayalneh Hora Higher Clinic Medical Director 6 Tadesse Legesse Hora Higher Clinic Head of Laboratory Services 7 Lidet Abera Hora Higher Clinic Laboratory Technician 8 Samuel Bekele Hora Higher Clinic Laboratory Technician 9 Chaltu Aberra Hora Higher Clinic Family Planning Focal Person 10 Chaltu Temesgen Hora Higher Clinic Family Planning Client 11 Dr. Tedla Mekonnen Tedela Medium Clinic Medical Director 12 Fantu Hassen Tedela Medium Clinic TB/HIV Coordinator 13 Sissay wolde Tedela Medium Clinic Laboratory Technician 54 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION No. Name Organization Position 14 Henok Tadese Tedela Medium Clinic Laboratory Technician 15 Meron Gezahegn Tedela Medium Clinic Customer Tigray Region 1 Yalem Tsegay Tigray-RHB Deputy Health of RHB 2 Zemenzeskidur Hangu Tigray-RHB Regulatory Core Process 3 Tekaly W/mariam Tigray-RHB Regulatory Core Process Case Team Coordinator 4 Berihu Mesfin Tigray-RHB Curative and Rehab. Core Process 5 Kahsu Iyasu PHSP-Tigray Coordination Office M&E Manager 6 Hasazi Amare PHSP-Tigray Coordination Office Program Officer 7 Selamawit Abraham PHSP-Tigray Coordination Office Nurse Mentor 8 Birhane Meressa PHSP-Tigray Coordination Office Pharmacy Mentor 9 Zeru G/medium PHSP-Tigray Coordination Office Program Coordinator 10 Almaz Mesfin Meskerem General Hospital-Mekele CEO 11 Kidanu Estifanos Meskerem General Hospital-Mekele Medical Director 12 Solomon Ghidey Meskerem General Hospital-Mekele ART Focal Person 13 Tesfay Assefa Meskerem General Hospital-Mekele TB Focal Person 14 Asmamaw Mirutse Kidus Higher Clinic-Mekele Medical Director 15 Shumeye Abreha Kidus Higher Clinic-Mekele TB Focal person 16 Berehanu Hayelom Kidus Higher Clinic-Mekele Laboratory Technician USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 55 ANNEX C. REFERENCES GENERAL Private Sector Health Policy Brief. PHSP, “PEPFAR Ethiopia In-country Reporting System (IRS): Annual Program Results, October, 2010–2011.” PHSP, Rapid Assessment of Involving Lower Clinics and Drug Vendors in Tuberculosis Suspect Identification and Referral, August 2011 PHSP, USAID, “Private Sector Health Program Performance Monitoring Plan,” October 2011. Farrell, Janet. Trip Report, October 2011. GH Tech, “Final Evaluation: The Private Sector Program in Ethiopia,” July 2009. PHSP, Associate Award Agreement, 2009. “Ethiopia Demographic Health Survey 2011,” March 2012. LABORATORY PHSP, “Performance of HIV Testing and Sputum ZN Microscopy in Private For-profit Health Facilities in Ethiopia (EQA).” PHSP, Meeting Minutes, 2/16/2012. PHSP, Meeting Minutes, 2/22/2012. PHSP, SWOT Analysis of Private. PHSP, AFB Internal Quality Control Log. PHSP, Meeting Minutes, 11/30/2011. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for St. Gebreal Hospital,” May 02-07 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Baruk Diagnostic Laboratory,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Girum Hospital,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Sr.Aklesiya Hospital,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Teklehymanot Hospital,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Adama General Hospital and Medical College,” May 2–7 2012. 56 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Medical Biotech Diagnostic Laboratory,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Hema Diagnostic laboratory,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Betel Teaching Hospital,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for Arsho Diagnostic Laboratory,” May 2–7 2012. PHSP, “Stepwise Laboratory Improvement Process Towards Accreditation (SLIPTA) for MCM Hospital,” May 2–7 2012. Procedure for CD4 Count Sample Transporter (Amharic). Venipuncture Procedures For CD4 Counts Sample Collection. HIV-1 Rapid Test, HIV-1/2-Stat-pak HIV-1 Rapid Test, UNI-Gold Antibody (Colloidal Gold)-KHB Rapid Test Procedure in CASE of Spill of Specimens in an AFB Microscopy Laboratory Smear Preparation Use of Protective Equipment in an AFB Microscopy Laboratory Sputum Collection, Handling, and Transportation. SOP of Waste Management PHSP, “Proceedings of the First National Medical Laboratory Week,” August 2012. PHSP, “Baseline Laboratory Capacity Assessment of Private and Non-governmental Health Facilities for Initiation of ART Services,” May 2011 PHSP, “External Quality Assessment Feedback on TB and HIV Laboratory Diagnosis in West Amhara Private Health Facility,” August 2012. TRAINING PHSP, Forms. PHSP, Training materials. PHSP, Training Plan. PHSP, Malaria Prevention and Control Training Module, November 2011. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 57 POLICY PHSP, “Service Under the Categories of the Ambulatory Service Standards.” PHSP, “Consolidated Training Report—Business Planning: Informed Decision-making for a Successful Future,” April–June, 2012. PHSP, “Engaging with the Private Health Sector in Africa Dar Es Salaam, Tanzania,” May 2012. PHSP, “Business Planning: Informed Decision-making for a Successful Future,” Executive Summary excerpt, June 2012. PHSP, PPM DOTS Implementation Guidelines, June 2012. PHSP, “Rapid Assessment of the Quality of Pre-service Education in Private Medical Colleges in Ethiopia,” August 2012. PHSP, “Regulatory Standard Setting and Information Delivery Directorate, FMHACA,” May 2011. PHSP, “Proceedings of the Regional Consultative Workshop on Clinics, Specialty Centers, Diagnostic Health Facilities Regulatory Draft Standards,” June 2, 2011. PHSP, International Experience Sharing Visit to Jordan, FMHACA in collaboration with PHSP, February 14, 2011. PHSP, Concept Note on PHSP Support in Licensure and Accreditation Process. PHSP, Brief Note on Roll-out of ART Program to Private Higher Clinics. Health Facility Regulatory Standards, Executive Summary, developed by Ethiopian FMHACA, September 2011. FAMILY PLANNING PHSP, Monthly Activity Plan and/or Achievements Template. PHSP, Reports of Family Planning Activities, May and June 2012. PHSP, Private Health Sector Program FP Implementation Strategy, July 2011. PHSP, “Family Planning Facility Readiness Assessment Report of SNNPR Facilities,” May 2012. PHSP, “Family Planning Facility Readiness Assessment Report of SNNPR Facilities,” May 2011. PHSP, To conduct quality comprehensive FP training with emphasis long acting FP methods, June 8, 2012. PHSP, Enhance the capacity of private sectors clinical service providers on provision of quality, February 20, 2012. PHSP, Enhance the capacity of private sectors clinical service providers on provision of quality, February 10, 2012. PHSP, To conduct FP service initiation in the selected PHSP Tigray region, 19/07/2012. PHSP, Conduct consensus building on FP/STI &Malaria program Tigray RHB, 10/02/2012. 58 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PHSP, Conduct consensus building on FP/STI &Malaria program Amhara RHB, 26/03/2012. PHSP, “PMTCT and FP Readiness Assessment Report of East Amhara Private Health Facility,” May 2012. PHSP, Training on Integrated Pharmaceutical Logistics System (IPLS). Pre-course and Post-course Knowledge Assessment for Training on Comprehensive FP. Long-acting Family Planning Methods Participant’s Handout, 2011. Checklist and Guideline of Family Planning. MALARIA Controlling Malaria Together (Amharic). Controlling Malaria Together (Tigrena). World Malaria Day (Amharina and Tigrena). PHSP, “Facility Readiness Assessment of the Private Health Sector to Implement Malaria Service (Amhara, SNNPR and Tigray region), Ethiopia,” August 2012. PHSP, Terms of Reference of the Celebration of World Malaria Day (WMD) 2012. PHSP, WMD 2012 Overall Coordinating Committee Meeting Minutes, No. 1, 2012. PHSP, WMD 2012 Theme: “Sustain Gains. Save Lives. Invest in Malaria,” April 25, 2012. PHSP, Malaria Control Technical Advisory Committee (TAC) Meeting Minutes, No. 1, January 24, 2012. PHSP, WMD 2012 Overall Coordinating Committee Meeting Minutes, No. 5, April 17, 2012. PHSP, WMD 2012 Overall Coordinating Committee Meeting Minutes, No. 3, April 3, 2012. PHSP, WMD 2012 Overall Coordinating Committee Meeting Minutes, No. 4, April 10, 2012. PHSP, “Facilitate Malaria Case Management Training for Health Professionals Working in Private Health Facilities,” 8/13/2012. PHSP, “Annual Performance Report, 2011/12.” PHSP, List of Facilities Supported in Malaria (SNNPR, Amhara and Tigray). PHSP, “Training of Trainer on Malaria Case Management Report,” December 12-14, 2011. Malaria Recording and Reporting Form. PHSP, Rapid facility readiness assessment result Malaria care service. PHSP, Schedule for WMD 2012 Celebration in Assosa town, Benshangule Gumez region, April 25, 2012. PHSP, Progress Update on Preparation of WMD 2012 Events. PHSP, Assessment Report on WMD 2012 Preparation to be Hosted by Benshangual Gumuzi Regional State, Asossa, March 18-23, 2012. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 59 PHSP, Malaria Control Technical Advisory Committee (TAC) Meeting Minutes, No. 4, February 29, 2012. PHSP, “Plan for Introduction of Rectal Artesunate Suppository and Artesunate Injections in Ethiopia,” (Draft), April 2012. PHSP, “PHSP Ethiopia Quarterly Internal Narrative Report of Malaria Program,” (Amhara Tigray and SNNPR). PHSP, “PHSP Ethiopia Quarterly Internal Narrative Report of Malaria Program,” (Amhara and Tigray). PHSP, Flow Chart for Diagnosis and Treatment of Malaria (Medium/Higher clinic and hospital). PHSP, Pediatric and Adult Anti-malaria Drug Dosage. The Glasgow Coma Scale for Adults and Older Children. PHSP, “Important Malaria Protection Manual,” (in Amharic). PHSP, “Conduct Strengthening Professional Practice and Legal Framework Enforcement for the Control of Modern Medicine, Amhara RHB,” 4/05/2012. PHSP, “Conduct Consensus Building On Family Planning/STI and Malaria, Amhara RHB,” 3/26/2012. PHSP, “PHSP Ethiopia Quarterly Internal Narrative Report of Malaria Program.” PHSP, Conduct Advocacy Workshop on Rational Drug Use and Strengthening Professional Practice of Drug Outlets, 8/13/2012. PHSP, Initiate Malaria Care Services in 20 Private Health Facilities of Amahara Region, 7/13/2012. PHSP, Attend Malaria Care Services Launching Event, Bahir Dar, Amhara Region, 6/10/2012. PHSP, Attend Malaria Data Management Training Using Arcview Software (Gis) Visiting Three PHF, 5/23/2012. PHARMACY PHSP, Art Drug Reconciliation Reporting Form for PHSP Supported Hospitals and NGO Clinics, June 2012. PHSP, Routine Feedback Reporting Format. PHSP, To Coordinate IPLS Training, Conducted in Adama, August 28–30, 2012. PHSP, To Coordinate IPLS Training, Conducted in Adama, April 21–23, 2012. PHSP, To Coordinate IPLS Training, Conducted in Mekele, May 11–13, 2012. PHSP, To Coordinate IPLS Training, Conducted in Kombolcha, July 12–14, 2012. PHSP, Disposal Communities Procedure Letter. PHSP, Integrated Pharmaceutical Logistics System (IPLS) Training in Amhara, Dire Dawa, and Harari and Seminar Invitation in Tigray, October 12, 2011. PHSP, To Coordinate IPLS Training, Conducted in Adama, February 27 –29, 2012. 60 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Ashenafi, Desta, Gulilat Teshome and Sami Tewfik, “Technical Report on Integrated Pharmaceutical Logistics System (IPLS) Training to Private Health Facility,” October 27–29, 2011. PHSP, Antiretroviral Treatment (ART) In-service Training for Pharmacists Pre-test Assessment, 10/02/2012. PHSP, Pharmaceuticals Logistics System in Health Facilities of Ethiopia (SOPM), June 2012. PHSP, Participant Course Workbook for the Integrated Pharmaceutical Logistics System in Ethiopia, June 2012. PHSP, Item Description. QUALITY MANAGEMENT PHSP, “Using Mobile Phones for Data Collecting Health Data: Experience of the PHSP in Ethiopia.” PHSP and Derebe Tadesse, “Trip Report-Sharing Experience of USAID|PHSP on Using Mobile Phone Technology for Collecting Data, Nairobi-Kenya,” December 2, 2011. PHSP, PHSP Year 3 Workplan (October 2011–September 2012), September 2011. PHSP, Annex B—Year 3 Workplan. PHSP, Annual Program Results (APR) (Oct. 2009–Sept. 2010) by PEPFAR Ethiopia In-country Reporting System (IRS) Reporting Template. PHSP, Annual Program Results (APR) (Oct. 2010 – Sept. 2011) By PEPFAR Ethiopia In-country Reporting System (IRS) Reporting Template. PHSP, Quarterly Supportive Supervision and Continuous Quality Improvement (CQI) Tool. PHSP, PHSP Year 1 Workplan (October 1, 2009–September 30, 2010). PHSP, PHSP Year 1 Workplan (October 1, 2009–September 30, 2010) with Activities, Indicators and Timeframe. PHSP, DQA Tools for PHSP, Draft, PHSP/Mesfin Eshetu (October 26, 2011). PHSP, PHSP Year 2 Workplan (October 1, 2010–September 30, 2011) with Activities, Indicators, Timeframe with Estimated Budget. TUBERCULOSIS Job Aids PHSP, Diagnostic Flow Chart of Pulmonary Tuberculosis for HIV Negative Patients. PHSP, Flow Chart for Follow-up on New Smear-positive Pulmonary TB Patients. PHSP, Algorithm for the Diagnosis of TB in Ambulatory HIV Positive Patients. PHSP, Algorithm for the Diagnosis of TB in Seriously Ill HIV Positive Patients PHSP, Management of Patients who are Initially Smear Positive and Interrupted TB Treatment for Less than 8 Consecutive Weeks. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 61 PHSP, Case Definitions. PHSP, Adult Anti-TB Drug Dosage. PHSP, Pediatric Anti-TB Drug Dosage. PHSP, WHO ART Treatment Supporters. PHSP, MDRTB Suspect Identification, Evaluation and Referral Procedure in Ethiopia, 2011. PHSP and Temesgen Assefa, Referring TB Patients from Private to Public Health Facilities in Ethiopia. PHSP with Yonas Yilma,?, Abenet Laykun and Derebe Tadesse, “Strengthening Partnership and Referrals: Connecting Public and Private TB-DOTS Health Facilities In Ethiopia.” PHSP with Temesgen Assefa, Abnet Laykun and Mohamed Dawd, “Acceptability of PITC Among Tuberculosis Patients in Private Health Facilities in Amhara Region, Ethiopia.” PHSP with Hands On Research &Training PLC, “Facility Readiness Assessment Expansion of TB-DOTS and HIV Counseling and Testing Services in Private Health Facilities,” May 2011. PHSP with Seyfu Abebe and Alemthehay Berhanu, “Baseline Assessment of Tuberculosis Medication and Laboratory Supplies Management at Selected PPM-DOTS Sites,” October 2010. PHSP with Alemayehu Deressa, “Eqa in Oromia Region; Panel Test (Proficiency Test) and Onsite Supportive Supervision For the 11 Mobile VCT Counselors In Nekemte Town in Two Sites,” January 16,2012. PHSP, Letters About Unwanted or Expired Pharmaceuticals Disposal. PHSP, Pamphlet About TB Treatment (In Amharic,Tigrigna, Oromiffa,?). PHSP, “Tuberculosis Fact Sheet, 2012 World TB Day” (In English, Amharic,Oromiffa). PHSP, Poster (2012 World Tb Day) (In Amharic, Tigrigna Oromiffa,?). PHSP with Derebe Tadesse, Yonas Yilma and Semunegus Mehrete, “Trip Report on Supportive Supervision and Review of Quality Management Practices in West Oromia TB/HIV Facilities,” January 2011. PHSP, “Annual Accomplishments of the Primary Health Sector Program in Addis Ababa City Administration,” Hamle 1, 2002–Sene 30, 2003” PHSP, Advocacy Workshop to Strengthen TB/HIV Service Related Referral Linkages Among Private and Public Health Facilities in the Eastern Regions of Ethiopia (Somali, DD, Harari And East Oromia). PHSP with Tadesse Ligidi, “EQA of AFB Microscopy and Rapid HIV Testing in Health Institutions (3rd Quarter, 2003 E.C.)” PHSP, “Annual Program Performance Report in Harari Region (2003 Efy),” August 2011. PHSP, “Annual Program Performance Report In Dire Dawa Region (2003 Efy),” August 2011. PHSP, Annual Program Performance Report In Amhara Region (2003 Efy),” October 2011. PHSP with Demeke Ashagre, “A Report On EQA For AFB Smear Microscopy And HIV Rapid Testing of Private Health Facilities in Snnpr,” October 2011. 62 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PHSP with Bayissa Bekele, “EQA of AFB Microscopy and Rapid HIV Testing in Health Institutions (1st Quarter, 2004 E.C). PHSP with Yehualaeshaet Bekele, Asfawesen G/Yohanes, Tesfai G/Kidan, “EQA of AFB Microscopy in Health Institutions (3rd Quarter, 2004 E.C).” “Contribution of Private Health Sector to the National TB Program in Ethiopia; Five Year Retrospective Analysis,” July 2012. PHSP With Yehualaeshaet Bekele, Asfawesen G/Yohanes, Tesfai G/Kidan, “PPM: Increasing Access and Quality of TB Care In Ethiopia.” PHSP, “Tuberculosis Public Private Mix Dot Short-course: Facility Support Transition Guideline,” August 2012. PHSP, Addis Ababa City Administration Review Meeting, September 25, 2011. PHSP with Yonas Yilma,Temesgen Assefa, “Trip Report, Participation and Poster Presentation at Union World TB Conference 2010,” January 23,2011. Facility Readiness Assessment Expansion of TB-DOTS and HIV Counseling and Testing Services in Private Health Facilities, June 2012. Justification For Revising The PPM-Dots Implementation Guideline, May 2012. PHSP, “TB-DOTS Referral Directory, Amhara Region.” PHSP with the A/A Health Bureau, “Joint HCT Cost Assessment Report of Private Health Facilities Providing HCT Service In Addis Ababa,” May 20, 2010. PHSP, “Clinical Seminar Report (May–June 2012),” May-June 2012. FMOH, “PPM DOTS Implementation Guideline,” (Draft), June 2012. FMOH, “Yearly Magazine, World TB Day 2011.” PHSP, “Rapid Assessment on Involving Lower Clinics and Drug Vendors in Tuberculosis Suspect Identification and Referral,” August 2011. PREVENTION OF MOTHER-TO-CHILD TRANSMISSION AND ANTIRETROVIRALS PHSP with Anteneh Tsegaye, “Expansion of PMTCT Of HIV Services to Private Higher Clinics: PHSP Strategy and Implementation Plan,” September 2010. “Expansion of HIV Care and Treatment/ART Service to Private Higher Clinics: Implementation Approach/Strategy,” (Draft), August 2010. PHSP with Parsa Sanjana, Anteneh Tsegaye and Tekleab Zaid, “Second Assessment of Readiness of Private Higher Clinics in Addis Ababa to Initiate Art Services,” April 2009. PHSP, “PHSP’s Community Care Model-communication for Improving PMTCT Outcomes.” PHSP, “PMTCT: Missed Opportunities and PHSP’s Care Mode,” April 2011. USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 63 PHSP, “Expansion Of HIV Care and Treatment (ART) Services to Private Higher Clinics in Addis Ababa,” May 2012. PHSP, “Expansion of HIV Care and Treatment (ART) Services to Private Facilities in Amhara,” May 2012. PHSP, “Expansion of HIV Care And Treatment (ART) Services to Private Facilities in Tigray,” May 2012. PHSP, “Art Rollout to Private Health Facilities (Brief Update),” December 22, 2010. PHSP, “Art Rollout to Private Facilities,” November 19, 2010. PHSP, “Art Rollout to Private Facilities,” November 4, 2010. PHSP, “Art Rollout to Higher Clinics,” October 8, 2010. PHSP, “Brief Note Regarding Expansion of ART Service to Private Higher Clinics.” PHSP, “Key Public Health Services Provided at PHSP Supported NGO Clinics.” PHSP, “Concept on Expansion of HIV Care and Treatment Services to Private Higher Clinics,” May 2009. PHSP, “ARV Drug Dispensing Practice in Private Higher Clinics.” HCT PHSP, Abt Associates Inc. Professional Service Agreement with ‘Arada Giorgis Higher Clinic’ for Mobile HCT. PHSP and Le Monde Health and Development Consultancy PLC, “Identifying Most-at-risk Population and HIV/AIDS Referral Services: Baseline Assessment for Mobile HIV Counseling and Testing Program in Tigray National Regional State of Ethiopia,” May 2011. PHSP and Le Monde Health and Development Consultancy PLC, “Identifying Most-at-risk Population and HIV/AIDS Referral Services: Baseline Assessment for Mobile HIV Counseling and Testing Program in SNNP Regional State of Ethiopia,” May 2011. PHSP and Le Monde Health and Development Consultancy PLC, “Identifying Most-at-Risk Population and HIV/AIDS Referral Services: Baseline Assessment for Mobile HIV Counseling and Testing Program in Oromia Regional State of Ethiopia,” April 2011. PHSP with Mohamed Dawd, “Women Promoters: Mobilizing Female Clients to Seek Mobile HCT,” June 2012. PHSP, Job Aids for Rapid HIV Testing. PHSP, Job Aids Finger Prick; Finger Preparation and Blood Collection. PHSP, “Analysis of VCT Data from Four Private Health Facilities Supported by PHSP In Addis Ababa City Administration,” June 2012. PHSP, Mobile Counseling and Testing Reporting Format, July 2010. PHSP with Abnet Takele,Tesfai G/Kidan,Semunegus Mihrete. “Targeting Ever-married as a High Risk Group with Mobile HCT.” 64 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION PHSP With Derebe Tadesse, Yehualeshet Bekele, Mohamed Dawd, Abye Shewarega, Tesfai G/Kidan, Semunegus Mihrete, Michail Biru,Bizuayehu Aklilu,Azeb Nigussie “Assessment of Client Satisfaction with Mobile HCT Services.” PHSP with Yehualeshet Bekele,Tesfai G/Kidan,Mohamed Dawd, Semunegus Mihrete, “Role of the Private Health Sector in HIV Counseling And Testing In Ethiopia.” PHSP with Yonas Yilma, Tesfayi G/Kidan, Asfawesen G/Yohannes, “Mobile HIV Counseling and Testing: Reaching Out of Key At-risk Population In Ethiopia.” Sex-Specific Determinants of HIV Prevalence Among Mobile HCT Clients. PHSP with Derebe Tadesse,Tesfig/Kidan, Asfawesen G/Yohannes, Mesfin Eshetu, Mohamed Dawd, Eillin Pierce, “Trend In HIV Prevalence Among High-risk Women Counseled and Tested at Mobile Testing Sites In Ethiopia (2007–2010).” PHSP With Daniel Alemu, “Report HCT Service Provision at Muger Cement Enterprise.” PHSP, “Mobile HCT Service Report From 5th Higher Education Institute’s Annual Sport Festival at Ambo University.” PHSP, VCT Promotion Brochures (In Tigrigna, Oromiffa, Amharic). USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 65 ANNEX D. INTERVIEW QUESTIONS REPORT FORMAT 1. Coversheet indicating the tile of the assessment (do we have a boilerplate format for this?) 2. Table of Contents 3. Acknowledgments 4. Acronyms 5. Executive Summary (3-5 pages) 6. Background 7. Scope and Methodology 8. Findings and Discussions 9. Here’s where we need to decide on how to cut the cake. – Policy – Quality – Demand – Access POLICY QUESTIONS MOHF 1. How do you see the role of public sector in providing health care in Ethiopia? – Do you have suggestions for how they can complement one another? – How do you see the private sector contribution to MDG/HSDP? – What are your recent changes in policies relative to the private sector? – What are your current health care priorities regarding the private sector? – How do you see the role of the development partners in the private sector? – Do you see any gaps at the regional/zonal/district levels? – How you develop and ensure regulatory standards? – Reference annual program report p30 2010/2011. – What resources do you have (staff &???)? – What is the structure in place? – How do you disseminate internal/external standards 66 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 2. Why is there no advocate for private sector in the public sector? Association of Private Physicians 1. How do you assist/ensure networking among private physicians? 2. How do you create an enabling environment? GEOGRAPHIC AND FINANCIAL ACCESS 1. How are you ensuring the private sector gets access to basic commodities? 2. What are constraints to private sector access (physician, entering supply chain, cost)? 3. How do you anticipate commodity needs? 4. How do you monitor/track central and regional stores? 5. Do you see any differences in consumption of commodities? (FMAPCO) 6. How involve private sector? What support you give the private sector 7. How do you work with Abt? What is the added value of Abt? 8. EHNRI? Ask Abt. 9. How increase commodities to private sector and how complement Abt/MOHF? What gaps exist, areas of collaboration? [DELIVER] 10. How does the private sector contribute to access? [RHB] SUSTAINED INCREASED QUALITY 1. How are you increasing the capacity of the private sector? 2. Competency, facilities, commodities, equipment, management, staffing, materials, training, in-service. [MOHF, RHB] 3. EHNRI ? Abt Laboratory QA 4. [RHB] 5. How do you licensing of private providers? 6. (Hospitals, clinics, labs, pharmacies) 7. Supportive supervision (how often?, methodology, what do you do with the information) 8. Do you provide training? What kind? Gaps? 9. Can you describe the referral system? How does it works? 10. What kind of support do you get from the MOHF? USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 67 DEMAND 1. Is there advertising? INTERVIEW CATEGORIES Central Government MOHF State Minister FMHACA PFSA FMAPCO EHNRI Non-governmental Marie Stopes International Association of Private Practitioners 68 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 69 ANNEX E. NATIONAL AND REGIONAL INDICATORS BY RESULT THROUGH YEAR THREE Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Result 2: Access- Increase access to public health service packages through assistance to expand TB, malaria, HIV/AIDS including STI and ARV, and FP/RH services in private health clinics and to increase sustained access and affordability to Ethiopians including mobile and workplace services for at risk populations Total number of private clinics with standard service packages (TB, HIV/AIDS, Malaria, FP services) which receive comprehensive PHSP support (training, supervision, lab EQA) TB/HCT Numerator:- Total number of private clinics with standard TB/HCT service package which receive comprehensive PHSP support Denominator:- N/A 106 34 36 61 59 65 65 ART Numerator:- Total number of private clinics with standard ART service package which receive comprehensive PHSP support Denominator:- N/A 0 0 0 0 0 10 10 Facilities with TB￾CT and Malaria (may also have FP/STI Numerator:- Total number of private clinics with standard TB-CT and Malaria service packages which receive comprehensive PHSP support Denominator:- N/A 0 0 0 0 22 22 70 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Facilities with TB￾CT and FP/STI (may also have malaria) Numerator:- Total number of private clinics with standard TB-CT and FP/STI service packages which receive comprehensive PHSP support Denominator:- N/A 0 0 0 0 0 22 22 Number of clients who received HIV counseling and testing in the private for-profit clinics, workplaces and university clinics Numerator: Number of clients who received HIV counseling and testing through TB/HCT sites. Denominator: N/A 59,263 34,53 4 31,611 0 37,654 65,977 50,037 Number of clients who receive counseling and testing services through MCT sites Numerator: Number of clients who receive counseling and testing services through MCT sites Denominator: N/A 53,586 19,11 7 18,404 0 0 0 0 % of clients who receive counseling and testing services in MCT sites who fit into one or more definition of at-risk population Numerator: Number of clients who received HIV counseling and testing through MCT sites who fit into one or more definition of at risk population. Denominator: Total number of clients who received HIV counseling & testing through MCT sites. 46% 50% 58% 0% 0% 0 0 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 71 Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan % of MCT clients who received HIV test result that are female* Numerator: Number of female MCT clients who received HIV test result Denominator: Total number of MCT clients who received HIV test result 35% 40% 41% 0% 0% 0 0 Number of TB patients diagnosed and treated in private, and workplace clinics Numerator: Number of TB patients diagnosed and treated in private, and workplace clinics Denominator: N/A 3971 3971 961 0 996 1319 1083 Number of TB patients diagnosed and referred from private facilities Numerator: Number of TB patients diagnosed and referred Denominator: N/A 8316 8316 6798 7058 6939 4863 6818 % of TB patients who had an HIV test result recorded in the TB register Numerator: Number of TB patients who had an HIV test result recorded in the TB register Denominator: Total number of TB patients diagnosed and treated 50% 70% 47% 80% 47% 100% 56% Number of HIV￾positive patients enrolled for pre￾ART/chronic care at private clinics Numerator: Number of HIV+ patients enrolled for pre-ART/chronic care Denominator: N/A 0 0 0 0 0 680 378 Number of HIV￾positive patients enrolled for ART (ever started) at private clinics Numerator: Number of HIV+ patients enrolled for ART (ever started) Denominator: N/A 0 0 0 0 0 400 223 Number of new modern FP acceptors (new for the facility) Numerator: Number of new modern FP acceptors Denominator: N/A 0 0 0 0 0 3761 16 72 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Number of revisit FP acceptors (second visit and above) Numerator: Number of revisit FP acceptors Denominator: N/A 0 0 0 0 0 Number of clients diagnosed and treated at private STI clinics according to national standards Numerator: Number of clients diagnosed and treated at private STI clinics Denominator: N/A 0 0 0 0 0 0 0 Number of malaria patients diagnosed and treated in private health facilities Numerator: Number of malaria patients diagnosed and treated in private health facilities Denominator: N/A 0 0 0 0 3300 11335 Result 3: Quality- improve the quality of health services and prevention interventions delivered at private health clinics, mobile and workplace settings; establish sustainable mechanisms for quality assurance/quality improvement in private health clinics and assist private educational institutions/GoE steward quality in pre-service nursing, laboratory and pharmaceutical programs. Number of private for-profit clinics, work place and university clinics receiving supportive supervision support for standard service packages (TB, HIV/AIDS, Malaria, FP/RH services) Numerator: Number of private for-profit clinics, work place and university clinics receiving supportive supervision support for standard service packages (TB, HIV/AIDS, Malaria, FP/RH services) Denominator: N/A 159 45 36 61 61 73 73 Number of private sector laboratories with testing kits and laboratory reagents with no stock-out in the past 3 months Numerator: Number of private sector laboratories with testing kits and laboratory reagents with no stock￾out in the past 3 months Denominator: N/A 34 36 61 73 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 73 Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Number of health workers (per category) trained by PHSP to deliver TB, malaria, RH/FP/STI and ART services according to national standards Comprehensive malaria /AFB/HIV for lab tech Numerator: number of lab technicians trained on Comprehensive malaria /AFB /HIV 41 69 46 AFB and rapid HIV Testing Numerator: Number of Lab Technicians trained on AFB and rapid HIV Testing Denominator: N/A 8 65 TB, TB/HIV, PICT Numerator: Number of health workers trained on TB, TB/HIV, PICT Denominator: N/A 25 106 56 VCT/PICT Numerator: Number of health workers trained on VCT/PITC Denominator: N/A 20 22 29 ART Numerator: Number of health workers trained on ART Denominator: N/A 38 33 FP/RH/STI Numerator: Number of health workers trained on FP/RH/STI Denominator: N/A 24 SPSS training for PHSP staff 1 74 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan PMTCT training for health care providers 52 Malaria case management TOT 5 Malaria case management training 43 DBS and rapid HIV test training for health care providers Hematology , clinical chemistry and CD4 14 Laboratory Quality management training Strengthening lab management to accreditation 1 Financial Management Numerator: Number of health workers trained on Financial Management Denominator: N/A 19 HR Numerator: Number of health workers trained on HR enominator: N/A 33 TOT for business planning for clinic owners Business planning training 26 IPLS training 89 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION 75 Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Number of participants who attend Inspection of Health facility standards 37 2 ART and PMTCT Data management training 13 TOT on Assessment Principles for Health Care Facility Surveyors/ Inspectors FP Couple Years of Protection (CYP) among FP acceptors by type of methods (long term methods) Number of contraceptives distributed within a program year, by type, multiplied by the average length of time they are effective. 0 0 0 0 % of ART patients alive and on treatment 12 months after initiation of antiretroviral therapy Numerator: Number of ART patients alive and on treatment 12 months after initiation of antiretroviral therapy Denominator: Total number of ART patients who start antiretroviral therapy 0 na na na na Number of integrated program (TB, CT and FP/STI) review meetings organized at regional level Numerator: Number of integrated program review meetings organized at regional level Denominator: N/A 3 2 4 10 76 USAID/ETHIOPIA: PRIVATE HEALTH SECTOR PROGRAM MID-TERM EVALUATION Indicators Definition of Indicators Benchmark/ Secondary Data Year 1 Plan Year 1 Performance Year 2 Plan Year 2 Performance Year 3 Plan Year 3 Performance Year 4 Plan Year 5 Plan Result 4: Demand- Improve demand for quality health service delivery through client education and establishment of functional referral systems to other health providers, both public and private. Number of attendants of National and Regional consensus building/sensitizati on workshops organized that informs public/private partnerships and networking Numerator: Number of attendants of National & Regional consensus building/sensitization workshops Denominator: N/A 54 0 400 468 Number of attendants of private/public referral advocacy workshops organized at regional levels Numerator: Number of attendants of private/public referral advocacy workshops organized at regional levels Denominator: N/A 303 0 2,280 388 For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge II Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com