FINAL EVALUATION: THE PRIVATE SECTOR PROGRAM IN ETHIOPIA July 2009 This publication was produced for review by the United States Agency for International Development. It was prepared by William Emmet and Christine Whalen through the Global Health Technical Assistance Project. Front Cover Photograph: HIV/AIDS Counselor preparing for work at a Private Sector Program Mobile Counseling and Testing Camp, Mojio, Oromiya Region, Ethiopia. Credit: Private Sector Program–Ethiopia, July 2007. FINAL EVALUATION: THE PRIVATE SECTOR PROGRAM IN ETHIOPIA ASSESSING THE FOUNDATION FOR A CONTINUUM FOCUSED ON PUBLIC-PRIVATE HEALTH SECTOR COLLABORATION IN ADDRESSING HIV/AIDS AND TUBERCULOSIS IN ETHIOPIA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 09–001–163) is available in printed or online versions. Online documents can be found in the Global Health Technical Assistance Project Web site library at www.ghtechproject.com/resources.aspx. Documents are also available via the Development Experience Clearinghouse (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521–1900 Fax: (202) 521–1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS–I–00–05–00005–00. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA i ACKNOWLEDGMENTS The Private Sector Program–Ethiopia (PSP–E) Evaluation Team thanks the United States Agency for International Development in Ethiopia for commissioning this evaluation of the Abt Private Sector Program Task Order and for selecting us to undertake this challenging task. The time, patience, goodwill, and depth of knowledge of the more than 85 persons interviewed as part of this evaluation made it possible for the team to understand the significance of the many and diverse reports and technical interventions associated with PSP–E’s multiple technical initiatives. The willingness of PSP–E’s staff to provide us with a full array of documentation and to share with us their thoughts on the project’s development and their perspectives on the present and the future was of primary importance in helping us to understand the complex issues involved in meeting PSP–E objectives. PSP–E staff’s patience and good humor in fully responding to our increasingly intense requests for clarification and information throughout the four weeks of this evaluation allowed us to feel less guilty as our inquiries multiplied. The importance of the contributions of U.S. Government officials in USAID/Ethiopia, most especially that of Mr. Brad Corner, PSP–E’s Contracting Officer Technical Representative, and Ms. Sophia Brewer, to this evaluation, in terms of framing its technical methodology and contributing to its findings, cannot be overstated. Finally, we would like to acknowledge our appreciation for the time set aside by the many representatives of Ethiopia’s stakeholder community to meet with us and to discuss their vision for the future of private-public sector collaboration in responding to the challenges of combating tuberculosis and HIV/AIDS in Ethiopia. Information provided by these host country counterparts enabled us to cut through superlatives and to focus on the reality. As a final note, we express our thanks and appreciation to the entire staff of the PSP–E for assisting us with the many logistical and administrative details associated with this evaluation. While many PSP–E staff contributed to this effort, we would like to single out Ms. Kathleen Poer, PSP–E Chief of Party, and Mr. Abenet Leykun, Deputy Chief of Party, for special notice and to thank them for their essential role in scheduling our interviews and in making certain that we did not forget to show up at the appointed hours. Finally, we express our appreciation to the staff of the Global Health Technical Assistance Project, most especially Ms. Tiera Kendle, for the administrative support they gave us throughout this interesting and challenging assignment. ii THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA iii ACRONYMS AA Addis Ababa AFB acid-fast bacilli AIDS acquired immune deficiency syndrome ALERT Africa Leprosy Rehabilitation and Training Institute APCT HIV/AIDS/STI Prevention and Control Team ARC AIDS Resource Center ART antiretroviral treatment CBOs community-based organizations CCM Country Coordinating Mechanism CD case detection CDR case detection rate COTR Contracting Officer Technical Representative CPFF cost-plus-fixed-fee CPT cotrimoxazole preventive treatment CT counseling and testing CTO Cognizant Technical Officer DHS Demographic and Health Survey DOTS directly observed treatment (for tuberculosis) DPCD Disease Prevention and Control Department EHNRI Ethiopian Health and Nutrition Research Institute EOP end of project EP TB extrapulmonary tuberculosis EQA external quality assurance EQC external quality control FMOH Federal Ministry of Health GF Global Fund GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GH Tech Global Health Technical Assistance Project HAART highly active antiretroviral treatment HAPCO HIV/AIDS Prevention and Control Office HBC home-based care HCT HIV counseling and testing HIV human immunodeficiency virus HRH Human Resources for Health IC infection control ICAP International Centre for AIDS Care and Treatment Programs IEC information, education and communication IP implementing partner IPT isoniazid preventive therapy iv THE PRIVATE SECTOR PROGRAM IN ETHIOPIA LOP life of project M&E monitoring and evaluation MAC Millennium AIDS Campaign MARPs most-at-risk populations (for HIV) MCT mobile counseling and testing MoU memorandum of understanding MSH Management Sciences for Health NGOs nongovernmental organizations NTLCP–E National Tuberculosis and Leprosy Control Program–Ethiopia OGAC Office of the U.S. Global AIDS Coordinator OI opportunistic infection OR operations research PCP4 Public-Private Partnerships Program PDA personal digital assistant PEPFAR President’s Emergency Plan for AIDS Relief PFTC periodic full-time consultant PHC/PHCU primary health care unit PIHCT provider-initiated counseling and testing PO Project Officer PPM–DOTS public-private mix for directly observed treatment for tuberculosis PSI Population Services International PSP Private Sector Program PSP–E Private Sector Program–Ethiopia PLWHA people living with HIV/AIDS PMTCT prevention of mother-to-child transmission of HIV RHB regional health bureaus RHAPCO Regional HIV/AIDS Prevention and Control Office SNNPR Southern Nations Nationalities and Peoples Region SS+/SS– sputum smear-positive/sputum smear-negative STIs sexually transmitted infections TB tuberculosis TBCAP Tuberculosis Control Assistance Project TB–DOTS directly observed treatment for tuberculosis TB/HIV TB and HIV THAC TB/HIV Advisory Committee THTC TB/HIV Technical Committee TX treatment USAID United States Agency for International Development VCT voluntary counseling and testing WHO World Health Organization THE PRIVATE SECTOR PROGRAM IN ETHIOPIA v CONTENTS ACKNOWLEDGMENTS ................................................................................................................................. i ACRONYMS .................................................................................................................................................iii EXECUTIVE SUMMARY ............................................................................................................................. vii PRINCIPAL FINDINGS ........................................................................................................................ vii LESSONS LEARNED ........................................................................................................................... x RECOMMENDATIONS ........................................................................................................................ xii SUMMARY .......................................................................................................................................... xv I. INTRODUCTION................................................................................................................................... 1 II. BACKGROUND .................................................................................................................................... 3 OVERVIEW OF TUBERCULOSIS AND HIV/AIDS IN ETHIOPIA ........................................................ 3 USAID/ETHIOPIA HEALTH FOCUS: TUBERCULOSIS AND HIV/AIDS ............................................. 3 PSP–E PROJECT OVERVIEW ............................................................................................................ 4 III. EVALUATION METHODOLOGY .......................................................................................................... 7 IV. FINDINGS ............................................................................................................................................. 9 PROGRAM MANAGEMENT ................................................................................................................. 9 PROGRAM IMPLEMENTATION ........................................................................................................10 POLICY DEVELOPMENT ...................................................................................................................13 WORKPLACE PREVENTION AND CLINICAL SERVICES ...............................................................13 PUBLIC-PRIVATE MIX—TB–DOTS ...................................................................................................16 V. LESSONS LEARNED .........................................................................................................................35 PROGRAM MANAGEMENT ...............................................................................................................35 WORKPLACE PREVENTION AND CLINICAL SERVICES ...............................................................35 PPM–DOTS AND TB/HIV ...................................................................................................................36 MOBILE COUNSELING AND TESTING ............................................................................................37 VI. RECOMMENDATIONS .......................................................................................................................39 PROGRAM MANAGEMENT ...............................................................................................................39 WORKPLACE PREVENTION AND CLINICAL SERVICES ...............................................................40 PPM–DOTS AND TB/HIV ...................................................................................................................40 MOBILE COUNSELING AND TRAINING ...........................................................................................43 VII. SUMMARY ..........................................................................................................................................45 LIST OF ANNEXES ....................................................................................................................................47 vi THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA vii EXECUTIVE SUMMARY In September 2004, the United States Agency for International Development Mission to Ethiopia (USAID/Ethiopia) issued a four-year task order under the Private Sector Program (PSP) Indefinite Quantity Contract (IQC) for the Ethiopia Positive Change: Public Private Partnerships Program (PCP4). The name of the program was later shortened to read the Private Sector Project–Ethiopia (PSP–E) before the task order was extended through September 20, 2010. Total funding for the project was $12,234,000. As specified in the Abt Associates, Inc. (Abt) task order, the objectives of the PSP–E are the following: To expand knowledge of and access to affordable and quality HIV/AIDS- and tuberculosis (TB)-oriented health care packages delivered by the private sector to mass markets of moderate and low income individuals and workplaces, with specific emphasis on three key strategies: 1. Fostering private sector partnerships to address HIV/AIDS and TB 2. Promoting social franchising to improve quality of private sector care in HIV/AIDS and TB 3. Highly targeted social marketing of HIV/AIDS prevention products and services to high risk groups.1 The purpose of this final evaluation was to complete an end-of-project evaluation to address the following tasks: 1. Assess how well technical assistance met the needs of private providers in delivering HIV prevention, HIV counseling and testing (HCT), and TB services under the contract. 2. Assess how well the needs of private sector clients are being met (broken down by age, gender, and HIV or TB status) by private health providers supported under the contract. 3. Examine the sustainability of assistance activities for the health system and their results under the contract. The substantive part of the evaluation took place in Addis Ababa, Ethiopia, April 6–29, 2009. The evaluation was undertaken by a two-person team through the Global Health Technical Assistance Project (GH Tech). During the evaluation, the team visited 25 program-affiliated sites and interviewed more than 85 program stakeholders, managers, subcontractors, and staff. PRINCIPAL FINDINGS Program Management In assessing the PSP–E’s approach to program management, the evaluation team noted and assessed the program’s application of a 13-step methodology focused on a transparent, participative, and measured development of each of the program’s three technical initiatives. Through its review of documentation and its discussions with stakeholders and project staff, the evaluation team judged that each of the steps was part of a critical continuum applied by project staff in their effort to develop a foundation for public￾private management of Ethiopia’s response to TB and HIV/AIDS. 1 With the 2006 infusion of a significant level of funding, USAID/Ethiopia and Abt undertook an analysis of project priorities, leading to an agreement by all parties that the PSP’s work would focus on the first of these three strategies. Accordingly, at the instruction of USAID’s COTR for the PSP–E, the evaluation’s scope of work does not include an assessment of strategies 2 or 3. viii THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Workplace Prevention and Clinical Services As the first major intervention initiated under the PSP–E Task Order, promotion of prevention and clinical services for HIV/AIDS counseling and testing at private sector workplaces served as a proving ground for the program’s approach to systematic managed development. With the development of a workplace assessment tool in March 2005 and its initial assessment of 25 private sector companies in April–June 2005, PSP–E eventually expanded its potential for workplace prevention to 84 companies. More than 42,000 employees were exposed to peer education messages. At the same time, it should be noted that the 26 companies that completed peer education activity and the 20 companies that continued to offer prevention activities to their employees at the time of the evaluation were only a fraction of the initial 84 companies assessed. With reference to the Workplace Clinical Initiatives, a December 2008 PSP–E assessment2 of TB/HIV services indicated that, with the exception of isoniazid preventive therapy (IPT) services, more than two￾thirds of those workplaces that began initiating selected services following PSP–E–sponsored training were continuing to offer these services. It is also worth noting that in those sites that actively offered clinical services, more than 6,000 clients had received counseling and testing for HIV/AIDS, and close to 2,000 clients had received treatment for TB. At the same time, it should be acknowledged that of the 56 sites in which staff were trained, only 18 were working with PSP–E in monitoring the quality of their services at the time of the evaluation. PPM–DOTS–HIV/AIDS The progress made in establishing the 90 sites offering a public-private mix for directly observed treatment for TB (PPM–DOTS) in three regions of Addis Ababa, Oromiya, and Amhara within the project’s short timeframe is commendable and is largely due to PSP–E’s facilitation of technical relationships between the public and private sectors. This relationship has been supported at all levels, from the national level to the regional health bureaus to woreda, subcity, and town health offices. However, supportive supervision systems were still in a process of maturation. The clinical training curricula and materials used in PPM–DOTS and provider-initiated counseling and testing (PIHCT) followed the national guidelines, although the time allocated for training was shortened. Logistics: Memoranda of understanding (MoUs) signed between the Regional Health Bureaus (RHBs) and the participating private clinics outline roles and responsibilities. Included in the RHBs’ responsibilities is the provision of a regular supply of drugs and reagents, and all recording and reporting formats. PSP–E played an important support role at times when supplies were limited or stockouts were imminent. Supportive supervision: PSP–E’s work on supportive supervision embraced both internal and external quality assurance mechanisms, ensuring that national guidelines were followed and monitored. Recording and reporting under PSP–E’s program of supportive supervision was carried out well and in adherence to standards. Advocacy workshops: PSP–E’s design and implementation of 12 advocacy workshops was effective in promoting the PPM–DOTS program broadly among stakeholders in the public sector, in improving trust among public and private sector stakeholders, and in addressing challenges to working collaboratively. These workshops led to PPM–DOTS clinic supervisors’ being invited to participate at regular government-sponsored semiannual TB meetings. 2 Dawit Abraham et al., Assessment of TB/HIV Services in Workplace Clinics in Ethiopia, Addis Ababa: USAID Private Sector Program–Ethiopia, December 2008. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA ix Laboratory: In many, if not most, PPM–DOTS programs, laboratory systems and services and links to clinical services represent major barriers to the expansion of quality directly observed treatment for TB (TB–DOTS). In the PSP–E project, all higher clinics visited had laboratory services on the premises. Laboratory records, for the most part, were adequately kept. PSP–E-supported laboratory environments ranged from very adequate to poor. In some sites, the evaluation team observed cramped laboratory quarters, poor or inadequate ventilation, or ventilation that puts both the laboratory staff and patients at risk for transmission of TB. Good infection control practices were not uniformly applied. With reference to HIV laboratory services, availability of the full range of test kits for PIHCT was limited in most sites visited. As a result, the program missed opportunities to effectively integrate TB/HIV activities and to foster good case management of TB patients. Supervision and external quality control (EQC) activities were being scaled up to all participating private labs, and results are promising. However, lack of transport and logistics for further scaling up may be hampered through inadequate funding support. Clinical mentoring: Originally intended as part of supportive supervision, PSP–E attempted to coordinate with clinicians to discuss problems presented many challenges. In response, PSP–E introduced clinical seminars. Defining the topics to be reviewed, timing, and frequency was supported in part through a survey of clinicians. Input from the PSP–E supervisory team’s field experiences also contributed to identifying topics for inclusion in future seminars. Case detection: PSP–E’s summative evaluation and subsequent quarterly reports indicated that rates of sputum smear-positive (SS+) patients were low—even lower than the national average and rates in the same regions in the primary health care unit (PHC). The proportion of SS+ among all pulmonary cases was 48 percent nationally, but was lower in the private sector clinics reporting. Consistently high rates of extrapulmonary TB (EP TB) were reported in the private sector clinics. One needs to consider the possibility of overdiagnosis of EP TB. The low rates of SS+ may be attributable to the quality of sputum provided for microscopy, although the external quality assurance (EQA) results show a high level of concordance in most clinics participating. Accordingly, the issue of low SS+ rates warrants further study. In addition, regular supervisory activities, clinical seminars, and review meetings are opportunities to address this issue. Referrals: PSP–E has made great strides in strengthening communication between the public and private sectors, changing attitudes and practices, and, ultimately, ensuring that patients diagnosed in the private sector are accepted in the public sector. To address the high rates of transferred-out patients, clients were encouraged to maintain the DOTs regimen at the diagnosing facility. If they were not able to do so, clients were referred to the public health facility closest to their home. Information, education, and communication (IEC) materials: The evaluation team visited the AIDS Resource Centre in Addis, and was introduced to most components of their interventions, including printed media and popular radio and television series. The team noted that few materials addressed TB or TB/HIV coinfection, despite TB’s being a major cause of morbidity and mortality among people living with HIV/AIDS (PLWHA). Counseling and testing for HIV/AIDS (CT) services: PSP–E has worked to strengthen HIV/AIDS counseling and testing (HCT) services in three different private sector settings. First, the project sought to strengthen existing HCT services in PPM–DOTS sites. Second, PSP–E worked with the RHBs to introduce CT services in PPM–DOTS sites which lacked HCT. Finally, the project also helped to strengthen or introduce CT services in facilities outside the PPM–DOTS group. Mobile Counseling and Testing (MCT) MCT coverage: From July 2007 through November 2008, a total of 69,174 clients were tested, of which approximately 35 percent were female. As of December 2008 PSP–E has implemented MCT services in x THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 40 towns in the Oromiya, Amhara, and Afar regions and in the Southern Nations Nationalities and Peoples Region (SNNP). Identification of most-at-risk populations for HIV/AIDS (MARPs): Less than 25 percent of HIV￾positives met USAID’s traditional, but limited, risk-group definitions. The importance of this finding to increasing case detection is that the PSP–E–sponsored MCTs succeeded in identifying or refining traditional USAID risk groups to include additional MARPS, such as widows/widowers, female petty traders, unemployed women, male day laborers, and male petty traders. Tracking referred clients: The mechanism for tracking referred clients was somewhat passive: PSP–E asked staff in nearby institutions to insert clients’ referral slips in boxes that PSP–E had placed in each institution. From March 2007 to August 2008, a total of 39,487 persons were tested with an infection rate of 5 percent. All HIV-positive clients received post-test counseling, and 87.0 percent (1,956 clients) accepted referrals to seek services at antiretroviral treatment (ART) sites. By collecting the referral slips within a two-month period following the end of the MCT sessions, PSP–E concluded that 520 (26 percent) of those who were referred reached the referral facilities within two months. Use of Ethiopian subcontractors: An important element of PSP–E’s program implementation strategy was its use of PSP–E–trained Ethiopian subcontractors to implement the program’s MCT initiative, to provide training to private health providers, and to support external quality control for private sector higher health clinics. The use of Ethiopian contractors to address many of the program’s technical support needs represents a significant and cost-effective shift in the standard donor project paradigm, which normally depends upon in-house project staff to provide such support. LESSONS LEARNED In assessing the PSP–E’s progress, the evaluation team has concluded that the following lessons can be drawn from its examination of the four principal program elements: Program Management Early engagement of stakeholders: Transparency and participatory development were vital to the program’s success in building a bridge of trust and collaboration between the private and public sectors. Systematic program development: PSP–E’s systematic application of its 10 steps to scaling up TB care and TB/HIV services constituted a thoughtful and inclusive approach to measured program development. Use of Ethiopian subcontractors: PSP–E’s innovative approach in using PSP–E-trained Ethiopian subcontractors to help implement the technical program responded to the need for a focused, cost￾effective strategy to address immediate program needs while, at the same time, laying the foundation for long-term sustainability. Attention to the policy environment: One PSP–E focus was identifying and addressing policy barriers to private sector participation in responding to national priorities in TB and HIV/AIDS service and care. PSP–E’s willingness and skill in becoming engaged in and supporting the national policy development process assisted the national and regional governments in more fully understanding and profiting from the true potential of the private sector. Workplace Prevention and Clinical Services The importance of listening to the client: As exemplified by PSP–E’s flexibility in implementing the workplace prevention initiative, listening to the client and addressing a private sector client’s constraints are of paramount importance. Difficulty of promoting prevention: As with any preventive health program, promoting HIV/AIDS and TB prevention in the workplace calls for employers and employees to set aside time from their work THE PRIVATE SECTOR PROGRAM IN ETHIOPIA xi schedules to fully understand the implications of current behavior patterns and ways to bring about effective behavior change. Accordingly, those responsible for organizing and implementing the program need to devote significant hours and energy to working with employers and employees to advocate for the necessary time to fully implement the training program for both peer educators and their workplace colleagues. In addition, program organizers need to build in sufficient time for ongoing support to the workplace following the formal training program. High patient load requirement for maintenance of quality clinical services: To be effective, clinical interventions from a workplace-by-workplace perspective require a reasonably large patient load in the interest of maintaining providers’ quality of services, including the quality of microscopy services. Accordingly, concentrating on workplaces with significant numbers of employees, or pooling services between companies, may prove to be the best investment in terms of being able to ensure the quality of those providing diagnostic services, treatment, and care. PPM–DOTS and TB/HIV Full participation of government at all levels: Staff in many PPM–DOTS sites are confident about their ability to carry on these services—if the RHBs and town health offices are able to provide a regular supply of TB drugs. Full collaboration with the government with respect to its commitment under the MoUs is therefore essential to the continued health of the PPM–DOTS and TB/HIV interventions. Development of MoUs with the regional governments and local authorities: MoUs were key tools in making PPM–DOTS a success. Under PSP–E, all MoUs enhanced the relationship between the public and private sectors by clearly setting out terms and conditions of the collaboration and the respective roles and responsibilities of each partner. Comprehensive approach to addressing the need to use the human resources for health (HRH) method for institutional development and sustainability: PSP–E training included a focus on technical, managerial, and financial management issues. PSP–E training helped to strengthen the skills of private health facility owners and managers in appropriate use of financial information for management decision making. Other trainings appropriately focused on building knowledge and skills in meeting staff needs, including creating enabling working relationships. Role of advocacy in strengthening the relationship between the public and private sectors: Three advocacy efforts—raising awareness among the public sector around PPM–DOTS, strengthening private￾public linkages, and promoting timely distribution of drugs and laboratory supplies—resulted in improved collaboration between the private and public sectors. The need for clarity in procurement of certain commodities: Lack of clarity on who should continue to provide test kits for HIV has resulted in confusion and frustration, articulated by many clinic owners. Some have already opted out of those provided by the RHB and are purchasing their own. Enhancing the capacity of the RHB and subcity/town health offices: PSP–E’s success in developing the capacity of government technical officers at all bureau levels underscored the importance of coordination in maintaining and scaling up services, especially with reference to supply chain management, data management and supervisory activities. Importance of supportive supervision and feedback: PSP–E’s focus on supportive supervision and feedback ensured that the importance of adherence to TB–DOTS and TB/HIV technical guidelines received appropriate attention and was effectively addressed. Demand for HCT services: Based on data collected by PSP–E, it would appear that there is a significant demand for HCT services within the private sector. xii THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Strong capacity of Ethiopian professionals: Local professionals and selected associations, such as the Ethiopia HIV/AIDS Counselors Association, proved able to provide services such as training and mentoring in areas such as HCT. However, professional medical associations, such as MAPPE (Medical Association of Private Practitioners of Ethiopia), require further attention to build their capacity to support critical system functions, including EQC, mentoring, and training. Increasing experience in PPM–DOTS and ART by working with counseling and testing (CT)-only sites: By working with CT-only sites, PSP–E has built the sites’ capacity and has been able to identify the best ones—those that can be engaged in expanding to PPM–DOTS and ART services. Facilitating integration of PPM–DOTS and HCT: Based on observations obtained from a significant number of stakeholders and on its own site visits, the evaluation team has concluded that PSP–E’s coordinated introduction and integration of PPM–DOTS and HCT services in selected primary health care sites has greatly enhanced the access, quality, and effectiveness of both health interventions. Mobile Counseling and Testing The key elements of mobile counseling and testing are supporting strong partnerships and awareness; building capacity; using data from rapid assessments to further define services to target high-risk populations; employing flexible and innovative approaches to address barriers; conducting referral and follow-up; and establishing or strengthening systems to maintain quality of services. Engaging local subcontractors: The importance of the PSP–E’s groundbreaking engagement of Ethiopian subcontractors was the key to much of PSP–E’s success. Developing a viable referral and tracking system: As it was not as strong as it might have been, attention to the MCT referral and tracking system and to ways of upgrading it should receive increased attention under PSP–E’s successor program. Developing an effective applied monitoring and evaluation (M&E) system: Despite PSP–E’s impressive array of consultancy reports focused on monitoring selected elements of the program, the evaluation team’s discussions with PSP–E’s staff resulted in a common agreement that the program’s current M&E system fell short of what was needed to both monitor the program’s overall progress and take effective action. Considering and responding to M&E requirements from a program’s first days of implementation should be considered essential to any program’s ability to periodically assess its progress and to adjust program components, where necessary, in the interest of ensuring that the program is both realistic about its defined goals and objectives and on target in meeting them. RECOMMENDATIONS Recognizing that the Private Sector Program–Ethiopia Task Order is scheduled for completion at the end of September 2009, the evaluation team offers the following actionable recommendations for moving forward on the trajectory established under PSP–E. Program Management Support Ethiopian subcontracting mechanism: The evaluation team recommends that the next iteration of the PSP–E (PSP–E II) continue developing the capacity and use of Ethiopian subcontractors. The evaluation team further recommends that PSP–E II act as a catalyst in forming a consortium of current PSP–E subcontractors and in helping the consortium develop proposals for sustained funding from donor agencies, such as the Global Fund to Fight AIDS, Tuberculosis and Malaria (Global Fund). Document PSP–E’s approach to building linkage and trust between the private and public health sectors: The evaluation team strongly recommends that project staff take the time and devote the THE PRIVATE SECTOR PROGRAM IN ETHIOPIA xiii necessary thought to capturing the essential elements of PSP–E’s successful approach to working with and engaging the public sector in developing an environment of mutual trust and collaboration. Continue advocacy for an improved private sector policy environment: The evaluation team recommends that, under PSP–E II, continued emphasis be placed on giving the federal and regional governments technical assistance in addressing current and emerging legislative and regulatory barriers to the private sector’s full engagement as part of Ethiopia’s response to the need for TB and HIV/AIDS prevention and care. Recruit senior-level technical assistance: The evaluation team recommends that providing the required level of senior international technical assistance be established as a prerequisite for the award of PSP–E II. Develop and implement a comprehensive plan for M&E: The evaluation team recommends that, under PSP–E II, program M&E address the goals and objectives of the project in a consistent manner, using established criteria, benchmarks, and targets, and report on these throughout the life of the program. Any changes or additions to the project, and ensuing M&E plans, should be adequately documented. Every effort should be made to standardize indicators across all funding sources—including the President's Emergency Plan for AIDS Relief (PEPFAR)—with progress monitored based on the standardized indicators. Workplace Prevention and Clinical Services Develop an enabling environment among workplace management: The evaluation team recommends that PSP–E II invest additional resources in developing effective advocacy strategies targeted toward senior company management. Develop a strategy for enlisting large companies: The evaluation team recommends that PSP–E II focus its recruitment efforts on companies with significant numbers of employees and potentially greater client loads. Consolidate or pool clinical services: Working with a reasonably large number of clients is important to maintaining quality services, especially for maintaining clinicians’ expertise in microscopy, diagnosis, and treatment. The evaluation team therefore recommends that PSP–E II develop a strategy to consolidate and/or pool services offered by participating private sector entities, thereby providing a large pool of potential clients to whom clinicians will be able to provide services. Develop strategies to increase financial support for preventive and clinical services: The evaluation team recommends that PSP–E II focus on working with companies to develop strategies to reduce provider costs associated with the provision of preventive and clinical services for TB and HIV/AIDS. One strategy worth considering is to work with insurance companies presently engaged by the private sector to include preventive and clinical services as a standard provision under existing insurance schemes. Develop long-range action plans for all recruited companies: The evaluation team recommends that, as PSP–E II expands its workplace preventive education program, more thought be given to developing a long-term, systematic approach to providing ongoing support for companies that have ―graduated‖ from the initial education program. PPM–DOTS and TB/HIV Hire a senior TB/HIV specialist: The evaluation team recommends that PSP–E II recruit a senior international TB/HIV specialist to help design and manage technical aspects of the program. xiv THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Hire a laboratory specialist: The evaluation team recommends that PSP–E II and indeed PSP–E identify a laboratory specialist who could also contribute to the ART initiative. Both the TB/HIV specialist and the laboratory specialist are essential to future activities operational research, to identify root causes of the low case detection rates and high rates of extrapulmonary TB. Clarify responsibility for the provision of test kits and laboratory reagents/supplies: The evaluation team recommends that PSP–E work with the private sector clinic owners and with representatives of the Federal Ministry of Health (FMOH) and the RHBs to ensure that the private clinics have a ready and timely supply of HIV/AIDS test kits and laboratory reagents. If it is determined and agreed upon with the government that the private sector needs to purchase these products and commodities, program managers and private sector center owners should consider a ―pooled procurement‖ approach that will ensure the quality of these products, as well as a regular supply of them, and result in affordable services for the clients. Future projects should plan for and include advocacy workshops: The evaluation team recommends that PSP–E’s approach to advocacy workshops continue to be supported by PSP–E and PSP–E II as an effective means of addressing challenges to continued growth and expansion of private sector PPM– DOTS and TB/HIV initiatives. Although some of the challenges to improved private/public sector collaboration are addressed in the regular TB and HIV semiannual meetings, the flexible, timely, and facilitative format of PSP–E–sponsored advocacy workshops has proven to be highly effective in providing public and private sector stakeholders with a process by which to introduce and address issues of current and emerging importance. Establish criteria for scaling up PPM–DOTS: The evaluation team recommends that in areas where PPM–DOTS activities have been implemented, scale-up should be linked to criteria such as an acceptable level of, or trend toward, positive treatment outcomes for patients under care in nearby PPM clinics, in addition to availability of systems support. Address referral issues: The evaluation team recommends that the PSP–E and its successor project develop mechanisms to more effectively track referrals of both TB and HIV/AIDS positives. Currently, the referral linkages between the private and public sectors remain tenuous with reference to the treatment of those individuals who are referred for TB–DOTS treatment as well as those clients who are diagnosed as HIV/AIDS positive. Accordingly, the evaluations team also recommends that PSP–E or its successor project consider developing and undertaking a small operational research study on a sample of patients that have been referred in all three areas—Addis Ababa, Amhara, and Oromiya. Place emphasis on infection prevention and control: The evaluation team recommends that, as clinic owners are the key to determining an effective infection control environment, suitably trained and experienced PSP–E/RHB staff continue to stress working with clinic owners on the development of MoUs and on training on technical and managerial priorities and on the provision of supportive supervision, with a strong emphasis on infection prevention and control. Assess and adapt IEC materials for TB and TB/HIV. The evaluation team recommends that educational materials be developed or adapted to address key areas of early diagnosis, links between TB and HIV, and the importance of completing treatment. These should include job aids that illustrate key messages for clients and materials for the public that can be used in clinic waiting rooms. The evaluation team also recommends that PSP–E work closely with and advocate for materials that address TB/HIV through partners of the Office of the U.S. Global AIDS Coordinator (OGAC), such as the AIDS Resource Center (ARC). Continue to strengthen and support established HCT services in PPM–DOTS: The evaluation team recommends that PSP–E’s efforts to integrate HCT and PPM–DOTS be continued and strengthened as a major component of the PSP–E successor program. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA xv Mobile Counseling and Training Continue interim support for the MCT initiative: The evaluation team strongly recommends that the concept and activities of the MCT receive continued USAID/Ethiopia support, at least on an interim basis, following the completion of PSP–E. Although PSP–E’s program design had not anticipated that the MCT initiative would be sustained through USAID/Ethiopia funding beyond PSP–E’s five-year timeline, the evaluation team, supported by the vast majority of stakeholders interviewed during the course of the evaluation, believes that the MCT initiative, as designed by PSP–E and as implemented by Ethiopian subcontractors, has made and will continue to make an important contribution to understanding and responding to the current incidence of HIV/AIDS among the nation’s most at-risk populations. Use the MCT services to expand knowledge base on MARPs: The evaluation team recommends that PSP–E program managers and the Government of Ethiopia continue to take maximum advantage of the MCT concept as a means of extending the nation’s ability to respond to the need for HIV/AIDS counseling and testing among Ethiopia’s most at-risk populations. As noted earlier in this report, the MCT, organized in a simple but well-conceived manner, serves an MARP client base that does not currently access health facilities for HIV/AIDS testing and counseling. Extend coverage into identified “hot spots” and underserved areas with high demand: During the evaluation interviews, a significant number of stakeholders recommended that the MCT expand its activities to include currently underserved but recognized periurban and rural HIV/AIDS hot spots. The evaluation team would like to lend its support to this recommendation. Support the development of an MCT-focused consortium among current subcontractors: As noted above, PSP–E’s program design had not anticipated that the MCT initiative would be sustained through USAID/Ethiopia funding beyond PSP–E’s five-year timeline. However, given the importance and success of the MCT and of the Ethiopian subcontractors’ development of their capacity to effectively manage the MCT technical process, the evaluation team recommends that USAID/Ethiopia support a concept through which the successor project to PSP–E would act as a catalyst in promoting and supporting the formation of a consortium of subcontractors and in building the consortium’s capacity to obtain donor funding support, outside of USAID, for a subcontractor-managed expansion of the current MCT initiative. SUMMARY In its final evaluation of the Private Sector Program–Ethiopia, the evaluation team has taken note of the following: 1. In establishing a strong base for PPM–DOTS and TB/HIV, the PSP–E, working closely with the Ethiopian Government and its private sector partners, has established Ethiopia’s reputation as a leader in identifying and supporting ways in which the private and public health sectors can work together toward a common goal. 2. In working to build a bridge of collaboration between the public and private health care sectors in Ethiopia, PSP–E has succeeded in establishing linkages of trust between the two sectors and in paving the way for continued cooperation among health providers, program managers, and policy champions from both sectors. 3. With its attention to detail and a systematic approach to the implementation of its initiatives, PSP–E has firmly established the basis for a continuum for the private sector’s contribution to quality-based TB and HIV/AIDS services. 4. As a key element to the PSP–E’s success, the program’s emphasis, from its onset, on transparency and on obtaining the participation of all stakeholders in both sectors and in the community at large represents a program management lesson of the highest importance. xvi THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 5. While the PSP–E has made considerable progress in working to establish meaningful and effective linkages with the public sector, the capacity of the Government of Ethiopia to develop and support mechanisms, policies, and guidelines to enable Ethiopia to fully profit from the potential contribution of the private sector remain a challenge that must be addressed now and in the future. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 1 I. INTRODUCTION In September 2004, the United States Agency for International Development Mission to Ethiopia (USAID/Ethiopia) issued a task order under the Private Sector Program (PSP) Indefinite Quantity Contract (IQC) for the Ethiopia Positive Change: Public Private Partnerships Program (PCP4). The name of the program was later shortened to the Private Sector Program–Ethiopia (PSP–E). Subject to availability of funds per annum, USAID/Ethiopia awarded a cost-plus-fixed-fee (CPFF) task order to Abt Associates, Inc. (Abt) for $2,800,000, to be obligated over the four-year performance period ending September 30, 2009. The Abt task order was subsequently extended through September 20, 2010, with additional funding of $9,434,000 for a total funding of $12,234,000. USAID/Ethiopia modified the Abt task order and its scope of work effective September 29, 2006, in agreement with Abt. The purpose of this end-of-project evaluation was to do the following: 1. Assess how well technical assistance met the needs of private providers in delivering human immunodeficiency virus (HIV) prevention, HIV counseling and testing (HCT), and tuberculosis (TB) services under the contract. 2. Assess how well the needs of private sector clients are being met (broken down by age, gender, and HIV or TB status) by private health providers supported under the contract. 3. Examine the sustainability of assistance activities for the health system and their results under the contract. Examine the institutionalization and sustainability of the results of the contract assistance activities. The substantive part of evaluation took place in Addis Ababa, Ethiopia, April 6–29, 2009. The evaluation was undertaken by a two-person team through the Global Health Technical Assistance Project (GH Tech) under Task Order No. 8071. 2 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 3 II. BACKGROUND OVERVIEW OF TUBERCULOSIS AND HIV/AIDS IN ETHIOPIA With an estimated current population of approximately 79 million, about 89 percent of Ethiopia’s potential health care clients have access to primary health care services.3 As of 2007/2008, there were 371 higher and 178 medium private health clinics in Ethiopia. In 2007/2008, the Government of Ethiopia estimates that there were approximately 40,000 new cases of smear-positive pulmonary TB, approximately 49,000 new cases of extrapulmonary TB, and approximately 142,000 total cases of TB in Ethiopia. Ethiopia further estimates a cure rate for TB positives at 67 percent. Based on World Health Organization (WHO) statistics for 2008, Ethiopia ranks seventh among 22 countries worldwide with high-burden TB incidence per year. The Government of Ethiopia estimates that in 2007/2008, there were approximately 1.3 million people living with HIV/AIDS (PLWHAs) in Ethiopia, for a prevalence rate of 2.1 percent, and approximately 35,000 individuals were diagnosed as being HIV positive. During the same period, 353 health facilities were providing HIV/AIDS services, HIV counseling and testing (HCT), antiretroviral treatment (ART), and services in preventing mother-to-child transmission of HIV/AIDS (PMTCT). A December 2008 report jointly prepared by Ethiopian Federal HIV/AIDS Prevention and Control Office (FHAPCO) and the Ethiopian Federal Ministry of Health (FMOH) estimates that approximately 129,000 patients are currently on ART. USAID/ETHIOPIA HEALTH FOCUS: TUBERCULOSIS AND HIV/AIDS4 USAID/Ethiopia support to the Ethiopian Tuberculosis and Leprosy Control Program (NTLCP-E) began in 2001 with the goal of strengthening the program. In consultation with experts at the TB and Leprosy Diseases Prevention and Control Team under the Disease Prevention and Control Department (DPCD) of Ethiopia’s Federal Ministry of Health, USAID/Ethiopia has developed a strategy to strengthen three regional referral laboratories in the Oromiya, Amhara, and Southern Regions and eight selected zonal laboratories within the regions. Between 2001 and 2003, USAID allocated approximately $1 million for interventions to strengthen laboratories. The involvement of the United States Government (USG) in HIV/AIDS programming began in 1992 with the condom social marketing program. Programming remained relatively small-scale until 1998. Since 2000, the USG has increased its HIV/AIDS activities in selected urban and periurban areas around the country, covering approximately four million people. Major government partners include FHAPCO; the Regional HIV/AIDS Prevention and Control Offices (RHAPCOs); the FMOH; regional health bureaus (RHBs); the Ministry of Youth, Sports, and Culture; and the Ministry of Labor and Social Affairs. While USAID/Ethiopia has worked extensively with private nongovernmental organizations (NGOs), its work with private for-profit organizations has been somewhat limited in scope—a limitation that the PSP–E was designed to address. 3 Information included in this background section comes from the Federal Democratic Republic of Ethiopia Ministry of Health, Health and Health Related Indicators, 2007/2008, Addis Ababa. 4 Summary information about the USAID response to tuberculosis and HIV/AIDS was obtained from the Abt PSP– E task order. 4 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA PSP–E PROJECT OVERVIEW Project objectives: As specified in the Abt task order, the objectives of the PSP–E are the following: To expand knowledge of and access to affordable and quality HIV/AIDS- and TB-oriented health care packages delivered by the private sector to mass markets of moderate- and low-income individuals and workplaces, with specific emphasis on three key strategies: 1. Fostering private sector partnerships to address HIV/AIDS and TB. 2. Promoting social franchising to improve quality of private sector care in HIV/AIDS and TB. 3. Highly targeted social marketing of HIV/AIDS prevention products and services to high-risk groups.5 Project financing: As shown in Table 1, which is based on financial data provided to the evaluation team by Abt Associates, of the total contract budget ($12,234,000, including the extension), all funds have been obligated. Of this amount, as of April 1, 2009, approximately $10,278,410, including accruals, had been expended, leaving approximately $1,955,590 for project expenditures going forward. With a monthly burn rate of approximately $325,000, it is estimated that essentially all obligated funds will have been expended as of September 30, 2009. Obviously, with an estimated burn rate resulting in a projected drawdown on all available funding, Abt will need to closely monitor all project expenditures as it completes its current task order. TABLE 1. PSP-E EXPENSES AGAINST OBLIGATIONS $ 12,234,000.00 $ 12,234,000.00 $ 10,278,410.00 $ 1,955,590.00 $ 436,018.00 $ 325,931.00 $ 1,955,586.00 $ 4.00 Contract budget (with extension) Projected balance as of September 30, 2009 Burn rate (April 08-March 2009) Projected expenditures (April 1-September 30, 2009) based on projected burn rate Total obligation (to date) Expenditures, including accruals (as of March 2009) Balance as of April 1, 2009 Projected burn rate (April 1-September 30, 2009) Project organization: As currently organized, the PSP–E comprises Abt as the prime contractor, with Population Services International, IntraHealth, and Banyan Global as subcontractors. Naturally, each organization draws on its own technical strengths in implementing the program. However, following discussions with representatives of each subcontractor, the evaluation team has concluded that under PSP–E’s Chief of Party (an Abt employee), technical responsibilities for the project were managed in a seamless and noncompetitive manner. Although the evaluation team notes a significant turnover in technical staff during the program’s first three years, Figure 1 shows that, with the exception of the senior HIV mobile counseling and testing (MCT) position, all technical and administrative posts were filled at 5 With the 2006 infusion of a significant level of funding, USAID/Ethiopia and Abt undertook an analysis of project priorities, leading to an agreement by all parties that the PSP would focus on the first of these three strategies. Accordingly, per USAID’s Contracting Officer Technical Representative (COTR) for the PSP–E, the evaluation’s scope of work does not include an assessment of strategies 2 or 3. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 5 Figure 1. PSP–E Organization Diagram Kathleen Poer Chief of Party Tseguye Gebremedhin Operations Manager Yonas Yilma PPM-DOTS & CT Team Leader Negussie Gebremariam F&A Manager 4 MCT Supervisors – Addis 1 MCT Supervisor - Amhara 4 Finance Support Staff 3 Admin Support Staff 1 Driver 2 office support staff Vacant MCT Team Leader Wasihun Andualem M&E Team Leader 2 Prevention Team Trainers 4 TB/HIV Regional Officers (Bahir Dar, Dessie, Jimma, Adama) 2 TB/HIV Addis Officers 1 CT-Only & Training officer Tekle-Ab Zaid ART Team Leader & Policy Advisor 7 drivers 1 Training Assistant Sr. Atsede Takele Prevention Team Leader 1 M&E Analyst -TB/HCT & MCT (vacant) 1 M&E Analyst – ART 1 M&E Assistant 2 Data Entry Assistants Abenet Leykun DCOP 3 ART Clinical Teams (Physician, Nurse, M&E Mentor) Pharmacist & Druggist Communications Officer TB/HIV Implementation Advisor Regional TB/HIV Experts Addis Amhara Oromia Business Skills Team (2) Project monitoring: When attempting to assess progress achieved against established targets within a results-oriented framework, the evaluation team was informed by PSP–E senior staff that the program was not governed by life-of-project or end-of-project targets. As an alternative program monitoring strategy, the PSP–E and USAID/Ethiopia technical staff together initially agreed to use annual indicator targets from the President’s Emergency Plan for AIDS Relief (PEPFAR). Later, during program Years 3 and 4, PSP–E management staff added internal PSP–E–specific targets to monitor their progress on activities introduced in 2007 following the significant increase in funding noted above. However, the evaluation team has concluded that the program did not use the annual targets for substantive project monitoring purposes, other than quarterly and annual reporting. Moreover, in reviewing the extent to which a results framework was used to monitor the program’s progress, the evaluation team was informed by USAID/Ethiopia mission that the results framework was no longer operative and should not be considered within the scope of the PSP–E evaluation. Accordingly, the evaluation team has determined that any attempt to assess the program’s attainment of targets as a measurement of progress against established objectives would have had questionable utility. However, as an alternative measurement of program progress, Table 2 (Chapter 4.1) illustrates PSP–E’s quantitative progress on key indicators associated with promoting, developing, and implementing PSP–E’s three technical private sector initiatives (workplace prevention and clinical services; PPM–DOTS and TB/HIV HCT; and MCT) within the regions of Addis Ababa, Oromiya, and Amhara. 6 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 7 III. EVALUATION METHODOLOGY In addressing the tasks indicated in the introduction to this report, the evaluation team, in consultation with the USAID/Ethiopia Cognizant Officer Technical Representative (COTR) for the PSP–E, adopted the following methodology: 1. Document review (April 1 and onwards): As indicated in Annex 1, the evaluation team reviewed more than 150 documents before and during the evaluation itself. 2. Team planning (April 6–7): Once assembled in Addis Ababa, the evaluation team began an intense two-day planning session to agree upon the evaluation’s technical parameters. 3. Information collection and key informant interviews (April 8–22): As indicated in Annex 2, the evaluation team personally interviewed 85 individuals representing a range of stakeholders, each of whom contributed to the evaluation team’s assessment of the PSP–E. In conducting the interviews, the evaluation team used standardized respondent discussion guidelines, one for the general set of respondents and a second for USAID/Ethiopia respondents (See Annex 3 and 4). Where feasible, the guidelines were distributed to each respondent well in advance of the interview. At the completion of each day of interviews, the evaluation team met to debrief each other on the results of the interviews, using a standardized interview summary form (see Annex 5) for this purpose. 4. Selected site visits (April 11–15): Faced with a limited time in which to assess all aspects of the project, the team was obliged to limit its field visits to a five-day trip to Amhara Region and to the towns of Bahir Dahr, Burie, and Debre Marcos. During these visits, the evaluation team focused its attention both on interviews with selected stakeholders at the regional and woreda level and on-site visits to private sector health clinics and to an MCT site in Debra Marcos. Before and after the five￾day field trip, the evaluation team also visited higher health clinics in Addis and Oromiya regions. 5. Preparation of preliminary debriefing (April 24): While preparing the first draft, the team continued to consult with key informants from PSP–E on issues requiring clarification. 6. Preliminary debriefing (April 24): During a one-hour session with USAID/Ethiopia staff and a three-hour session with PSP–E staff, the team solicited suggestions for ways in which the report could be strengthened and modified prior to the formal delivery of the first draft to USAID/Ethiopia on April 29. Constraints and gaps: The evaluation team has concluded that there were three principal constraints on the team’s efforts, each of which may have resulted in gaps preventing the evaluation from being truly comprehensive: Time allocated to document review: The team was assigned only three days to review and analyze the significant amount of documentation associated with the project. The team believes that the assessment would have benefited from more time being allocated for this. While the team expended every possible effort to ensure that the assessment was truly evidence-based, it acknowledges that it may have missed a number of key points or issues. Time allocated to field visits: While the evaluation team is highly appreciative of the time and effort expended by PSP–E staff to arrange and coordinate the field visit to Amhara Region, it believes that the brief time available overall for field visits lessened its ability to fully assess the progress achieved by PSP–E in its implementation of the program throughout the three regions. As a final point, the evaluation team was unable to fully assess the quality of PSP–E interventions. To do so would have required more time and certainly additional expertise on specific issues. 8 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 9 IV. FINDINGS While USAID/Ethiopia awarded the Private Sector Program–Ethiopia (PSP–E) task order in late September 2004, effective startup of project activities was delayed until late February 2005. This was reportedly at the request of USAID/Ethiopia, due to ―increased Mission activities and PEPFAR planning.‖ 6 Accordingly, this report focuses on assessing progress achieved from February 2005 through the end of September 2008.7 In subsequent paragraphs, this report will assess progress achieved on each of PSP–E’s three principal initiatives: (1) Workplace HIV Prevention and Clinical Services (Workplace); (2) Public-Private Mix– TB–DOTS–TB and HIV/AIDS (PPM–DOTS–TB/HIV) and counseling and testing (CT) only; and (3) Mobile HIV Counseling and Testing (MCT). In summarizing the evaluation team’s assessment of overall progress, this section will report on the following three cross-cutting themes critical to the progress of each of these technical components: (1) program management; (2) program implementation; and (3) policy development. PROGRAM MANAGEMENT In assessing the PSP–E’s approach to program management, the evaluation team noted and assessed the program’s application of a 13-step methodology focused on a transparent, participative, and measured development of each of the program’s three technical initiatives. As assessed by the evaluation team through its review of documentation and its discussions with stakeholders and project staff, each of the steps illustrated in Figure 2 were part of a critical continuum applied by project staff in their effort to develop a foundation for public-private management of Ethiopia’s response to TB and HIV/AIDS. 6 Abt Associates, PSP–E Annual Report, October 1, 2005–September 30, 2005. Addis Ababa: PSP–E. 7 As noted in the introduction to this report, the scope of work for this evaluation calls for an assessment of progress achieved through September 2008. However, where appropriate and in the interest of being able to provide a view of future opportunities for progress, the evaluation team will report on selected achievements through the first quarter of Year 5 (October 1, 2009–December 31, 2009). 10 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Figure 2. PSP–E’s 13 Key Steps to Scaling Up TB Care and TB/HIV Services Service Delivery Supervise and mentor Advocate for PPM and referral Promote services Initiate sites Capacity Building Link to supply chain Prepare sites Conduct clinical and systems training Conclude memoranda of understanding/ subcontracts Foundation Identify referral network Select service sites Conduct rapid assessment Build consensus with all partners In implementing these steps, PSP–E’s focus on working with key stakeholders, from federal to regional to district levels, throughout the life of the project was consistently cited by respondents to the evaluation as significantly contributing to building a bridge of confidence and trust between the public and private sectors. The importance of PSP–E’s success as a catalyst in developing trust—an indispensable step in PPM (TB and TB/HIV) services at the regional and subregional levels—cannot be overstated and is to be commended. PROGRAM IMPLEMENTATION As illustrated in Figure 3, PSP–E’s approach to implementing each of the program’s three principal initiatives called for a systematic application of a three-step process involving, first, policy development, planning, and assessment; second, piloting, implementation, and evaluation; and, where appropriate and indicated, scale-up. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 11 Figure 3. PSP–E Implementation Process Prevention Plan Implement Add simplified strategy WP Clinical Implement Integrate with PPM-DOTS private facilities PPM-DOTS Develop policy/Plan/Pilot Implement Integrate VCT CT-only Implement MCT Plan/Assess Implement PPM-ART Assess/Policy Implement 1 2 3 4 5 Years A second strategically important element of PSP–E’s program implementation strategy was its use of PSP–trained Ethiopian subcontractors to implement the program’s MCT initiative, to provide training to private health providers, and to support external quality control for private sector higher health clinics (please refer to Annex 6 for a complete list of subcontractors’ roles and responsibilities). The use of Ethiopian contractors to address many of the program’s technical support needs represents a significant and cost-effective shift in the standard donor project paradigm, which normally depends upon in-house project staff to provide such support. An added benefit of PSP–E’s use of Ethiopian subcontractors rests with its potential for developing a national and sustainable resource of experienced subcontractors capable of responding to the private sector’s future technical support needs. It is important to note that PSP–E also provided financial management skills training to PPM–DOTS site owners, two of whom were MCT subcontractors. The MCT subcontractors received coaching as part of the review of the cost and technical proposals. While it was not formal training, this coaching should enhance the contractors’ ability to effectively act as a resource for future private sector development. A third important element of PSP–E’s program implementation strategy was the development and application of training programs and supportive supervision protocols to address all aspects of the program’s initiatives. While the scope of this evaluation allowed too little time for the evaluation team to adequately assess the quality of the training and all supportive supervision protocols, discussions with private sector providers and with Ethiopian subcontractors indicated that the training and supervision provided by PSP–E staff contributed to their adherence to national protocols. However, in assessing the extent to which training and supervision was effectively carried out by project staff, the evaluation team would note the absence of a senior technical adviser whose expertise and experience would have served to support and develop the technical capacity of PSP–E’s professional staff, especially in PPM–DOTS and TB/HIV. The evaluation team judges that the process employed by PSP–E in implementing its initiatives has resulted in a significant level of achievement. As indicated in Table 2, after less than four years of full￾scale activity and little more than two years since receiving the additional infusion of significant funding noted in the background section of this report, PSP–E’s level of documented achievement provides solid evidence to support the validity of PSP–E’s implementation strategy. 12 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA TABLE 2. PSP–E PERFORMANCE BY INITIATIVE, OCTOBER 2004–SEPTEMBER 2008 Initiative 1: Workplace Sites Cumulative # of workplaces that received PSP–E prevention support 82 # of peer educators and facilitators trained 1,632 # of people reached through PE rollout sessions (including alternate strategy) 46,712 # of workplace sites providing DOTS 18 # of clients receiving CT 6,289 # of clients on anti-TB treatment 1,342 Initiative 2: TB DOTS-TB/HIV Sites # of private sites providing DOTS 90 # of private sites providing CT 107 # of PPM sites under supportive supervision 90 # of health workers trained in TB–DOTS, TB/HIV, provider-initiated counseling and testing, acid-fast bacteria (AFB), and HIV testing 379 # of counselors and lab technicians trained on finger prick (new HIV testing algorithm) 204 # of clinic owners trained in financial management 112 # of clients on anti-TB treatment 4,184 # of clients receiving CT 83,363 # of TB patients receiving CT 698 Treatment success rate (# of patients completing treatment + cured in 20 pilot PPM sites) 79.2 % of clients who default on DOTS 1.7 # of HCT External quality control (EQC) samples collected 342 % of concordant HCT EQC results 99 Initiative 3: Mobile HCT Sites # of MCT sites which PSP–E directly supervised 155 # of people receiving voluntary CT at mobile sites 61,307 % of HIV-positive clients referred to ART sites 56.9 % of HIV-positive clients referred for TB screening at follow-up sites 15.7 # of HIV-positive clients who reached ART referral sites (confirmed referrals) 520 # of EQC samples collected for rechecking 2,120 % of VCT EQC samples with concordant results at mobile sites (blind rechecking) 99.2 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 13 POLICY DEVELOPMENT In reviewing PSP–E’s strategy for program implementation, the evaluation team has noted the program’s attention to the importance of support for the development of policies designed to enhance public-private partnership and to establish a consensus on technical issues critical to improved PPM services at regional and subregional levels. For example, relatively early on in the project, PSP–E played a major role in working with the FMOH and key stakeholders in the development of the August 2006 implementation guidelines for PPM–DOTS. Development of this policy document represented a first and crucial step to the rollout of PPM–DOTS in the workplace, followed by the launching of PPM–DOTS private sector pilot sites in Addis and Oromiya and later by a significant rollout of PPM–DOTS services to a total of 90 sites in the Addis Ababa, Oromiya, and Amhara regions. More recently, PSP–E technical staff have been instrumental in the development and approval of a national policy to authorize private higher health clinics to provide HIV-positive clients with ART, using nursing staff to dispense the free antiretrovirals. As a result, it was expected that as of May 21, 2009, PSP–E–trained staff from 22 private higher health clinics in Addis Ababa region would begin providing ART on a pilot basis. A second and important element of PSP–E’s approach to improving the private-public sector policy environment and to building a collaborative and informed bridge between the private and public sectors was its secondment of technical advisers to both the federal and regional areas. During the program’s early stages, PSP–E seconded a technical adviser to assist the FMOH in developing the PPM–DOTS implementation guidelines and to work with the FMOH on developing support for and engagement with PSP–E as it embarked on its PPM–DOTS pilot stage. Over time, especially with the assignment of Global Fund-financed advisers, seconded support from PSP–E was shifted from the FMOH to the three RHBs in Oromiya, Amhara, and Addis Ababa regions. PSP–E has continued to second staff to the regions for the purpose of supporting each regional bureau in a process focused on engaging, supervising, and providing stewardship and technical support for private health providers. A final component of PSP–E’s efforts to engage the public sector and stakeholders in the private sector policy debate is represented by PSP–E’s presence on the FHAPCO Voluntary Counseling and Testing (VCT) Technical Working Group, the Addis Ababa TB/HIV Task Force, and the ART Task Force. While the evaluation team is aware of the work accomplished by PSP–E’s participation on the ART Task Force, the evaluation team was unable to confirm the significance or impact of PSP–E’s participation on the other two technical groups. WORKPLACE PREVENTION AND CLINICAL SERVICES As the first major intervention initiated under the PSP–E Task Order, promotion of prevention and clinical services for counseling and testing for HIV/AIDS and for TB/DOTS at private sector workplaces established a proving ground for the program’s approach to systematic managed development. As defined, the objective of the PSP–E workplace initiative was to expand knowledge of, and access to, affordable high-quality public-private sector HIV/AIDS and TB services. On the preventive side, the initiative was designed to assist employees and their families in engaging in positive behavior change conducive to the prevention and treatment of TB and HIV/AIDS. As previewed earlier in this report’s overview of the program’s development, technical staff responsible for developing the workplace initiative followed a systematic and measured approach involving assessment, design of training tools, advocating for private sector workplace support, development of MoUs with participating workplaces and, finally, rollout of activities. Accordingly, following the development of a workplace assessment tool in March 2005 and its initial assessment of 25 private sector companies in April–June 2005, PSP–E eventually expanded its workplace prevention assessment to 84 companies. Of these, 26 completed a peer education cycle guided by PSP–E trainers using a training 14 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA curriculum developed by PSP–E and delivered to workplace employees by PSP–E–trained peer educators from each workplace. All workplace training programs were governed by a memorandum of understanding (MoU) between each workplace and PSP–E. Once the MoU was signed, PSP–E provided selected company staff with a five-day training to prepare them to serve peer educators. The project sought to train one peer educator for every 35–50 employees. It then assisted companies in forming peer education groups. The peer education courses, initially planned to include a series of classes given once per month over eight months, focused on topics related to TB, HIV/AIDS, and other sexually transmitted infections (STIs), including prevention, care, and support of people with these diseases. Over time, the length of the monthly classes was reduced, at the request of the companies, from one hour to less than 45 minutes. At all stages of the education program, PSP–E provided supportive supervision as well as information, education, and communication (IEC) material. At the completion of the course, PSP–E arranged for an appropriate ―graduation‖ ceremony. Although the limited time allocated to the evaluation only permitted the evaluation team to visit one active workplace clinical site (Commercial Bank) and two active preventive sites (Ghion Hotel and the Ministry of Federal Affairs), a review of documentation and a discussion with the site staff would indicate that the intensity of the PSP–E’s first attempt to reach out to the private sector workplace community may well have opened the door to PSP–E’s MCT and PPM–DOTS initiatives in the three target regions. With reference to the workplace clinical initiatives, a May 2008 PSP–E assessment8 of TB/HIV services indicated that, with the exception of isoniazid preventive therapy (IPT) services, more than two-thirds of the workplaces that initiated selected services following PSP–E–sponsored training were continuing to offer those services (Table 3). At the same time, it should be acknowledged that of the 56 sites that received training, only 18 were engaged in working with PSP–E in monitoring the quality of their services at the time of the assessment. It is also worth noting that in those sites that actively offer clinical services, more than 6,000 clients received counseling and testing for HIV/AIDS, and close to 2,000 clients received treatment for TB. TABLE 3. WORKPLACE CLINICAL INITIATIVE: MAY 2008 ASSESSMENT OF SUSTAINED SERVICES (CPT = COTRIMOXAZOLE PREVENTIVE TREATMENT) Service Number of facilities that initiated service % of facilities assessed that ever initiated services (n = 51) Number of facilities still providing services as of May 2008 % of initiating facilities still proving services as of May 2008 HCT 22 43.1 15 68.2 TB– DOTS 22 43.1 16 72.7 PIHCT 9 17.6 8 88.9 CPT 10 19.6 7 70.0 IPT 7 13.7 4 57.1 In addressing issues associated with implementation of the clinical services workplace initiative, the evaluation team noted that many of those issues affecting PSP–E’s PPM–DOTS and TB–HIV/AIDS initiatives were presaged early on in this first PSP–E initiative. For example, the need to improve referral 8 Dawit Abraham et al., Assessment of TB/HIV Services in Workplace Clinics in Ethiopia, Addis Ababa: Private Sector Program–Ethiopia, December 2008. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 15 and information systems, problems with retention of trained staff, the need to strengthen capacity in supportive supervision and M&E, and the importance of collaboration with the government in ensuring the availability of a regular supply of quality-assured drugs are all issues that first became evident in PSP–E’s implementation of its clinical workplace activity. With reference to workplace prevention activities, Table 4 indicates that PSP–E’s considerable investment in this initiative resulted in a significant number of employees being exposed to peer education messages. At the same time, it should be noted that the 26 companies that completed peer education activity and the 20 companies that were continuing to offer prevention activities to their employees at the time of the evaluation were only a fraction of the initial 82 companies assessed. Time management appears to represent an issue of paramount importance impacting the progress of workplace prevention activities. Simply stated, the amount of time requested from company management to complete the full schedule of eight monthly peer education sessions for all a company’s employees appears to have been a major constraint to a company’s acceptance of the program. PSP–E management staff have reported that while private sector companies were generally interested in receiving technical assistance to improve the capacity of their clinical services in addressing TB and perhaps HIV/AIDS, the idea of devoting company time to communicating prevention messages had considerably less appeal. This reluctance of employers to fully embrace the benefits of prevention over treatment may well explain why PSP–E’s forecasts of the numbers of workplace employees who would be reached through peer education proved overly optimistic.9 As evidence of PSP–E’s ability to adjust to the needs of their clients, PSP–E’s focal person for the workplace environment responded to senior management and employees’ concerns about time management by considerably reducing the length and duration of employee educational sessions. The program also initiated review meetings and refresher training for peer educators. In the opinion of the evaluators, in this regard PSP–E was a model for ways in which a private sector workplace initiative can effectively respond and adjust to emerging requirements of the private sector. 9 In the briefing received by the evaluation team from the PSP–E team, PSP’s Chief of Party, Kathleen Poer, reported that PSP had expected to reach 30,000 employees during PSP’s second year of operations (2005–2006) as opposed to the 9,000 actually reached. Similarly, the disparity in expectations versus results for the third year in operation was 35,000 vs. 17,000. Only in the fourth year of operations did the PSP come close to a realistic forecast of 18,000 vs. 17,000. Presumably, the PSP–E’s new leadership had taken note of past years’ forecasts in significantly lowering their expectations. TABLE 4. ACHIEVEMENT OF PEER EDUCATORS Number of companies assessed Number of companies that completed peer education activity Number of companies not completing peer education Number of companies with active prevention activity Number of employees trained as peer educators Number of employees who attended at least one session Male Female 82 26 18 20 1,113 519 42,521 16 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA PUBLIC-PRIVATE MIX—TB–DOTS Meeting the Needs of Private Providers Policies and guidelines: To enable private practitioners to provide quality TB–DOTS and TB/HIV services, national policies are required. As mentioned in Section 4.1, PSP–E was closely involved in the elaboration of the Ethiopia’s National PPM–DOTS implementation guidelines, paving the way for the private sector’s involvement. As previously mentioned, PSP–E seconded a technical adviser to the Central Unit of the National TB and Leprosy Control Program (NTLCP-E). Experience from the implementation of PPM–DOTS in the workplace enhanced PSP–E’s credibility as it moved into implementing PPM–DOTS in private sector facilities in selected regions. Algorithms and protocols contained in the guidelines have been printed up as posters and job aids and were observed in the vast majority of clinics. Staff have found them to be very useful. Processes to establish PPM DOTS services: The process of establishing PPM–DOTS sites is illustrated in Figure 2. Minimum criteria for site selection were agreed to and applied to interested clinics. Out of 25 sites assessed in the first pilot phase in two regions—Addis Ababa and Oromiya—20 were selected, including 2 specialized hospitals, 2 general hospitals, 15 higher clinics, and 1 medium clinic. Figure 4. Number of Health Providers Trained in Private Facilities, by Type of Training Package (PSP–E, October 2006–December 2008) 0 50 100 150 200 250 300 350 400 M&E (ART) Clinic Data Clerks Comprehensive ART & M&E for Nurses Supportive Suppervision on TB/HIV Basic ART & M&E for Physicians and Nurses Basic VCT for Counselors Rapid HIV, AFB & QA for Lab Techs Financial Management & Bookkeeping New HIV Testing Algorithm (Finger Prick) Basic TB/HIV,TB-DOTS & PIHCT 22 38 42 57 122 209 214 283 398 Number of Health Providers trained by Type of Training Package in the Private Facilities (PSP-E, Oct 2006 - Dec 2008) The clinical curricula and materials used in PPM–DOTS and PIHCT training follow the national guidelines. Integrating the training modules minimized the training time and resulting in a more efficient training schedule. This process was subject to an external evaluation, and the content was similarly reviewed. Continuous attrition of physicians, nurses, and laboratory technicians has necessitated increasing the number of trainings for PPM–DOTS. In each clinic visited, the evaluation team observed that only a few of the staff members have received PPM–DOTS training. For clinics that have experienced attrition of trained staff, problems have arisen in maintaining the consistency of services. For example, lack of continuity of trained staff affected quality of care in clinical areas, with respect to diagnostic algorithms; in programmatic areas, such as in the quality of recording and reporting; and in patient education. In response, PSP–E staff responsible for regular supervisory visits have spent a significant amount of time educating the new staff until they are able to attend the next scheduled training. Although it may not be feasible to train all clinic staff, there need to be more trained staff to balance the staff who have not yet been trained. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 17 As previously mentioned, clinic owners and selected staff had an opportunity to receive training in business development; participants who were interviewed were very appreciative of this experience. Moreover, those interviewed requested that PSP–E address areas such as in human resource management. Of the 90 PPM–DOTS sites in the three target regions, 20 sites in Addis have been selected as pilot sites for introducing ART. Physicians, nurses, and data entry clerks and have already been trained in ART. A series of ―advocacy workshops‖ in the Amhara and Oromiya RHBs contributed significantly to building trust and collaboration between the public and private sectors. The advocacy workshops were a forum for discussing challenges such as referral linkage, reporting on outcomes of transferred cases, and supply chain management. These workshops enabled the two sectors to agree on shared roles and responsibilities. The impact of the workshops was evident in the Amhara RHB’s decision to include PPM–DOTS sites in the region’s semiannual program review meetings. A total of nearly 700 persons attended advocacy workshops held in the Amhara and Oromiya regions. Consumables/supplies and logistics management: MoUs signed between the RHBs and the participating private sector clinics outlined roles and responsibilities of each partner. The RHBs’ responsibilities included providing a regular supply of drugs and reagents, establishing all recording and reporting formats, and giving supportive supervision. PSP–E has also given support at times when supplies were limited or stockouts were imminent. The clinics visited by the evaluation team were generally well supplied with drugs for the treatment of TB. Many clinics are in the vicinity of the local health office, facilitating ready access to drugs for new patients. One clinic expressed concern with having to travel to the local health authority to collect regular supply of treatment and kits, which was inconvenient. In other clinics, stocks of TB drugs were available. Storage of TB drugs varies across clinics. In some, those drugs were kept in a locked cabinet and brought to the DOTS treatment room on a daily basis; others kept them in the DOTS treatment room. Most drugs were 7–9 months away from their expiry dates. Although forms for drug consumption were available, few were filled out. Most clinics had experienced stockouts of the HIV rapid-test kits. Some of the clinics had received the first of three test kits (KHB) but lacked Stat Pack and UniGold for confirmatory testing. Testing policies also varied among clinics. Some clinics initiated testing, but then referred clients to the health center for the confirmatory tests. Other clinics purchased in-market test kits, while still others simply did not and will not test and, instead, only referred patients to health centers. The PPM national guidelines list the products/consumables to be provided to the PPM clinics free of charge and includes laboratory reagent. Although these clinics may have been provided with necessary supplies at the outset of PSP–E’s PPM–DOTS activities in Addis and Oromiya, many clinic laboratories are now engaged in purchasing their own supplies to replenish stocks. In Amhara, the RHB refused, from the outset, to provide reagents and consumables. Problems with the quality of these products have been cited by the directors of the regional laboratories in Bahir Dar and Adama, with the result that some regional laboratories are now advocating for pooled procurement of quality products by the RHB. Supportive supervision: Supportive supervision embraces both internal and external quality assurance mechanisms, ensuring that national guidelines are implemented, that recording and reporting is carried out to standard and that appropriate laboratory and overall facilities are adequately maintained. Earlier PSP–E consultancy recommendations suggested that the region provide intensive supportive supervision during the first 3–6 months, and then quarterly thereafter, to each of the PPM–DOTS sites. PSP–E has been very responsive to the needs to the private sector facilities. For example, PSP–E has been proactive in ensuring that level of drugs, appropriate recording and reporting formats and other materials are available to all sites. Instances where, for example, health facilities ran short of test kits for HIV appear to have been caused by a lack of clarity on who should be providing the required supplies. 18 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA PSP–E has invested a significant amount of time and effort in defining and implementing a supervisory management system and in refining tools for supportive supervision. These tools focus on several major program areas: logistics management; laboratory external quality assurance (EQA); recording and reporting; and clinical outcomes focused on cure rates and rates of TB patients tested for HIV. PSP–E’s role also extends to strengthening the RHBs and subcities’ and towns’ health offices through training and active facilitation of joint quarterly supervision. As illustrated earlier in this report (Figure 1, PSP–E Organization Chart), the supervisory team comprises eight health professionals: four are based in PSP–E’s Addis Ababa headquarters, one acts as team leader, two oversee TB/HIV activities in Addis, and one oversees sites in Oromiya as well as program-wide CT. PSP–E headquarters staff also provide additional support to the regions on an ad hoc basis. In addition, two program officers are based in East and West Amhara, and two in East and West Oromiya. These personnel are responsible for supportive supervision as well as communication between the zones, woredas, and town health offices. They also organize events such as advocacy workshops, trainings, and clinical seminars in their respective regions and monitor staff attrition at the facility level. Three additional staff are seconded to RHBs in Amhara, Oromiya, and Addis Ababa. These persons act as focal points within the RHBs and greatly facilitate the partnership between PSP–E and RHBs, and between the RHBs and the participating facilities. An average of 40 percent of their time is dedicated to PSP–E activities and 60 percent to other RHB activities. The evaluation team found that consistently across all those interviewed at facilities, at the RHBs, and at town health offices, PSP–E’s regional technical support activity is greatly appreciated for its contribution to improving the quality of services. Some respondents reported that, while most supervision is generally a joint activity in collaboration with local government officers, PSP–E staff has occasionally undertaken supervision without their government counterparts. Action plans are developed during each supervisory visit and any need for follow-up is discussed. These action plans are left in binders at facility level, and a copy is kept with the Program Officer. However, the evaluation team noted that that not all clinics were able to provide action plans associated with the last supervisory visit. Semiannual review meetings were planned for Year 5; as of the end of the first quarter of Year 5, PSP–E staff had facilitated one review meeting in each region or subregion. The evaluation team also noted the recent use, on a pilot basis, of electronic handheld personal digital assistants (PDAs) to enhance supportive supervision at the PPM–DOTS sites and facilitate data collection. Input into PDAs by field supervisors has mirrored the existing paper-based system in recording the major findings of a supervisory visit and assisted in providing a rapid appraisal on-site of issues requiring follow-up. After each supervisory visit, PDA data has then been electronically downloaded by email to PSP–E’s Addis headquarters; results and any further analysis were intended to be fed back to the respective regions. Theoretically, the advantages of this technology lie in increasing the ease and efficiency of the supervisory activities and of data management. Limitations may include the effort needed to make any changes to the tool—for example, expanding it to address newer areas of TB control, such as a referral system, or components of TB/HIV program management. PSP–E has put mechanisms in place to closely monitor this tool, as the PDA could be used to manage the implementation and scaleup of ART. Given that use of the PDA is a new development, the evaluation team was unable to assess its progress as a program initiative. Laboratory services: As one regional laboratory director stated: ―In any program, the laboratory services are the most neglected.‖ Indeed, coincident with the establishment in 2001 of the Global Drug Facility, which addressed lack of access to a quality assured drug supply, the quality of laboratory systems and services was identified as the major barrier to expansion of TB–DOTS. In Ethiopia, all PSP–E–supported THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 19 higher clinics that were visited by the evaluation team had laboratory services on the premises. For those clinics visited, however, laboratory environments ranged from very adequate to poor. In some sites, the evaluation team observed cramped laboratory quarters, poor or inadequate ventilation, or ventilation that put both the laboratory staff and patients at risk. In a few instances, issues with appropriate waste disposal were noted. In addition to including laboratories during regular facility supervisory activities, PSP–E staff facilitated supportive supervision specific to the laboratories in concert with the RGBs and regional reference laboratories. During these visits, slides were collected and reports of the supervision and external quality control (EQC) of participating clinics were made available to center staff. However, of those sites visited, not all were able to produce these results. Examples of major findings from Amhara West Region are described Table 5. Although regions use the same checklist for supervision, there is no consistent approach to reporting, making any kind of comparison between regions difficult. TABLE 5. MAJOR FINDINGS FROM SUPPORTIVE SUPERVISION OF PPM–DOTS FACILITIES CARRIED OUT BY THE REGIONAL LABORATORIES IN AMHARA* Amhara (first round)* Only 13/22 labs provided slides for EQA Availability of standard operating procedures 22.7% Appropriate smear and staining technique 36% Safety in AFB microscopy 54% Staff trained in AFB and Quality Assurance 77% Adequate sputum collection procedures 72% Adequacy of AFB reagents 81% Total smear-positive cases 11.4% Well-functioning microscope 100% Concordance rate 98.5% *Presentation on the regional laboratory TB program activities, Bahir Dar, February 2009. Clinical mentorship and clinical seminars: Originally intended as part of supportive supervision, PSP– E’s attempts to coordinate with clinicians to discuss problems have presented many challenges. This is not surprising, given that PPM–DOTS services are well integrated into the busy clinical work; with the heavy patient load and the nature of private practice, staff time is at a premium. In response to this, PSP– E introduced clinical seminars. The topics to be reviewed, timing, and frequency of the seminars were defined based in part on a survey of clinicians. Valuable input from the PSP–E–supervisory team’s field experiences has also contributed to clarifying specific needs for technical upgrades in future clinical seminars. The evaluation team noted that clinicians interviewed valued these seminars and also the opportunity to join with their counterparts from the RHBs and local health officers; as in other circumstances, these gatherings appeared to have enhanced collaboration between the public and private sectors. Tools developed to support providers: Since the inception of the PPM–DOTS program, PSP–E has developed and refined several tools: a site assessment tool used to select sites for PPM–DOTS; MoUs establishing formal relationships between the RHBs and the private facilities; supportive supervision tools, described earlier; and referral registers, now used in all PPM–DOTS facilities to register all patients referred to PHC facilities to begin treatment. All tools are well known to the staff interviewed. Clinical 20 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA owners and practitioners are particularly enthusiastic about the MoUs that have been signed: formalizing their relationships with the RHBs has greatly increased their ability to contribute to TB control, although recognizing that many patients diagnosed are referred to the PHC services. Meeting the Needs of Clients Case detection: Case detection begins with the identification of TB suspects and referral to diagnostic services. The role of the laboratory and its network has already been discussed earlier in this report. Since the initial implementation in 20 pilot sites in Oromiya and Addis Ababa, PSP–E has expanded services to a total of 90 sites, including 35 new sites in Amhara region. The PSP–E summative evaluation of the PPM–DOTS initiative and subsequent quarterly reports continue to show low rates of sputum smear-positive (SS+) patients, even lower than the national average and rates in the same regions in the PHC. The proportion of SS+ for all pulmonary cases is 48 percent nationally, but is lower in the private clinics, along with consistently high rates of extrapulmonary TB (EP TB). Several reasons could partially but not fully account for the higher rates of EP TB. First, the HIV epidemic may affect the type of TB contracted. Second, some patients who have both pulmonary sputum smear-negative (SS–) TB and EP TB may be erroneously classified as EP TB cases. One needs to consider the possibility of overdiagnosis of EP TB. Third, the low rates of SS+ may be attributed to the quality of sputum provided for microscopy, although the external quality analysis (EQA) results show high levels of concordance in most clinics participating. Regular supervisory activities, clinical seminars, and review meetings are opportunities to address these findings. Still, this needs to be further studied. Referrals: PSP–E has made great strides in strengthening communication between the public and private sectors, changing attitudes and practices, and, ultimately, ensuring that patients diagnosed in the private sector are accepted in the public sector. The evaluation team’s observations in sites visited confirmed that most clinics are referring the majority of their newly diagnosed TB patients (all forms of TB) to PHC services. These cases, therefore, are not registered nor reported through the PPM–DOTS program. Many clinics have maintained informal registers since the beginning of the PPM–DOTs program to record newly diagnosed patients and note to which primary health care unit (PHCU) they were referred. Recently, however, PSP–E has developed and distributed a new program register to systematically record information on all patients referred. Recent data from half of the 90 clinics are shown in Table 6. To address the high numbers of ―transferred out‖ patients (Table 7), patients are counseled to ascertain their ability to attend the private facility for DOT; if they cannot do so, they are referred to the nearest PHC facility closest to their home. In responding to problems encountered when public sites refused to accept patients diagnosed and referred by private sites, PSP–E worked hard to facilitate a change in attitudes and practices of the public services. According to interview responses at all sites, acceptance by the public sector of diagnoses made in the private sector has greatly improved, and in only rare instances is this still problematic. Table 6. TB Patients Diagnosed and Referred, or Registered and Reported as Cases Detected, for Quarter 2 Year 5 (January–March 2009) # of regions # of health facilities SS+ SS– EP TB Total Diagnosed and referred 3 45 (90) 400 894 1,049 2,344 100% 50% 17% 38% 45% 100% Information from CD in cohort during same period 3 82 (90) N/A N/A N/A 934 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 21 However, despite the large number of cases being diagnosed and referred to other sites for treatment, no mechanism has been developed to track these referred patients, or to clarify case-holding activities in the referral sites for these patients. In addition, the current PPM–DOTS operational manual for scaling up TB and TB/HIV activities does not address ways to manage referrals. If one considers that, in the most recent quarter (Table 6), 2,344 newly diagnosed TB patients were referred, the potential contribution to case detection is potentially enormous. Moreover, with no follow-up, the inability to track referrals constitutes a significant lost opportunity that, if not addressed, could well jeopardize the true potential of the PPM– DOTS initiative. This is especially true for patients diagnosed with SS+ TB, who are actively transmitting disease in their communities, and those with TB/HIV coinfection, who risk higher mortality if not placed in treatment quickly. Treatment outcomes: Review of the cohort data from PPM–DOTS sites in Addis and Oromiya (Table 7) reported in the summative evaluation of February 2008 describes treatment success for the sites as 56.5 percent, compared to 81 percent and 86 percent from the respective regions overall. These major differences are attributable to the numbers of patients who began but did not complete treatment and who most often were transferred, for whatever reason, to PHC facilities. Further analysis done by the consultants during their summative evaluation reveals important differences in both defaults and death rates; default rates are about 3 percent higher in the PPM–DOTS facilities, and the death rates are almost three times the regional rates. Both of these results were erroneously reported as similar in the summative evaluation report. There does not appear to be any evidence to substantiate higher death rates in the private health clinics, although anecdotal reports suggest that patients who are either dissatisfied with the PHC services or seriously ill seek care in private facilities, thus potentially biasing the results. Another reason for the high death rates reported here could be the quality of the data. This needs to be carefully followed up on. Case detection is reported on a quarterly basis, but although data for treatment outcomes are registered quarterly as well, they are collected and collated only annually. It is quite probable that there has been considerable improvement in some of the outcome indicators in the pending cohort, due to action taken by PSP–E to mitigate the numbers of transferred-out cases. *not included in the cohort analysis Transfers during TB–DOTs: Patients who are admitted into treatment but cannot continue to receive DOT at the PPM–DOTs site are referred to the health facility nearest to their dwelling. Only the total number of cases transferred out is reported to the woreda health office, as opposed to individual cases for follow-up. According to staff interviewed at the PPM–DOTS sites, rarely do they receive reports of treatment outcomes of patients transferred out from their service. As the PPM–DOTS program in Ethiopia TABLE 7. TREATMENT (TX) OUTCOMES FROM TWO REGIONS—ADDIS ABABA AND OROMIYA—FOR 2007, AS REPORTED IN THE SUMMATIVE EVALUATION OF PPM–DOTS, FEBRUARY 2008 COMPARISONS Q1 Q2 Q3 Q4 Total Addis/Oromiya TX completed 2 74 169 46 291 (44.3%) N/A Cured 11 23 25 11 80 (12.2%) N/A TX failure 0 0 2 0 2 (0.3%) 0.98% / 1.0% Defaulter 7 12 22 15 56 (8.5%) 5.4% / 6.6% Death 8 21 39 24 92 (14.0%) 5.9% / 5.0% Transferred out 19 30 49 38 136 (20.7%) 4.1% / 6.2% Transferred in * 0 2 3 7 12 * N/A Total evaluated 47 160 306 234 657 N/A TX success # (TX success rate) 13 (28%) 97 (61%) 194 (63%) 57 (24%) 371 (56.5%) Addis/Oromiya 81% / 86% 22 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA is a new intervention, the feasibility of feedback mechanisms for transferred clients should be addressed, at least for a sample of patients transferred. Problems with reporting back outcomes of transferred patients are common in public health care systems in many countries. Some countries have established quarterly review/monitoring meetings at the district or subdistrict levels where data on transferred patients can be shared among the staff members of the different facilities. This approach has addressed some, but not all, of the gaps in information. Any effort to address this here in Ethiopia will need to take both the public and private services into consideration. Systems for tracking defaulters: Some PSP–E reports describe tracking of defaulters as weak by virtue of the nature of the private sector service delivery and linkages with the public sector to ensure DOT. During the site visits the evaluators reviewed both new and old registers. In the most recent data contained in new registers, there appeared to be few defaulters. Other than attempts to contact the defaulter or their contact person by phone, no other recourse is currently set up. TB/HIV collaborative services: WHO defines a three-strategy approach to fostering TB/HIV collaboration that consists of 12 well-defined steps.10 Table 8 summarizes the WHO approach and indicates areas in which PSP–E activities in PPM–DOTS sites are in support of the approach. TABLE 8. WHO’S FRAMEWORK FOR RECOMMENDED TB/HIV COLLABORATIVE ACTIVITIES VIS-À-VIS TB/HIV COLLABORATIVE ACTIVITIES UNDERTAKEN BY PSP–E AND PPM–DOTS SITES TB/HIV collaborative activities in PSPE–E–supported sites through PPM–DOTS, PPM–DOTS in the Workplace, MCT, and HCT Comments A. Establish the mechanisms of collaboration A.1 Coordinating bodies for TB/HIV at all levels X Participation at all three regional TB/HIV advisory committees and more recently at the national level. A.2 Surveillance of HIV prevalence among TB patients X All PSP–E–supported PPM–DOTS facilities provide PIHCT; results are recorded in the new TB/HIV registers, collected regularly through quarterly reports, and reported to the regional RHB. A.3 Joint TB/HIV planning X Through participation at the RHBs’ annual planning activities in addition to integrated supervision. Technical Working Groups are not yet in place /functional at the zonal and woreda health offices. A.4 Monitoring and evaluation X Standardized indicators are collected according to national requirements. New national TB register incorporates TB/HIV indicators: offered testing, acceptance, results, referral, and CPT. B. Decrease the burden of TB in PLWHA B.1 Establish intensified TB case finding X PSP–E established ICF in MCT sites as part of risk assessment in all HIV-positive clients. On the risk assessment card, only two questions appear—cough for more than two weeks, and night sweats—out of the original 10 Stop TB Department and Department of HIV/AIDS, A Guide to Monitoring and Evaluation for Collaborative TB/HIV Activities, Geneva: WHO, 2004 WHO/HTM/TB/2004.342 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 23 TABLE 8. WHO’S FRAMEWORK FOR RECOMMENDED TB/HIV COLLABORATIVE ACTIVITIES VIS-À-VIS TB/HIV COLLABORATIVE ACTIVITIES UNDERTAKEN BY PSP–E AND PPM–DOTS SITES five. This was due to poor response to those symptoms. Only national recording formats are used in the HCT-only sites, and these do not contain risk assessment questions for TB, although all PSP–E–supported facilities train staff in TB/HIV. B.2 Introduce IPT INH is not currently available through the national program, although some private workplace sites have previously offered INH. B.3 Ensure TB infection control (IC) in high￾risk settings X Only a few aspects of TB IC are being implemented—and not consistently—in both private clinical and laboratory settings. C. Decrease the burden of HIV in TB patients C.1 Provide HIV testing and counseling X Most TB patients are offered HIV testing through PIHCT. Results for Q1 of Year 5 (January–March 2009): uptake of is 53%. Of those tested, 32.7% were HIV-positive. C.2 Introduce HIV prevention methods PSP–E is currently working with PSI to distribute condoms to the private facilities and MCT sites. Post-test counseling includes prevention and risk reduction messages for both HIV-positive and HIV-negative clients. C.3 Introduce CPT Not yet available through the NTP, but some physicians are prescribing CPT to coinfected patients. Patients referred to ART are provided CPT as part of the ART package. The NTLCP-E does not provide CPT. C.4 Ensure HIV/AIDS care and support X Coinfected TB patients are referred to ART sites closest to their home. For each region, PSP–E provides service directories to respective PPM–DOTS sites. Most PPM–DOTS sites continue to provide TB care after referral to ART. C.5 Introduce ART Currently all coinfected patients are referred for ART assessment and initiation. PSP–E is preparing 20 PPM– DOTS sites in Addis Ababa to initiate ART. During site visits in all regions, the evaluation team observed that most patients are offered HCT. However, as indicated in Table 9, less than 53 percent accepted being tested. 24 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA TABLE 9. TB/HIV IN THE PRIVATE HEALTH FACILITIES: QUARTERLY REPORT FOR THE PERIOD JANUARY 1–MARCH 31, 2009 Region # of facilities # of TB patients (all forms) # tested for HIV # who were HIV￾positive Prevalence of HIV (%) among those tested Oromiya 31 270 165 47 28.5 AA 16 342 156 53 34 Amhara 35 322 174 62 35.6 Total 82 934 495 162 32.7 Data provided by PSP–E Information, communication, and education (IEC) materials: The evaluation team visited the AIDS Resource Centre (ARC) in Addis Ababa and were introduced to most components of their interventions, including printed media and popular radio and television series. The team noted that few ARC materials addressed TB or TB/HIV coinfection, despite TB’s being a major cause of morbidity and mortality among PLWHA. Missed opportunities at various levels in the health centers themselves included a lack of patient awareness of ways to decrease the spread of TB. For example, posters in waiting rooms that address cough etiquette and cough hygiene are important in preventing infection, especially in those PPM–DOTS sites that will be introducing ART. During site visits, the evaluation team noted that few clinics had printed materials for TB and TB/HIV available. PSP–E is currently undertaking the design and printing of IEC materials—posters and brochures—for PPM–DOTS sites. HCT services: PSP–E has worked to strengthen HCT services in three different private sector settings. First, the project sought to strengthen existing HCT services in PPM–DOTS sites. Second, PSP–E worked with the RHBs to introduce CT services in PPM–DOTS sites which lacked HCT. Finally, the project also helped to strengthen or introduce CT services in facilities outside the PPM–DOTS group. In Year 2 of the project, the primary focus was on establishing a PPM–DOTS pilot in 20 sites in Oromiya and Addis Ababa. In December 2006, the Government of Ethiopia launched a one-year Millennium AIDS campaign (MAC) with the goal of increasing HCT utilization nationwide. As a result of PSP–E’s support for PPM–DOTS, the RHBs of Addis Ababa and Oromiya asked for PSP–E assistance to engage private facilities in HCT activities. In each region, PSP–E’s took a region-specific approach tailored to the state of HCT services in that region. In Addis Ababa, PSP–E strengthened existing private HCT services by providing supportive supervision, EQC, refresher training, rapid-test kits and vacutainer sets, better recording and reporting tools, and M&E training, as well as active social mobilization. In conjunction with the Ethiopian HIV/AIDS Counselors’ Support Association, PSP–E deployed mentors to help improve counseling skills. In just one quarter, from October to December 2007, 40 sites supported by PSP–E in Addis Ababa served 45 percent of the total clients tested in the city’s 178 total HCT sites. Through three different phases of the MAC campaign, PSP–E established relationships with 67 sites in Addis Ababa which deliver HCT services but are not PPM–DOTS sites. PSP–E is now providing these sites with continuing support. In Oromiya, the RHB had launched PPM–DOTS services in 2006 before it had actively engaged the private sector in HCT service delivery. The MAC encouraged the Oromiya Bureau to seek PSP–E help to engage private clinics in providing HCT services. Accordingly, PSP–E worked with the Oromiya Bureau to establish services as a pilot in 10 sites by providing the sites with same package of support as had been given in Addis Ababa. In addition, the project initially seconded counselors to the pilot sites. This pilot THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 25 demonstrated the quality of private sector HCT services to the RHB and also showed the benefits of collaboration. Based upon the pilot, the RHB issued licenses to the 10 private facilities. PSP–E trained the staff of these facilities, as well as staff at another 25 PPM–DOTS sites, as counselors, with the result that PPM–DOTS sites in Oromiya have now integrated HCT services into their TB programs. In Amhara, only 12 of the 35 PPM–DOTS sites had HCT services. PSP–E worked with the RHB to expand CT services to the entire group of 35 PPM sites by establishing an MoU, providing training, and introducing tools and systems for better recording and reporting. The project is now working with the RHB to help license the 35 PPM–DOTS sites to deliver HCT services. In all three regions, PSP–E is providing continued support for CT services in PPM–DOTs and HCT-only sites through provision of integrated supervision, basic VCT training, guidelines and protocols, recording and reporting formats, EQC, and systems strengthening through linkages with the local health office. The data in the two tables below (Tables 10 and 11) highlight the contribution of these two initiatives over the last quarter and demonstrate the potential for PSP–E to contribute to expanding access to HCT and integration of TB/HIV services in Ethiopia through the private sector. TABLE 10. HCT SERVICES DELIVERED IN 71 PPM–DOTS SITES, JANUARY 1, 2009, TO MARCH 31, 2009 AA = ADDIS ABABA) Region # of facilities # of HCT clients tested # of HCT clients who tested positive for HIV Seroprevalence of HIV(%) Male Female Total Male Female Total Male Female Total Oromiya 35 2,465 2,581 5,049 227 285 512 9.2 11 10.1 AA 16 4,702 5,050 9,752 347 456 803 7.4 9 8.2 Amhara 20 3,446 3,863 7,309 362 433 795 10.5 11.2 10.8 Total 71* 10,613 11,494 22,110 936 1174 2110 8.8 10.2 9.5 Data provided by PSP–E. Results shown here are independent of PIHCT for TB patients. TB/HIV results are shown separately. * Only 71/90 PPM–DOTS facilities reported due to shortage of HIV test kits 26 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA TABLE 11. HCT SERVICES DELIVERED IN 57 PRIVATE SECTOR POLICLINICS SUPPORTED THROUGH PSP–E* SITES FROM JANUARY 1, 2009 TO MARCH 31, 2009 Region # of Health Facilities VCT Clients Total Tested HIV-positive clients HIV Rate Male Female Total Male Female Total Male Female HIV rate Addis Ababa 55 3,715 5,115 8,830 227 349 576 6.1 6.8 6.5 Oromiya 2 116 207 323 9 21 30 7.7 10.1 9.3 Total 57** 3,831 5,322 9,153 236 370 606 6.2 6.9 6.6 Data provided by PSP–E. *These facilities are private sector (medium and higher) clinics located in Addis Ababa and Oromiya that provide policlinic services, but not PPM–DOTS or ART services. ** Only 57/67 facilities reported, due to a shortage of HIV test kits and the fact that some of the clinics had closed. Working towards Sustainability and Institutionalization To become sustainable, PSP–E–II should identify and continue to build the capacity of subcontractor organizations to undertake supportive activities, such as training, EQA, and mentoring. Also vital is strengthening the public sector to ensure that the systems are in place to support HCT, such as supply chain management, participation in training, and supportive supervision and M&E. Ensuring clarity of roles and responsibilities between the public and private sectors, as well as maintaining transparent and open communication, will continue to facilitate quality services. The progress made in establishing the 90 PPM–DOTS sites in three regions in the short time period is commendable and is largely due to PSP–E’s facilitation of the relationship between the public and private sectors. This relationship has been supported at all levels, from the national level to the RHBs and to woredas, subcities, and town health offices. However, supportive systems are still in the process of maturing. Private providers are generally confident about their ability to carry on if PSP–E project is not continued. However, many providers expressed concern regarding the public sector entities’ ability to maintain their roles in providing a regular supply of drugs, test kits, and other consumables, in addition to undertaking regular supervisory activities. Mobile HIV Counseling and Testing With the introduction and rapid scaleup of free ART in most countries affected by HIV/AIDS, including Ethiopia, the benefit of HIV counseling and testing has become much more apparent. Although HCT is integrated into general health services in Ethiopia, the uptake has been low, with an even lower uptake among at-risk populations. There appear to be many reasons for this: the results of initial PSP–E assessments of perceptions and barriers to HCT in July 2007 indicated that low uptake of HCT was due to facilities’ hours of operation, distance, long waiting times, and perceived lack of confidentiality. PSP–E developed, pilot-tested, implemented, and subsequently scaled up MCT in urban and semiurban areas along four of the major transport corridors linking Ethiopia to Kenya, Djibouti and Sudan. PSP–E’s MCT program has striven to deliver high-quality, confidential counseling and testing services and referral to HIV care treatment and support services targeting most at-risk populations (MARPs). USAID/Ethiopia THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 27 defines MARPs as including commercial sex workers, females involved in cross-generational and/or transactional sex, males engaged in transactional sex (including clients of commercial sex workers), young men with multiple sex partners, highly mobile workers, separated or divorced individuals, and pregnant women. National standards and guidelines have been adopted for training curricula, recording and reporting formats, protocols, and laboratory testing algorithms. MCT TEAMS are present in two sites per town. Each site is managed by a team of individuals: the site coordinator who generally supervises both sites; one health educator; one laboratory technician, who also oversees both sites in each town; a receptionist; five to six counselors; support staff; and security guards. Rapid-test kits are provided and logistics are facilitated by PSP–E, in partnership with the local stakeholders. Feedback mechanisms have been systematically applied to assess results through QA, with results of the QA fed back to the MCT teams and to the stakeholders in a timely way. Key activities have included rapid assessments in each town; consensus building with stakeholders; site selection within the towns; community mobilization to inform communities and create demand; implementation; M&E and QA activities for all aspects of the services, including the laboratory. The process and timelines associated with the MTC, well outlined in the PSP–E’s MCT operational manual,11 entail three sequential phases: (1) consensus building, design and preparation; (2) service implementation; and (3) post-activity follow-up. Roles and responsibilities are well outlined for each stakeholder, as reflected in the MoUs between the RHB and PSP–E. Details covered by the manual include aspects of setting up and implementing the referral system; subcontracting services, community mobilization; actual organization and implementation of the MCT services; QA; supervision; EQA; and M&E. The manual is a result of over one full year of implementation and lessons learned in the process. Meeting the needs of private providers PSP–E devoted significant effort to applying systematic processes to assess, pilot, refine, and scale up MCT services. Moreover, the project implemented continuous quality improvement in addition to using results from M&E to adjust services. Geographic locations of MCT sites are found in Annex 7. Highlights of the MCT effort, described in PSP–E documentation, include the results of the three rounds of assessments carried out to guide the design of MCT; rapid baseline assessments of towns in all participating regions; and copies of recording and reporting formats. In managing the MCT, PSP–E ensured that basic equipment, including tents, furniture and supplies, and all recording and reporting formats, was available. In addition, PSP–E assessed, trained, and supervised subcontractors responsible for providing MCT services. Finally, PSP–E was responsible for bridging and reinforcing collaboration between public and private providers, as well as between different levels of the health systems (RHB and town health offices), and linking them with a vast array of stakeholders at the community level. In all instances, PSP–E devoted appropriate and significant attention to detail, building the local capacity to provide services through subcontract arrangements to four local service providers; ensuring quality assurance of program activities; and developing mechanisms to monitor quality and provide regular and timely feedback to staff and communities. Interviews conducted by the evaluation team included local stakeholders, town health officials, and subcontractors. (For a complete list of all stakeholders, town health officials and subcontractors interviewed as part of this evaluation, please see Annex 2.) Meeting the needs of communities and clients 11 Private Sector Project–Ethiopia, Operational Manual for Implementing Mobile Counseling and Testing Services in Ethiopia, Addis Ababa: PSP–E, December 2008. 28 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Initial assessments undertaken on each new area provided information critical to the identification of specific high-risk groups. Pilots were then undertaken to develop and test the MCT mechanism. In all instances, PSP–E engaged stakeholders in supporting each town’s MCT schedule of programs. Once an agreement was reached with town officials, PSP–E undertook the following steps to launch the program: 1. Ethiopian private subcontractors were trained and employed to manage each round of MCT. To ensure confidentiality and to reassure clients being tested, all contractors employed to manage the MCT in a specific region were recruited from a region different from that in which the MCT was scheduled. (Please see Annex 6 for a completed list of all subcontractors). 2. PSP–E organized and managed a community mobilization effort to publicize each scheduled round of MCTs in each town. 3. Once trained by PSP–E technical staff, subcontractors implemented the MCTs during periods of one to two weeks per town in two separate locations in each town, fielding a staff of five to six counselors, a laboratory technician, and a receptionist. With the change in testing protocols to a finger prick rather than venipuncture, the laboratory technician worked with both teams to provide EQA and technical assistance to counselors. 4. EQC was carried out initially through blind rechecking of serum samples and, following the change in the protocol, through proficiency tests of each counselor’s ability to effectively read the results of the finger prick tests. In all instances, the results of the EQC were at or very close to 100 percent. 5. Following the completion of each round of the MCT, PSP–E attempted to confirm the referrals. 6. Finally, at the completion of the 2008 round of services, PSP–E organized review sessions with each contractor to improve service quality. Similar meetings were held with local groups to develop enhanced collaboration for the 2009 rounds. As indicated in Figure 5, from July 2007 through November 2008 a total of 69,174 clients were tested, of whom approximately 35 percent were female. As of December 2008, PSP–E had implemented MCT services in 40 towns in the Oromiya, Amhara, SNNP, and Afar regions. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 29 Figure 5. Total MCT Clients Tested by Gender and Age Category 0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000 18,000 < 15 15 - 19 20 - 24 25 - 29 30 - 34 35 - 39 40 - 44 45 - 49 50 + Age Category < 15 15 - 19 20 - 24 25 - 29 30 - 34 35 - 39 40 - 44 45 - 49 50 + F 6 7,406 6,312 3,676 2,146 1,851 1,169 782 1,118 M 5 8,344 15,727 8,067 4,306 2,717 1,867 1,360 2,314 PSP-E Mobile CT - Amhara, Oromiya and Afar July 10, 07 -Nov 9, 08 Total Clients Tested by Gender and Age Category A total of 69,174 clients were tested and 24,466 clients (35.4 % ) were females. The success of the interventions was enhanced by the quality of data used to design services and by careful planning and implementation of these services, followed by frequent and timely feedback to subcontractors and local government officials. PSP–E established systematic assessment methods and mechanisms for continual performance improvement to identify and address the needs of the MARPs. This underscores the need to be nimble, flexible, and innovative in terms of how, when, and where MCT services were offered. Client-sensitive services: In its review of PSP–E’s extensive documentation on the MCT and its field observations, the evaluation team noted that PSP–E and its subcontractors systematically adhered to guidelines established in the PSP–E operational manual. Through sensitization activities and discussions with many key stakeholders, PSP–E selected sites in each town to facilitate access to at-risk populations, establishing hours of operation adapted to each town to maximize uptake of services. The importance of establishing M&E systems from the onset is underscored in the findings of the 2007 report summative evaluation,12 where an assessment of client intake in the initial period of implementation of MCT in Oromiya indicated that only one-fifth of the respondents were female, suggesting a lack of mobilization activities and hours targeted to females as well as fear of stigma associated with participation. According to the data in this early report, the positivity rate in females was twice that of males, and ―separated‖ clients were almost three times more likely than married clients to be HIV-positive. These finding encouraged PSP–E and stakeholders to tailor services to ensure that more flexible hours were established. As a result, a number of MCT sites provided moonlight services or woman-HCT days and engaged females to promote the availability of MCT services by distributing coupons to women, ensuring their 12 PSP–Ethiopia, Implementation of MCT in Ethiopia: Preliminary Lessons from Oromiya Region, Addis Ababa: USAID Private Sector Program–Ethiopia, December 2007. 30 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA priority in accessing MCT services. Although the evaluation team noted improvements in the MCT’s ability to attract women, women continue to be underrepresented, representing only one-third of clients taking advantage of MCT services. This underscores the need for enhanced community mobilization efforts targeting women. Defining the MARPs: In information provided to the evaluation team, PSP–E characterized all MCT clients served between July 2007 and November 2008 by sociodemographic criteria, sexual behaviors, history of STI, and reasons for seeking MCT services. Details of this population’s HIV infection rates were also provided and compared well with the 2005 Demographic and Health Survey (DHS) findings for urban Ethiopia. HIV infection rate was 5.5 percent, with significantly higher infection rates among females than males (8.6 percent vs. 3.6 percent), confirming that the HIV epidemic is well established in the urban population and at disproportionately higher rates in women. Further analysis unveiled consistently high rates of HIV infection among female sex workers as well as in daily laborers and petty traders, the unemployed, and the widowed and divorced (both sexes in all). Young females 15–24 years of age have been among the most affected section of the population. The HIV infection rates documented in these populations are 4–12 times higher than the national single point prevalence of 2.1 percent. However, based on the information gathered during the MCT initiative and as illustrated in Figure 6, of those clients tested, females in the 30–34 year age group represented the highest percentage of HIV￾positives. Figure 6. PSP–E Mobile CT Amhara, Oromiya, and Afar 15–19 20–24 25–29 30–34 35–39 40–44 45–49 50+ Male 0.4% 1.2% 4.7% 7.7% 10.5% 9.6% 8.4% 4.9% Female 2.6% 7.6% 14.6% 16.3% 13.1% 11.2% 9.3% 8.8% Total 1.4% 3.0% 7.8% 10.5% 11.6% 10.2% 8.7% 6.2% THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 31 At the same time, as indicated by Figure 7, less than 25 percent of HIV-positive people met USAID’s traditional—but limited—risk-group definitions. The importance of this finding to increased case detection is that the MCT succeeded in identifying or adding to traditional USAID risk groups, including such new MARP definitions such as: Widowed women Female petty traders Unemployed women Widowed males Male day laborers Male petty traders Unemployed males Figure 7. Proportion of HCT Clients Fitting In Each USAID Definition of At Risk Group As indicated by Figure 8, less than 1 percent of the MCT clients fit four or more of USAID definitions cited in Figure 7, with 15 percent fitting only one of the definitions. The major implication for the future associated with PSP–E’s MCT initiative is that its approach to reaching out to an expanded population during the MCT initiative succeeded in identifying and responding to a significant number of formerly unrecognized cohorts of the at-risk population. 32 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Demand for services: To ensure quality of services offered in MCT, PSP–E has adopted national standards in counseling and testing, establishing limits on the number of clients each counselor can see in a given period of time. As currently specified in the operational manual, HIV counselors are limited to 15–20 clients per day. While the evaluation team judges that this limitation is highly appropriate, respondents to the evaluation noted that the imposed limitation on the number of clients per counselor resulted in a significantly large number of individuals being turned away on any given day. While those that were turned away were given priority for the next day of operations, respondents expressed a plausible but undocumented concern that many of those who were turned away did not return to be tested. Referrals: In developing comprehensive referral directories of public and private services for each region, PSP–E helped clients and providers of MCT programs to select the most appropriate center for follow-up of HIV-positive clients. The directories provide each organization’s name, contact person, type (e.g., Government of Ethiopia, NGO, private), services offered, hours of operation, and service eligibility criteria. Although many providers interviewed said that the directories had been useful for patient referrals, it does not appear that the directories have been updated. In addition, data on those actually referred are imprecise and require clarification. In theory, 100 percent of all HIV-positives should have been referred. In discussing the available data with PSP–E, the evaluation team was unable to arrive at an agreement on a precise or near-precise number of HIV-positives actually referred. This problem occurred because of discrepancies between (1) the HCT register kept at the site, which is the source used to compile daily and weekly reports to the woreda health office, and (2) the client intake form, which was subsequently analyzed in the PSP–E office in Addis Ababa. Client intake forms show a lower rate of referral of HIV-positives than do the registration logs. PSP–E is now working to improve accuracy of data recording on client intake forms. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 33 Tracking referrals: The mechanism for tracking referred clients is currently somewhat passive. PSP–E asked staff in nearby institutions to insert clients’ referral slips in boxes that PSP–E had placed in each institution. While the initial findings were low, there does seem to be a trend in improving the tracking of referrals. From March to August 2008, a total of 39,487 persons were tested with an infection rate of 5 percent. All HIV-positive clients received post-test counseling, and 87.0 percent (1,956 clients) accepted referrals to seek services at ART sites. By collecting the referral slips within a two-month period following the end of the MCT sessions, PSP–E was able to confirm that 520 (26 percent) of all those that accepted referral slips reached the referral facilities within two months. Referral rates: Reportedly, during the first quarter of Year 5 (October–December 2008), the referral rate increased to 70 percent. Recent data on MCT post-test counseling reveal that the majority (>75 percent) of HIV-positive clients were referred to different services, including ART (72 percent), follow-up counseling (23 percent), TB clinics (19 percent), hospitals (11. 3 percent), and social services (3.8 percent). Client satisfaction: In April 2009, PSP–E conducted an exit interview with MCT clients in 12 randomly selected service sites in the Oromiya, Amhara, and SNNP regions and Dire Dawa City Administration, with the objective of assessing the quality of services and the level of satisfaction of MCT clients. A semi-structured questionnaire was applied to a random sample of clients from these sites. Sociodemographic characteristics, including age and gender breakdown were reported from the 756 client-respondents featured in Table 12. Results of the survey indicate a high level of satisfaction (98.9 percent) with the services, the settings, and the infrastructure (95.8 percent). Clients generally agreed that that the amount of information provided to them during pretest counseling was adequate to decide whether to pursue testing (98.5 percent), to accept their results (98.1 percent) and change their behavior to reduce future risk of being exposed to HIV. Reaching communities and clients through social mobilization and IEC materials: Through partnerships and in collaboration with stakeholders, PSP–E social mobilization has included several activities, including printed materials (posters, banners, and flyers), roadside shows, community conversations, and megaphone announcements. Because much of this material is printed in Amharic and other languages, the evaluators were unable to evaluate their content. PSP–E has invested time and effort to characterize how clients became aware of these services, using lessons learned for planning future activities. For example, in Oromiya (December 2007), among all clients interviewed, posters were mentioned as the most effective means of communicating the importance and availability of the MCTs. Younger clients reported that road side shows and peer referrals were most effective in promoting knowledge on the availability of MCTs. TABLE 12. RESULTS OF CLIENT EXIT SURVEY IN 12 MCT SITES, MARCH 2009 Gender Number Percent Male 407 53.8 Female 349 46.2 Age Number Percent 15-24 418 55.3 25-34 213 28.2 35-44 66 8.7 45-54 23 4.4 >55 23 3.0 No response 3 .4 34 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Sustainability and Institutionalization The geographic coverage and the large number of clients tested under PSP–E–sponsored MCT services in Ethiopia would indicate that PSP–E’s MCT program represented an important contribution to monitoring HIV incidence with the PSP–E regions in Ethiopia. PSP–E data from the MCT intervention also represent an important evidence-based reference on the effectiveness of an MCT intervention. Although MCT has been implemented by local subcontractors, PSP–E’s role has been indispensable in terms of organization, logistics, routine monitoring of quality of services, and M&E. Accordingly, the evaluation team would recommend USAID/Ethiopia’s continued support of the PSP–E–designed MCT concept. THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 35 V. LESSONS LEARNED PROGRAM MANAGEMENT In assessing PSP–E’s response to the program’s stated objectives, the evaluation team has concluded that the following lessons can be drawn from PSP–E’s approach to the program management: Early engagement of stakeholders: As noted earlier, PSP–E’s transparent and participatory engagement of key stakeholders from the onset of the program development process represented a key to the program’s success in building a bridge of trust and collaboration between the private and public sectors. Systematic program development: PSP–E’s systematic application of its 13 steps to scaling up TB care and TB/HIV services constituted a thoughtful and inclusive approach to measured program development in an environment calling for a rapid scale-up of activities, with little if any historical reference on which to rely. Use of Ethiopian subcontractors: PSP–E’s innovative approach to the use of PSP–E–trained Ethiopian subcontractors to implement its technical program responded to the need for a focused, cost-effective strategy to address immediate program development needs while laying the foundation for long-term sustainability. Attention to the policy environment: In identifying and addressing policy barriers to private sector participation in the provision of national priorities in TB and HIV/AIDS service and care, PSP–E’s focus on engaging and supporting the national policy development process assisted the national and regional governments in more fully understanding and profiting from the true potential of the private sector. Importance of senior-level technical guidance: Although the PSP–E’s professional staff have exhibited dedication and professionalism in responding to their assigned tasks, increased exposure and access to a senior-level technical adviser with international experience and expertise would more fully equipped them to develop and sustain their capacity as technical advisers, both during the life of the program and in the future. WORKPLACE PREVENTION AND CLINICAL SERVICES In assessing PSP–E’s response to the program’s stated objectives, the evaluation team has concluded that the following lessons can be drawn from PSP–E’s approach to the implementation of its workplace prevention and clinical services initiative: The importance of listening to the client: As exemplified by PSP–E’s flexibility in shortening class periods during its implementation of the workplace prevention initiative, listening to the client and addressing a private sector client’s constraints is of paramount importance. Promoting prevention is difficult: As with any preventive health program, promoting HIV/AIDS and TB prevention in the workplace calls for employers and employees to set aside the time required from their work schedules to fully understand the implications of current behavior patterns and ways to bring about effective behavior change. Accordingly, those responsible for organizing and implementing the program need to devote significant hours and energy to working with employers and employees, advocating for the time needed to fully implement the training program for both peer educators and their workplace colleagues. In addition, program organizers need to build in sufficient time for ongoing support to the workplace following the formal training program. Developing a capacity for effective counseling: As employees within a workplace are reluctant to be tested for HIV because of fear of discrimination and stigma, clinical practitioners would benefit from the 36 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA development of a capacity to effectively promote the importance of early detection and access to treatment, care, and support. Workplace initiatives are high investment but low yield: As currently implemented, PSP–E’s clinical and preventive workplace initiative has proven to be high in investment and low in yield. However, the payoff in terms of client sensitization, the reduction in stigma and discrimination, and change in behavior may well be worth the effort. Maintenance of quality clinical services requires high patient load: To be effective, workplace clinical interventions from a workplace-by-workplace perspective require a reasonably high patient load, especially with reference to quality of microscopy services. Accordingly, a concentration on workplaces with significant numbers of employees, or pooling services between companies, may prove to be the best investment in terms of ensuring the quality of those providing diagnostic services, treatment, and care. PPM–DOTS AND TB/HIV In assessing PSP–E’s response to the program’s stated objectives, the evaluation team has concluded that the following lessons can be drawn from PSP–E’s approach to the implementation of its PPM–DOTS and TB/HIV initiative: Full participation of the government at all levels is essential: As noted earlier, PSP–E’s approach to engaging the government in building a bridge between the private and public sectors was a key to the program’s success over a decidedly limited period of time. This point warrants reiteration with respect to the project’s PPM–DOTS and TB/HIV initiative. Staff in many PPM–DOTS sites are confident about their ability to carry on these services—if the RHB and town health offices are able to provide a regular supply of TB drugs. Advocacy workshops were key to building trust and addressing critical challenges: Raising awareness among the public sector around PPM–DOTS, strengthening private–public linkages, and timely distribution of drugs and laboratory supplies all improved collaboration between the private and public sectors. The need for clarity in procurement of certain commodities: Lack of clarity on who should continue to provide test kits for HIV has resulted in confusion and frustration articulated by many clinic owners. Some have already opted out of those provided by the RHB and are purchasing their own. Importance of capacity development of the RHB and subcity/town health offices: Enhancing government officials’ ability to work effectively with the private sector is critical to any future efforts to maintain and scale up services, especially with regard to the supply chain management, data management, and supervisory activities. Importance of supportive supervision and feedback: PSP–E has made major efforts to provide regular supervision for basic aspects of TB–DOTS. This is vital to performance improvement. PSP–E has supported providers’ efforts to improve performance through problem identification and the development of action-oriented plans, with follow-up tailored to the individual clinics and addressed in other venues, such as clinical seminars. Flexibility in defining the supervisory checklist: During the establishment of the PPM–DOTS program, the supervisory checklists needed to be somewhat flexible to accommodate newer areas or areas that needed enhancement, such as infection control and laboratory systems and services. Addressing human resources for health (HRH) is an issue of recognized importance: In addressing the problem of staff retention, a PSP–E consultant has highlighted the emerging recognition of HRH as a concern for the PSP–E and for the PSP–E successor project. In identifying that the private sector’s success in retaining trained staff goes far beyond the importance of monetary remuneration and extends to THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 37 the issue of staff being recognized for their technical contribution, the PSP–E consultant helped frame the argument for an important initiative for the future. Demand for HCT services: While much of the PSP–E emphasis in TB–DOTS and TB/HIV was placed on the intricacies associated with launching a private sector TB–DOTS initiative, effectively integrating HCT with TB–DOTS represents a significant opportunity to address the demand for HIV/AIDS counseling and testing. Strong capacity of Ethiopian professionals: As noted earlier in this report, the PSP–E’s work with Ethiopian professionals and selected associations has demonstrated the important resource that these professionals and their organizations represent. Ethiopian professionals are uniquely placed to provide services such as training, EQC, and mentoring. This point warrants reiteration with respect to the project’s PPM–DOTS and TB/HIV initiative. Work with CT-only sites prepares them to implement PPM–DOTS and ART: By working with CT￾only sites, PSP–E has built their capacity and has identified the best ones, which can be engaged in expanding to PPM–DOTS and ART services. MOBILE COUNSELING AND TESTING In assessing PSP–E’s response to the program’s stated objectives, the evaluation team has concluded that the following lessons can be drawn from PSP–E’s approach to the implementation of its MCT initiative: Importance of thorough planning and management for establishing and scaling up MCT sites: Through its development and management of the MCT initiative, PSP–E has demonstrated the effectiveness of a thoughtful, systematic, and simplified approach to responding to the needs of high-risk patients for HIV/AIDS counseling and testing report. Key elements of the PSP–E approach have included the following: supporting strong partnerships and awareness; building capacity; using data from rapid assessments to further define services to target high-risk populations; employing flexible and innovative approaches to address barriers; conducting referral and follow-up; and systems to maintain quality of services. Importance of documentation: Through 18 months of MCT service provision, PSP–E has carefully documented the process and clearly defined roles and responsibilities for all partners, thus ensuring that, in the future, similar initiatives will have a clear record upon which to build and develop. In the delivery of MCT services, PSP–E’s attention to the importance of documentation has also contributed to ensuring needed feedback to PSP–E staff partners and communities. Reaching out to HIV-AIDS MARPs: Through its approach to expanding access to added categories of vulnerable at-risk individuals, the PSP–E has demonstrated the feasibility of ensuring that persons most at risk are provided with the opportunity to assess and respond to their need for prevention and treatment. Engaging local subcontractors: The importance of the PSP–E’s groundbreaking engagement of Ethiopian subcontractors is discussed in detail earlier in this report. Based on the evaluation team’s discussions with all of Ethiopian subcontractors, it is clear that entrepreneurship among selected private sector providers is thriving. These subcontractors are also involved in many types of activities in their own areas, such as expansion of their services to hospital settings, strengthening and expanding laboratory capacity to serve their many clinics and hospitals and, in one instance, building academic institutions associated with their clinical facilities. Providing all clinic owners with business training has supported these endeavors. 38 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 39 VI. RECOMMENDATIONS In recognizing that the Private Sector Program–Ethiopia Task Order is scheduled for completion at the end of September 2009, the evaluation team offers the following actionable recommendations for moving forward on the trajectory established under PSP–E: PROGRAM MANAGEMENT Support Ethiopian subcontracting mechanism: Given the assessed success of the PSP–E approach to the use of the Ethiopian subcontractors, the evaluation team recommends that the next iteration of the PSP–E (PSP–E II) continue to build on developing the capacity and use of Ethiopian subcontractors. The evaluation team further recommends that PSP–E II act as a catalyst in forming a consortium of current PSP–E subcontractors and in assisting the consortium in the development of proposals for sustained funding from donor agencies, such as the Global Fund, and from international and national entities. Document approach to building linkage and trust between private and public health sectors: In their rush to complete the program’s current portfolio of activities before the end of the Task Order, PSP– E staff might well neglect the importance of documenting their approach to building linkages and trust between the public and private sectors. Accordingly, the evaluation team strongly recommends that project staff take the time and devote the necessary thought to capturing the essential elements of PSP– E’s successful approach to working with and engaging the public sector in developing an environment of mutual trust and collaboration. Continue advocacy for an improved policy environment: While PSP–E has made significant progress in helping to enhance the public-private sector policy environment, the evaluation team recommends that, under PSP–E II, continued emphasis be placed on giving the federal and regional governments technical assistance in addressing current and emerging legislative and regulatory barriers to the private sector’s full engagement in Ethiopia’s response to TB and HIV/AIDS prevention and treatment. Establish linkage mechanism with public sector CAs: In the team’s evaluation of PSP–E, there was little evidence of collaboration between PSP–E and USAID collaborating agencies working on TB and HIV/AIDS within Ethiopia’s public sector. Accordingly, the evaluation team recommends that, under PSP–E II, USAID/Ethiopia take the lead in developing opportunities, such as joint public-private sector collaborating agency (CA) review meetings, through which the two sector’s CAs can work together in addressing the multitude of issues of common interest and importance to both sectors. Toward this end, the evaluation team also recommends that USAID/Ethiopia continue to support, through CA secondments, equipment donations, and supportive supervision, the Federal Ministry of Health and the Regional Health Bureaus as the Government of Ethiopia works with CAs to coordinate assistance between the public and private sectors. Recruit senior-level technical assistance: As noted earlier in this report, PSP–E failed to recruit the international senior technical assistance necessary to provide optimal technical direction to the PSP–E and to develop the capacity of PSP–E’s current technical advisers. According, under PSP–E II, the evaluation team recommends that providing the required level of senior level international technical assistance be established as a prerequisite for the award of PSP–E II. In the event that it proves difficult to recruit a long-term adviser of sufficient qualifications and expertise, the evaluation team would recommend that, as an alternative to a long-term adviser, a strategy for engaging a long-term periodic adviser (LTPA) committed to extended (one month or more) and periodic (once per quarter) technical assistance assignments be explored. Work with partners to address HRH issues for all cadres of professionals: As noted earlier in this report, PSP–E has taken the first tentative step in addressing the importance of HRH through its brief study on issues associated with employee retention within the private sector. The evaluation team 40 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA recommends that the program’s preliminary focus on retention be continued and expanded to include other critical HRH issues, such as job satisfaction, in-service supervision and support, and career development. WORKPLACE PREVENTION AND CLINICAL SERVICES In recognizing that the Private Sector Program–Ethiopia Task Order is scheduled for completion at the end of September 2009, the evaluation team offers the following actionable recommendations for moving forward on the continuum established by workplace interventions under PSP–E: Develop an enabling environment among management: As noted above, gaining support for workplace prevention initiatives requires the full and informed support of workplace managers. Accordingly, the evaluation team recommends that PSP– E II invest additional resources focused on developing workplace prevention advocacy strategies targeted toward senior company management. Develop a workplace training program strategy focused on an expanded base of significantly large companies: To ensure a greater return on the investment in training associated with clinical services, the evaluation team recommends that PSP–E II focus its recruitment efforts on companies with significant numbers of employees and thus potentially greater client loads. Consolidate or pool clinical services: In the event that large companies are not easy to recruit on their own and to ensure a greater patient load and to thereby work toward maintaining quality in the delivery of clinical services, especially with reference to microscopy and diagnostic and treatment services, the evaluation team recommends that PSP–E II develop a strategy to consolidate and/or pool services offered by participating private sector entities. Develop strategies to increase financial support for preventive and clinical services: Given that engaging in preventing and providing clinical services for TB and HIV/AIDS is a relatively new concept for the private sector, the evaluation team recommends that PSP–E II focus on working with insurance companies to develop strategies to reduce provider costs for those services. While the evaluation team understands that PSP–E has been unsuccessful in enlisting the support of insurance companies in providing coverage for preventive and clinical services, the evaluation team recommends that PSP–E II continue to explore ways to convince insurance companies of the ―value added‖ to their participation in the promotion of preventive initiatives and of early detection and treatment of both HIV/AIDS and TB. Develop long-range action plans for all recruited companies: While the PSP–E has achieved documented progress in enlisting private sector companies to support prevention education for its employees, the program has paid limited attention to the needs of private sector companies who have successfully completed the educational program. Accordingly, the evaluation team would recommend that, as PSP–E II expands its workplace preventive education program, more thought be given to developing a long-term and systematic approach to providing ongoing support for those ―graduated‖ companies. PPM–DOTS AND TB/HIV Hire a senior TB/HIV specialist: Although the PSP–E’s progress within a short timeframe is impressive and noteworthy, the absence of a senior staff TB/HIV specialist represents a significant program weakness and needs to be addressed in the follow-on to PSP–E. A full-time TB/HIV specialist, in serving as a mentor to other PSP–E staff, will also represent the program in its development of a bridge with senior government technical officials, will serve as a technical resource on issues such as infection control and TB diagnosis, and will lead the project in the design and implementation of operations research projects to address issues such as case management, TB and HIV/AIDS CT program integration, and patient referral. In the event that it should prove difficult to hire a full-time specialist for this position, THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 41 PSP–E II should consider engaging the services of a periodic full-time consultant (PFTC) to provide concentrated (one to two months) periodic assistance each year over the life of the future project. Hire a laboratory specialist: As many issues in improving laboratory services for TB and HIV/AIDS remain outstanding, the evaluation team recommends that PSP–E II recruit a full-time senior laboratory specialist. As the program expands to include the provision of ART, a full-time laboratory specialist will assume added importance. As recommended above, the program could consider contracting services for a laboratory expert if hiring a full-time specialist is not an option. Improve diagnostic capability: Improving private sector clinic diagnostic capacity is of importance not only to clinical but also to laboratory services. Of immediate importance is the fact that developing diagnostic capacity will entail including the provision of technical assistance and a definition for operations research, with a focus on identifying root causes of the low case detection rates and high rates of EP TB. Conduct operations research on referrals of diagnosed patients: As the vast majority of patients diagnosed in PPM–DOTS sites are referred to the PHC facilities for registration into the TB–DOTS program, the program should consider developing operations research initiatives to address issues such as the movement of patients, the time between diagnosis and registration, and, if possible, the treatment outcome. Priority should be given to sputum smear-positive patients and those that are coinfected with HIV. Added funds should be allocated to this activity as a priority matter. Clarify responsibility for the provision of test kits and laboratory reagents/supplies: The evaluation team recommends that PSP–E and PSP–E II focus on ways of ensuring that the Government of Ethiopia adheres to its MoU supply commitments. If the private sector should need to purchase these products and commodities, PSP–E II should consider promoting a ―pooled procurement‖ approach among providers that will ensure the quality of these products and a regular supply and result in affordable services for the clients. Establish criteria for scale-up of PPM–DOTS: Before expanding PPM–DOTS further, the evaluation team would recommend the following: With partners, define and agree on the expected minimum level of achievement with regard to treatment outcomes in the PPM–DOTS sites in each region. This agreement should focus on including treatment outcomes of patients under PPM–DOTS in the surrounding geographic areas, as well as the availability of support systems, such as supportive supervision of the program and laboratory components, and EQA; Together with partners, develop and undertake a small operational research study on a sample of patients who have been referred in all three areas. Based on these findings and in coordination with partners, develop a system for tracking patients; pilot-test; and, based on lessons learned, scale up as part of the PPM–DOTS program; Define and implement a needs assessment to quantify the availability of commodities and supplies at central and regional levels; Ensure that any new MoUs explicitly state the roles and responsibilities of all stakeholders, especially with reference to commitments on commodities needed for the PPM–DOTS sites. Pay attention to infection control: While of importance in all sites, infection control will take on added importance for those sites that will introduce ART services. All PPM–DOTS facilities should have a TB infection control plan which deals with administrative, environmental, and personal protection aspects. These are well defined in the new WHO/CDC document on TB IC and available on the WHO Web site, as well as in the new version of Guidelines for Prevention and Transmission of Tuberculosis in Health 42 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA Care Facilities, Congregate and Community Settings in Ethiopia (April 2009). PSP–E and any follow-on should ensure that this is a priority. As clinic owners are the key to determining an effective IC environment, the evaluation team further recommends that suitably trained and experienced PSP–E/RHB staff continue to emphasize working with clinic owners on the development of MoUs; on training, focused on technical and managerial priorities; and on providing quality supportive supervision. Ensure that the quality of supervision is maintained: The evaluation team recommends that PSP–E and PSP–E II update and increase the level of knowledge of the supervisory staff, especially in areas such as IC and laboratory services. Regular review meetings for the supervisors based in the central PSP–E office and the regions should be considered as an approach to updating skills as well as a way to review progress, discuss issues, and resolve problems. Implement a periodic program update initiative: Currently, clinical seminars are available but are targeted mainly to physicians. As opposed to direct supervision, program monitoring meetings bring together the TB–DOTS staff from the operational levels—in this case, the PPM–DOTS sites—along with program supervisors. This is an opportunity to review reports, discuss issues, find solutions to common problems, and update skills, as well as introduce new topics such as patient education, improved management of TB/HIV coinfected individuals, and integration of services. All staff, including DOTS nurses and laboratory technicians, should be encouraged to attend. This initiative could also serve as an incentive for staff and could contribute to staff retention. Strengthen the regional laboratories: The evaluation team would recommend a number of steps to improve laboratory services. These steps would include: (1) providing logistical support for the regional laboratories, especially as the program scales up to other sites, (2) supporting a quarterly review meeting in each region in addition to EQA and EQC—this is an approach used very successfully in several countries to improve the quality of services in the laboratory network, and (3) supporting the Ethiopian Health and Nutrition Research Institute (EHNRI) in developing a standard reporting format for the regions to address the lack of consistency in current EQA and EQC reports. This initiative would also facilitate comparison between regions; enable the national reference laboratory to produce one national report; provide a basis for identifying common problem areas; and focus attention on solving problems that are common throughout the country. Improve Annex B of the Laboratory Supervision Tool: The evaluation team recommends that improvement of Annex B of the Laboratory Supervision Tool should be undertaken in partnership with the regional laboratory staff and should also include infection prevention and control. Following these changes, staff will need to be brought up to date to ensure that the new information/skills become institutionalized during regular supervision. Conduct a pre-seminar and post-seminar assessment for each clinical seminar: The evaluation team recommends the use of systematic pretests and post-tests of clinical seminars as essential tools for assessing their quality and providing a focus for areas to be covered in subsequent seminars. Monitor adherence to MoUs: While MoUs are developed with the best of intentions, the evaluation team recommends that continued progress of the program would be enhanced by a periodic and participatory review of the initial conditions of all MoUs in the interest of monitoring adherence to the MOUs and of adjusting MoUs in the light of emerging issues and requirements. Assess and adapt IEC materials for TB and HIV: Many countries, with the support of United States Government funds, have invested in the development of IEC materials that can be adapted and used in the PPM–DOTS settings in Ethiopia: in clinic waiting rooms, as teaching aids for the DOTs nurse, and as materials that can be provided to patients and their families on the importance of continuing treatment, cough hygiene, and other priorities. Similar resources of didactic materials that could effectively be employed within Ethiopia’s private sector exist for HIV/AIDS prevention and mitigation, both THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 43 internationally and within Ethiopia through the innovative efforts of USAID/Ethiopia’s ARC. Accordingly, the evaluation team recommends that educational material be developed or adapted to address key areas of early diagnosis, links between TB and HIV/AIDS, and the importance of treatment completion. Continue to strengthen and support established HCT services in PPM–DOTS: As noted earlier, PPM–DOTS initiatives provide an opportunity to address clients’ potential need for HIV/AIDS counseling and testing. Accordingly, the evaluation team recommends that continued efforts be directed towards integrating HCT as a standardized component of the private sector’s PPM–DOTS services. Develop a strategy to reduce staff attrition: While it was beyond the focus of this evaluation, staff retention and turnover, especially in the private sector, was repeatedly raised by the evaluation’s respondents as a human resource and health service management issue of critical importance to private health service managers and owners. Accordingly, the evaluation team recommends that the follow-on project to PSP–E develop an operations research initiative to assess, monitor, and respond to the issue of staff retention among private health facilities. MOBILE COUNSELING AND TRAINING Continue interim support for the MCT initiative: The evaluation team strongly recommends that the concept and activities of the MCT receive continued USAID/Ethiopia support, at least on an interim basis, following the completion of PSP–E. Although PSP–E’s program design had not anticipated that the MCT initiative would be sustained through USAID/Ethiopia funding beyond PSP–E’s five-year timeline, the evaluation team, supported by the vast majority of stakeholders interviewed during the course of the evaluation, believes that the MCT initiative, as designed by PSP–E and as implemented by Ethiopian subcontractors, has made and will continue to make an important contribution to understanding and responding to Ethiopia’s current incidence of HIV/AIDS among the nation’s most at-risk populations. Use the MCT services to expand the knowledge base on MARPs: In the Ethiopian context, the MCT, organized in a simple but well-conceived manner, serves clients who may not go to health facilities for testing. It has also provided new insight into the composition of MARPs that has not previously been described. Accordingly, the evaluation team would strongly recommend that USAID/Ethiopia continue to support this initiative under PSP–E II. Extend coverage into identified “hot spots” and underserved areas with high demand: During the evaluation’s interviews, a significant number of stakeholders recommended that the MCT expand its activities to include currently underserved but recognized periurban and rural HIV/AIDS ―hot spots.‖ The evaluation team would like to lend its support to this recommendation. Support the development of an MCT-focused consortium among current subcontractors: As noted above, PSP–E’s program design had not anticipated that the MCT initiative would be sustained through USAID/Ethiopia funding beyond PSP–E’s five-year timeline. However, given the importance and success of the MCT and of the Ethiopian subcontractors’ development of their capacity to effectively manage the MCT technical process, the evaluation team recommends that USAID/Ethiopia support a concept through which the successor project to PSP–E would serve as a catalyst in promoting and supporting the formation of a consortium of subcontractors and in building the consortium’s capacity to obtain donor funding support, outside of USAID, for a subcontractor-managed expansion of the current MCT initiative. 44 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 45 VII. SUMMARY In September 2004, USAID/Ethiopia awarded a $2.8 million four-year task order to Abt Associates and its three consortium partners. The stated purpose of the Private Sector Program–Ethiopia was to expand knowledge of and access to affordable, high-quality private sector HIV/AIDS and TB services. In September 2006, the Ethiopian USAID Mission provided approximately nine million additional dollars to the task order to enable PSP–E to extend the program for an additional year so that the program could expand and build upon its activities in workplace prevention and clinical services, PPM–TB–DOTS and HIV/AIDS, and mobile testing and counseling. It is the assessment of this evaluation team that, based on all available evidence, PSP–E has succeed admirably in establishing the framework for progress on all three of the program’s principal initiatives. In its final evaluation of the PSP–E, the evaluation team has taken note of the following: 1. In establishing a strong base for PPM–DOTS and TB/HIV, the PSP–E, working closely with the Ethiopian Government and its private sector partners, has established Ethiopia’s reputation as a pathfinder in identifying and supporting ways in which the private and public health sectors can work together toward a common goal. 2. In working to build a bridge of collaboration between the public and private health care sectors in Ethiopia, PSP–E has succeeded in establishing linkages of trust between the two sectors and in paving the way for continued cooperation among health providers, program managers, and policy champions from both sectors. 3. With its attention to detail and a systematic approach to the implementation of its initiatives, PSP–E has firmly established the basis for a continuum for the private sector’s contribution to quality-based TB and HIV/AID services. 4. As a key element to the PSP–E’s success, the program’s emphasis, from its onset, on transparency and participation with all stakeholders—in both sectors and in the community at large—represents a program management lesson of highest importance. 5. While the PSP–E has made considerable progress in working to establish meaningful and effective linkages with the public sector, the capacity of the Government of Ethiopia to develop and support mechanisms, policies, and guidelines to enable Ethiopia to fully profit from the potential contribution of the private sector remains a challenge that must be addressed now and in the future. 46 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA THE PRIVATE SECTOR PROGRAM IN ETHIOPIA 47 LIST OF ANNEXES (AVAILABLE SEPARATELY) Annex 1. Bibliography Annex 2. List of Evaluation Respondents Annex 3. PSP–E Evaluation Interview Guidelines for Respondents Annex 4. PSP–E Evaluation, USAID Interview Format Annex 5. PSP–E Evaluation Interview, Summary Format Annex 6. Subcontractors Table, PSP–E Annex 7. PSP–E MCT Towns Annex 8. PSP–E Participants in Financial Management and Bookkeeping Course Annex 9. PSP–E PEPFAR M&E Framework Annex 10. PSP–E PPM–TB–DOTS Sites Annex 11. PSP–E Workplace Clinical Sites Annex 12. PSP–E Active and Inactive Workplace Sites (Prevention) Annex 13. PSP–E Indicators and Progress Report, October 2004–December 2008 Annex 14. Briefing Presentations on PSP–E Annex 15. Evaluation Scope of Work Annex 16. Evaluation Work Schedule Annex 17. Team Meeting Debriefing Annex 18. Preliminary Debriefing 48 THE PRIVATE SECTOR PROGRAM IN ETHIOPIA For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com