REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT AUGUST 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Benson Barh, Selam Kebrom, Cecelia Morris, Mbuyi Mutala, and Bruce Grogan (Team Leader) through the Global Health Technical Assistance Project. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT DISCLAIMER The authors‘ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 11-01-513) is available in printed or online versions. Online documents can be located in the GH Tech website library at http://resources.ghtechproject.net. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT i CONTENTS ACRONYMS ......................................................................................................... iii EXECUTIVE SUMMARY ....................................................................................... v I. BACKGROUND ............................................................................................... 1 Purpose of Assessment ............................................................................................................... 1 Methodology ................................................................................................................................. 1 Liberia Health Sector .................................................................................................................. 2 RBHS Project Overview ............................................................................................................. 5 Project Status ................................................................................................................................ 6 II. FINDINGS AND ISSUES ................................................................................. 9 Type: Programmatic and Technical ......................................................................................... 9 Type: Management/Administration ........................................................................................11 Cross-Cutting: ............................................................................................................................13 III. DISCUSSION: ................................................................................................. 15 IV. CONCLUSIONS ............................................................................................. 17 V. RECOMMENDATIONS:................................................................................ 19 Recommendations for the MOHSW .....................................................................................19 Recommendations for USAID .................................................................................................20 Recommendations for RBHS ...................................................................................................20 APPENDICES APPENDIX A. SCOPE OF WORK .................................................................... 23 APPENDIX B. PERSONS CONTACTED ......................................................... 33 APPENDIX C. LIBERIA PERFORMANCE BASED FINANCING (PBF) ASSESSMENT REPORT (FINAL DRAFT, JUNE 2011) ........ 37 APPENDIX D. DETAILED LIST OF KEY FINDINGS ..................................... 47 APPENDIX E. INTERVIEW QUESTIONNAIRES........................................... 55 APPENDIX F. RELEVANT USAID INTERMEDIATE RESULTS ................... 65 APPENDIX G. WORK PLAN AND TRAVEL SCHEDULE ............................ 67 APPENDIX H. POWERPOINT PRESENTATION TO MINISTRY OF HEALTH AND SOCIAL WELFARE ....................................... 69 ii REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT iii ACRONYMS AIDS Acquired immune deficiency syndrome ANC Antenatal care BCC Behavior change communication BPHS Basic Package of Health Services CFSNS Comprehensive Food Security National Survey CHAI Clinton Health Access Initiative CHSWT County Health and Social Welfare Team CHT County health team COP Chief of party CM Case manager CMO Chief medical officer DCOP Deputy chief of party DHS Department of Health Services DFID United Kingdom Department for International Development EBSNM Esther Bacon School of Nursing and Midwifery EHT Environmental Health Technician EmONC Emergency obstetric and neonatal care EPHS Essential package of health services EPI Expanded Program on Immunization FBO Faith-based organization FP/RH Family planning/reproductive health gCHV General community health volunteer GH Tech Global Health Technical Assistance Project GOL Government of Liberia HHP Health and hygiene promotion HIV Human immune deficiency virus HMIS Health Management Information System IEC Information, education, and communication INGO International non-governmental organization IPT2 Intermittent preventive treatment (of malaria), 2nd dose IRC International Rescue Committee ITN Insecticide-treated net JHUCCP Johns Hopkins University Center for Communications Programs iv REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT JSI John Snow Inc. LQAS Low Quality Assurance Sample M&E Monitoring and evaluation MOHSW Ministry of Health and Social Welfare MOU Memorandum of understanding MSH Management Sciences for Health MTI Medical Teams International NGO Non-governmental organization NHSWP 2011–2021 National Health and Social Welfare Plan OIC Officer in charge PA Physician‘s assistant PCT Program coordination team PBC Performance-based contract PBF Performance-based financing PPP Public-private partnership RBHS Rebuilding Basic Health Services RN Registered nurse SBCC Social and behavior change communication SOPs Standard operating procedures TB Tuberculosis TNIMA Tubman National Institute for Medical Arts TPM Team planning meeting TTM Trained traditional midwife WASH Water, sanitation, and hygiene REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT v EXECUTIVE SUMMARY The purpose of this non-quantitative assessment was to review the performance and the progress of the five-year Rebuilding Basic Health Services (RBHS) project, which is now at its mid-point, and assess the appropriateness of the project design. The assessment identified factors enabling or impeding the effective implementation of different components of the project, with emphasis placed on the performance-based contracting (PBC) component. The assessment advised the U.S. Agency for International Development (USAID)/Liberia on any necessary redirection of strategies or priorities that could modify the approaches currently used and/or provide suggestions on any possible expansion of the project. More specifically, the assessment team evaluated the progress made in achieving the three intermediate results of the RBHS Project. The methodology for assessment included interviews, data collection, and review of resources. Eight years after emerging from two prolonged and devastating civil wars, Liberia is beginning to make slow but measurable progress on a range of economic and social outcomes. The impact of the conflicts on the health sector was as severe as on any other, and included loss of staff, destruction of infrastructure, disruption of health programs, and diminishment of resources, along with the resultant increased dependency on international donors. The cornerstone of the Government of Liberia‘s (GOL) post-conflict national health policy, as formulated and endorsed in 2007, has been the rollout of a basic package of health services (BPHS) to all citizens. This BPHS is composed of a number of evidence-based, affordable health interventions designed to reach the population through a network of health clinics, health centers, and first-level referral hospitals across Liberia. The RBHS project is the United States Government‘s flagship bilateral health project to support the Ministry of Health and Social Welfare (MOHSW). The RBHS project is a $62 million-five￾year cooperative agreement (2008–2013) with John Snow Inc. (JSI), Research and Training, in collaboration with JHPIEGO, Johns Hopkins University Center for Communication Programs (JHUCCP), and Management Sciences for Health (MSH). The project supports the following USAID strategic objectives: Increased access to basic health services through improved provision of quality health services and adoption of positive health behaviors; increased quality of health services through improving infrastructure, health workforce and systems performance by enhancing capacity to plan, manage, and monitor a decentralized health system; and youth informed and networked on reproductive health. This evaluation concluded that the RBHS project is making significant progress toward achieving its goals and objectives. Specifically, the strengths of the project include: 1. The introduction of a data-driven culture 2. Evidence-based service delivery and supportive supervision 3. Some improvement in quality care 4. Timely provision of supplies and services 5. An active community volunteer network 6. A strong in-service training program. vi REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT The weaknesses of the project include: 1. Insufficient progress in building the capacity of the county health and social welfare teams (CHSWT) and mid-level management in the MOHSW 2. Lack of coherent strategy for measureable progress 3. Insufficient attention to infrastructure requirements and health care financing as per the original project description During the final two years of the project, RBHS will shift its focus away from service delivery toward a concentration on building the capacity of the MOHSW, including the CHSWTs, facilitating an effective management transition of the PBC program, and supporting the transition toward decentralizing MOHSW management . The overarching recommendations of the assessment team are that RBHS: 1) continue to support the MOHSW in the development of policies, procedures, and guidelines required for both a successful transition toward decentralization and MOHSW management of PBCs, and 2) develop the capacity of the CHSWTs to effectively supervise PBCs. It is the team‘s recommendation that RBHS employ a mentoring, or counterpart, approach to building the capacity of the senior, mid-level, and county-level members of the MOHSW. The team also recommends that the focus on health outcomes be pursued through a vision of health service delivery that stresses the importance of systems-thinking, which links all stakeholders from the facility through the ministry level. Another specific recommendation includes the need for an integrated health education program that employs all available and appropriate technology. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 1 I. BACKGROUND PURPOSE OF ASSESSMENT The purpose of this assessment was to review the performance and the progress of the five￾year RBHS project, which is now at its mid-point, and assess the appropriateness of the project design relative to the pending transition of responsibility to MOHSW. The assessment identified factors enabling or impeding effective implementation of different components of the project, particularly as they relate to the transition. Transition entails the gradual transfer or hand-over of the RBHS project management responsibilities of sub-partners from JSI to the central MOHSW for future ownership and management, as per the agreement between the GOL and USAID. It is understood that during the last two years of the project, RBHS will gradually cease providing health services directly and will instead provide support to central MOHSW for management of contracts for service delivery and CHSWT support. This support, in the form of technical and managerial assistance, will focus on skill sets to manage implementing partners. This major shift in the project‘s course is referred to as the transition throughout this report. The objective of this evaluation was to advise USAID/Liberia on any necessary redirection of strategies or priorities that could modify approaches that are currently being used and/or provide suggestions on any possible expansion of the project. More specifically, the evaluation team was expected to assess the progress made in achieving the three intermediate results of RBHS: 1. Increased access to basic health services through improved provision of quality health services and adoption of positive health behaviors 2. Increased quality of health services through improving infrastructure, and health workforce and systems performance by enhancing the capacity to plan, manage, and monitor a decentralized health system 3. Youth informed and networked on reproductive health The review included a cursory look at the progress made in relation to the operation of the six key principles in the program description: 1) participation; 2) partnership; 3) capacity building; 4) gender sensitivity; 5) youth focus; and 6) data driven. Additionally, the assessment team was to identify the lessons learned, provide suggestions for the future direction of Liberia‘s health initiatives to ensure a comprehensive and consistent maternal and child health strategy, and determine new areas for technical support relating to the upcoming transition. METHODOLOGY This is a qualitative evaluation designed to collect and analyze key stakeholders‘ opinions, perceptions, and experiences with the RBHS Project. The quantitative point of departure for this assessment was RBHS-generated data and other stakeholder information obtained from MOHSW, the Clinton Health Access Initiative (CHAI), and the World Bank (e.g., Health Management Information System [HMIS], Comprehensive Food Security National Survey (CFSNS), Expanded Program on Immunization [EPI] coverage, Low Quality Assurance Sample [LQAS], etc.). The RBHS Semi-Annual Report presented quantitative and self-reported data regarding the project‘s performance and progress toward meeting its goals and objectives. The RBHS data were drawn from the HMIS and crosschecked for accuracy. Other sources of data, though informative, were less useful. The Food Security and Nutritional Status could not be 2 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT directly correlated with the HMIS and RBHS data, given the survey nature of the data collection. The EPI coverage data were last collected in 2009 and are outdated. And the LQAS program recently released a draft report that is being studied by RBHS in order to identify and explain data results that conflict with the RBHS data. RBHS will report this to USAID upon completion of the review. The information provided by other stakeholders ranged from assessments by the World Bank of the PBC model1 to descriptive data collected during interviews with the MOHSW, United Nations, Global Fund, CHAI, and the Carter Center, during which the various steps the MOHSW is taking to develop revised policies, procedures, and regulations, as well as the efforts being made to collect real time service delivery data and annual accreditation data in an MOHSW partnership with the CHAI, were identified. The methodology used for collecting and analyzing the required information included: team planning meeting (TPM), document review, interviews with key stakeholders,2 site visits to RBHS health facilities, and direct observation by team members. This assessment fulfills conditions set forth by USAID‘s Evaluation Policy, which requires that each operating unit conduct at least one performance evaluation for each large project. In line with the policy, the evaluation team was composed of members with relevant subject matter expertise including program design, monitoring and evaluation (M&E), policy development, community health education, social and behavior change communication, community mobilization, human capacity building and training, nursing, midwifery, antenatal care (ANC), health system management and training. The inclusion of two Liberian health professionals also afforded an important socio-political context upon which to base findings. This study also supports USAID Forward‘s procurement reform by providing a basis from which to transfer the responsibility of health service delivery to the host government in a manner that minimizes the potential for diminishing quality of service. LIBERIA HEALTH SECTOR Eight years after emerging from two prolonged and devastating civil wars, Liberia is beginning to make slow but measurable progress on a range of economic and social outcomes. An illustrative example of this slow but discernable progress is the strong commitment by the GOL through the MOHSW to rebuild and reform its health system with the support of bilateral and multilateral assistance. They have done so by rolling out a basic package of health services intervention designed to reach all Liberians through the existing network of health centers, clinics, and first-level referral hospitals all over Liberia. Early indications of progress suggest that there have already been improvements in some important health outcomes. Infant and child mortality rates have reduced since earlier in the decade and now compare favorably with regional rates, as shown in Table 1 below. 1 World Bank, July 2011. 2 Key stakeholders included USAID/Liberia staff, MOHSW leadership, CHSWTs, health facility staff, and other donors/international agencies. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 3 Table 1. Key Health Indicators for Liberia (Source: RBHS Program Description) Health Indicator Liberia Regional Average Global Average Infant Mortality Rate (Department of Human Services [DHS] 2007) 72 97 47 Under-5 Mortality Rate (DHS 2007) 111 169 68 Maternal Mortality Ratio (DHS, 2007) 994 1100 400 The MOHSW has emerged as one of the strongest and most effective government entities in Liberia. Over the past four years, the MOHSW has demonstrated strong leadership and vision, developed a sound national health policy and plan, and collaborated effectively with its partners. While the health sector will require substantial external assistance for years to come, it is clear that the MOHSW is taking the lead on setting national policies, strategies, and plans. In July of 2011, the ministry released a new National Health and Social Welfare Plan (NHSWP 2011– 2021) consistent with the GOL‘s Poverty Reduction Strategy. The goal of this NHSWP 2011–21 is to improve the health and social welfare status of the population of Liberia on an equitable basis. As per the plan: Sustained leadership, stakeholder commitment, resources and effort are needed to achieve this by: 1) increasing access to and utilization of a comprehensive package of quality health and social welfare services of proven effectiveness, delivered close to the community, endowed with the necessary resources, and supported by effective systems; 2) making health and social welfare services more responsive to people’s needs, demands, and expectations by transferring management and decision-making to lower administration levels; and 3) making health care and social protection available to all people in Liberia, regardless of their position in society, and at a cost that is affordable to the Country.” The NHSWP 2011–21 expanded the BPHS into the Essential Package of Health Services (EPHS) to include eight additional areas: 1. Environmental/occupational health 2. Neglected tropical diseases 3. Non-communicable diseases 4. School health services 5. Prison health services 6. Emergency health 7. Mental health 8. Eye care The EPHS is an approach to health planning in post-conflict developing countries that aims to concentrate available resources on interventions that promote cost-effective and efficient ways of improving service delivery. While before the war the government was the main provider of health care—next to a number of faith-based organizations (FBOs)—during and after the war, international non-governmental organizations (INGOs) became main providers in a health system that was in disarray. Against a backdrop of governmental policy to decentralize, it is the MOHSW‘s vision that the CHSWTs will be responsible for deciding whether to continue contracting out for services or to deliver 4 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT these services themselves. This is the long-term vision, anticipated to take at least 10 years. In the near term, however, the MOHSW envisions that INGOs and local non-government organizations (NGOs) will continue to play an important role in assisting the CHSWTs in taking up this responsibility until the time when the CHSWT‘s have the capacity to implement themselves—if they so choose. The transition of management responsibility, including central contracting for the PBC‘s to the MOHSW, is a short-term solution. Eventually, the CHSWT‘s will have contracting authority. All donors and implementing partners use the same service delivery approach in Liberia: the BPHS (and now EPHS). There are, however, alternative models for funding this approach: a purchaser-provider model and direct government provision. The ―purchaser-provider‖ models are governed by contracts between fund holder and implementing agency. In the present case, the fund holder is the MOHSW with funding from the Pool Fund or from other donors. The present ―providers‖ are NGOs, except in Bomi County where they are not using NGOs as implementing partners but rather provide services through the CHSWT. In the future, they may be NGOs or the CHSWT themselves. The majority of present contracts are performance-based. RBHS (USAID Fund) and Pool Fund are similar models, though slightly different. RBHS contracts NGOs to support health service delivery in a number of counties. From the start this scheme has used PBC as the implementation vehicle. RBHS has been successful in creating detailed operational procedures and guidelines. In the RBHS model, targets are negotiated with the INGOs and set by RBHS for each contract, which includes several health facilities in a county. Contracts do not have countywide coverage. Performance is judged by monitoring and evaluating the implementing partners‘ previously agreed upon administrative indicators and service indicators, all of which are consistent with performance-based contracting. The Pool Fund was established within the MOHSW in 2009, mainly with funds from the United Kingdom Department for International Development (DFID) and Irish Aid, to support health service delivery. It contracted Bomi CHSWT to provide health care in 20 health facilities as a pilot on ‗contracting in.‘ By March 2010, the Pool Fund also contracted four INGOs to provide BPHS service delivery to 87 health facilities in six counties. All INGOs partnered with a national NGO. The MOHSW intended these contracts to be PBCs, but they didn‘t set up mechanisms to actually assess performance, nor did they pay bonuses during the first year. The Pool Fund Bomi experience is using the CHSWT to provide health care, whereas the RBHS model contracts with INGOs to implement the program. Since the geographic coverage of the Pool Fund differs from that of RBHS, there is no overlap in coverage areas. However, both employ the same data collection and information systems, as well as the same organizational structure. The MOHSW‘s decision to assume the RBHS scheme to manage service delivery for PBCs by June of 2012 reflects its commitment to assume management responsibility for performance￾based financing (PBF). This will more than double the MOHSW‘s direct involvement in PBC, hence meeting the need for good institutional arrangements, harmonized approaches between the different PBC schemes, and an operational plan that is widely understood by all stakeholders. A recent joint RBHS and World Bank assessment of the Liberian experience with PBCs concluded that they are well functioning, have produced detailed procedures, function with flexibility, and are effective at building the capacity of their implementing partners. However, the report found that there were some significant gaps: 1) there is a large need to build capacity at the county and central levels of the MOHSW; 2) there is a need for a formal agreement, possibly a memorandum of understanding (MOU), between the central and county entities; and 3) there is also a technical gap when it comes to supervising/mentoring the REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 5 performance of health service providers in the districts, in addition to requisite managerial and technical capacities lacking at the CHSWT level.3 Before the implementation of the PBC models, the MOHSW established a Program Coordination Team (PCT) to coordinate partners and resources for implementing the NHSWP 2011–2021. The PCT consists of the four deputy ministers supported by technical experts, and is led by the chief medical officer/deputy minister of health services. The PCT can and should play a key role in addressing PBC-related needs. The Policy on Contracting foresaw a comprehensive system for M&E of the NGO contracts to be established within the MOHSW, which would ensure that results met internationally agreed upon standards of financial management. Additionally, the MOHSW set up a PBC steering committee consisting of representatives from MOHSW departments and other stakeholders. More recent was the appointment by the MOHSW of a PBC focal point, under the Health Financing Department. Additional activities are under way within the MOHSW to develop—with RBHS support—a wide range of requisite policies, guidelines, and procedures, including human-resource–related requirements and system revisions, infrastructure guidelines, and the development of an operational manual for use in PBC management. Under the NHSWP 2011–2021, the MOHSW has committed to establishing appropriate structures (Program Management Unit, PBF Technical Team, PBF Steering Committee) to oversee PBF, improve communication with partners, strengthen existing systems of contract M&E, improve data verification processes, and establish transparent incentive systems. But at present, the management of the Pool Fund claimed that it is not prepared to assume complete responsibility for PBC program implementation.4 There is a limited support infrastructure of policies, procedures, operational guidelines, and human resources in place. Although many of these are under development (e.g., human resources, financial management, operations manual for managing and implementing PBCs), they have not been approved for use as of yet. Additionally, the staff capability to manage the contracts is limited, and there are still significant training needs associated with financial management. Although PBCs will be managed from the central level, supervisory responsibilities will reside with the relevant CHSWT. The long-term vision of the MOHSW is that the responsibility for managing performance-based partnerships will be shifted to the CHSWTs once administrative and evaluation processes have been developed. There is no specific timetable established for this devolution. At present, there are administrative, technical competency, and human-resource constraints on MOHSW‘s present ability to assume responsibility for PBF, and therefore they transfer part of that responsibility to the CHSWTs for oversight of the PBC agreements. RBHS PROJECT OVERVIEW The RBHS project is the U.S. Government‘s flagship health bilateral project in support of the MOHSW. The $62 million project, funded by USAID, is a partnership among JSI Research and Training, Jhpiego, JHUCCP, and MSH. The project is active in select districts in Bomi, Grand Cape Mount, Lofa, Bong, Nimba, River Gee, and Monsterrado counties The RBHS project has specific responsibilities in the areas of maternal and child health, family planning/reproductive health, malaria, human immunodeficiency virus 3 See Appendix C for the MOHSW Liberia PBF Assessment Report, Final Draft, June 2011. 4 Interview with Esther Mwanveza, Program Manager of the Pool Fund, July 2011. 6 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT (HIV), and water and sanitation, with the focus on: strengthening and extending service delivery through performance-based contracts to NGO partners; strengthening Liberia‘s health system in human resource management, infrastructure, policy development, M&E, disease prevention; promotion of health behavior change communication; and community mobilization. Implementation of RBHS is over a five-year period, 2008–2013, and is guided by a three-pronged strategic approach: 1. Strengthening and extending service delivery through performance-based grants to NGO partners 2. Strengthening Liberia‘s health system in the areas of human resource management, infrastructure, policy development, and M&E 3. Preventing disease and promoting more healthful behaviors through behavior change, communication, and community mobilization The RBHS project is implemented to achieve the following results: Increased utilization and coverage of priority health services, especially maternal, neonatal, child, and family planning/reproductive health Expanded availability of and access to services, including at community level (e.g., case management of childhood infections, family planning) Improved quality of health services, including improved health worker performance Increased adoption of healthful behaviors by community members Strengthened training institutions for mid-level health care providers Strengthened health systems, especially in the areas of health management information systems, M&E, and drug management Increased technical and management capacity of the MOHSW at central and county levels Improved health infrastructure PROJECT STATUS RBHS is overseeing the delivery of the BPHS at 112 facilities and their surrounding communities in seven counties through five PBCs with four NGO partners—and a grant to a fifth NGO partner. RBHS continues to document improvements through a range of important indicators, especially those related to maternal and reproductive health, and malaria. In terms of management and administration, it appears to the assessment team that RBHS is responding appropriately to the Liberian situation with flexibility and consistency by effectively using the capabilities of partners and supervising implementing partners; conducting joint supervision of health care delivery; introducing a data driven culture in the health system that is likely to enhance opportunities for good planning and decision-making; and providing capacity building opportunities such as pre-service and in-service training of health workers. According to RBHS5, systems strengthening activities have continued to improve both pre￾service and in-service training, supported rollout of HMIS, and made foundational contributions to the NHSWP 2011–2021. Behavior change communication (BCC) activities have included a national insecticide-treated net (ITN) campaign, facility- and community-level activities, and capacity building of national staff. RBHS also provides ongoing support to MOHSW efforts to implement PBF, including in those areas that RBHS currently supports. In many of its system 5 See Appendix H for detailed reporting on all indicators. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 7 strengthening efforts, RBHS has both provided technical expertise and attempted to build capacity at both central and county levels. Through this collaborative approach, RBHS reported the following achievements in the most recent semi-annual report to USAID (March of 2011). The team observed that there were some disagreements regarding specific numbers associated with some indicators. The team heard some disagreements at the final out-briefing for the MOHSW, but the consensus is that the results are still significant and positive. These reported results include: All 112 RBHS—supported health facilities providing BPHS included in the MOHSW‘s 2011 accreditation surveys met national accreditation standards (score: 100%) with an average score of 88%. The top two nationally scoring NGOs were RBHS partners Medical Teams International (MTI) with 92% and Africare with 91%. 66% increase in facility-based deliveries. 164% increase in couple-years of family planning protection. 69% increase in pregnant women receiving a second dose of intermittent preventive treatment of malaria (IPT2). Treated 112,750 children for malaria, averting an estimated 2,255 deaths. Reached 84% of the target population with messages on ITNs and documented a utilization rate of 78% among respondents and 80% of their children in households that owned an ITN. Tested 16,337 individuals for HIV, increasing by 67% from July 2010 through March 2011. Improved already high administrative performance, so that by the latest quarter 100% of facility staff were paid on time and 100% of HMIS reports were submitted on time. Participated actively in 24 national working groups, task forces, and steering committees. Made substantial contributions to the development of the Country Situational Analysis Report for the Health Sector, and drafts of the NHSWP 2011–2021 and EPHS. Despite these achievements, RBHS reported several areas in which performance and results were not as expected. These included: Delays in obtaining a USAID drug waiver and problems with the implementation of national supply chain standard operating procedures (SOPs) may be contributing to stock-outs. The team was informed that RBHS is actively working with USAID to identify ways to expedite the drug procurement process, as well as with the MOHSW on the finalization of the national supply chain standards. Some community-level BCC activities have been slow to start due to holdups in production of key information, education, and communication (IEC) materials and job aids caused by lack of local production capacity/expertise and delays in the training of community health volunteers. Emergency obstetric and neonatal care (EmONC) services have not progressed as far as planned because of inadequate infrastructure, equipment, staffing, and the postponement of the release of the MOHSW‘s own roadmap for the reduction of maternal mortality. RBHS continues to work with the MOHSW and partners on these issues. Among the most challenging issues faced by RBHS is the absorptive capacity of counterparts within the MOHSW and CHSWTs. RBHS takes its role in building capacity of MOHSW staff seriously, but several counterparts lack capacity, willingness, or time; therefore, the transfer of skills has been sub-optimal in many instances. 8 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 9 II. FINDINGS AND ISSUES The following are summaries of what the assessment team deemed to be ―key‖ findings obtained through the integrated data collection process.6 The findings as presented here do not reflect the opinions of the assessment team but rather are, the team hopes, an accurate distillation and packaging of the complete list of findings as recorded by the team. TYPE: PROGRAMMATIC AND TECHNICAL Taking into account the future direction of the project, there are two segments of critical issues/findings in the programmatic/technical area namely: 1. Strengths/achievements/successes of the project that must be built upon to maximize impact as the project moves toward the transition period. 2. Weaknesses/short-comings/hurdles that must be addressed during the period of transition in order to guarantee gradual scale-up and sustainability. The issues reported by RBHS, sub￾partners, CHSWTs, and health facility staff—and then validated by the evaluation team through field visits and review of documents—are described below. Strengths: RBHS: 1. Project introduced a data-driven culture in the national health care delivery system at all levels with the goal of informing planning and decision-making. 2. The patient load in the facilities visited in three counties demonstrated an increase in demand for services in catchment areas. This is an evidence of change in health-seeking behaviors since the inception of the RBHS activities, and the services must be meeting the needs of the care seekers. Services particularly mentioned were general care, ANC, delivery services, and delivery short stay in the facilities. 3. Generally, delivery of the BPHS is on track. This can be attributed to commitment and coordination at all levels (MOHSW, RBHS, USAID, sub-partners, CHSWT) to achieve the intended results. 4. RBHS is effectively delivering BCC/health promotion activities using a two-pronged approach: 1) Integrated activities addressing a wide range of health topics including six in the BPHS: maternal and newborn health, child health, adolescent sexual and reproductive health, communicable disease control, emergency care, and mental health; and 2) A series of intense campaigns for limited duration on one or two specific health issues, e.g., malaria, HIV/acquired immune deficiency syndrome (AIDS). These approaches complement and reinforce each other in that they target audiences at the household, health facility, community, county, and national level using media (radio programming) and interpersonal communication activities using a selection of IEC materials (Liberia RBHS SBCC Strategy Framework). 5. RBHS trained gCHVs and motivated them in various ways to sustain their enthusiasm in the dissemination of basic health information in communities. GCHVs also encouraged care￾seeking behavior, especially among pregnant women, and sometimes escorted them to health facilities for delivery. The network of community health volunteers (gCHVs, health and hygiene promotion [HHP], and trained traditional midwives [TTMs]) represent a key 6 See Appendix D for the complete and detailed list of key findings. 10 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT factor in the success achieved so far. They are motivated through provision of materials, supplies, training, and supportive supervision. 6. One of the strengths of the project is regular refresher training with the injection of additional skills to both the facility staff and the community health volunteers. 7. Service providers at the facilities received regular supportive supervision. This includes: feedback on previous visit, onsite training on various topics, technical and logistical/material support including IEC/BCC materials provision, and psychosocial support. This level of support for facility staff motivates them to enhance service delivery and improve outcomes. 8. RBHS support to central MOHSW includes: seconding a staff to the mental health department, liaising closely with the Health Promotion Unit, and helping in the establishment of a Community Health Unit directly at MOHSW. 9. RBHS is supporting the MOHSW in the development and production of standardized protocols, guidelines, and policy documents that would harmonize service delivery and enhance outcomes. 10. RBHS invites CHSWTs to join workshops with health care providers from RBHS-supported facilities in order to transfer the knowledge of policies, procedures, and operational guidelines, and thus increase the CHSWT‘s ability to assume a more active supervisory role. Additional benefits include the development of relationships between the CHSWT and the facility staff. 11. RBHS helps in making sure drugs and family planning commodities reach other health facilities. This will avoid stock-outs and help these facilities meet standards and enhance service delivery. 12. RBHS provides evidence-based service delivery through sub-partners. During field visits, facility staff, sub-partners, and CHSWT spoke of service delivery in terms of numbers (e.g., increase in clinic attendance, outcome of facility-based deliveries). 13. Both CHSWTs and sub-partners are making deliberate efforts to effect good coordination of activities through the coordination meetings, joint supervision, and on-time collection of reports from clinics for CHSWT and transporting Global Fund drugs for tuberculosis (TB) and AIDS control from CHSWT to clinics. MOHSW: 1. MOHSW employs a participatory, inclusive approach in the development of key policies, plans, and programs. 2. All health facilities visited were staffed in accordance with the MOHSW‘s BPHS. MOHSW, with the support of RBHS, is working on the development and production of standardized treatment protocols and guidelines to be provided to facilities for harmonization of activities and outcomes. 3. MOHSW has positively adopted a data-driven culture introduced by RBHS that would be used to inform good management and decision-making in service delivery. 4. MOHSW is fully committed to the delivery of quality and affordable health care services with the technical and managerial support/mentoring by RBHS. Weaknesses: RBHS: 1. To the extent that dual reporting of project staff does exist to MOHSW through the CHSWT and the sub-partners, it is time consuming for the officer in charge (OIC) and may affect data quality and validity. 2. In some instances, the recruitment of community health volunteers is carried out without the community involvement and engagement. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 11 3. Project design was reported by RBHS and sub-partners to be too ambitious. The targets, objectives, and indicators are reportedly too many. With a ladder progression approach in the implementation, sub-partners and facility staff are overwhelmed, and less attention is given to CHSWT capacity building. TYPE: MANAGEMENT/ADMINISTRATION Project management and administration is the basis upon which a project relies for the organization and coordination of activities in keeping with certain rules and regulations, with the purpose of achieving defined objectives. Management and administration lays the groundwork for support of the implementation of the project‘s technical activities. Given the complex nature and the scope of RBHS project, and taking into account the critical importance of management and administration functions during the transition and beyond, the assessment team took a critical look at JSI management of RBHS project. Like in the previous section, the team identified the project‘s strengths, as well as weaknesses to be addressed, during the transition in order to ensure continuity and sustainability. These issues were discussed in meetings with RBHS and with sub-partners, CHSWT, and facility staff during field visits. Strengths: RBHS: 1. JSI has developed a strong partnership with key stakeholders including MOHSW, USAID/Liberia, sub-partners, and other donors to support MOHSW in building its health system and implementing its national health plan. 2. JSI has put in place a strong team of experienced professionals serving as Senior Management Team of RBHS project. This team is directly supported by JSI Home Office and Partners. 3. JSI plans to transition several key positions in the management team, e.g., COP, DCOP, BCC advisor from expatriates to Liberian nationals during Year 3 of the project (through a counterpart approach). 4. JSI has successfully introduced an innovative, cost-effective strategy for financing health service delivery and health system strengthening in RBHS-supported areas through regular communication and monitoring, to ensure compliance with USAID management rules and regulations. 5. JSI has developed a strong M&E plan as a management tool that will enhance MOHSW governance. 6. Incentive paid to facility service providers by RBHS was reported to be regular, timely, and largely appreciated. 7. RBHS has successfully managed to establish a network of community health volunteers for community outreach service. The assessment team ascertained that the network is visible in the community through field visits. 8. Training, logistical support, and payment of performance-based bonuses provided by RBHS to the facility staff and non-cash incentives to the community health volunteers contributed to motivating staff/volunteers and keeping the project on track. 9. RBHS provides facilities with adequate equipment and supplies on a regular basis. No case of a drug stock-out was reported during site visits. MOHSW 1. The MOHSW affirms a central leadership role and commitment to transparency and accountability, which has resulted, in part, in several donors having made contributions directly to the Pool Fund. 12 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 2. The MOHSW‘s effective collaboration with partners enhanced its commitment and capacity to strengthen the health system and improve service delivery. 3. The MOHSW has developed a NHSWP 2011–2021, as well as established appropriate management structures and tools designed to oversee implementation of the health strategy. 4. The MOHSW has built a strong financial and accountability system within the ministry that will be an asset during the transition for transparency and sustainability. 5. The MOHSW decision to adopt the RBHS scheme within the ministry is an indication of its commitment to decentralization. Weaknesses: RBHS: 1. The RBHS project was designed to implement vertical funding, and yet it implemented horizontally in an integrated fashion. Each funding stream has its own set of indicators. This is cumbersome for sub-partners. There is a need to focus only on the strategic specific indicators as determined by the MOHSW. 2. As agreed to by RBHS, the lack of a baseline comprehensive assessment (programmatic, managerial, and administrative) of all CHSWTs by RBHS prior to commencement of project activities makes the identification of programmatic gains difficult to quantify and assess. 3. Health facilities whose PBCs do not meet their administrative indicators are also penalized. This is unfair and depicts a weakness of the project, as it has the potential to de-motivate health providers in affected facilities. 4. Field staff at visited facilities referred to delays in receiving drug supplies from MOHSW as a demotivating factor. 5. The role, responsibility, and position of county coordinator in the county health structure of the RBHS project are not clearly defined. Besides, not to provide office space for the coordinator in one county is to weaken cooperation and collaboration between CHSWT and sub-partners, much to the disadvantage of the project and its beneficiaries. 6. The level of effort required to cover all areas and health facilities by only an RBHS county coordinator for supervision is overwhelming and might likely isolate him further from the CHSWT. 7. There is no systematic information sharing by RBHS with other international organizations, particularly the INGOs and UN agencies. This precludes RBHS from having access to information from other organizations that could be used to strengthen the project. 8. RBHS has too many components for implementation, including managing sub-partners; innovating finance approaches; strengthening service delivery; scaling up access to BPHS; providing essential medicines and improving drug management; mobilizing and informing communities; delivering services at community level; renovating health facilities; promoting public-private partnership; and building institutional and human capacity and other vertical interventions such as malaria, HIV, TB, and nutrition. Some components such as infrastructure renovation, financing, and promoting public-private partnership are potentially diverting focus from key technical aspects. 9. There is lack of communications capacity between health facilities and sub-partners. This undermines timely information on notifiable diseases, reportable diseases, and referrals. 10. In some instances, CHSWT in one county reported that sub-partners initiated strategies and activities without consulting CHSWT for direction and guidelines on specific issues. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 13 MOHSW: 1. At the level of the central MOHSW, there is no budget for the Community Health Service Division and the Health Promotion Division, two key divisions in enhancing implementation of project activities at the community level. The implications are that service delivery may suffer and that there is no consistency between and among the MOHSW, CHSWT, and sub￾partners. For example, according to the revised MOHSW plan, TTMs are no longer supposed to do deliveries, but rather are being directed to assess pregnant women for danger signs and escort them to facilities. 2. At present, the MOHSW staffing pattern favors central administration over the CHSWT level (e.g., most health facilities visited are understaffed). Health care providers are not uniformly identified on the MOHSW payroll. 3. The success and sustainability of the project depends heavily on community involvement and behavior change amongst the care-seekers, yet the health promotion division at the MOHSW central is weakly represented in all the CHSWT structure and function. 4. As agreed to by RBHS and MHSW in discussions with the team, dual reporting, if it exists, to CHSWT and to sub-partners by project facility staff is time consuming for OICs. 5. Delay in disbursement of the quarterly subsidy to CHSWTs makes it difficult for them to motivate the network of community volunteers. This also impedes regular supervision of activities and reduces the probability of reaching targets. 6. The reported lack of budget lines for the Community Health Services and the Health Promotion divisions at MOHSW central filters to the CHSWT level, hampering adequate delivery of community health services (e.g., BCC/health promotion at the household, individual, and community levels). 7. There are difficulties with motivation and retention of staff in all facilities (RBHS-run and government-run). One contributing factor to low motivation is the low level or lack of income. Although incentives are not the only motivating tool, the incentives currently paid to health workers are based on civil service ceilings, which are low. Such low incentives do not attract health workers to work in rural areas. Alternatively, many choose to work in private facilities in Monrovia where there is potential for more income. 8. The lack of a clear MOU or another administrative instrument between the MOHSW, the CHSWT, and sub-partners negatively affects the working relationship between CHSWT and sub-partners, which in turn affects CHSWT performance. 9. Currently capacities vary widely from county to county. In certain cases, such as the county health officer and hospital medical director, the requirements of simultaneously being clinicians, managers, and administrators are cumbersome and should be separated. 10. Future CHSWT composition might include the county health officer, a hospital medical director, a community health department director, a county health service administrator, a county surveillance officer, a county health education officer, and a county M&E coordinator. CROSS-CUTTING: The cross-cutting issues identified in the Scope of Work (SOW) have been discussed in the technical/programmatic and management/administration sections. The assessment team deemed it necessary to focus only on RBHS strengths in this section. RBHS Strengths: 1. RBHS is well integrated with the MOHSW, particularly at the county, facility, and community level where activities are being implemented. Two good examples of how RBHS has made a significant difference in how the MOHSW operates are the introduction of a 14 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT data-driven culture at all levels in the health system and the strengthening of the HMIS. The establishment of a culture of use of information for planning and decision-making is improving service delivery quantitatively as reported by facility staff during site visits. If maintained, this culture will strengthen service delivery in a sustainable manner beyond the life of the project. 2. RBHS‘s planned strategy to transition senior management functions from expatriate staff to Liberian nationals during Year 3 of the project is a strong support to the MOHSW assuming leadership and a rational move toward sustainability. Other elements of progress toward sustainability include institutionalization of PBF and standardization of health policies, procedures, and capabilities at all levels. 3. One good example of work done by RBHS to strengthen national institutions, professional associations, and NGOs is the Pre-service Education Initiative (PSE), which was designed to improve the secondary/undergraduate curriculum of mid-level health care providers, the educational environment in both the classroom and clinical sites, and the overall management of selected teaching institutions. Developed in collaboration with the MOHSW, professional, and regulatory bodies, and other key stakeholders, PSE is directed primarily at improving the level of education for trainee registered nurses (RNs). Physician‘s assistants (PAs), CMs, EHTs at two schools (The Tubman National Institute of Medical Arts [TNIMA] and the Esther Bacon School of Nursing and Midwifery [EBSNM]). PSE has already led both educational and clinical standards for the training institutions. According to RBHS, several of the strategies, standards, and tools developed by PSE have already been adopted by six other training institutions for mid-level health workers, as well as the associated professional boards and associations. 4. To ensure coordination and synergy with the GOL, JSI has developed a partnership with key stakeholders including the MOHSW, USAID/Liberia, sub-partners, other donors (e.g., Pool Fund) to support the MOHSW in building its health system, developing and implementing its NHSWP 2011–2021, and thus contributing to USAID/Liberia intermediate results 1 & 2 and assisting the GOL in meeting its health millennium development goals. The operationalization of this partnership is still evolving in areas such as data sharing. 5. RBHS chose to address youth- and gender-related issues within several components of the project, including reproductive health, HIV/AIDS, maternal and child health, and in-facility deliveries. Addressing youth and gender issues within established key components affecting adolescent girls and boys, as well as women‘s lives, is more practical, cost-effective, and likely to have an impact since these components are already receiving much attention. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 15 III. DISCUSSION: The USAID-funded RBHS project appears to be a good example of cooperation between the GOL and the U.S. Government. RBHS demonstrates the desire of both governments to work together in implementing activities leading toward strengthening the Liberian health system in the post-war period and improving the lives of people severely affected by the 14 years of social unrest. The health and lives of men, women, children, and youth are significantly being positively affected by the services of the project, as reported by the staff of the project‘s health facilities that were visited in Cape Mount, Bong, and Nimba Counties. RBHS reports that the project has made notable achievement in service delivery both at the health facility and community levels. A deputy MOHSW minister lauded the project as being ―ahead of the curve‖ in its endeavour to strengthen the Liberian health system and to improve service delivery all over the country. The fact that RBHS is a recognized partner with the government and other development partners bodes well for the project‘s ability to continue to show leadership at the central level and below. The project at this point appears to be on track. It is important to acknowledge the Liberian context within which the project is being implemented. The health sector was making significant gains in improving the lives of its citizenry before the 1989 civil war. The socio-economic decline that followed eroded all the gains made in previous years. The health sector in Liberia continues to evolve in the post-war period. Health services during the war years were funded by the UN system and the international community. Presently, the MOHSW is taking a strong leadership in decision-making and delivering the health sector activities. Many critical policies, plans, criteria, organizational units, management, structures, and implementation of plans are being developed. Some are near completion and others are adopted as the NHSWP 2011–2021. The RBHS project is being implemented in an environment where expectations are high at all fronts, the health system is being rehabilitated and/or reconstructed, and plans and policies are being formulated. This situation creates both an opportunity to contribute to the process for the development of a sustainable health system and a challenge to deal with many factors, realities, and policy limitations in navigating through the process. In Liberia the modus operandi at this moment in time calls for flexibility, patience, and understanding as the enabling environment for health service delivery is being crafted. The RBHS has already had one revision in response to the evolving situation, and is about to experience another to direct its final two years of effort in ways that are supportive of the government‘s commitment to a path of decentralization self-control. Further complicating the equation is the fact that many of the programmatic and technical areas in which RBHS has, can, and is expected to make significant contributions are still being defined within the context of the MOHSW‘s new strategy and governance plans. It is clear to the assessment team that RBHS and all stakeholders need to adjust their thinking to assume more of a ―systems‖ approach to health care governance and the realization of desired health outcomes throughout the country. Projects need to be designed with targets that accurately reflect systemic considerations in all areas including but not limited to: planning, coordination, program integration, outcomes, management, M&E, and the consistent institutionalization of policies, procedures, and capabilities, not only at the central level, but also at the county, district and facility levels. The network of community health volunteers is one of the major successes of the RBHS project. For a project whose achievements depend on the improvement of health-seeking behaviors and the adoption of healthier behaviors by the community, the role of this category of 16 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT community health workers is paramount. They are needed to effect changes at the individual, household, and community levels. However, the project needs to take a step further in ensuring active community involvement and commitment, which will pave the way to sustainability of the project‘s activities. Therefore, the communities should be consistently involved in the recruitment and motivation of these volunteers. Moreover, the issue of motivation of these volunteers needed a detailed discussion in a national health forum. An analytical review of the gCHVs scope of work will help MOHSW and RBHS decide how to motivate them. For example, facility staff in Grand Cape Mount County give cash incentive to TTMs in addition to the non-incentive from the project. Each woman escorted by a TTM to deliver at the facility is asked to pay $LD350. This amount is then given to the TTM by the facility staff. This motivates TTMs to escort pregnant women to the facility for delivery. This community decision to motivate volunteers should be also discussed in order to find innovative ways for motivating volunteers. Finally, RBHS has introduced a data-driven culture into the health system. Staff also show off their newly earned technical skills. This is a laudable achievement. However, if this newly acquired skill is not nurtured and sustained it may become counterproductive. In the next phase of the project, emphasis should be placed on quality of service, quality of data generated, quality of support given, and the number of meetings and trainings conducted. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 17 IV. CONCLUSIONS The evaluation team documented that the RBHS project has made significant progress in meeting its objectives, with the weakest areas being associated with capacity building, infrastructure development, and public-private partnership (PPP). The evaluation team also documented that: 1. RBHS receives strong technical support from USAID and the Mission, and has been very responsive in turn. 2. RBHS sub-partners are committed, engaged and supported by RBHS. 3. CHSWTs are committed and engaged as well, but are still frustrated by the inadequate technical capacity building and logistical support they receive. 4. Coordination and collaboration between RBHS sub-partners and CSWHTs are not working properly. 5. Insufficient attention is given to the quality of services provided. 6. Ultimate output of the project is not expressed in terms of ―health outcomes,‖ such as changes in health profiles of communities or families, but rather in terms of numbers of a unit of measurement defining an action—not an impact. The assessment team recognizes a need for the development of detailed implementation plans to guide activities at all levels, both during the transition and beyond. These plans must be integrated within and between all levels of the health care delivery system and must be supported by an interrelated set of policies, procedures, guidelines, rules and regulations, and other systemic and institutional arrangements necessary to create and maintain a supportive, enabling environment. In the light of the above findings, issues, discussion, and conclusions, it appears that RBHS has a significant role to play in a number of areas critical to the ultimate creation of an effective and sustainable health care delivery system, both owned and operated by the MOHSW, in Liberia. Regarding the MOHSW‘s assuming responsibility for managing the PBC program, the assessment team is deeply concerned that there may be deterioration of the quality of services if the plan is not implemented at a pace that is responsive to the capacity of both central MOHSW staff and CHSWTs to manage. The transition should be carefully managed and timed to prevent deterioration of service delivery. Another major theme of the findings is related to the need for the MOHSW to develop systems that will help recruit and retain health workers, particularly those assigned to rural areas. This is especially important for the MOHSW‘s commitment to decentralization. In order to institutionalize a rational human resource infrastructure, many issues need to be revisited— including salary levels and incentives for service. The structure of the CHSWTs should be revisited to ensure that necessary skills and capabilities will be in place and able to function effectively within a decentralized system. Some possible county health system functions may include: planning, data collection, analysis and consolidation, M&E, surveillance, stakeholder coordination, health education, service delivery, and financial management. 18 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT Building the capacity of the health care delivery system in Liberia will also require strengthening the academic training, including curricula and in-service training programs, and programs in health care-related fields, in order to produce more qualified graduates in areas such as nursing, midwifery, public health, physician assistant work, etc. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 19 V. RECOMMENDATIONS: The following recommendations reflect the assessment team‘s belief that the approach used by RBHS and other implementing partners in Liberia should reflect a shared commitment to systemic reform and systemic capacity building. Therefore, the team offers these recommendations in the belief that, (1) activities are all interrelated, and that the cumulative impact on decentralization and the transition to government ―ownership‖ is what needs to be orchestrated, and (2) the ultimate measure of success will be in terms of both the sustainable provision of quality health care and the measurable health outcomes. Some of the recommendations, particularly those to be undertaken at the central MOHSW level, are identified as a recognition that progress may already be being made in some of these areas by other parties, including the development of policies and procedures to guide human resource management in the ministry, financial management, etc. In such cases, the team believes that RBHS can and should make contributions, as they have in so many ways to date. By being active and visible on a daily basis, RBHS can work to ensure that the county health care delivery system is not only represented but also, and perhaps most important, is functionally integrated within the Liberian health care system. RBHS has an important role as a ―bridge‖ between the county health care delivery system and the central ministry, facilitating the institutionalization of the relative roles. Recommendations are made for the MOHSW, USAID, RBHS, and grouped into three categories: programmatic/technical, management/administrative, and cross-cutting. RECOMMENDATIONS FOR THE MOHSW Programmatic/Technical 1. Consider the need for additional public health indicators that measure specific health outcomes for use at the central and county levels. 2. Enhance the human, financial, and logistical capacity of the Health Promotion Division at the central and county levels to allow it to take leadership in rolling out BCC activities during the transition and beyond. The MOHSW should consider expanding the translation of current and future BCC/health promotion programs in local Liberian languages for a broader reach and efficiency. 3. Review the previous RBHS experience using cell-phone free airtime and messaging targeting women and youth, assess what went wrong in the partnership with the cell phone company, negotiate a new contract adequate for both parties, and revive this activity, which is likely to reach a large portion of the target audiences nationwide where there is network. 4. Develop a pilot community wellness program to include mental health modules (surveillance, education, and services). Management/Administrative 1. Take leadership at all levels in the management and coordination of the RBHS project activities and effect actual coordination among stakeholders and donors. 2. Strengthen public heath leadership at the central and county levels to ensure continuity in and integration of policies, procedures, capabilities, and operations from central down through facility levels, with particular attention being given to mid-level managers and CHSWTs. 3. Design a model CHSWT structure and areas of functional responsibility. 20 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 4. Develop a county-wide facility and equipment infrastructure development, facility management and maintenance plan, and equipment maintenance plan, as well as related local servicing capability. 5. Revise human resource training, recruitment, and retention policies and programs to ensure equitability, consistency, and competency at all levels of the health care delivery system in Liberia. Such things as salary scales, incentives, and academic training programs must be considered and, if appropriate, revised. 6. The structure of the CHSWTs should be revisited to ensure that necessary skills and capabilities will be in place and able to function effectively within a decentralized system. Combined functions of county health officers and hospital medical directors as managers, administrators, and clinicians should be separated out and assigned to others as appropriate. RECOMMENDATIONS FOR USAID Programmatic/Technical 1. Consider reducing and focusing project indicators in order to reduce the administrative burden, and focus on specific health outcomes and quality of service. 2. Consider narrowing RBHS scope of operation by removing the infrastructure, health financing, and PPP development elements from the Project Cooperative Agreement and assigning them to other entities that specialize in each of those areas. This will allow RBHS to focus on targeted outcomes and quality of service. The transferred activities should, however, be closely coordinated with the RBHS project to facilitate planning and support effective service delivery and health outcomes. Management/Administrative 1. Negotiate with the MOHSW a reasonable (one or two year) transition period, continue capacity building support, and see what political change will bring. 2. Emphasize collaboration between RBHS and other USAID health projects in order to realize synergies and cumulative impacts achievable through leveraging. RECOMMENDATIONS FOR RBHS Programmatic/Technical 1. Conduct jointly with the MOHSW a comprehensive assessment of the CHSWT and the whole county health sector‘s (CMO), county health team [CHT], administrators, network of volunteers, other local stakeholders) capacity and potential to determine the capacity￾building needs to be addressed in order to support the MOHSW policies and objectives, as well as strengthen community health strategies and services. 2. Implement a mentoring program at MOHSW central through a counterpart approach to strengthen the technical, managerial, and information management capacity of MOHSW central in senior positions including: deputy CMO/assistant minister for curative services, deputy CMO/assistant minister for preventive services and managers/directors of strategic programs and divisions including the Health Promotion Division, the Community Health Services Division, the Family Health Division, the National AIDS Control Program, National TB/Leprosy Control Program, the Environmental Health Division, and other key management units. Technical capacity building should specifically include information technology at all levels. 3. Facilitate the MOHSW‘s strengthening of the HMIS through coordination between central, mid and low levels. Mechanisms for the generation of accurate, timely, and qualitatively REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 21 relevant data need to be strengthened and institutionalized through strong supervision, M&E, and HMISs. 4. Work collaboratively with the MOHSW to establish a strong training unit at the central level to oversee and coordinate all training/capacity-building needs at all levels 5. Conduct an assessment of the highly touted Bomi experience with PBC in order to identify lessons learned and applicability to other counties in Liberia. 6. Support and increase current MOHSW Health Promotion Division IEC/training/outreach capacity (using the counterpart approach) in designing and implementing nationwide social and behavior change communication (SBCC) and integrated health promotion programs in keeping with the National Health Communication Strategy. These programs should build on past BCC programs implemented in Liberia and particularly draw upon the current RBHS SBCC effort, which has, according to the MOHSW, the CHTs and health facility staff, contributed a fair share of appreciable results in terms of increase in demand of services and improvement in health-seeking behaviors. The programs should use all available media (formal and informal), technology, IEC materials, and outreach capabilities to reach both the vulnerable and the general population at the individual, household, health facility, community, county, and national levels. RBHS should also support the capacity of the MOHSW Health Promotion Division to produce relevant and cost-effective IEC materials locally. 7. Develop and implement a sound, long-term BCC M&E plan that would go beyond the use of the dip-stick approach, which only measures reach/exposure at one point. 8. Intensify, reinforce, and expand the current RBHS-run vertical campaigns and integrated health promotion activities by engaging women, men, and youth at homes, health facilities, schools, churches, mosques, women associations, market places/events, youth clubs, sport events, workplaces using peer education, radio spots, audio drama series, listening clubs, community dialogues, and other social networks on key issues such as reproductive health/family planning, HIV/AIDS, and water, sanitation, and hygiene (WASH). 9. Capitalize on ongoing advocacy with the leadership at national level, and develop a model of community mobilization and empowerment through the evidence-based Champion Community approach on issues like WASH and malaria. 10. RBHS revised work plan should have a perspective of the interrelatedness of the complete health care delivery system, and, as such, should be developed according to the following guidelines: – Consistency with the NHSWP 2011–2021 and two-year County Operational Plans. – Stakeholder involvement of all significant parties in each administrative level (central, county, district, facility) to include the administration, service provider, end users, private sector, CHSWT, public health, and curative leadership, etc. – Management/Administrative 11. Communicate implementation plans with the curative service at the MOHSW to augment cooperation and service delivery. 12. Facilitate MOHSW development of an administrative instrument (an MOU) between sub￾partners and CHSWTs with clear roles and responsibilities, strategies, targets, indicators, and a monitoring plan. 13. Support the MOHSW to develop standardized management instruments, including supervision checklist and tools, mechanism for accurate data collection and management, and a national referral system. 14. Support the MOHSW‘s External Aid Coordination Office by sharing information on a regular basis with other organizations, particularly NGOs and UN agencies within an 22 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT MOHSW framework, with the goal of ensuring consistency of programs with adopted plans at all levels. 15. Contribute to the review and revision of a structured salary scale for all health providers in the public sector. Variables to be considered include annual salary adjustments based on performance reviews and incentive program for staff serving in rural areas. Similarly RBHS should facilitate a thorough review by the MOHSW of strategies that will promote and sustain the motivation of the network of community health volunteers and ensure their applicability. 16. The role of RBHS county coordinator needs a clear definition with respect to the position in the CHSWT, organogram, and scope of work. This will address the problem of collaboration that is much needed between these two entities to support the achievement of the project‘s objectives. The pending hire of county capacity coordinators represents a significant response to the need for coordination and collaboration. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 23 APPENDIX A. SCOPE OF WORK Global Health Technical Assistance Project GH Tech Contract No. GHS-I-00-05-00005-00 USAID/Liberia Rebuilding Basic Health Services (RBHS) Year 2 Assessment Scope of Work (Revised: 06-21-11) I. TITLE Activity: USAID/Liberia RBHS Project Assessment Contract: Global Health Technical Assistance Project (GH Tech), Task Order No. 01 II. PERFORMANCE PERIOD The period of performance will run approximately on or about July 11, 2011 through August 31, including approximately 3–4 weeks of in-country work (beginning around July 18, 2011). III. FUNDING SOURCE The funding source will be through USAID/Liberia field support funds (up to $107,088). The balance will be funded by AFR/SD. IV. BACKGROUND The Rebuilding Basic Health Services (RBHS) project is a five-year Cooperative Agreement (2008–2013) with JSI Research and Training, in collaboration with JHPIEGO, the Johns Hopkins University Center for Communication Programs (JHUCCP), and Management Sciences for Health (MSH). Following a modification of the project in 2010, the project has three main intermediate results: 1) increased access to basic health services through improved provision of quality health services and adoption of positive health behaviors; 2) increase the quality of health services through improving infrastructure, and health workforce and systems performance by enhancing capacity to plan, manage, and monitor a decentralized health system; 3) youth informed and networked on reproductive health. V. PURPOSE The purpose of this assessment is to review the performance and the progress of the five-year Rebuilding Basic Health Services (RBHS) project, which is now at its mid-point, and assess the appropriateness of the project design. The assessment shall identify factors enabling or impending effective implementation of different components of the project. The assessment will also advise USAID/Liberia on any needed redirection of strategies or priorities, which might modify currently utilized approaches and/or suggest further expansion of the project. More specifically, the evaluation team is expected to assess the progress made in achieving the three intermediate results. The review should also include a cursory look at the progress made in relation to operationalizing the six key principles in the program description. 24 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT Additionally, the assessment team shall identify lessons learned and provide suggestions for the future direction of Liberia‘s health initiatives in order to assure a comprehensive and consistent maternal and child health strategy, and determine new areas for technical support. The team will allocate approximately 85% of its effort to assessing RBHS project accomplishments, and JSI‘s management approach, and the other 15% will be allocated to making recommendations for the future direction of health initiatives in Liberia. The team members will, through interviews, data collection, and review of the resources, provide answers to the following questions: VI. ISSUES TO INVESTIGATE Programmatic/Technical Is the RBHS project on the right track to achieve the results targeted? What major changes, if any, need to be made? Are there any significant gaps? – Are the project goals and objectives realistic or overly ambitious? Are there too many components to be implemented under one broad umbrella? What are the strengths and innovative activities being undertaken that should be continued/emphasized and may be a best practice for other USAID development activities? – What are the most notable successes (exceeding expectations)? What were the major shortcomings or failures? What were the challenges or changes in circumstances that explain these successes or failures? How effective has the RBHS project been in improving capacity to meet the growing demand for basic health services and in promoting healthy behavior change as these relate to the results sought by priority USG and GOL programs: 1) HIV/AIDS; 2) Maternal and Newborn Health; 3) Child Health; 4) Malaria; 5) TB; and 6) Family Planning and Reproductive Health (FP/RH)? – To what extent was the project‘s effectiveness helped or hindered by its integrated funding (six program elements) and design? What are the special challenges it faces as an integrated program? Were there any significant missed opportunities? What were they? How well has RBHS been able to utilize core competencies of its partners (JSI, MSH, JHPIEGO, JHUCCP)? How well has RBHS been able to manage its sub-grantees/contractors to maximize health impact through the delivery of the Basic Package of Health Services (BPHS) (e.g., providing technical and operational guidance to systematize/harmonize activities and scale-up best practices and lessons learned)? – Are all components of the BPHS delivered through the sub-grantees being adequately met? How can service delivery be further improved? Has the program learned any lessons about a ―ladder of progression‖ for phasing in elements of the BPHS? Is there a harmonized approach that could be developed from the different experiences to date? What have been the implications of standardization of the BPHS? Has this expanded the capacity of NGOs that traditionally had a more limited focus? Has it stretched NGOs positively—or too rapidly? Is the training approach used by RBHS reasonable given the Liberian context? Please describe some of the challenges and how RBHS has responded? REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 25 What has been learned about health-seeking behavior and creating demand in Liberia? How effective has RBHS been in learning the Liberian context and in applying it to IEC/BCC message content and delivery? How successful has the program been at building a national HMIS capacity, both at the central level and linked to the county level? Comment on the degree to which RBHS has integrated key gender and youth concepts into its overall programming. What are the strengths and weaknesses? Cross-Cutting How well does the project respond to the government‘s desired direction for Liberia? Is the structure of the project well integrated with the MOHSW program to assist the GOL in meeting its health MDGs? Cite any examples of where the program made significant differences in how the MOHSW operates, results obtained, and its reform and direction. What elements of the program are making progress toward sustainability? What are not, and what else could be done? Describe the work done to strengthen national institutions, professional associations, and NGOs. What are the major results, challenges, and recommendations? How does the program complement the work of other donors, NGOs, and MOHSW health programs? Any missed opportunities and what recommendations? What mechanisms are in place to ensure coordination and synergy with GOL, other donors (specifically Pool Fund, GAVI, and GFATM), and other United States Government supported activities? How effective are these? What recommendations? Management How well is the overall administrative and implementation structure working to manage and carry out project objectives? – How well is the RBHS team, including management structure and staff positions, interacting productively with the AOTR and AO, USAID health team? Discuss relative strengths and weaknesses? How is the current program being managed (both technically and financially)? Discuss the degree to which this management approach adequately documents decisions made, accomplishments and changes. Discuss any challenges to the management‘s approach that affect outcomes. How well does communication flow between the prime and sub-grantees? What are the successes and challenges? Discuss any recommendations for improvement. How effectively is RBHS managing the planned transition in project leadership to a Liberian COP? How effectively has USAID been able to manage the RBHS project and provide needed management and technical direction? What have been barriers or shortcomings? Future Direction What are overall impressions of the RBHS project and recommendations for current and future programming? What are the three key lessons learned that the Mission should focus on when developing its upcoming follow-on projects and implementing the new CDCS and GHI strategy? 26 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT How can the project achieve a successful rollout to other areas not targeted by the current RBHS project? In other words, how can RBHS project‘s successes best be institutionalized nationwide? What recommendations would you make regarding future plan or approach for sustainability of service delivery when external funding, which is now very significant, inevitably declines? How have the funding earmarks (POP, ID) affected the program? How well has the program been able to meet the requirements of the earmarks and report on them? Has there been, or does there need to be, a course correction to match the program with the funding categories? How have the program activities been perceived by beneficiaries and stakeholders (end￾users, NGOs, MOH, UN agencies and other donors), the Mission, and the embassy? What have been the drawbacks of the US visibility and/or invisibility? VII. METHODOLOGY AND PROCEDURES The evaluators should consider a range of possible methods and approaches for collecting and analyzing the information required to assess the evaluation objectives. The methodology will include, but not be limited to: team planning meeting (TPM), document review, key informant interviews (including USAID/Liberia staff and GOL and other donors/international agencies), site visits to several of the 108 health facilities assisted by the project, 2 training schools, and direct observation. Existing Data Sources The team will review briefing materials that will be provided by USAID/Liberia including but not limited to the following: Technical Assessments of RBHS, BASICS, and other groups (2004–2010) Draft Mission Strategy document 2006–2009 RFP for the RBHS Cooperative agreement RBHS (2008) and Amendments RBHS first and second year work plans RBHS first and second year annual reports PMP of the Mission Mission Semi-Annual Performance Reports Government of Liberia key documents (PRS, National Health Policy and Plan, Roadmap to Maternal Health, Situational Analysis, Nutrition Strategy and Plan, Sexual and Reproductive Health Strategy, etc.) Liberia DHS 2007, MIS 2009, and Comprehensive Food Security and Nutrition Survey 2010. 2011 LQAS survey report Team Planning Meeting (TPM) The assessment team will start their work with a two-day planning meeting prior to the onset of key stakeholder meetings and field work. The purpose of the TPM will be to clarify team roles and responsibilities; to develop the work plan and methodology; and to create a timeline and action plan for completing the deliverables. In the meeting, the team will specifically: Share background, experience, and expectations of each of the team members for the assignment; REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 27 Formulate a common understanding of the assignment, clarifying team members‘ roles and responsibilities; Agree on the objectives and desired outcomes of the assignment; Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; Revisit and finalize the assessment timeline and strategy for achieving deliverables; Develop and finalize data collection methods, survey questionnaire, and guidelines; Develop preliminary outline of the team‘s report and assign drafting responsibilities for the final report. During the TPM, an in-briefing with USAID/Liberia will be held to discuss expectations of the assessment. Data Collection: The information collected will be mainly qualitative guided by a key set of questions. Information will be collected through personal and/or telephone interviews (rarely) with key contacts, through document review, and through field visits. The full list of stakeholders and contacts will be provided. Additional individuals may be identified by the evaluation team at any point during the final evaluation. Key informant interviews will include but not limited to: RBHS program managers and sector specialists in the field USAID/Washington and USAID/Liberia technical team members GOL/MOHSW counterparts Donors (World Bank, UNDP, UNICEF, DFID, GFATM, WHO, EU) Project directors for other USAID projects such as MCHIP, HS20/20 etc. Staff from selected partner NGOs of RBHS County level local leaders, administrators, stakeholders RBHS beneficiaries Field visits: The team will coordinate with USAID/Liberia to prepare for and conduct site visits while in￾country, and to interview key informants at these sites. Site visits will be conducted in Bong County (three days) to see one sub-grantee: Africare; in Nimba County (five days) to see two subgrantees: EQUIP and International Rescue Committee (IRC); and in Grand Cape Mount (two days) to see MTI. Briefing/final debriefing meetings with USAID/Liberia Staff: The evaluation team will meet with the USAID/Liberia health team to review the scope of the final evaluation, the proposed schedule, and the overall assignment. The initial briefing will also include reaching agreement on a set of key questions and will take place over one day (or could be incorporated into the TPM). At least two days prior to ending the in-country evaluation, the team will hold a debriefing with USAID to present the major findings and recommendations of the evaluation. These recommendations will focus on the accomplishments, weaknesses, and lessons learned in the program, including recommendations for improvements and increased effectiveness and efficiency of the RBHS program. 28 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT VIII. SKILLS AND LEVEL OF EFFORT Team Composition A Team Leader, with expertise in health systems development, public health management, and/or institution building, who will be an international consultant with extensive USAID program implementation and evaluation experience, must possess proven skills in assessment and analysis of post-conflict/transitioning development programs. She/he must have a proven track record supervising teams in the field and producing high quality and concise reports, as well as extensive experience working in Africa and similar fragile/post-conflict settings. The team leader will: Finalize and negotiate with USAID/Liberia the evaluation work plan; Establish evaluation team roles, responsibilities, and tasks; Develop data collection instruments/questionnaire Facilitate all necessary meetings in the U.S. and in Liberia; Ensure that the logistics arrangements in the field are complete; Coordinate schedules to ensure timely production of deliverables; Coordinate the process of assembling individual input/findings for the evaluation report and finalizing the evaluation report; Lead the oral and written preparation and presentation of key evaluation findings and recommendations to USAID/Liberia. Two Local Consultants with broad knowledge of Liberian health issues will assist in key informant interviews, data collection, qualitative instrument preparation, and analysis of collected data. Combined qualifications should include: expertise in maternal, child, and newborn health; family planning and reproductive health; infectious disease prevention and control. Additional expertise in WASH and health systems strengthening (especially human resource capacity development) is preferred. An international expert in BCC/IEC will assist the Team Leaders in the duties above—in addition to expert analysis related to BCC/IEC component of RBHS. IEC/BCC expert should have a proven track record of successful project assessment and evaluation—preference given to experience evaluating USAID projects. An internal USAID program officer will join the team to support logistics and coordination, as well as provide technical expertise. With an MPH, the program officer will provide expertise in health systems strengthening. This fifth team member is funded by USAID and will not require financial support from GH TECH. Level of Effort An illustrative table of Level of Effort (LOE)* follows: Activity Team Leader Local Consultant 1+2 BCC Advisor Preparation and pre-field work (remote work) 5 days 4 days 4 days Travel to Liberia 2 days 0 days 2 days Team Planning Meeting (TPM) (in-country work) 1.5 days 1.5 days 1.5 days * A six-day work week is authorized when consultants are working in country. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 29 Activity Team Leader Local Consultant 1+2 BCC Advisor Briefing Meeting with USAID/Liberia (in-country work) 0.5 day 0.5 day 0.5 day Interviews with key informants (in-country work) and Site Visits (in-country work) 12 days 12 days 12 days Document Review (in-country work) 1 days 1 days 1 days Drafting of Evaluation Report and any necessary interview follow-up (in-country work) 3 days 3 days 3 days Debriefing Meeting with USAID/Liberia (in-country work) 1 day 1 day 1 day Travel- Return Home 2 days 0 days 2 days Finalizing Report (remote work) 5 days 3 days 3 days Total LOE (estimated) 33 days 26 days 30 days IX. LOGISTICS The Mission will assist in arranging local meetings and provide some transportation assistance for appointments in Monrovia. GH Tech will provide transportation to and from Liberia and arrange for local lodging and transportation (as needed). X. DELIVERABLES AND PRODUCTS Deliverables 1. A written work plan prepared during the TPM and submitted to the Mission for review and approval before fieldwork and key informant interviews begin. 2. A draft report outline prepared during the TPM. 3. A Mission debrief meeting that will be held before the team‘s departure and prior to the submission of the draft report. The team will prepare a PowerPoint presentation for this event. 4. Prior to departing Liberia, a draft report addressing key performance findings, conclusions, recommendations and lessons learned will be submitted. Feedback from the final debriefing will be incorporated into this draft report. The mission will have 10 days following the submission of the draft report to respond and provide written comments and feedback to GH Tech. 5. The final report will be due five days after receipt of the comments from USAID/Liberia. It will be the property of USAID. Dissemination of relevant findings will occur through official channels at local (Mission, USG and stakeholders) as well as Washington level. Some of the findings may be used for country operational planning. The report shall not exceed 30 pages, excluding the annexes. – The revised final unedited report will be provided to the mission five days after the comments are received. 30 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT – Once the mission signs off on the final unedited report, GH Tech will have the documents edited and formatted and will provide the final report to USAID/Namibia for distribution (five hard copies and CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. This will be a public document and will be posted on the USAID/DEC and the GH Tech websites. Suggested Format for report Executive Summary Table of Contents List of Acronyms Introduction Background Methodology Finding & Issues Conclusions Recommendations Lessons learned References Annexes (institutions visited, persons interviewed, etc) GH Tech will provide the edited and formatted final document approximately 30 business days after USAID provides final approval of the report. GH Tech will provide six hard copies along with an electronic final copy. The final report will be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the GH Tech project web site (www.ghtechproject.com). XI. RELATIONSHIPS AND RESPONSIBILITIES In-country, the evaluation team will report to Randolph Augustin, health team Leader. They will also work with other members of the USAID/Liberia health team in preparing and drafting the required documents. GH Tech will provide: International travel to and from the consultant‘s point of origin and Liberia. GH Tech will provide full-fare economy. GH Tech consultant per diem and lodging expenses. Country cable clearance. Reserve hotel/guest house accommodations in country. USAID/Liberia will provide: Visitors will not have an EA and therefore will need to work out of their hotel/lodging or a designated workspace, to be determined. They will need prior approval to bring any laptop into the USAID office for any meetings or briefings. Local sim card for cell phone, but consultant(s) will provide their own cell phone and purchase airtime. Arrangements/scheduling for in-country site visits. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 31 USAID/Liberia will provide a Mission car and driver for use by GH Tech Consultants only when other USG staff members accompany them. When no United States Government staff members accompany consultants, they will use taxis. Prior to In-country Work: USAID/Liberia will undertake the following: Consultant Conflict of Interest. To avoid conflicts of interest (COI) or the appearance of a COI, review previous employers listed on the CV‘s for proposed consultants and provide additional information regarding any potential COI. Background Documents: Identify and prioritize background materials for consultants and provide them to GH Tech as early as possible prior to teamwork. Key Informant and Site Visit Preparations: Provide a list of key informants, list of health facilities and suppliers, and suggested length of field visits for use in planning for in-country travel and accurate estimation of country travel line items costs (i.e. number of in-country travel days required to reach each destination, and number of days allocated for interviews at each site). Lodging and Travel: Provide information as early as possible on suggested lodging and identify a person in the Mission to assist with logistics. During In-country Work: USAID/Liberia will undertake the following while the team is in country: Mission Point of Contact: Ensure constant availability of the Mission Point of Contact person(s) to provide technical leadership and direction for the consultant team‘s work. Meeting Space. Provide guidance on the team‘s selection of a meeting space for interviews and workshops (i.e., USAID space if available, or other known office/hotel meeting space) if appropriate. Meeting Arrangements and Field Visits. While consultants typically will arrange meetings for contacts outside the Mission, support the consultants in coordinating meetings with stakeholders and organizing site visits. Formal and Official Meetings. Arrange key appointments with national and local government officials and accompany the team on these introductory interviews (especially important in high-level meetings). Other Meetings. If appropriate, assist in identifying and helping to set up meetings with local development partners relevant to the assignment. Facilitate Contacts with Partners. Introduce the team to project partners, local government officials, and other stakeholders, and where applicable and appropriate, prepare and send out an introduction letter for team‘s arrival and/or anticipated meetings. Following In-country Work: USAID/Liberia will undertake the following once the in-country work is completed: Timely reviews: Provide timely review and approval of the draft/final draft reports. XII. MISSION CONTACT PERSON Anna McCrerey, Health Officer, USAID/Liberia will serve as point of contact until July 5th, at which time Selam Kebrom, Program Officer and Evaluation Team Member will take over. 32 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 33 APPENDIX B. PERSONS CONTACTED MS YAH ZOLIA MOHSW /Acting Deputy Minister for Planning, Research and Development MOMOLU SIRLEAF MOHSW /Head of External AID Coordination Unit MS JESSIE E. DUNCAN MOHSW /Deputy Minister of Curative Service MOSES PEWU MOHSW / Deputy Minister of Preventive Service ESTHER VORDZORGBE Pool Fund RANDOLPH AUGUSTIN USAID Liberia Health Team Leader NOE ROKOTONDRAJAONA USAID Liberia President‘s Malaria Initiative Advisor AUGUSTIN MOBA USAID Liberia SELAM KEBRON USAID Liberia contact SOPHIE PARWON USAID Liberia ANNA MCREREY USAID Liberia RICHARD BRENAN RBHS COP ZAIRA ALONZO RBHS DCOP ROSE MACAULEY RBHS Technical Team Leader MARION SUBAH RBHS Education & Training Advisor JOSHUA OFORI RBHS BCC Advisor BAL RAM BHUL RBHS M&E Dir TEAH DOEGMAH RBHS BCC P.O. DAVID FRANKLIN RBHS Mental Health Advisor MIKE MULBACH RBHS M&E BEDAR H. FARKAT RBHS TAREK M. HUSSAIN UNICEF Health Program Coordinator JOSEPHINE FREEMAN UNICEF MCH JULIE GARON UNICEF Health Consultant JESSICA LOWDEN World Learning Project Director LEONARDO THOMAS Africare ANTHONY YEAHPALAH Africare MARY A. TUCKER Africare ROLAND GASTLOR Africare JAMES KALLIE Africare ALOYSIUSNYANDU Africare 34 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT SCOTT VICTOR Africare DAVID M. MEMBAH EQUIP GENEVIE NUAH EQUIP ABRAHAM TOZAY EQUIP OLIVE TEAH EQUIP GAYFOR BARNAR EQUIP LOUDRINA DONKOH EQUIP VICTOR SUAH EQUIP KRISTEN CAHILL EQUIP TEFERI BEYENE MTI GEORGE KAINE RBHS JERRY ZANGAR MTI JULIUS GARBO Cape Mount CHT EDWARD MASSAQUIN Cape Mount CHT LAWRENCE MOORE Cape Mount CHT MATTHEW PAASEWE Cape Mount CHT ERIC KAPLEE Cape Mount CHT GEOFFREY GBARTE Cape Mount CHT THERESA ALPHA Cape Mount CHT VARLIE SIKAMARA Cape Mount CHT RAYMOND HOLDER Cape Mount CHT ABRAHAM WILES Cape Mount CHT MORRIS LASANMA Cape Mount CHT WILLIAM PEWU Cape Mount CHT GARFEE WILLIAMS Bong CHT GERTRUDE COLE Bong CHT JESSICA WALKER Bong CHT RUFUS Bong CHT NYANDAY DORBOR Medena Clinic LORETA COLLINS EQUIP VARNEY FMEREMAN Medena Clinic ZUANA GRAY Medena Clinic GOENNA BROWN Medena Clinic VARNEY GRAY Medena Clinic REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 35 DUALLAU HOWE CHO/Bong PHIDERALD PRATT UNFPA HANNAN BESTMAN IMAD CLEMENT LUGALA WHO JUAN CASANOVA EU MOSES MASSAQUOI Clinton foundation DAVID LOGAN Global Fund Nicolas Low IRC Wilson Ballah IRC Dr. Folaranmi Ogunbowale IRC Vekeh Donzo IRC 36 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 37 APPENDIX C. LIBERIA PERFORMANCE BASED FINANCING (PBF) ASSESSMENT REPORT (FINAL DRAFT, JUNE 2011) REPUBLIC OF LIBERIA MINISTRY OF HEALTH & SOCIAL WELFARE LIBERIA PERFORMANCE BASED FINANCING (PBF) ASSESSMENT REPORT (Final draft, June 2011) 38 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT INTRODUCTION This report aims to capture the findings of a joint mission of Ministry of Health and Social Welfare (MOHSW), World Bank, and Rebuilding Basic Health Services (RBHS), which took place in early May 2011 to assess current functioning of Performance Based Contracting (PBC) in the context of Liberia‘s post-conflict health reconstruction. The assessment was to lead to recommendations regarding improved institutional and implementation arrangements for PBC as well as an operational manual. The cornerstone of Liberia‘s new, post-conflict national health policy as formulated and endorsed in 2007 is the rollout of a Basic Package of Health Services (BPHS) to all citizens. This BPHS is composed of a number of well-proven and affordable health interventions that are to reach the population through a network of health clinics, health centers, and first-level referral hospitals all over Liberia. Interventions and required inputs (equipment, drugs and number and level of staff) have been determined by level of health facility. Based on these inputs all health facilities can be assessed in an annual accreditation exercise. While in the past the government was the main provider of health care, next to a number of faith-based organizations (FBOs), during and after the war international non-governmental organizations (INGOs) became main providers in an otherwise dilapidating health system. Against a backdrop of governmental policy to decentralize, it is the vision of the MOHSW that the county health teams (CHTs) will be responsible for health care provision in their counties. For the time being, however, it is envisaged that INGOs will play an important role in assisting the CHTs to take up this responsibility. At the moment, the rollout of the BPHS takes place through different implementation mechanisms. All mechanisms may be found in different parts of the country, which means that a particular county may be covered by more than one mechanism. Broadly, there are currently three mechanisms, based on a purchaser-provider split model, next to direct government provision. These first three mechanisms are governed by contracts between fund holder and implementing agency. The majority of these contracts is or is meant to be ‗performance based,‘ whereby a portion of the payment is determined by achieved results. These three mechanisms are: RBHS RBHS is a USAID sponsored program, implemented by John Snow Inc (JSI). One of its components entails contracting INGOs7 to support health care delivery in a number of countries. The scheme started by mid-2009, and 4 NGOs were contracted to provide support to 95 health facilities in 6 counties.8 From the start this scheme uses ‗PBC.‘ Pool Fund The Pool Fund was established in 2009, mainly with funds from DFID and Irish Aid. The Pool Fund secretariat is based within the MOHSW. By the end of 2009, the Pool Fund contracted Bomi CHT to provide health care in 20 health facilities, as a pilot on ‗contracting in.‘ By March 2010 the Pool Fund also contracted INGOs for support to the delivery of the BPHS. In total 4 NGOs were contracted to support 87 health facilities in 6 counties. All INGOs partnered with a 7 RBHS also provided a grant to Merci to implement BPHS in 15 facilities in River Gee county; in many aspects similar to the other contracts, but not formally PBC. 8The contracted INGOs by county are: Africare (Bong, 16 facilities), EQUIP (Nimba, 23 facilities), IRC (Lofa, 19 facilities), IRC (Nimba, 12 facilities), and MTI (Grand Cape Mount/Bomi/Montserrado, 25 facilities) REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 39 national NGO. These contracts were intended to be ‗PBC‘ but during the first year the mechanisms were not in place to actually assess performance and no bonuses were paid. European Union Early 2011, the European Union concluded similar contracts for BPHS support with NGOs in three counties. These contracts are not performance based. Public facilities that are not contracted under any of these three schemes are still run directly by the government using a more traditional input model. NATIONAL HEALTH POLICY ON CONTRACTING As addendum to the National Health Policy, a National Health Policy on Contracting for Health Service Delivery was established in 2008. The policy describes the rationale, guiding principles, and initial management arrangements. The contracting policy states that the MOHSW will lead the PBC in the health sector and will establish adequate regulatory capacity in the fields of legislation, standards setting, inspection, and operational guidance. Standardized contracts, reporting, and monitoring tools will be developed. A long-term strategic institutional plan aimed at establishing adequate regulatory capacity will be formulated and a dedicated unit will be established within the Ministry to oversee this effort. A mixture of approaches will be tried that include contracting in, when one level of government contracts with another, contracting out, when a partner is contracted with complete authority over all resources (human, material, and financial) to provide health services, and management contracting, when a partner is contracted to provide management services alone over government resources. The policy further states that, while the primary objective of the reform process is to build the capacity of the CHTs, PBC shall initially be managed from the central level, with active participation of the relevant county. Once administrative and evaluation processes have been developed, the intent is that responsibility for management of performance based partnerships will be gradually shifted to the CHTs, in the context of intended further decentralization. The MOHSW has created a Program Coordination Team (PCT) to coordinate partners and resources for implementing the National Health Policy and Plan. The PCT consists of the four deputy ministers supported by technical experts, and is headed by the chief medical officer/deputy minister of health services. This is a higher decision-making body. The Policy on Contracting foresaw a comprehensive system for monitoring and evaluation (M&E) of the NGO contracts to be established within the MOHSW to ensure results that will meet internationally agreed upon standards of financial management. When later on the Pool Fund started to issue performance-based contracts to the NGOs and Bomi County, this system was not really in place yet. The financial management was taken care of by the Pool Fund secretariat, but technical support for real PBC, including the set up of appropriate M&E systems, was not really established. A PBC steering committee was set up consisting of representatives of a range of MOHSW departments and others. More recent was the appointment of a PBC focal point, under the health financing department. During the assessment, while discussing the right place within the MOHSW from where less fragmented, stronger support to PBC could be given, it was suggested to establish a dedicated PBC unit within a still-to-be-established program management unit (PMU). In the National Health Plan, a PMU has been proposed, but up to date no resources have been dedicated to its establishment and function. The PMU is to be headed by a senior technical staff member of the 40 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT MOHSW. He/she would be joined by a Pool Fund representative, a Global Fund representative, and representatives of other programs/initiatives. It had been noted, that while quite a few other countries introduce PBC with the set up of institutional arrangements and drafting of an operational plan that preceded the implementation of PBC contracts, the dynamics in Liberia had led to a reverse situation. There is still a need to draft an operational plan for the PBC scheme to come to full fruition. This assessment report is to assess the institutional arrangements and current PBC implementation strengths and gaps to form the basis for the operational manual, which is to be based on best practices adapted to the situation in Liberia. The assessment became all the more relevant because of a recent decision to bring the RBHS scheme within the MOHSW, to start in a year‘s time. This will more than double the MOHSW‘s direct involvement in PBC and hence the need for good institutional arrangements, harmonized approaches between the different PBC schemes, and an operational plan that is widely understood by all stakeholders. THE METHODOLOGY A joint team composed by the MOHSW, the USAID/RBHS program and the World Bank was set up to conduct interviews, consultative meetings and technical deliberations, and provide the findings and recommendations on how the PBC is implemented. A sample of PBC partners, including the MOHSW and implementers was selected based on the criteria of representation. At the central level, meetings took place with the minister and deputy ministers in charge of health financing and health services, the Pool Fund accountant, the USAID health coordinators, and the RBHS project team. For both PBC schemes, the Pool Fund and the RBHS, three headquarters of the implementer NGOs were visited. For both, the team selected a high, middle and low performing NGO (or rather ‗contract,‘ since several NGOs have more than one contract). Among the Pool Fund contractees Save the Children UK, Merlin, and Bomi CHT were selected, and among RBHS contractees: Medical Teams International (MTI), Africare, and International Rescue Committee (IRC). POOL FUND Save the Children Merlin Bomi County RBHS MTI Africare IRC In addition three counties were visited to see the CHTs and a number of clinics. These were: Montserrado CHT and a clinic supported by SCF; Bomi CHT; Vortor community health clinic and Beh town health clinic; and Grand Cape Mount CHT and Madina health clinic. For all interviews a topic guide was established to ensure inclusion of all relevant aspects: 1. General for all: a) Strengths and gaps b) Key recommendations on how PBC can work better. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 41 2. Specific: a) Indicators, content, contribution to Millennium Development Goals (MDGs) (4, 5, 6, and maybe 7 and 1c), participation in design of the targets, the payment modality, the reports/disbursement b) NGOs and government: collaboration with CHT, relation with MOHSW, inclusion of RBHS contracts into the Ministry c) Operations at health facility level: management of HR, drugs and supplies etc and how staff incentivised d) Demand side issues: access (financial/geographical, outreach) e) Opinion regarding incentives within the contracts: stick, carrots or both, side-effects?, combine with PBF at facility level? (sub-contracts between NGO and facility) f) M&E and verification: independent verification The Findings The findings from documentation, interviews, and field visits have been compiled and will be presented using four headings that are key functions of PBC: (i) regulation; (ii) purchasing (fund holding); (iii) payment; and (iv) M&E and verification/validation. For comparison‘s sake we contrast findings within the RBHS scheme with that of the Pool Fund, with a focus to highlight what needs further strengthening within the MOHSW to make the Pool Fund contracting mechanism truly ‗PBC,‘ which then at the same time will prepare the ground for the hand-over of current RBHS-led contracts to the MOHSW. However, this may mean that many positive effects of the new Pool Fund mechanism are not well represented in this report. For instance, during our field visits we could clearly see many positive changes in health care delivery thanks to wider and streamlined availability of resources, of which the populations in the respective areas will benefit. While several contracts also entail support to county hospitals, in our assessment and report we focused on the bulk of the contracts, the support to primary care facilities. Situational Assessment of Regulation9 RBHS: RBHS has strong capacity to design a well-functioning PBC scheme. They have created detailed procedures on the functioning and operation of the PBC scheme. RBHS is flexible in changing indicators, targets, and bonus payments, with a dynamism that is required of the Liberian context and demands. RBHS has technical and administrative staff that are responsible for carrying out the required work, and to amend the scheme as necessary in order to accommodate the situation, and to improve their organizational and management systems. RBHS has also been successful in training their implementing partners (NGOs) to implement PBC in their respective counties. One of the weaknesses of RBHS regulation is the ability to transmit capacity to the CHT to manage performance-based contracts. There is low knowledge of PBC at the CHT level. The scheme relies heavily on intermediary implementing partners. Targets are set for each indicator, by county and for each health center. The target is, however, consistent for each health center, and does not take into consideration the catchment population of each health facility, or the context of each health facility. These targets need to be refined for more specificity, adjusting at least for catchment population. Additionally, one could consider how measuring performance each quarter contributes to reaching the targets. For instance, the targets do not need to be consistent across all four quarters. 9 Regulation includes the design capacity (guiding document, contracting/management tools), policy￾making, building of PBF implementers, and monitoring of the scheme. 42 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT If RBHS implementing partners do not meet their administrative indicators, the health facilities under their purview are also penalized. This is a design weakness, because if the health facilities perform well, yet the implementing partner does not meet its own targets, the facilities will nonetheless feel the effects of a reduction in bonus. Pool Fund: The Pool Fund is accountable to the MOHSW. In our assessment, some main weaknesses in Pool Fund regulation exist. These are: 1. No one person or unit ―owns‖ the PBC scheme in the Ministry 2. The MOHSW lacks a technical team that can develop and govern a PBC program. The function of the technical team should be to propose implementation options for policymakers. Currently there is only one technical focal point in the Ministry. It is necessary to build the capacity of a team of technical people to undertake the work and have strong decision￾making authority, and accountability for the scheme. Currently, many staff from different departments contribute to the technical aspects of PBC, but are not fully dedicated. Therefore, no guidelines, indicators, incentive, and payment structures have been developed. There is also a technical gap when it comes to supervising/mentoring the performance of health service providers in the districts. Without a strong capacity to support the implementation of the performance-based contracts, the Pool Fund policy will continue not to be PBF, but rather traditional input-based financing. One example of this constraint is the Ministry‘s limited ability to sensitize and train the CHT on PBC operations. As a result, there is a low level of understanding at the CHT level regarding what a true PBC system is, and how it works. The MOHSW lacks a policy committee that can deliberate on policies regarding the implementation, maintenance, and troubleshooting for the existing PBC scheme. There is a PBC steering committee for the Pool Fund, which includes Ministry officials, NGOs, and other donors. The PBC steering committee is supposed to govern the Pool Fund PBC scheme, and give high-level guidance and oversight on operations. To our knowledge, this group has irregularly met in the last year, which resulted in limited review of policies, progress, and achievements, and inability to provide options for recourse. One example of the consequences of weak policy is that the role of the CHT is not well defined, nor is it understood. Although the role of the CHT is critical because of their support of direct service delivery and as overseers of the PBC scheme, nowhere is their role documented, nor specified in any contract. Additionally, though the Ministry contracted NGOs, they lacked technical guidance on how to implement and operate a PBC program. The current strengths include fund management. The accountability tools are well designed and operational. The Pool Fund Procedures Manual contains several templates, which include: project implementation plan (by objective and activity), budgeting template, disbursement schedule template, and a pro forma invoice. However, the Pool Fund Procedures Manual is weak on the programmatic and performance elements of PBC. It does not explain how performance is assessed, or how bonus payments are linked to performance. The Manual is heavily geared toward an input-based financing paradigm. Additionally, there is no specificity in the document to hold NGOs and service providers accountable for their performance. Our assessment indicates that the MOHSW is quite focused on how to manage the Pool Fund resources and accountability. The MOHSW seems to be less concerned about instituting the PBC scheme. Another strength is that the MOHSW has established a list of indicators for the PBC program, and the relative weights for each indicator (decided in a January 2010 workshop). These indicators very closely mirror the RBHS indicators. They have nationally set targets for those indicators, though these will have to be updated to reflect the current year, and targets should REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 43 be set for each of the counties, and each health facility. In the upcoming 10-year strategic plan, there are targets set for each county. This can be used as a reference document to establish targets for the county. Situational Assessment of Purchasing (Fund Holding) RBHS: RBHS manages funds so that the money is disbursed to health service providers through NGO implementing partners. Money (neither the regular monthly incentives, nor the bonus) is not directly transferred to the Ministry, or to the CHTs. Money flows well, and field assessment indicates that the funding does not stall, and the transfer from NGO to health facility staff is not problematic. Health facility staff now receive monthly ‗incentives,‘ on time, on an individual basis from the implementing partner. Pool Fund: The PCT in the MOHSW manages the Pool Fund. Money is moving smoothly to Bomi County, but there are some occasional delays. Sometimes, this is because the counties are not providing justification on time, which delays assessment and the disbursement of money. However, currently the Pool Fund disburses two months of funding in advance, so as not to create bottlenecks in service delivery. Situational Assessment of Payment RBHS: RBHS selected indicators for key services according to the importance of key health services from the Liberia national BPHS, which can also help them in meeting their USAID Performance Management Plan (PMP). After the request for proposals (RFP) is issued, implementing partners propose a budget that will cover their activities. If the health facilities that they support meet the targets of their service delivery indicators, they receive a bonus of 6% of that quarter‘s budget. This bonus is shared with staff at each of the facilities that are supported by that NGO. Bonuses are also shared with the CHT, though there is no formal agreement to do so. Clinic staff bonuses are paid according to their salary level. They receive a bonus that is directly dependent upon the percentage of indicators that they met, and that is proportional to their salary. The implementing partners do not have any formal written Memorandum of Understanding (MOU) or agreement with the CHT, nor the health facility on the level of bonus that is tied to performance. NGOs also have set administrative indicators and targets, which they must meet. If administrative indicators are not met, an NGO is liable to lose up to 5% of their originally proposed budget. Examples of these administrative indicators include: 1. Accurate reporting submitted in a timely manner 2. Number of joint (with CHT) supervision visits NGOs are reluctant for their bonus to be contingent upon things they cannot directly control. As such, they are only responsible for administrative indicators. There is a different weight assigned to the various indicators, which are agreed upon at the beginning of the quarter. There is nothing tied to payment in the current scheme, which protects the delivery of non￾incentivized services in the BPHS. Accreditation and a quality assessment are only conducted once per year, as it is such a work intensive endeavor. We do not know whether accreditation and quality assessment scores are used as indicators to assess performance. Pool Fund: The Pool Fund also has selected indicators to monitor performance. These are drawn from selected services in the BPHS. There is a set budget, and fixed amounts obligated to each implementer (Bomi CHT and four NGOs). In the plan, 5% of the budget was designated to be the bonus payment. The CHT/NGOs could lose 5% if they didn‘t perform, or gain 5% if they reach their targets. 44 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT In the current situation, whether or not the CHT/NGOs perform, they receive the full amount of their budget. Payments for inputs are disbursed quarterly. Bonuses for performance were planned for yearly disbursement, but this has not yet happened. CHTs have agreed upon administrative indicators that they must meet. There is an existing quality assessment tool that was used to assess quality and was implemented in all facilities last year. Full payment to the Bomi CHT and NGOs has always been proffered, without regard to the achievement of selected indicators. Situational Assessment of M&E, and Verification/Validation RBHS: In the current system, health facilities provide the quarterly reporting for all BPHS services that they delivered in that quarter to the implementing partner. The implementing partner in that county extracts the numbers of services of selected PBC indicators. The implementing partner and the CHT will visit each facility to validate the recorded information from the registers. Then that information is sent to RBHS. RBHS County Coordinators select three facilities at random to verify data. If there are inconsistencies, the implementing partner and RBHS revisit the facility to settle the discrepancy. In the verification process, if there are data inconsistencies, bonus payments are not made. An implementing partner is responsible for a number of health facilities in a county. For instance, MTI supports 22 health centers in Grand Cape Mount County. In certain quarters, though some facilities fail to reach their targets, other facilities may achieve above their targets, such that MTI still meets the targets of their indicators. In this case, the facilities that did not achieve will not be paid their bonus, and MTI retains that portion. There is no formula for payments to those who support the PBC program at the CHT. Each person receives a certain amount, which can vary by quarter, depending on the staff cadre that he/she is in. The CHT staff is not held accountable for service delivery targets. If the health facility achieves, and the implementing partner achieves the overall target, then CHTs are also given a bonus. CHTs do have to report on health system strengthening indicators (i.e., no stock-outs). RBHS convenes implementing partners quarterly to present performance data for that quarter. The data are also disaggregated by implementing partners. Best practices are shared. For low￾performing implementing partners, a plan is created to improve performance. MOHSW representatives are also invited to provide consultation. Pool Fund: In counties where the Pool Fund operates, the CHTs rely on the regular reports from health facilities. There is limited capacity to verify data. At the time of our visit, for instance, there was no M&E officer in Bomi CHT. Joint supervision visits are conducted and during that time, it is possible that services delivered are monitored, however, data validation does not routinely occur and performance is not tied to payment. Recently, verification of health center reports began as a team from the Central level visited facilities in Bomi. For the five NGO-run clinics in Bomi County (a subcontract between Bomi CHT and the AHA), the AHA district coordinators visit each clinic to collect and record services delivered. However, similar to the CHT, there is limited verification of the integrity of the recorded data, since payment is not tied to performance. Neither RBHS, nor the Pool Fund, currently conduct counter-verification of data (visits at the household level to ensure that (1) the patient exists, and (2) this patient received the recorded service. There is also no third-party assessment of the impact and progress of the PBC scheme. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 45 RECOMMENDATIONS 1. Establish a functional, robust, and dedicated technical team within the proposed MOHSW PMU to support programmatic aspects of a nationally-led PBC scheme. 2. Functionalize the PBC steering committee, which is comprised of staff from health services, health financing, and health planning departments. The team can be improved by: a) Revisiting the terms of reference, composition, and accountability of this team b) Convening regular meetings c) Reviewing the quarterly reports from the implementing partners, giving feedback, and suggesting actions for recourse d) Ensuring that all streams of funding (i.e., Global Fund, EU, etc.) are all harmonized under the PBC scheme 3. Ensure that all services in the BPHS are protected by instituting a balanced scorecard approach, so that non-incentivized services and quality of services are not neglected. 4. Harmonize the bonus payment mechanism from the two existing schemes (RBHS and Pool Fund). This includes the amount that is paid and the selected indicators. 5. Strengthen the existing verification mechanism so that CHTs and implementing partners are paid bonuses according to performance. 6. Implement counter-verification (verification at the household and community level) to ensure that the data recorded at health facilities is accurate and valid. 7. Conduct independent operational research to ascertain the impact of the MOHSW national-led PBC program. 8. Develop a comprehensive national PBC operational manual, which will describe the institutional and implementation arrangements. 9. Dedicate a portion of MOHSW budget to support the national PBC scheme, to encourage sustainability of the scheme. This will also engender donor confidence in the nationally-led PBC scheme. 10. Clarify the role of the CHT. 11. Develop a recruitment and training plan to build the MOHSW capacity to design and implement a national PBC scheme. 12. Increase IEC at the service delivery level (CHTs and health facilities) regarding the PBC mechanism, so that they can be motivated to achieve performance targets for bonus payment. 46 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 47 APPENDIX D. DETAILED LIST OF KEY FINDINGS ISSUES: More collaboration with the Pool Fund is needed. There is a disconnect between the higher levels and lower levels in the MOHSW that needs to be reconciled. RBHS needs to share information with other organizations. Transition will seriously affect work/performance at facilities if MOHSW does not carry on with timely pay/incentive. MOH staff lack computer skills. There is a need for a better retention policy. Placement of the right people in the right position is lacking. There is a need for clear objectives/outcomes for capacity building. Vertical funding for integrated projects is unrealistic. Capacity building is very fluid and not well defined. Partners are over engaged because of an effort to meet indicators and deadlines. There is no ladder progression because of the intensity of the program. There are too many indicators to deliver. There is no comprehensive assessment of CHT to implementation of project‘s components. There are no clear MOUs with CHT. Targets need to be established with MOH at central level and translated down to CHTs. There‘s a disparity in training of RBHS workers and government health workers due to funding. There is a strain in the interpersonal relationship between the Grand Cape Mount RBHS county coordinator and the county health team. Grand Cape Mount RBHS county coordinator is left out of management meetings, especially when budget is discussed. Grand Cape Mount RBHS county coordinator RBHS has to stay at MTI because of office space at CHT. How does RBHS prepare for transitioning to the GOL and ensuring sustainability? What major achievements can be used as a basis for scale up? How should they be prioritized? Government of Liberia Issues: – MOHSW is not seeing any capacity building of CHT by contractors (How do they do it?) – There is a very weak community component in the project (poor community involvement/initiative in reducing preventable diseases). – Capacity building should focus on the improvement of a set of skills, e.g., data collection/data management for good planning (despite resources input and collaboration with NGOs data collected is of poor quality). – There is a need for strong collaboration between NGOs and CHT in data management. 48 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT – Recommendation: review of PBC between RBHS and NGOs. Lots of resources are invested in capacity building at the MOHSW by other donors (EU, World Bank etc.), but there are still a lot of gaps, especially at lower and middle levels. RBHS is a very difficult concept to work on. PBC is very labor intensive. Top priorities: easy structure, building basic skills, e.g., IT and retention of qualified staff. Large capacity building needs of CHTs: – Planning – Vision – Standard functionality/ core competencies – Standard organizational structure – Performance indicators – Data collection, analysis, verification capability – Supply chain management capability – Financial management – Human resources management – Monitoring and evaluation – Logistics management – Administration – Computer skills – Training capability – Transport for staff to move around county – Contract management skills – Procurement – Communications There‘s a lack of coordination (between who?) There is no capacity building of mid- and low-level bureaucrats There is no county level procurement training. Quality of data collected is lacking. County is not viewed as a ―system‖ that works together. Is RBHS plan consistent with MOH 10–Year Plan? How will the bonus/incentive program be affected by the transition and what will impacts be? How to retain volunteer network? Senior level staff at MOH need mentoring /coaching, e.g., PMU. Indicators for PBCs need to be revisited. There is no time for NGOs to engage in capacity building. Quality of services may deteriorate without adequate planning, capacity building, support to CHTs. There is a need for GOL leadership and commitment (MOU?) CHT is not hitting targets. REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 49 CHTs want access to Pool Funds. Government does not keep up with regular payroll. CHT need M&E capacity building Facilities are in county too small to allow CHT to hit targets (need facility expansion). Timeliness of getting drugs, equipment, other is lacking. CHTs need training for a whole team. Need better communication capacity between facilities and other stakeholders, district, Ministry, other). They need salary support from GOL after transition. To add value to what is being developed now, RBHS needs to focus on HR development, health education, M&E, PB bonus, BCC capacity building/training, quality assurance. New approach: new strategy needs to be done jointly with MOHSW, strong collaboration with partners is necessary. To prepare for transition, RBHS staff is going to sit at Ministry for transfer of skills (if RBHS to relocate at MOHSW). Gaps in the revised work plan are programmatic in RBHS perspective. RBHS work at community level need to focus on health education and community mobilization, and the rollout of community case management. Staffing is done more at central level, not at community level. There is no clear policy on community health (MOHSW revising community health strategy). There is no budget for the community health division at MOHSW. There is a lack of leadership in that division (someone has been appointed now). Staff in health promotion at county level is not dedicated. RBHS has reservations in the process of selecting CHV and supervising them. RBHS to engage with MOHSW to solve these problems, i.e., review all documents produced by MOHSW and provide feedback, strengthen community health area, and revise community health strategy. RBHS project design/objectives too ambitious. RBHS needs to trim PPP, health financing at community level, and renovation of infrastructure. There are issues with procurement, e.g., five-month delays in drug procurement. They need to scale up specific activities, e.g., community case management, HIV/AIDS prevention and treatment. There is a critical issue with transition: government salary. There is an issue with the use of community radios: not done in local languages. PBC can motivate GCHV but CHT can‘t because of untimely subsidy from MOHSW. RBHS PROGRAM SUCCESS IN MEETING OBJECTIVES Achieved first two goals: 1. Service Delivery a) Support MOH 50 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT b) Deliver evidence-based services/ data-driven culture introduce by RBHS in country c) Data is being used now for planning and decision-making d) Coordination meetings e) Joint supervision f) Increase deliveries at health centers/facilities 200 to 600 g) HMIS h) Increase performance – Community Services a) Hiring right social community workers b) Hiring the rightGCHVs c) Hiring the rightCHPs d) Doing household health promotion e) Data rechecked with intense supervision f) Retention support for volunteers Less success at building capacity of MOHSW Volunteer network Supervision Rapid payment/timeliness Improvement in service delivery Community engagement Timely drug supply Increased patient load Increase in deliveries at facilities Strengths: Bonuses for performance that were high and a one-cent fine for below-standard performance Bonus shared with facilities Data-driven culture Strong supervision Improve quality and efficiency Built capacity of partners Made partners and employee adhere to a data-driven culture Strengths from clinic perspective: – Incentive is adequate and on time – Drugs are regular and on time – There is provision for training – Joint and supportive supervision – Improved health system REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 51 – Working with community service providers Volunteer network Data management culture-/evidence-based interventions Strong office of financial management at the MOHSW Relationship with USAID has been good in general but some tensions sometimes arise (macro management) RBHS acknowledges technical support from USAID RBHS has been very responsive to USAID RBHS on track in reporting CHT works collaboratively with RBHS (regular coordination meetings, joint supervision) Delivery of BPHS at facilities improved in 2010 Expanded service to community through cordial interaction with the community RBHS strengths include timely implementation of project activities, good coordination with CHT, regular joint supervision, timely pay of staff, capacity building of staff (performance), provision of gasoline for motorcycles, training M&E staff Weaknesses/ Gaps: Building capacity of MOHSW and CHTs Lack of MOUs Human Resource don‘t have over-arching capacity building strategy Need to do a situational analysis Weaknesses from the clinic‘s perspectives: – Grand Cape Mount health providers are paying rent for housing leaving little to live on – Facility space (need expansion of clinic) – Dual reporting (one for CHT and another for NGO/PBC, and reports ask for similar if not same info) – Not enough money – RBHS thinks MTI is sometimes not responsive (example of refrigerator repair at one facility) Data management M&E Capacity building (human, systems, infrastructure) Training Inconsistency of capacity levels of contractors Coordination Inflexible funding Low salaries Not enough logistical support (travel, gas) RBHS did not put emphasis on building capacity at CHT level 52 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT FUTURE DIRECTION: Project was well designed at onset given Liberia‘s needs and situation at the time, but needs to be revisited, knowing what it was before Proposal should match county plan MOH should take indicators and reduce them to strategic targets and demand results For community health services to become sustainable motivation, mechanism should be in place Donor sharing lessons learned Incentives for health workers should be good and on time Development of integrated county health plans Ease into transition Go back to basics—nuts-and-bolts of what an office needs to function Develop and implement a better retention policy Build strong volunteer networks Consistency with 10-Year Plan Support GOL/ MOH capacity development: – Get right people in right positions – Strong mid-level management – Clear objectives – Internal capacity development plan to be developed – Build strong education, training, and outreach capability, including internal training Wait one to two years for transition. In meantime, build capacity and see what political changes occur. Need mentorship in MOH (PMU) and CHTs Get GOL to pay on time Improved incentive structure Rationale to approach on community health Let priority activities go on Continue support of PBC, PBF (institutionalize it) Issues with operationalizing the PMU at the MOHSW Capacity building at CHT should focus on HR, HMIS, drug management Capacity being built at the MOHSW includes guidelines, follow up, supervision USAID should set clear overall objective in Liberia and determine where RBHS and other NGOs fit Coordination of RBHS and partners on specific activities should be done complementarily to avoid overlap Health promotion should be coordinated between the MOHSW, RBHS, contractors Building the capacity of CHT data unit in accurate data collection and management through strong data verification and analysis Training needs at CHT include M&E (data collection), HMIS staff (improving IT/computer skills) REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 53 CHT challenges in managing facilities (10) include infrastructure extension, drugs, medical equipment, funding not easy to get from the MOHSW Challenges with transition: need to start with skill-based capacity building now in management, planning, data collection, service delivery, use of communications devices Recommendations for capacity building at CHT: financial management, HR, logistics, administrative, technical Health education needs at CHT: logistics/mobility, IEC material from RHBS/MOHSW, training of HHP, CHP, GCHV on key topics, i.e., Malaria, diarrhea, breastfeeding FOLLOW UP ITEMS: RBHS partners feedback: validation process very painful IEC materials available at community level (journey of hope kit, chest kit, cards) BCC strategy: vertical (national campaigns) and community (based on clinic records and community needs) Indicators of impact of BCC: quarterly deep stick survey, assessment of reach (exposure/number of people sleeping under net), number of Malaria case declining 54 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 55 APPENDIX E. INTERVIEW QUESTIONNAIRES QUESTIONS TO GUIDE DISCUSSIONS Draft: 07.19.2011 Overview: The objective of data collection is to inform USAID of progress toward transition to GOL, including identification of gaps, needs, and constraints. Data will be collected through individual and group interviews with key stakeholders, including but not limited to: GOL RBHS World Learning International and Local NGOs USAID and other donors The approach is to focus on relevant issues associated with core elements of the RBHS as it relates the ultimate successful and sustainable transition to the GOL. These issues are divided into the following categories, each of which is followed by a series of key questions to be to be posed. Programmatic/Technical Is the RBHS project on the right track to achieve its goals and objectives? What are the strengths and weaknesses of the project? What are the targeted results of the RBHS project? Are the project‘s goals and objectives realistic? What are the significant gaps identified in the implementation of the project‘s components? What changes do you think could be made to correct the gaps? Why? 56 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT What are the project‘s most notable successes? What contributed to the successes? Are there any major shortcomings/failures? What are the impeding factors? How effective has the RBHS project been in building the capacity of the MOHSW at the central, county, facility and community levels to meet the post-war growing demands for basic health services? How effective has the RBHS project been in improving capacity to meet the growing demand for basic health services and in promoting healthy behavior change, as these relate to the results sought by priority United States Government and GOL programs: 1) HIV/AIDS; 2) maternal and newborn health; 3) child health; 4) Malaria; 5) TB; and 6) family planning and reproductive ealth (FP/RH)? What have the constraints been? To what extent was the project‘s effectiveness helped or hindered by its integrated funding (six program elements) and design? What are major challenges facing RBHS in implementing an integrated project? Are there any identified significant missed opportunities in the design and implementation of the RBHS project? How well has RBHS been able to utilize core competencies of its partners (JSI, MSH, JHPIEGO, and JHUCCP) to maximize results? How well has RBHS been able to manage its subcontractors to maximize health impact through the delivery of basic package of health services (BPHS)? Provision of technical and operational guidance? Scale up of best practices and lessons learned? REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 57 Are the objectives of all the components of the BPHS set for the subcontractors being met? How can the performance (service delivery) of the subcontractors be improved? What lessons has the RBHS project learned in using a ladder progression approach for phasing in elements of BPHS? Has the standardization of the BPHS positively or negatively affected the performance of NGOs? How does this affect the performance of health facilities? How will this affect transition? Is the training approach used by RBHS adapted to the Liberian context? Describe some of the challenges encountered in using the RBHS training approach. How has RBHS responded to these challenges? How successful has RBHS been at building a national HMIS capacity both at central and community levels? To what extent do you think RBHS has integrated key gender and youth concepts into its overall programming? Does the MOHSW have a national SBCC strategy? Is this strategy in keeping with the SBCC planning process? Is RBHS using this strategy? 58 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT How relevant and effective have the BCC/IEC messages and materials been in influencing health seeking behaviors and the adoption of positive behaviors at the individual, household, and community levels? Are there any issues in the development of IEC materials? How is RBHS addressing these issues? Are there any identified gaps in the development and delivery of messages? How is RBHS addressing these gaps? What are the most appropriate and effective channels used to reach the target audiences? Does RBHS training/capacity building approach include SBCC capacity building of partners and providers? What can be done to improve BCC/IEC delivery at the community level? Are there indicators relative to the adoption of positive behavior in the communities attributable to RBHS BCC/IEC delivery? Cross-Cutting How well does the project respond to the government desired direction for Liberia? Is the structure of the project well integrated with the MOHSW program to assist the GOL in meeting its health MDGs? Cite any examples of where the program made significant differences in how the MOHSW operates, results obtained, and its reform and direction. What elements of the program are making progress toward sustainability? What are not, and what else could be done? REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 59 Describe the work done to strengthen national institutions, professional associations, and NGOs. What are the major results, challenges, and recommendations? How does the program complement the work of other donors, NGOs and MOHSW health programs? Any missed opportunities and recommendations? What mechanisms are in place to ensure coordination and synergy with GOL, other donors (specifically Pool Fund, GAVI, and GFATM), and other United States Government supported activities? How effective are these? What recommendations Management How well is the overall administrative and implementation structure working to manage and carry out project objectives? How well is the RBHS team, including management structure and staff positions, interacting productively with the AOTR and AO, USAID health team? What are the relative strengths and weaknesses? How is the current program being managed (both technically and financially)? Discuss the degree to which this management approach adequately documents decisions made, accomplishments, and changes. Discuss any challenges to the managements approach that affect outcomes. How well does communication flow between the prime and sub-grantees? What are the successes and challenges? Discuss any recommendations for improvement? How effectively is RBHS managing the planned transition in project leadership to a Liberian COP? 60 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT How effectively has USAID been able to manage the RBHS project and provide needed management and technical direction? What have been barriers or shortcomings? How is data used to inform management decisions? Future Direction What are overall impressions of the RBHS project and recommendations for current and future programming? What are the three key lessons learned that Mission should focus on when developing its upcoming follow-on projects and implementing the new CDCS and GHI strategy? How can the project achieve a successful rollout to other areas not targeted by the current RBHS project? In other words, how can the RBHS project‘s successes best be institutionalized nationwide? What recommendations would you make regarding a future plan or approach for the sustainability of service delivery when external funding, which is now very significant, inevitably declines? How have the funding earmarks (POP, ID) affected the program? How well has the program been able to meet the requirements of the earmarks and report on them? Has there been, or does there need to be, a course correction to match the program with the funding categories? How have the program activities been perceived by beneficiaries and stakeholders (end-users, NGOs, MOH, UN agencies and other donors), the Mission, and the embassy? What have been the drawbacks of the US visibility and/or invisibility? REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 61 Capacity Building How good is the understanding of the human resource situation at the ministry of health, especially the composition and characteristics of HR department? What are the needs? Skill sets? When? Is there a needs assessment and an associated implementation plan? How good is the health information system? How many are working and where they are based? What are their skill sets? Job titles? How many involved in FP, Malaria, and reproductive health? Where are they based? Are they distributed throughout the country? Who is setting standards and monitoring performance, motivation level, pay or non-incentive schemes? What is the capacity of nursing and midwifery programs to take more students? How many staff have attended the in-service training? Who are the funders of specific training programs? How is capacity building strategy tied to your mission, organizational structure, and activities at RBHS? 62 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT Do you see a need for enhancement of institutional coordination with the government? Questions for Ministry Officials: 1. Overarching Questions: a. What is being done inside the Ministry to prepare for transition: i. Human resources requirements at all levels ii. Staffing patterns iii. Education levels iv. Recruitment v. Retention vi. Salary scales vii. Planning processes viii. Setting of indicators ix. Policy development x. Identification of specific health outcomes xi. Ministry level organization structure xii. Where does most senior public health official sit xiii. What are capacity building needs of the Ministry at all levels xiv. Who are major partners (other Ministry‘s? Donors? Other?) How is donor coordination managed? Effectively? Needs for improvement? xv. How are activities of donors and various implementing partners aligned with 10-Year strategy and 2-year strategies at county and district Levels? REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 63 xvi. What is the MOH doing to get ready to manage performance-based contracts? xvii. Will the Ministry monitor the NGOs? 2. What has RBHS accomplished? Strengths? Weaknesses? 3. What should RBHS focus on during the next two years? 4. Are there established procedures to access the Pool Fund? 5. Are staff in health facilities on MOH Payroll? If not, why not? 6. How will the MOH motivate community health workers? 7. Will the MOH provide ―incentives‖? 8. What entities in MOH are responsible for coordination and management of PBC? 9. What role does Ministry have in monitoring and evaluation? 64 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 65 APPENDIX F. RELEVANT USAID INTERMEDIATE RESULTS 66 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 67 APPENDIX G. WORK PLAN AND TRAVEL SCHEDULE GLOBAL HEALTH TECHNICAL ASSISTANCE PROJECT GH Tech Contract No. GHS 1-00-05-00005-00 USAID/Liberia Rebuilding Basic Health Services (RBHS) Year 2 Assessment Work Plan July 18: Team planning meetings including scheduling of field trips with RBHS team July 19: Debrief with USAID for clarification and focus and conclusion of team planning meetings Finalize team assignments and areas of responsibility Submit work plan, schedule, and questions to USAID July 20: Finalize questions for data collection, methodology to be used in data collection, and meet with RBHS team and World Learning July 21-22: Site visits in Grand Cape Mount July 23-25: Monrovia work/meetings with stakeholders available during National Holiday period July 26-30: Site visits to Nimba County and Bong County/return to Monrovia night of 30th August 1-2: Meetings with Government of Liberia and other key stakeholders August 3-5: Draft report August 6: USAID debriefing and draft report submission August 8: Ministry debriefing and team departure 68 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 69 APPENDIX H. POWERPOINT PRESENTATION TO MINISTRY OF HEALTH AND SOCIAL WELFARE REBUILDING BASIC HEALTH SERVICES (RBHS) YEAR 2 ASSESSMENT Purpose The purpose of this subjective assessment is to review the performance and the progress of the 5-year Rebuilding Basic Health Services (RBHS) project, which is now at its mid-point, and assess the appropriateness of the project design relative to the pending transition of responsibility to the Government of Liberia Methodology Subjective evaluation designed to collect and analyze opinions, perceptions and people‘s experiences with the RBHS Project The methodology included: Team Planning Meeting (TPM), document review, interviews with key individuals and organizations (including USAID/Liberia staff, Ministry of Health and Social Welfare leadership, County Health Teams, facility staff, other donors/international agencies), site visits to health facilities assisted by the project, and direct observation by team members. RBHS PROJECT: INTRODUCTION United States government‘s major initiative in support of the MOHSW. Funded by USAID, RBHS is a partnership among JSI Research and Training, JHPIEGO, the Johns Hopkins University Center for Communication Programs (JHU CCP), and Management Sciences for Health (MSH). The implementing partner is John Snow Inc. The project is active in select districts in Bomi, Grand Cape Mount, Lofa, Bong, Nimba, River Gee and Monsterrado counties INTERMEDIATE RESULTS Intermediate Result 1 (IR1): Increased access to basic health services through improved provision of quality health services and adoption of positive health behaviors. Activities: specific responsibilities in the areas of maternal and child health, family planning/reproductive health, malaria, HIV and water & sanitation, TB and nutrition Intermediate Result 2 (IR2): Increase the quality of health services through improving infrastructure, health workforce and systems performance by enhancing capacity to plan, manage and monitor a decentralized health system Activities: strengthening and extending service delivery through performance-based grants to NGO partners, strengthening Liberia‘s health system in areas of human resource management, infrastructure, policy development, monitoring and evaluation, preventing disease, promoting more healthful behavior change communication and community mobilization. Intermediate Result 3 (IR3): Youth informed and networked on reproductive health (Merged into IR1) 70 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT RBHS: PROGRESS TO DATE SOURCE: RBHS SEMI ANNUAL REPORT (MARCH 2011)
 Intermediate Result 1: 88% average score in the MOHSW‘s 2011 accreditation surveys. RBHS partners Medical Teams International (MTI) and Africare ranked first and equal second respectively at national level. 66% increase in facility-based deliveries. Tested 16,337 individuals for HIV, increasing by 67% from July 2010 through March 2011. 69% increase in pregnant women receiving a second dose of intermittent preventive treatment of malaria (IPT2). 164% increase in couple-years of family planning protection Treated 112,750 children for malaria, averting an estimated 2,255 deaths. Reached 84% of the target population with messages on ITNs and documented a utilization rate of 78% among respondents and 80% of their children in households that owned an ITN. Intermediate Result 2: Improved already high administrative performance, so that by the latest quarter 100% of facility staff were paid on time and 100% of HMIS reports were submitted on time. Participated actively in 24 national working groups, task forces, and steering committees. Made substantial contributions to the development of the Country Situational Analysis Report for the Health Sector, and drafts of the National Health and Social Welfare Policy, National Health and Social Welfare Plan, county health plans, and Essential Package of Health Services. KEY FINDINGS Key Strengths of Project Introduction of data driven culture Evidence based service delivery Supportive supervision Some improvement in quality of care Supplies and incentives provided on time Visible community volunteer network In service training Key Weaknesses of Project Capacity building of CHT‘S and MOHSW mid-level management Little attention to infrastructure development (facility repair and expansion, wells, etc.) Little attention to health care financing Reporting mechanisms sometimes delay financing by USAID Focus on numbers, not much attention on quality of service and health outcomes REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 71 Key Systemic Issues Impacting Future Direction Central: For decentralization, lack of integrated action plans and work plans, policies, procedures necessary for systemic health system reform (enabling environment) Lack of adequate capacity (human, infrastructure, organizational) Lack of understanding of county level capacity to effectively supervise and specific service needs Additional challenges that decentralization will bring for management of the health care system at central and county levels County Level Lack of integrated action plans, policies, procedures, goals and objectives (enabling environment) Lack of capacity (human, infrastructure, organizational) Lack of community ownership DISCUSSION Evolving Context: Decentralization MOHSW management of performance based contracting CHT supervision Many activities already underway to address needs at national level RBHS making major contributions at all levels Lack of ―systems‖ approach that has interrelatedness and consistency throughout Lack of understanding of county level needs CONCLUSIONS RBHS project has made significant progress in meeting its objectives with the weakest areas being associated with capacity building of the CHT‘s, infrastructure development and health care financing Transition to decentralization should be gradual, strategically planned and implemented only as MOHSW capacity at all levels is sufficient (midlevel and CHT management, human resources, policies, procedures, etcetera) MOHSW must address the issue of training, recruitment and retention of all categories of health care workers, particularly those assigned to rural areas Need mechanism of coordination and collaboration between MOHSW and PCB as well as between CHT and PCB under leadership of MOHSW Large need for capacity building at all levels particularly mid-management an supervisory levels Need to harmonize CHT structure, composition and scope of work 72 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT RECOMMENDATIONS: GENERAL GUIDELINES Alignment with MOHSW 10-year strategic plan and 2-year implementation plans including all county and district plan Stakeholder involvement of all key parties in each administrative level (central, county, district, facility) to include administration, service providers, end users, private sector, CHT, public health and curative leadership, etcetera) Continuity in and integration of policies, procedures, capabilities and operations from central down through facility levels Focus on health outcomes and quality of service not just number of events, individuals or cases KEY RECOMMENDATIONS: CENTRAL MOHSW Implement longer, phased in decentralization plan consistent with capacity, at all levels, of MOHSW to manage it. Design and implement a nationwide BCC and integrated health education program using all available technologies Strengthen and support the Community Health Service and the Health Promotion Divisions DEVELOP STANDARDIZED SUPERVISION CHECKLIST AND TOOLS Develop strategies to promote and sustain the motivation of community health volunteers Develop and implement a revised salary scale for all health providers in the public sector Design and implement a national referral system Develop incentive program whereby additional training opportunities can be offered to health providers in rural areas who have served more than two years. Specific training opportunities and programs may or may not be available in Liberia but should, in all cases, be consistent with health sector work force needs and projections Strengthen existing professional and technical training programs (both public and private) in such areas as curriculum development, evaluation criteria, continuing education, licensing and certification, etcetera). Institutions to include: Accredited nursing and midwifery schools, Schools of physician assistants, School of Environment; School of Pharmacy, U.l.; School of Medicine, U.l.; Health Sciences Department, Cuttington University; Cuttington University Graduate School (mph program Develop new public health indicators at the county and district levels (environmental health and mental health, for example) REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT 73 Develop administrative instrument (MOU) between PBCS and CHTS to identify roles and responsibilities, strategies, targets, a monitoring plan and reporting requirements KEY RECOMMENDATIONS: CENTRAL RBHS: Support existing MOHSW capacity building activities through a counterpart approach to providing TA for transfer of skills Contribute to the design, implementation and review of implementation plans to ensure continuity, consistency and integration of policies, procedures, and goals from central level down through facility Assess existing experience with academic training programs to develop lessons learned Support MOHSW in development of long-term training plan in response to human resource needs in the health sector Support MOHSW in development of standardized supervision, checklist and tools Support MOHSW in development of nationwide BCC and integrated health education program Additional Support to MOHSW in: Strengthening of community health service and health promotion divisions Development of revised salary scale Developing network of local volunteers Development of national referral system KEY RECOMMENDATIONS: CENTRAL USAID: Coordinate inputs of other USAID programs with MOHSW activities Support MOHSW needs for more effective donor coordination in support of 10-year plan and related implementation plans at all levels Assist MOHSW with development of template administrative tool (i.e. an MOU) between central and CHTS KEY RECOMMENDATIONS: COUNTY LEVEL MOHSW: Focus on development of capacity of integrated county health care delivery system including integration with national plans, goals and objectives Conduct a comprehensive assessment of the capacity, potentials and weaknesses of the CHT‘s and other county stakeholders as soon as possible to identify capacity building needs Design model county health team and identify specific areas of functional responsibility Strengthen a standardized mechanism for accurate data collection and management 74 REBUILDING BASIC HEALTH SERVICES (RBHS): YEAR 2 ASSESSMENT Assess and strengthen current bcc capcity of pbcs and chts Expand and strengthen current bcc vertical campaigns and integrate activities with radio spots and IEC materials Introduce new methods such as audio drama series, listening clubs, cell phones and community dialogue Conduct an assessment of the Bomi County experience with PBC in order to identify lessons learned and applicability to other counties in Liberia KEY RECOMMENDATIONS: COUNTY RBHS (CONTINUED): Assist MOHSW with assessments of: Bomi county experience with PBC in order to identify lessons learned Capacity of PBCS and CHTS (managerial, technical, supervisory) Capacity, potential, strengths and weaknesses of CHTS Support the MOHSW with developing capacity of county health teams Assess RBHS in-service training activities and support to training institutions in order to identify lessons learned to be shared with MOHSW Assist MOH in strengthening public health leadership at county and district levels Support MOHSW in development of county facility and equipment infrastructure development, facility and equipment management and maintenance plans KEY RECOMMENDATIONS: COUNTY LEVEL USAID: Develop pilot community wellness program to include mental health modules (surveillance, education and services) Facilitate effective coordination between MOHSW, RBHS and other relevant USAID projects RECOMMENDATIONS: GENERAL/CROSS-CUTTING Infrastructure, health financing and public-private partnership development elements of the project should be removed from the Cooperative Agreement and assigned to another entity specializing in each of those areas For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com