DISCLAIMER Ibustibeate ratur aspelestia deliquia sitae etumquis sande ni nume nectur reped quat. Et enis in commo officat vollabo. Ut volorum sit excesto te venimax imoluptae nulpa quissit faceste non ped moloriate molorenit mo totat res eaquid minctatur ant laborporum. SYSTEMS FOR HEALTH (S4H) EVALUATION OF ACCOMPLISHMENTS AND LESSONS LEARNED 2014-2019 October 2019 CONTENTS LIST OF ABBREVIATIONS 1 BACKGROUND 2 GENERAL PROJECT OVERVIEW AND KEY AREAS OF FOCUS 2 EVALUATION PURPOSE AND QUESTIONS 2 METHODOLOGY 3 S4H APPROACH TO HEALTH SERVICE UTILIZATION AND QUALITY IMPROVEMENT 3 QUESTION 1: WHAT ARE THE CHANGES IN THE UTILIZATION OF KEY HEALTH SERVICES RELATED TO NUTRITION, FAMILY PLANNING, AND MATERNAL, NEONATAL AND CHILD HEALTH (MNCH) AT HEALTH FACILITIES? 4 MATERNAL, NEONATAL AND CHILD HEALTH (MNCH) SERVICES 4 FAMILY PLANNING AND REPRODUCTIVE HEALTH (FP/RH) 5 NUTRITION 7 LESSONS LEARNED 8 QUESTION 2: WHAT ARE THE CHANGES IN THE QUALITY OF KEY HEALTH SERVICES IN (NUTRITION, FAMILY PLANNING, AND MNCH) AT THE HEALTH FACILITIES? 8 CAPACITY BUILDING AND TRAINING 8 COMMUNITY PERCEPTIONS OF QUALITY AT HEALTH FACILITIES 10 LESSONS LEARNED 11 QUESTION 3: HOW DID S4H CONTRIBUTE TO ANY CHANGES IN THE QA/QI CULTURE AND PROCESSES AT THE FACILITY LEVEL? 11 CHANGES IN QA/QI CULTURE AND PROCESSES 11 QUESTION 3A: STRENGTHS AND WEAKNESSES OF QA/QI SYSTEM 13 FIXED AMOUNT AWARDS (FAA) 14 LESSONS LEARNED 15 QUESTION 4: HOW DID S4H CONTRIBUTE TO STRENGTHENING THE LINKAGES AND ENGAGEMENT BETWEEN THE CHPS AND THE COMMUNITY 15 S4H SUPPORT FOR CHPS IMPLEMENTATION STEPS 16 CHMC SUPPORT AND FUNCTIONALITY 17 HOME VISITS AND COMMUNITY MEETINGS 18 QUESTION 5: HOW HAS THE INITIAL PROJECT DESIGN CHANGED OVER TIME (E.G., MODIFICATIONS), AND WHY?18 ADJUSTMENTS TO PROJECT DESIGN AND IMPLEMENTATION 19 LESSONS LEARNED FROM IMPLEMENTATION 20 CONCLUSIONS 21 RECOMMENDATIONS FOR FUTURE PROGRAMMING 22 ANNEX 1: LIST OF INTERVIEWEES 24 3 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV LIST OF ABBREVIATIONS ANC Antenatal Care BEmONC Basic Emergency Obstetric and Newborn Care CEmONC Comprehensive Emergency Obstetric and Newborn Care CHMC Community Health Management Committee CHC Community Health Committee CHO Community Health Officer CHPS Community-Based Health Planning and Services CHV Community Health Volunteer DDHS District Director of Health Services DHIMS District Health Information Management System ENA Essential Nutrition Actions ENC Essential Newborn Care ETAT Emergency Triage Assessment and Treatment FAA Fixed Amount Awards FP Family Planning GHS Ghana Health Service HC Health Center IMNCI Integrated Management of Neonatal and Childhood Illness IUD Intrauterine Device IYCF Infant and Young Child Feeding LARC Long-Acting and Reversible Contraceptives LSS Life Saving Skills MICS Multiple Indicator Cluster Survey MNCH Maternal, Neonatal, and Child Health MOH Ministry of Health PTFU Post Training Follow-Up QA Quality Assurance QI Quality Improvement RHD Regional Health Directorate S4H Systems for Health SDHO Sub-District Health Officer URC University Research Co., LLC USAID United States Agency for International Development USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 4 BACKGROUND GENERAL PROJECT OVERVIEW AND KEY AREAS OF FOCUS The USAID Systems for Health (S4H) Project was a five year (2014-2019) $75 million project funded by USAID/Ghana intended to improve health service delivery by strengthening systems vital to ensuring service access and quality. To achieve this objective, S4H services focused on the Community Health Planning and Services (CHPS) program, mobilizing communities, and building public-private partnerships to maximize coverage. Working in close partnership with the Ghana Health Service (GHS), S4H areas of concentration included reductions in preventable child and maternal deaths, unmet need for family planning services and childhood mortality and morbidity from malaria, as well as improvements in the nutritional status of children under five and pregnant women. The project sought to enhance vital health-system building blocks while maximizing service coverage by scaling up evidence-based, high-impact interventions at the national level and in USAID’s five focal regions: Greater Accra, Central, Western, Northern, and Volta (See figure 1). University Research Co., LLC (URC) managed the project and worked in partnership with PATH, Plan International, Results for Development Institute, and local Ghanaian partners. EVALUATION PURPOSE AND QUESTIONS USAID/Ghana asked Evaluate for Health (Evaluate) to carry out this evaluation, working from already existing data, primarily from Evaluate’s Baseline, Midline, and Endline studies,1 to learn about the contributions of the S4H project in the five USAID focal regions in Ghana from 2015-2019. This evaluation extracted findings and lessons learned from S4H implementation in key areas of interest for which Evaluate data were available, including workforce development (especially supportive supervision) and its contribution to the quality of care; the culture of Quality Assurance (QA) and Quality Improvement (QI); overall CHPS support and engagement with the community; and service delivery uptake (e.g., nutrition, family planning, and maternal/neonatal/child health). USAID/Ghana intends to review this evaluation to help inform future programming decisions about how it can best support continued improvements within the Ghana health system. The evaluation questions examined data from the period 2015-2019 when S4H activities were implemented across the five USAID focal regions. This evaluation focused on the critical health services mentioned above delivered at community-level health facilities (e.g., health centers and CHPS). The five evaluation questions are: 1. What are the changes in the utilization of key health services related to nutrition, family planning, and maternal, neonatal, and child health (MNCH) at the health facilities? 1 USAID Systems for Health 2 pager, “Maternal, Neonatal, and child health”, The 2018 Year review, https://www.urc￾chs.com/sites/default/files/urc-s4h-mnch-2018.pdf Figure 1: Systems for Health’s Regions of focus 5 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV 2. What are the changes in the quality of key health services in (nutrition, family planning, and MNCH) at the health facilities? a. What support did S4H provide towards improving the quality of any of these services? 3. How did S4H contribute to any changes in the QA/QI culture and processes at the facility level? a. What are some of the current strengths and weaknesses of the QA/QI systems? 4. How did S4H contribute to strengthening the linkages and engagement between the CHPS and the community? 5. How has the initial project design changed over time (e.g., modifications), and why? Based on the answers to these questions, the evaluation draws conclusions on what should be done differently in future programs and why and makes recommendations for future interventions. METHODOLOGY This evaluation primarily used secondary data analysis to answer the evaluation questions, including existing qualitative and quantitative data from previous USAID evaluations, project data (e.g., indicator data, special studies, the status of CHPS implementation data, etc.), and DHIMS2 data. The evaluation team also collected primary data from a small sample of key informants through qualitative interviews (e.g., S4H project staff, GHS, and USAID- see list of interviewees in Annex A). The team collected this primary data to fill in the gaps and add more specific S4H context into the analysis after it completed the review of available secondary data. Data analysis focused on examining the changes over the last four years in USAID’s five focal regions, to which S4H provided support. Analyses output from quantitative data are summarized into charts indicating trends for critical indicators. When applicable, the team used quantitative baseline/endline data to compare focal regions versus non-focal regions for critical indicators. The team re-analyzed qualitative data from the baseline/endline using Nvivo12 to extract relevant information to answer the evaluation questions. S4H APPROACH TO HEALTH SERVICE UTILIZATION AND QUALITY IMPROVEMENT Systems for Health (S4H) worked to improve equitable access to, demand for, and use of high-quality, high-impact health services, with a focus at the primary care level (i.e.. CHPS zones). The S4H approach focused on supporting the GHS to coach former trainees, institutionalize the competencies learned during training, and strengthen implementation of process improvement at facilities. By using data from previous visits and DHIMS2, the approach sought to facilitate on-site mentoring that focused on specific competencies and challenges in targeted facilities or geographic areas. Core elements of the process included data-driven planning, counseling, constructive feedback, and supportive problem-solving. S4H also constructed 26 CHPS compounds and rehabilitated 50 facilities in two regions - Volta and Northern. The project’s approach also involved using the GHS CHPS Implementation Guidelines2 to improve the functionality of targeted CHPS zones across all five regions and engaging GHS counterparts to systematically use data to plan, prioritize, and implement programs, especially in the face of limited resources. The project promoted the use of disaggregated data, emphasizing district and facility-level values to target interventions to facilities and districts with the highest service delivery gaps, improving 2 USAID Systems for Health 2 pager, “Family Planning and Reproductive Health”, The 2018 Year review https://www.urc￾chs.com/sites/default/files/urc-s4h-fprh-2018.pdf USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 6 the efficiency and effectiveness of resources in achieving desired health outcomes. This approach was also intended to build the skills of GHS leaders and providers to systematically and continuously use data to implement adaptive interventions to address challenges and to support sustained systems-level change. QUESTION 1: WHAT ARE THE CHANGES IN THE UTILIZATION OF KEY HEALTH SERVICES RELATED TO NUTRITION, FAMILY PLANNING, AND MATERNAL, NEONATAL AND CHILD HEALTH (MNCH) AT HEALTH FACILITIES? According to comparative data from the Evaluate midline and endline studies, nationally, health centers on average served most clients in their outpatient departments, with a mean of 1,086 clients in the two￾month period before the survey. Children were the second largest client group. CHPS zones on average served 321 clients in the two-month period before the survey, with largest number of clients being children. MATERNAL, NEONATAL AND CHILD HEALTH (MNCH) SERVICES S4H supported CHPS strengthening activities to increase access to and improve the provision of MNCH services, including antenatal and postnatal care, child growth monitoring, and immunization. Since CHPS providers also conduct community-level education to promote the utilization of these services, the project facilitated the provision of patient-centered maternity care through establishment of pregnancy schools at facility and community levels.3 As of October 2019, there have been 215 pregnancy schools implemented in Western and Greater Accra regions, with a total of 23,753 participants. S4H collaborated with GHS to carry out evidence-based MNCH service strengthening, including Essential Newborn Care (ENC), Integrated Management of Neonatal and Childhood Illnesses (IMNCI), Life-Saving Skills (LSS), and Emergency Triage, Assessment, and Treatment (ETAT). Project interventions focused on knowledge and skills development integrated with leadership and QI methods training, post training follow-up (PTFU) visits, integrated on-site support visits, and shared learning platforms. By the end of FY2018, 106 midwives had been trained in LSS and 243 health workers received follow up visits for ENC while 147 health workers received follow-up visits for IMNCI. This report compared baseline to endline data on deliveries, ANC services, stock-out indicators, described below. CHPS and Health Center Deliveries: Since S4H started in 2015, there were increases in the number of CHPS and Health Centers across all regions providing deliveries. CHPS in focal regions had higher increase from 25.8% to 44.2%, compared to non-focal 23.8% to 28.3%. (Figure 2). A factor behind the increase in CHPS zones conducting deliveries could be policy changes that assigned midwives to more CHPS zones and allowed some trained (CHOs) to conduct deliveries at these lower-level facilities. At the same time, traditional birth attendants have been more likely to coordinate with nurses and to refer their clients to facilities for delivery in recent years. The average number of deliveries in facilities reporting performing deliveries in the two months before the survey increased for both types of facilities, in all regions. Deliveries in CHPS zones in the focal 3 USAID Systems for Health Project Annual Report October 1, 2017–September 30, 2018 7 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV regions increased from 6.2 to 10.4 on average and at health centers increased from 28 to 36.4 deliveries, on average. ANC Services: The number of facilities providing ANC services increased in both focal and non-focal regions from baseline to endline. For CHPS, focal regions at the endline had a 7.9% increase over baseline (65.5% to 70.7%) in the number of facilities providing ANC services, while non-focal regions had an 8.2% increase over baseline (57.9% to 62.7%). At health centers, the non-focal regions had a slightly better improvement (92.1% to 98.6%) compared to the focal regions (92.7% to 97%). Sulfadoxine-pyrimethamine: CHPS recorded an increase in mothers receiving at least two doses of sulfadoxine-pyrimethamine from 83.2% at baseline to 89.5% at endline in focal regions and 87% to 93.6% in the non-focal areas. Health centers also had an increase, but they were smaller than CHPS. Stock-outs of BEmONC commodities: Overall, health centers experiencing stock-outs of essential BEmONC commodities increased between 2017 and 2019. However, in focal regions the increase was smaller. For parenteral antibiotics, in focal regions 6.9% experienced stock outs in 2015 which increased to 13.4% in 2019. In non-focal regions stock outs increased from 10.6% to 17.7%. Magnesium sulfate for preeclampsia increased significantly from 8.7% to 22.4% experiencing stocks outs in the last two months in focal regions, compared to non-focal region 4.7% to 17.3%. HCs with parenteral antibiotics available on the day of the interview increased from 77% to 96.6% in non-focal regions between 2017 and 2019; however, there was a decrease in the availability in focal regions from 88.6% to 84.2%. The availability of these drugs, which were outside the scope of S4H, buy can affect the quality of the services. FAMILY PLANNING AND REPRODUCTIVE HEALTH (FP/RH) S4H supported the GHS to increase access to FP/RH by building provider capacity to offer long-acting reversible contraceptives (LARCs) at health centers and CHPS. S4H’s training followed the introduction of the task-shifting policy by the MOH/GHS to attain universal access to quality reproductive health care. Before 2014, LARCs were provided only by doctors and some trained midwives. S4H provided on-site coaching for providers in implant and intrauterine device (IUD) insertion and removal, FP counseling, and data and commodity management, regional resource teams, and FP preceptorships to provide technical support for FP interventions. Overall Systems supported over 1,000 midwives and nurses in 105 districts trained in Years 1–3 in FP counseling and LARC through on-site coaching visits targeted to areas with the most need.4 The project addressed systemic service delivery challenges that affect access to and utilization of services through QI approaches and shared learning (e.g. bringing facility teams together to work on common indicators, identify performance gaps around those indicators, craft improvement aims, identify root 4USAID Systems for Health 2 pager, “Nutrition”, The 2018 Year review, https://www.urc-chs.com/sites/default/files/urc-s4h￾nutrition-2018.pdf USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 8 causes and performance barriers, and design local interventions to address the issues identified). Shared learning began in 2017 in six sub-districts in the Western Region and expanded in 2018 to 24 districts in the four other regions. The project also created user demand through community meetings and health promotion activities. The evaluation compared baseline to endline data on contraceptive provision, stock-outs, family planning method indicators, described below. Contraceptives provision: Overall, the provision of contraceptives increased among CHPS zones to complement ongoing family planning counseling. By 2019, nearly all CHPS zones and health centers offered both family planning counseling and contraceptives, with an increase at endline compared to baseline, especially among CHPS (84.2% to 96.3%). Nonetheless, secondary data suggest that, nationwide, most women still do not use any family planning method. Specifically, per the 2017-18 Ghana Multiple Indicator Cluster Survey (MICS), 73 percent of women between the ages of 15 and 49 who were married or in a union did not use any family planning method (MICS 2018). There was a decrease from 2017 to 2019 in the number of clients receiving contraceptives for the first time at CHPS in the focal regions (19.8 to 13 average clients in the last two months). The non-focal areas had a smaller decrease (10.5 to 9.9 average clients). In health centers, the non-focal areas had a considerable decline from an average of 57.2 to 32.2, while for the focal regions was 43.6 to 42 average clients in the last two months. LARCs: Although almost all CHPS and health centers reported providing long-acting contraception methods, there was a decrease across all regions in the number of CHPS providing at least four modern methods of family planning on the day of the interview, with the decrease in focal and non-focal regions from 75% in 2017 to 58% in 2019 in focal regions. Health centers, however, showed increases, especially in the focal region, with 69% in 2017 vs. 87% in 2019. This contrasts with S4H project data, which shows that across 105 districts S4H provided training for, from 2015 to 2018, the number of yearly new LARC (e.g. IUD, Implant) users nearly doubled, from 44,783 to 91,8315 . FP commodity stock-outs: CHPS experiencing stock out of any of the 6 family planning commodities (e.g., injectable contraceptive, condoms, hormonal implant, combined oral pill, progestogen-only pill, or intrauterine device), decreased in focal regions from 80% to 76%, while non-focal regions decreased from 79% to 69%. Qualitative interviews from the Evaluate endline identified several critical barriers to the availability of family planning services. These include inadequate or inappropriate infrastructure (for example, lack of private rooms for counseling and service delivery such as the provision of IUDs), equipment such as exam tables, the availability of staff with the right training, and the availability of medication. For example, a CHO explained that lack of space and privacy makes it impossible for her CHPS compound to provide family planning services. Further, the availability of reporting forms and client record books for FP has had a significant effect on the quality of services provided to clients, especially for follow-on visits where knowing a client’s 5 Systems for Health, PMP data, FY2018. 9 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV previous history has a bearing on the quality of services offered. KII respondents from S4H observed that during the period of the interventions: “Clients come to the facility, and they are provided with the FP method, but all the demographics and the (associated) reproductive health and medical data that is to be collected on the client are not obtained. As a result, there is no information for follow up on the client. Even if the client goes back to the same facility and meets the same provider, there is nothing to refer to; to help give the client holistic counseling. So those perhaps were not within our mandate to provide, but it affected the quality of the implementation of the interventions”. NUTRITION S4H worked to improve the nutritional status of children under 5 and pregnant and lactating women through training and on-site support visits to build health workers’ capacity to provide quality services (e.g., anemia prevention and case management, Essential Nutrition Actions, Infant and Young Child Feeding, and lactation management). Shared learning and quality improvement activities also supported facilities to improve the quality and availability of nutrition services. To reduced missed opportunities to improve nutritional status, nutrition was integrated into MNCH training, coaching visits, and quality improvement activities.6 By the end of FY2018, the project had trained 4,300 people in child health and nutrition. In addition, in total 3,599 health facilities had established capacity to manage acute undernutrition7 . S4H project data and baseline to endline data comparisons regarding nutrition are described below. In the fourth year of the project, S4H followed up with 55 providers who had received trainings to assess gaps and provide coaching. S4H reported that providers showed good retention of skills in clinical assessment (87%) and counseling (86%) on anemia. Skills with more room for improvement included hemoglobin testing (68%) and treatment for anemia (56%). Many sites lacked testing equipment, making it difficult for providers to practice and retain testing skills and to properly diagnose anemia.8 At the endline, the recording of infant and young child feeding (IYCF) counseling data occurred in fewer than half of the CHPS zones and health centers. In general, reporting on some of these nutrition indicators was still a challenge—particularly in CHPS zones. S4H project data indicates that a new CHPS supervision tool supporting coaching for CHOs on nutrition assessment and counseling for children was rolled out in May 2018. At that time,, of 79 CHOs observed, nutrition assessment and counseling came out as a major area that needed strengthening at the CHPS level. Only 37% of CHOs checked the Mid￾Upper Arm Circumference (MUAC) to help assess nutritional status. S4H used “the number of facilities conducting Severe Acute Malnutrition (SAM) admissions” from routine GHS DHIMS 2 data as a proxy measure for the indicator “number of facilities with capacity to manage acute malnutrition”. The data showed a steady rise from baseline of facilities admitting severe acute malnutrition cases. In FY 2017 1,885 facilities managed acute malnutrition cases representing 6 USAID Systems for Health Project Annual Report October 1, 2017–September 30, 2018 7 Data Source Systems for Health FY 19 Quarter 3 Performance Indicator Table 8 USAID Systems for Health 2 pager, “USAID Systems for Health”, Project Overview, https://www.urc￾chs.com/sites/default/files/urc-s4h-project-overview-2018.pdf USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 10 about three times the number of facilities which offered the service in FY 2016( 616). However, there was a sharp fall in FY 2018 to 703 facilities.9 LESSONS LEARNED Lessons learned related to the utilization of services as shared by KIIs included: - The utilization of vital health services is influenced by the quality of care provided by the health facility. When clients are satisfied with the quality of care, there is increased utilization of health services. - Using QI approaches to identify and address systemic service delivery challenges that affect access and utilization of health services led to improved utilization of health services. - Training and subsequent follow-up visits to providers resulted in improved quality and, in turn, increased utilization by clients. - NHIS accreditation status and reimbursement protocols have a significant effect on the gatekeeper system for the utilization of health services. Although some facilities (CHPS and Health Centers) had qualified personnel, they could not dispense some specific medications because they were not accredited to administer those medications. Therefore, clients bypassed these gatekeeper facilities within their geographic reach to seek care at other facilities. that could administer needed medications. QUESTION 2: WHAT ARE THE CHANGES IN THE QUALITY OF KEY HEALTH SERVICES IN (NUTRITION, FAMILY PLANNING, AND MNCH) AT THE HEALTH FACILITIES? CAPACITY BUILDING AND TRAINING Systems supported the GHS to coach former trainees, institutionalize the competencies learned during trainings, and support the implementation of process improvement at facilities throughout the five regions. Using data from previous visits and DHIMS2, on-site mentoring focused on specific competencies and challenges in targeted facilities or geographic areas. Coaching visits also targeted facilities and districts that are participating in shared learning to support the implementation of change ideas and enhanced results. The project worked to ensure frontline workers get one-on-one time with supervisors through supportive supervision, integrated coaching, and post-training follow-up visits. 10 9 USAID Systems for Health Project Annual Report October 1, 2017–September 30, 2018 10 Note data was missing from three of the non-focal regions: Ahafo, Bono and Eastern. 262 395 616 1885 703 0 500 1000 1500 2000 FY 2014 FY 2015 FY 2016 FY 2017 FY 2018 NUMBER OF FACILITIES FISCAL YEAR FIGURE 3: NUMBER OF FACILITIES WITH CAPACITY TO MANAGE ACUTE MALNUTRITION 11 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV S4H supported the application of GHS Supportive Supervision Guidelines which provide a harmonized approach to supervision at all levels within the health sector, including individual supportive supervision, integrated supportive supervision for health care services, and integrated supportive supervision of management systems. S4H had trained a total of 845 regional and district level supervisors as of June 2019. The trainees included senior and middle level managers/clinicians, physician assistants, midwives and health information officers. By the end of FY2018, S4H had trained 39,000 health care workers which included, over 13,000 coaching visits carried out with GHS to health facilities, with 20,500 health workers trained in infection prevention and control. By the end of the fourth year, 2,333 facilities in 114 districts received at least one integrated coaching visit, and 877 facilities received two or more.11 The evaluation compared baseline to endline data the capacity building indicators, described below. Training combined with supportive supervision: There was a substantial decline between baseline and endline in the percentage of facilities in which staff had received training combined with supportive supervision in the last 12 months. Facility survey data revealed decreases in training on topics related to malaria, malaria data tracking, maternal and child health, nutrition, and management-related topics. However, in most cases, these decreases occurred between midline (2017) and endline (2019). At endline, almost half of CHPS zones and more than half of health centers nationwide indicated that they had unmet priority training needs. The decline in staff trained in these topics was likely due to (1) a shift after midline to training frequency that followed the GHS guidelines of training every three years and (2) S4H programming shift in the USAID focal regions from training to supportive supervision. The decision to shift away from focusing on just trainings came from a review of the project after its second year which resulted in a strategy adjustment to enable S4H to achieve it’s expected results. S4H opted to deliver more on-site training, such as supportive supervision. The adjusted approach proved more effective in building capacity through continuous learning. Another consideration was the high cost of residential training. By switching to a more decentralized approach of working out of its regional offices and traveling to health workers rather than taking them out of work, S4H achieved substantial cost savings. (See section below on adjustments to project design for me details). “It became very apparent that there was a need to provide continuous contact with the providers, so the emphasis in terms of implementation moved a bit more into supervision in the various forms. We had supportive supervision, we had the post-training follow-up, we had the integrated coaching; all were ways of providing follow-up to ensure that providers are able to replicate their skills in their working environment and where you would identify any challenges and could help them in collaboration with their district teams or regional health management teams to address those.” (KII S4H Staff) Adherence to Protocols: GHS staff interviewed at the district, and facility-level were of the firm view that one difference the S4H project made was teaching the habit of adherence to protocols. They stressed that adherence to protocols led to an improvement in the quality of services provided. 11 USAID Systems for Health 2 pager, “Quality improvement/leadership Management”, The 2018 Year review. https://www.urc￾chs.com/sites/default/files/urc-s4h-qil-2018.pdf USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 12 Although protocols were always available, staff previously did not pay full attention to these guidelines. A district director explained that “For the past two years, the word adherence has stuck in my head. …. I knew IMNCI but did not fully follow the IMNCI protocols. Protocols provide quality assurance. It (Protocols) was there, but now for me, the change I have seen is adherence If you are adhering to protocols and you are supervised it will ensure your quality of care will improve” (KII, DDHS, Greater Accra) Closely linked to adherence to protocols was the constant interaction between district-level facilitators and providers at the facility level. According to GHS staff interviewed in Greater Accra Region, the project changed strategy and ensured that they had six weeks back-to-back interactions, including training, interaction with facilitators, coaching, and peer-to-peer learning. GHS also assigned “regional parents” to serve as champions within the various zones. This constant interaction led to improved quality of care among the staff who were involved in the process. GHS views this process as highly sustainable. S4H and GHS staff at the district and facility level noted that at times, the choice of health workers to serve as champions or selected for training was not appropriate. GHS staff at the district level explained that sometimes the selection of health workers is made at the regional or national level instead of the district level. and results in the choice of health worker trainees who might be the right cadre, but who are not offering at the facility the kind of service addressed by the training. COMMUNITY PERCEPTIONS OF QUALITY AT HEALTH FACILITIES The endline findings indicated that clients at the CHPS zones were very happy with the quality of care that they received. Some clients equated the quality of care and treatment provided by the CHPS zones to service they would expect to receive at higher-level facilities such as hospitals. Clients cited the following factors contributing to their satisfaction: positive staff attitudes and the respect CHOs showed them, the quick service delivery they received, and the quality of treatment provided at their CHPS. An additional positive attribute mentioned by some clients was the ability to receive treatment on credit at their CHPS zone and pay when they have the money. A significant improvement reported by clients at endline was an increase in the number of skilled deliveries conducted in the CHPS zones. Other clients indicated the availability of female midwives at CHPS compounds was a considerable improvement. The non-availability of supplies and medication in CHPS zones was a significant factor that negatively affected client satisfaction regarding services received. While some CHPS clients reported they were provided with all the medicines required for their treatment at the CHPS, most clients across all five focal regions said that some essential medications such as acetaminophen (paracetamol) or other supplies were not available at the CHPS, and clients were asked to purchase them from the open market. At the health center level, clients were also satisfied with the quality of care and services they received, including the access to laboratory services. One service they flagged as lacking is the provision of blood transfusions. Most of the clients who patronized a health center were content with the attitudes of the health center staff, describing them as friendly, patient, and respectful. 13 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV In line with GHS infrastructural improvements at health centers over the last two years (e.g. rehabilitation, boreholes, etc.), especially in Northern and Volta Regions, many clients of health centers reported noticeable changes, including the renovation of existing structures, building new structures, provision of water and electricity, and purchases of equipment, supplies, and commodities, among other changes. The availability of medicines, on the other hand, remains a challenge at some health centers, and clients would like to see significant improvements in the availability of supplies and other commodities. However, DHIMS data for the community score cards assessments (Table 1), shows lower scores in focal regions compared to non-focal regions. Focal regions with the highest overall score were Central (83) and Savannah (85.6), while the highest ranked non-focal regions were Upper West (96.3) and Upper East (86.7).12 Overall the highest scoring area across all regions was “Leadership and Management”, followed by “conducting of home visits by CHO/CHN” and “NHIS assessment.” Table 1: Community Assessment Scores from January to September 2019 Region Availability of medicines Conducting of home visits by CHO/CHN Conducting of home visits by CHW/CHV Leadership and management NHIS Assessment Total facility score Total for Focal Regions 68.1 80.4 67.8 86.7 78.1 79.6 Total Non-Focal Regions 70 84.2 75.8 92.5 90 85.6 LESSONS LEARNED Lessons learned related to the quality of services shared during KIIs included: - Support for adherence to protocols, supportive supervision, coaching, and shared learning sessions were very effective in enhancing the quality of services provided at the facility. - The core cadre or providers for an intervention should be identified from the onset; if not, projects will spend resources on providers who cannot implement what they have been taught. - Involving communities and national level authorities deepens sustainability and provides a clear direction as to what they want in terms of technical assistance. - Shared learning was an essential tool for improving the quality of care. It allowed for ownership of interventions, peer learning, and sharing of best practices on what works and what will not work. It has advantages over the original peer review mechanism where facilities only react when the period for assessment is getting close. - The project from the onset was distracted by issues around who to work with and the roles of institutions like the Ministry of Health, GHS, and the various divisions within the GHS. The situation contributed to the initial lack of ownership for project interventions, which affected the quality of care. - Post Training Follow Up (PFTU) enables supervisors to address within a short time basic stumbling blocks that affect the delivery of quality services. S4H Staff have observed that for FP, simple changes are meaningful, like rearranging the service area. – The project learned by doing, making mistakes, building relationships, and empowering the people (health workers) themselves to do the necessary interventions.” 12 Defined as having an active QA/QI team that met at least once in the previous three months and a QA/QI action plan in place. USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 14 QUESTION 3: HOW DID S4H CONTRIBUTE TO ANY CHANGES IN THE QA/QI CULTURE AND PROCESSES AT THE FACILITY LEVEL? CHANGES IN QA/QI CULTURE AND PROCESSES S4H worked to integrate quality improvement (QI) with leadership and management (LM) strengthening, recognizing that service delivery gaps often cannot be completely closed without empowering leadership and management to address the issues. S4H support in QI/LM focused on helping GHS leaders strengthen their skills in collaborative planning, continuous use of data, coaching and mentoring, and implementation of follow-up actions. With the support of over 450 GHS Improvement Coaches, shared learning activities took place in 75 districts across the five supported regions. Eighty-nine districts with trained coaches implemented improvement projects. S4H also awarded Fixed Amount Awards (FAAs) to each Regional Health Directorate (RHD) to implement a regionally specific, leadership-led QI project aimed at improving critical maternal, newborn, and child health (MNCH) indicators. 13 The evaluation compared baseline to endline data for the QA/QI indicators described below. QA/QI Action Plans: In early 2019, 39.3% of health centers in focal regions, and 40% in the non-focal areas reported having active QA/QI plans,14 representing a substantial increase relative to baseline (22.3% and 29.9%). There were also important decreases between baseline and endline in the percentage of CHPS zones reporting that they did not have a QA/QI action plan in place: the percentage fell from about half of CHPS at baseline to one-third at endline. Among health centers, the percentage fell from about half to one-quarter of facilities. Overall, these findings show strong improvements in formal QA/QI activities between baseline and endline. Endline data from qualitative interviews suggested that the increase in the implementation of QA/QI action plans and activities, and the spread of QI to CHPS zones have been aided by increased availability of all types of health personnel and equipment, as well as training to enhance knowledge of how to implement QA/QI at health facilities. Training: Several CHOs and Sub-district Health Officers (SDHOs) also reported that they had received some form of training in the last two years to enhance their knowledge of how to implement and ensure QA/QI at health facilities, e.g., S4H training on Plan, Do, Study and Act. A few SDHOs and DDHSs mentioned that monitoring visits they conducted helped to support QA/QI as well. Several CHOs agreed that the visits were beneficial and enabled them to learn on the spot. One DDHS mentioned that S4H had trained them to conduct monitoring visits. However, some CHOs and SDHOs think that more could be done to enhance the implementation of QA/QI activities at facilities, and report that at times they are limited by lack of resources. 13 https://www.urc-chs.com/sites/default/files/urc-s4h-commmob-2018.pdf 14 Number of CHPS Zones indicated by the Ghana Health Service DHIMS 2 for the focus regions 15 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV Implementation of sanitation practices: In terms of measures to prevent and control infections, there was an overall increase from baseline to endline in application of measures related to sanitation, sterilization, and disposal, and ways of dealing with contagious clients among both types of facilities. Many of the measures examined were available in well over half of CHPS zones at endline and over 85 percent of health centers. Percentages in the focal region CHPS zones were often higher than those in non-focal regions. QA/QI Protocols: A minority of CHPS zones and health centers do not have any written QA/QI protocols, but they reported that they do have routines they follow to ensure QA/QI, such as hand washing between clients or using a new glove for each client. There was only one CHO out of the 20 interviewed who reported they had no knowledge regarding QA/QI, despite being at their facility for some time. Data Use: CHPS using QA/QI nutritional information to improve services in the previous two months, among facilities with nutrition included as a topic in their action plan, increased since baseline. In focal regions the increase was from 37.1% at baseline to 89.9% in the focal regions, compared to a smaller change in the non-focal areas of 53.9% at baseline to 55.3% at endline. All DDHSs interviewed reported using DHIMS2 data to track the performance of facilities and to inform their decision-making, notably as concerns posting of health worker staff, immunization coverage, planning community outreach, distribution of equipment, and financial disbursements. Health workers at the district and facility level explained that S4H interventions produced a significant shift in their attention to data. “Now people are talking data… People know that if there is not data, if you don’t have data to speak to activities, you haven’t done it” (KII, District Respondent, Greater Accra) This approach also built the skills of GHS leaders and providers to use data systematically to implement adaptive interventions to address challenges, which supports sustained systems-level change. Per the endline data, there was an increase in CHPS using data for decisions in all regions (Figure 5). Implementing QA and QI also included enhancing client-patient relationships and communications. CHOs interviewed at endline expressed the importance of having strong relationships with community members so they could feel comfortable coming to the CHPS zones. In addition, CHOs have incorporated 70% 64% 60% 54% 44% 73% 64% 62% 69% 57% 52% 58% 48% 54% 44% 77% 58% 67% 73% 54% 0% 20% 40% 60% 80% Plan community outreach Help allocate resources Improve supply chain and logistics Help develop action plans Identify training needs Figure 5: Percent of CHPS using Monthly Reports Data for Decisions in the Last 12 months Focal regions  Baseline Focal regions  Endline Non focal regions Baseline Non focal regions Endline USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 16 education, sensitization, and follow-ups of clients at the facility and in the community as part of the implementation of QA and QI. QUESTION 3A: STRENGTHS AND WEAKNESSES OF QA/QI SYSTEM Over the last five years, S4H made a focused effort to strengthen the use of QA/QI approaches for health care service delivery. Some critical strengths of the current QA/QI systems identified by respondents include: Improved capacity to implement quality improvement projects. This was supported through the fixed amount awards, which provided regions with funding to implement projects to improve selected health outcomes and with technical support to track and report on results. Improved use of data for decision making at all levels as a result of shared learning. At endline, more than half of CHPS zones in focal regions reported having used data generated for monthly reports in the previous 12 months for all five key purposes covered in the survey. “it [QI Intervention] also helped in ensuring that they looked at their own data and not just transmitting it to the next level...they actually looked at their own data and used it to inform a lot of actions they want to be doing after the shared learning sessions, so data use improved drastically, and I could also say that it is also one of the best practices we can get” (Respondent KII S4H). Although there have been improvements in the QA/QI systems, there are still areas that need strengthening and create challenges. Lack of Resources: Financial challenges and lack of logistics were cited as significant reasons that prevented improvement coaches from undertaking this activity as part of their quarterly routine monitoring visits. The high turnover of health workers as a result of internal staff changes and transfers contributes to a QA/QI knowledge vacuum. “We did the training, and we planned with them their routine monitoring visits which are supposed to be quarterly. But for obvious reasons, funding was a challenge, no fuel, no vehicle, so that so they weren’t able to do it on their own.” (QA/QI specialist S4H) Need for Expansion: Based on the recent successes, there is a need for expanding the QA/QI interventions to involve more health workers at the facility level. Some health workers shared that although some workers are incorporating practices, there are still people at the facility that are not engaging and see it as the responsibility of those that attend the learning events or workshops. FIXED AMOUNT AWARDS (FAA) Leadership-led Quality Improvement (QI) Fixed Amount Awards (FAA) were set up as performance￾based grants, with the regions receiving payments for meeting outcome indicator targets in select districts within each region. They focused on improving key maternal, neonatal, and child health indicators. After one year of implementation, all five regions made impressive improvements in their process and/or outcome indicators. Three of the five regions met or exceeded at least one of their outcome indicator targets (Table 2). Table 2: Region FAA Indicators June 2018 – May 2019 17 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV Region FAA Indicator June 2018 May 2019 Volta (6 districts) Neonatal mortality per 1,000 live births 9.65 5.3 Stillbirth rate per 1,000 births 16.12 12 Western (4 districts for skilled delivery and 2 for stillbirth) Skilled delivery coverage 33.3% 45% Stillbirth rate per 1,000 live births 23.35 14 Greater Accra (11 hospitals) Stillbirth rate per 1,000 live births 22.9 20.8 Northern (10 hospitals) Maternal mortality ratio per 100,000 live births 159 109.9 Under 5 malaria case fatality 0.23 0.24 Central (6 districts) Neonatal mortality per 1,000 live births 10.2 10 During the FAAs, S4H provided support through regular visits and check ins on activities. The regions implemented the project with the S4H regional team working alongside them. The regions had to overcome challenges which provided an opportunity for learning. For some regions this approached worked better than for others, but the primary advantage for all regions was to put GHS in charge of the design (e.g. writing the proposal) and of the implementation of the project. LESSONS LEARNED Lessons learned related to QA/QI shared by KIIs included: – GHS leadership engagement and commitment from the national, regional and district levels was crucial in implementing most of the QA/QI interventions in all five regions. – The team-based strategy used in implementing shared learning sessions made learning very effective since health workers are supposed to be working in teams. – There are existing structures in some regions, districts and hospitals where key staff are designated as focal QI persons. Strengthening these structures will ensure that all regions will have enough quality improvement specialists to support districts and sub-districts. – Moving trainings from classrooms to onsite increased the number of health workers reached and made the intervention more sustainable. – Regional ownership of QA/QI activities is critical both in terms of cost-effective training and local field capacity to provide support to districts and sub-districts, as demonstrated through the leadership led fixed amount award intervention. USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 18 – According to S4H staff, LSS training only covers four of the seven signal functions of the Basic Emergency Obstetric and Neonatal Care (BEmONC). If projects want to achieve BEmONC targets, they must be trained on the other three key functions. – S4H staff noted that although the IMNCI intervention has been implemented since the 2000s, it lacks needed coordination and standardized approach, especially when compared with the essential newborn care intervention. QUESTION 4: HOW DID S4H CONTRIBUTE TO STRENGTHENING THE LINKAGES AND ENGAGEMENT BETWEEN THE CHPS AND THE COMMUNITY Systems provided technical assistance to 483 targeted CHPS zones to help the GHS, communities, district assemblies, and other partners to advance along 15 steps of CHPS functionality (Table 3), many of which contribute to improved linkages between CHPS and their communities. S4H also worked to engage CHPS stakeholders through community meetings and events, built CHO capacity through internships and on-site training, and shared learning to identify and address service delivery gaps. In the Northern and Volta regions, S4H constructed and renovated health facilities to eliminate geographical barriers to care.15 Systems championed community mobilization and participation and gender integration throughout the construction of the CHPS Compounds. Community members were involved in all stages of the construction from the discussion of building design, siting and approval of final construction with contractors were brought to the field to hand over of the compounds. Capacity building for the Community Health Management Committee (CHMC) was another critical activity supported by S4H. S4H SUPPORT FOR CHPS IMPLEMENTATION STEPS S4H supported 483 out of 2,54916 CHPS zones (19%) in the five regions to progress along the 15 steps of CHPS implementation and strengthened community engagement through the implementation of the community scorecard. As of the end of June, the 483 CHPS across the five regions had completed an average of 13 out of 15 steps (Figure 6). 15 A CAP is an action plan created at the community level focusing on many different areas for improvement, such as education, markets, agriculture and the like. Some communities incorporate health-related action steps in their CAP, while other communities have a separate CHAP just for health-related activities. 16 Table 3: Steps of CHPS Implementation 19 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV CHOs and CHMCs in 477 CHPS zones (71 districts) received orientation on the community scorecard, with 449 supported to assess their CHPS zones. Out of the 449 zones conducting assessments, 439 developed action plans to address identified challenges. Preliminary baseline results from reporting zones revealed excellent scores on indicators such as caring, respectful, and compassionate care, as well as waiting time for the provision of health care service. However, milestones around the deployment of Community Health Volunteer (CHVs) were achieved by only 141 out of the 483 CHPS (Figure 7). Interviewees noted that CHVs are an essential feature of the CHPS and it is essential to ensure that their role is strengthened in CHPS implementation. In 60 CHPS zones (12%) community entry was not done which has implications for ownership and sustainability of these CHPS zones. CHMC SUPPORT AND FUNCTIONALITY An analysis of baseline and endline data provided the following insights into the functionality of CHMCs. There was a substantial increase in the proportion of CHPS zones with a CHMC since baseline: at endline, more than 97% of CHPS zones nationwide had a CHMC, increased from 62.8% at baseline. Qualitative data indicated that most CHMCs were functioning per official guidelines, including meeting regularly and working to improve health in their communities. 9.3 6.3 7.2 8.2 9.4 13.5 14.7 14.0 13.1 13.4 0 5 10 15 20 Central (118) Greater Accra (107) Northern (75) Volta (113) Western (70) AVERAGE STEPS COMPLETED REGION Figure 6: Average Steps Completed as at June 2019 by Region Baseline FY19-Quarter 3 USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 20 CHMC members reported that building the relationship between the community and the CHPS was an essential part of their role. CHMC members mentioned that they need to continue to give community members more education on the work CHPS staff do and encourage them to support the CHPS, as well as encourage the nurses to be patient with community members. CHMCs reported continuing challenges that hampered their effectiveness, including the lack of a regular funding stream for activities they would like to undertake to help their community’s health, lack of transportation for meetings, inadequate information sharing, inexistent transport means for clients, and requirements to prioritize their livelihoods over their CHMC responsibilities. Nonetheless, CHMCs reported some ways they have helped their CHPS zone, such as organizing durbars and leading community sanitation efforts. They have also fostered positive relationships between health workers and community members. At endline, most CHOs agreed that the effectiveness of their CHMC had improved considerably since baseline. CHMC members noted that they receive no stipend for their work which posed challenges since they must choose between the work of the CHMC and work for their livelihood. Most CHMCs reported that their community had a community action plan (CAP), often with health activities integrated into it.17 The most common health activities reported to be included in the CAPs and related Community Health Action Plan (CHAPs), according to the CHMCs, were sanitation efforts (especially cleaning the community), clean drinking water (for example, boreholes, pipe repair, etc.), and toilet facilities. Other priorities included working to fight malaria with bed nets, creating a dumping point, and working on preventing teenage pregnancy. Less common efforts included focusing on getting a midwife for the community and working to find the money for an ambulance. HOME VISITS AND COMMUNITY MEETINGS Home visits, follow-up visits and community meetings were recognized by CHPS stakeholders as important measures to increase linkages and engagement between CHPS and their communities. From baseline to endline, the average number of routine home visits (in the previous two months before the survey) more than tripled among health centers and CHPS nationwide, with the larger increase in the number visits being in the non-focal regions. There was little change in the percentage of facilities conducting visits; the change came from the increase in the number of home visits by each facility. In 2019, staff at all CHPS zones and health centers had conducted at least one home visit in the two months before the survey (this includes routine home visits, follow-up visits, and special care visits), representing a slight increase over baseline rates. These increases could be due to increased application of the National CHPS Implementation Guidelines related to systematic scheduling of home visits. The average number of follow-up visits—which are not routine and are thus a measure of the responsiveness of health workers—also increased in CHPS zones and health centers in all regions. In CHPS zones the average increased from 10.5 to 19, while in non-focal the average was 10 to 20 visits (Figure 8). 17 21 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV Changes in the average number of postnatal home visits did not change substantially across different facilities and regions, however, school visits for CHPS zones (e.g., health promotion activities or general health checks) dropped from 7.4 to 3.1, while in non-focal regions they increased from 2.8 to 3.4. At endline, 84% and 87% of CHPS zones in all regions had held at least one community health meeting in the quarter before the survey. Among CHPS zones that had conducted a community health meeting, the proportion of facilities reporting that the CHO was vital in planning and organizing the last community health meeting increased in focal regions from 59% to 69%, wherein non-focal regions it decreased from 62% to 47%. One reason for this could be the S4H support of the National CHPS Implementation Guidelines 15 steps. Nationwide, there were increases in the percentage of CHPS zones where CHVs, CHMCs and other community leaders organized durbars between the baseline and endline. QUESTION 5: HOW HAS THE INITIAL PROJECT DESIGN CHANGED OVER TIME (E.G., MODIFICATIONS), AND WHY? The S4H initial design was very ambitious. Through the five years of implementation S4H learned lessons from the challenges faced in carrying out the project's different interventions and approaches and adapted its approaches based on this learning. Overall, the design and objectives of the project remained the same over the five years. However, changes were made in the strategies and approaches to implementation over time. Because of the complexity of the project design, its wide geographic coverage, and the need to start implementation quickly, the initial focus of the project was on training. This was also the initial interest of the project’s key stakeholder, GHS, based on its understanding of the project. “Everybody was expecting the project to focus more on disease-specific outcomes, but the vision was to really strengthen the health systems. [The project] had to change approach and work at all levels of the health system instead of concentrating on the lower levels.” (KII Respondent) ADJUSTMENTS TO PROJECT DESIGN AND IMPLEMENTATION At the end of the second year, the project held an internal implementation review which indicated that the project was not on track to achieve the expected end-of-project results. The project critically reviewed the challenges they were facing that appeared to affect long-term project results. USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 22 “We were not seeing the results that we envisaged… despite the fact that it is a health system strengthening project; the ultimate outcome is the improvement the health status of Ghana, so we wanted to see the indicators really moving in the right direction” (KII Respondent) “We needed to show results that health system strengthening will eventually lead to results, and so that really made us take a second look and decide that we had to change the focus and some of the approaches in other to attain results.” (KII Respondent) Based on the project’s review of results data, experiences and workplan, S4H acknowledged that they could not achieve the health system strengthening results if the implementation focus continued to be predominantly on training. One of the main changes coming out of the project review was the shift from focusing on residential training to building capacity through on-site training and more direct technical support. The trainings were not yielding the desired results, and there was a need to follow up on those participants who had been trained to ensure that they applied their training on the job. Shifting towards a capacity building continuum with a focus on on-site and post-training support allowed the project to assist the full facility team rather than sending just one person to training. It also allowed the project to provide specialized support to different districts. While S4H shifted to on-site support in Year 3, the first two years of the project had served to lay the foundation for technical activities by updating technical guidelines and training materials. Beginning in Year 3 and ramping up in Year 4, the project focused on sustaining and complementing gains in provider competency through GHS-led on-site coaching and mentoring The project also moved to decentralize activities to its regional teams, empowering them to work closely with GHS counterparts in the region to jointly design and implement activities. S4H also shifted focus from trying to work everywhere and on everything to a focus on areas and districts facing the most significant challenges. S4H continued to work in all the districts in the five targeted regions, but concentrated on a few priorities at a time The new strategy also tied in leadership development with results. The project redirected efforts to focus on QI activities, working on improvement coaching training and building a relationship and day-to-day engagement with project counterparts. The GHS was very receptive to this new approach, especially after review of results data. “In the leadership quality improvement everybody sees themselves as a leader so when you are in the health or a district you see yourself as a leader and you make efforts to change the situation at your end, and we’ve seen facilities, districts and regions all implementing this kind of approach and they are demonstration results and the data that we get here is also showing improving trends.” (KII Respondent) S4H also adjusted use of its resources. working to find ways to use resources more efficiently due to budget cuts and uncertainty about future budget availability. Also, residential training was extremely costly but wasn’t contributing to the overall results. The new approach was less expensive and allowed more people from the facilities to be involved. LESSONS LEARNED FROM IMPLEMENTATION 23 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV KIIs with key S4H staff and stakeholders identified the following suggestions for future project design and implementation based on lessons learned from the project. – Start with a clear capacity building plan which provides a comprehensive, clearly defined strategic capacity building effort, with sustainability in mind from the outset. In the first few years of the project, S4H trained many providers but did not have a clear strategy for post-training support and for capacity building along a continuum. Plans for team-based on-site support at health facilities were not implemented early enough. This gap was addressed by the shift in strategy during the project’s third year, which was later. – Engage the national level. Although the project supported a large number of national-level activities (particularly technical working group meetings), more work was needed to engage the national level in project activities in the regions to ensure diffusion of ideas. – Address issues of ownership. A significant initial challenge was building ownership for project interventions at the GHS district and facility levels. – Take time to manage relationships. This project depended on the relationship with GHS for its success. The relationship was not strong at project outset, which affected implementation and results. More regular and collaborative consultations were needed to build trust and to gain buy-in. “Now we trust them [S4H] more. When you interact with people, after a while you become so familiar with them that they are no longer strangers. Sometimes when we see them, we see them as part of the health system and we are able to open up and have discussions with them. You don’t see them as strangers anymore.” (Ningo Prampram, DDHS, Endline) – Manage expectations. It took time for S4H to find their way and to get their counterparts on board with a different way of doing things. In the beginning, the project felt pressure to get activities started. When it was apparent that training would not be an effective means to achieve results, S4H had to manage counterpart expectations and convince them of the value of on-site support/coaching to improve provider competency and facility readiness. – Integrate Improvement Coaches. These coaches worked as champions and helped facility teams identify and prioritize health facility challenges and implement changes to address them. S4H worked with the GHS to transition this work to intra-district shared learning. Facility teams within districts and across levels of care worked together to resolve a particular health challenge. Shifting the work within the district significantly cut costs and achieved better facility and provider coverage. – Identify and engage champions. S4H engaged open-minded leaders and managers, explained implementation strategies to them and sought their advice on how to resolve issues. – Adapt strategies to conform to new policies and guidelines. The project had to change strategy to conform to new GHS policies and guidelines to ensure that training materials, coaching guidelines and technical support are developed in areas where they were lacking. – Consider systemic Challenges. S4H realized that there were many systemic challenges related to quality, service provider attitude, health financing, the management structures within all the levels, etc. Strategies are needed to address these challenges as well as the capacity issues. USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 24 – Address the demand side of the intervention. More attention was needed to improve community mobilization, demand generation, and interpersonal communication aspects of the project. S4H did not initially see this as part of their mandate, thinking that other projects would work on the demand side “The GHS structure as it is with its limited resources cannot do much to engage communities to increase demand for services, and so we struggled … eventually, community mobilization for CHPS came in, but it could still not adequately address the demand side, working with the people to know what they want so that the service providers provide that. You work to create the demand, and then you supply so there is a holistic approach. That was a big gap.” (KII Non-S4H Staff) “The project should be tailored towards the community more than we, the health staff; we participate, but most of the program should be within the community. They [future project] should liaise with the opinion leaders, they [opinion leaders] should know more about it [interventions] because they are the front line of the community and [community members] will listen to them more, and that will make the work easier…the change in behavior is difficult so it should be tailored towards the community members. (KII CHO) CONCLUSIONS Over the last five years, S4H has supported GHS in strengthening the health system in Ghana, especially in the areas of MNCH, family planning and reproductive health, and QA/QI. S4H has supported these improvements through building capacity, promoting learning and ownership, providing grant opportunity for improving health results (e.g. FAA), and integrating leadership. Key improvements since S4H began work in 2015 include Access to services: An increase in the availability of ANC services, especially at CHPS; increased access to LARC and FP counseling, and increases in the average number of home visits conducted by CHPS zones (routine home visits, follow-up home visits, and clients needing special visits). QA and QI : More than half of health centers have an active QA/QI team, and two-thirds of CHPS zones and three-quarters of health centers had a QA/QI plan at the endline in 2019; many CHPS zones and health centers conduct a range of QA and QI activities, including those related to hygiene and safety, supplies, and client satisfaction in their formal plans. Quality of services: Although hard to measure, S4H contributed to building strong health worker capacity which contributes to improved quality of health service delivery. S4H provided GHS with different interventions at multiple levels and across different sectors. Although data showed a decline in the number of health workers receiving training between baseline and endline, this is likely due to the shift to providing more on-site and targeted TA. Based on the results of the project and per KII respondents, this adjustment was necessary to enable S4H to take learning to a higher level and ultimately build lasting capacity. In addition to supporting the improvement of health services related to MNCH, Family planning and QA/QI, S4H also worked on cross-cutting areas to support health systems strengthening and sustainability. Over time, S4H was successful at building relationships at all levels of GHS. DDHSs and DAs reported good relationships and positive experiences with S4H during the endline survey. Once the 25 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV project took a more regional approach at the beginning of the third year, S4H was able to capitalize on the relationships they built and provide support through innovative interventions such as the FAA and QI projects, which allowed GHS to take a lead on the work and build ownership and sustainability. The project also successfully promoted the use of data for decision making across the different levels of GHS from district to facility. This approach also built the skills of GHS leaders and providers to systematically use data to implement adaptive interventions to address challenges and to support sustained systems-level change. This practice helped to improve the efficient and effective use of resources in achieving desired targeted health outcomes. Although there have been many successes, S4H also faced challenges that were beyond the project control, including availability of human resources, funding constraints, and supply chain issues, which also affect the achievement of health outcomes. S4H demonstrated the capacity to learn from their successes and mistakes and to apply adaptive management to improve results. The key lessons learned about building relationships ownership, targeting future programs, and cross-cutting interventions are documented int this report and should be considered when designing future health systems strengthening programs. RECOMMENDATIONS FOR FUTURE PROGRAMMING The following recommendations are proposed for future programming, based on KIIs with key S4H staff, USAID and GHS. 1. Future projects should enable flexible programming based on the needs of specific regions, districts and facilities, instead of trying to provide all interventions to all districts. S4H promoted the use of disaggregated data, emphasizing district, and facility-level values to target interventions to facilities and districts with the highest service delivery gaps, improving the efficiency and effectiveness of resources. By targeting their efforts, they were able to see improvements in service delivery results in all five regions. 2. Health sector stakeholders such as GHS should be involved in the design phase (e.g., co-design) of future interventions to enhance ownership and to help manage expectations. Involving stakeholders (e.g. MOH/GHS) in the design phase of the project would ensure their input and priorities are communicated early on in the process and that the local context is factored into the design. 3. Future programs should continue to put GHS as the lead of intervention design and implementation. Having GHS as the decision-maker lead to stronger ownership and a much better chance for the work to be sustained. 4. USAID should continue to create opportunities for development partners and stakeholders to plan for sustainability with the government (e.g., upcoming GHS/USAID/UNICEF joint meeting). Maintaining high-level discussions with the government helps to promote understanding and get their buy-in for current and future work. 5. USAID should continue funding grants for GHS to design and lead their own project implementation, such as through the fixed amount awards. The FAA provided an opportunity for GHS USAID.GOV USAID SYSTEMS FOR HEALTH: LESSONS LEARNED | 26 to develop and improve different skills through learning by experience while receiving TA support. This not only builds capacity but also delivers results in service delivery areas from the project interventions. 6. Health system strengthening programming should be cross-cutting and should be included in all health project approaches, rather than in one large, complex project. Complex project designs can affect results. 7. Transition and sustainability plans should be part of the design of the project instead of waiting for the last two years of the project to start sustainability planning. Having a clear plan at the beginning is also an opportunity also to explore cost-sharing or other avenues to encourage long-term investment and sustainability. 8. Coordination and collaboration with partners should be encouraged to ensure that the required data collection tools for service delivery are available to ensure continuum of care to clients. Since data are key and the data collection tools are not available, it will be necessary for a future project to also look at ways of addressing the issues of inadequate registers and other data collection tools within the health system. Recommendations for areas to target future programming shared by KIIs: 1. Future projects should support GHS in aligning the prescribing practices with NHIS reimbursements, protocols, and procedures to ensure that health workers who are qualified to administer a particular medication can do so even when the health worker is at a lower level facility. 2. Future projects should have pay equal attention to the community and demand generation interventions in addition to concentrating on the supply side (health workers). Although building capacity is essential, more focus on generating the demand at the community level is needed. 3. Project interventions should consider advance care for sick newborns since the current newborn strategy 2019 – 2023 goes beyond the primary essential newborn care and takes into consideration advanced care for sick newborns. S4H was unable to make any significant change in terms of prematurity and other ill newborns apart from asphyxia. 4. Future projects should sustain the gains made in FPRH by collaborating with GHS and partners to open more preceptor sites at the facility level. 5. Future projects should equip community health nurses to be able to provide FPRH services, especially LARC, within the homes of clients in the community where feasible. Provide LARC services at the community level will increase access to these services. 6. Future projects should work to strengthen adherence to nutrition assessment protocols for determining the nutrition status of children. With only 37% of health workers using the MUAC tape to assess the nutrition status of children, adherence to the nutrition assessment protocol is currently weak. Children under 5 will therefore not receive optimal service since this vital step is missed. 27 | USAID SYSTEMS FOR HEALTH: LESSONS LEARNED USAID.GOV ANNEX 1: LIST OF INTERVIEWEES List of interviews conducted for report: No Position Institution 1. District Director of Health Services – Krowor GHS 2. The staff of Dawenya CHPS GHS 3. CHN Prampram PolyClinic GHS 4. Senior M&E Advisor S4H 5. Senior FPRH Advisor S4H 6. Senior MNCG Advisor S4H 7. Nutrition Specialist S4H 8. Technical Director URC (Past Chief of Party) S4H 9. Senior Technical Advisor CHPS Coordinator URC 10. Senior Quality Improvement Advisor S4H 11. Public health specialist for USAID health office USAID