June 2018 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Principal Investigator Lydia Aziato, Emmanuel Mahama, and Gwynne Zodrow, Management Systems International, A Tetra Tech Company. Evaluation on Optimal Methods of Health Care Worker Training in Ghana Evaluation on Optimal Methods of Health Care Worker Training in Ghana Contracted under AID-641-Q-14-00001 / AID-641-TO-17-00002 USAID/Ghana Evaluate for Health Project DISCLAIMER This report is made possible by the support of the American people through the United States Agency for International Development (USAID). The contents are the sole responsibility of the Management Systems International and do not necessarily reflect the views of USAID or the United States Government. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 1 CONTENTS Acknowledgments......................................................................................................................... 3 Acronyms ....................................................................................................................................... 4 Executive Summary ...................................................................................................................... 6 Evaluation Design and Methods............................................................................................................................6 Key Findings and Conclusions...............................................................................................................................7 Recommendations.................................................................................................................................................10 Evaluation Purpose and Questions............................................................................................ 13 Introduction ...........................................................................................................................................................13 Evaluation Purpose................................................................................................................................................13 Evaluation Background............................................................................................................... 14 Ghana’s Health System.........................................................................................................................................14 Implementing Partners .........................................................................................................................................15 Evaluation Methods and Limitations ......................................................................................... 16 Sampling ..................................................................................................................................................................18 Data Management and Analysis Approach........................................................................................................18 Limitations..............................................................................................................................................................19 Ethical Clearance...................................................................................................................................................20 Findings, Conclusions and Recommendations ......................................................................... 20 Findings...................................................................................................................................................................20 Conclusions.................................................................................................................................. 43 External Factors That Influence Effectiveness of Training..............................................................................45 Recommendations ...................................................................................................................... 46 Annexes........................................................................................................................................ 48 Annex A: Evaluation Statement of Work .........................................................................................................48 Annex B: Evaluation Methods and Limitations.................................................................................................52 Annex C: Additional Tables ................................................................................................................................55 Annex D: Quantitative Tests Conducted .........................................................................................................68 Annex E: Data Collection Instruments..............................................................................................................68 Annex F: Disclosure of Any Conflicts Of Interest ..........................................................................................99 Annex G: References..........................................................................................................................................100 LIST OF TABLES: Table 1. Types and Topics of Relevant IP Trainings for Study ............................................................15 Table 2. Total Number of Interviews per Region .................................................................................18 Table 3. Distribution of Cadre of HCWs Interviewed By Training Type ............................................. Table 4: Percentage of budget per training.............................................................................................34 Table 5: Steering Committee Membership.............................................................................................49 Table 6: Study Deliverables.......................................................................................................................51 Table 7: Getting To Answers....................................................................................................................54 Table 8. District and Regional Distribution of Health Facilities Sampled..........................................55 Table 9. Distribution of Respondents According To Their Occupation...........................................55 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 2 Table 10. How Often Respondent Attends Training............................................................................56 Table 11. Respondents’ Responses to Environmental Factors Influencing the Work of HCWs.....................................................................................................................................57 Table 12. Knowledge Scores Obtained Per Knowledge Questionnaire Answered ........................61 Table 13: Total Knowledge Score Obtained By Respondents............................................................63 Table 14. Distribution of Trainee Knowledge Mean Scores Based on Questionnaire Answered................................................................................................................................63 Table 15. Distribution of Trainee Knowledge Mean Score Based On Training Models.................63 Table 16. Respondents’ Rating of Training Models...............................................................................63 Table 17. Respondents’ Rating of Training Implementing Partner .....................................................64 Table 18. Respondents’ Rating of Training Audience ...........................................................................65 Table 19. Respondents’ Rating of Training Topics................................................................................66 Table 20 : Respondents’ Rating of Training Duration...........................................................................67 LIST OF FIGURES: Figure 1. Distribution of Trainees Sampled............................................................................................21 Figure 2. Participant Age Groups.............................................................................................................21 Figure 3. Participants’ Occupation Profile...............................................................................................21 Figure 4. Trainee Mean Knowledge Scores per Training Model.........................................................22 Figure 5: Training Type Mean Rating .......................................................................................................25 Figure 6: Years Respondents Participants in Relevant Training ..........................................................26 Figure 7: Training Topic Mean Rating......................................................................................................27 Figure 8: Training evaluation framework with nested levels and situational factors.......................36 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 3 ACKNOWLEDGMENTS The research team is grateful to the USAID/Ghana Health, Population and Nutrition Office and its implementing partners for their contribution to the design and findings of this report, as well as providing needed data throughout the process. We especially would like to thank the members of this study’s steering committee who made substantive inputs into the study to ensure its success. We also thank the staff of Ghana Health Service (GHS) for their support and consent to interview participants in this study. Finally, we thank the Evaluate for Health staff for their support throughout the evaluation process. Cover Photo Credit: JHPIEGO OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 4 ACRONYMS CHN Community Health Nurse CHO Community Health Officer CHPS Community-based health planning and services CHV Community Health Volunteers CHW Community Health Workers DHIMS District Health Information Management System DHS Demographic and Health Survey ENC Essential Newborn Care EPCMD End Preventable Child and Maternal Deaths Evaluate Evaluate for Health GHS Ghana Health Service HCW Health Care Worker HIV Human Immunodeficiency Virus HPNO Health, population & Nutrition (USAID/Ghana) HQ Headquarters ICD Institutional Care Division IMNCI Integrated Management of Neonatal and Childhood Illnesses IP Implementing Partner IYCF Infant and Young Child Feeding JHPIEGO Johns Hopkins Program for International Education in Gynecology and Obstetrics KII Key Informant Interviews LDHF Low-dose high frequency MCSP Maternal Child Survival Program MOH Ministry of Health OSCE Objective Structured Clinical Examination OTSS Outreach Training and Supportive Supervision PMTCT Preventing Mother-Child-HIV transmission SPRING Strengthening Partnerships, Results, and Innovations in Nutrition Globally (USAID) TOT Training of Trainer USAID United States Agency for International Development WHO World Health Organization OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 5 Operational Definition of Key Terms Health Care Worker (HCW): Community-based health care providers who deliver curative and preventive health services to clients in the CHPS zone/community. Community Health Nurses (CHN), Community Health Officers (CHO), and Midwives. Effectiveness of training: refers to (1) the ability of health care workers to answer correctly questions related to training content and (2) health care worker assessment of the relevance of training content from specific modules to the requirements of their jobs. Training: Organized activity aimed at impacting information and/or instructions to improve the recipient's performance or to help him or her attain a required level of knowledge or skill (Business Dictionary, 2018). Training model: HCW training approaches being implemented by GHS with support from USAID/Ghana through their implementing partners. Training models in this study include traditional, training-of-trainer, supportive supervision, mentoring, internship and e-learning. Traditional training model: A residential or non-residential workshop-style training delivered usually over 1-5 days by a technical specialist in specific topics related to the functions of the target CHPS zone personnel Training-of-Trainers model: Training of trainers who are expected to carry out follow-up training for the target CHPS zone personnel Supportive Supervision training model: On-site supervision from an experienced health care worker or supervisor to provide feedback and correction on skills/protocols application. Supportive supervision is also known as outreach training and supportive supervision (OTSS) for malaria-related training. Mentoring training model: Clustering of HCWs who meet as a group with a licensed midwife for advice and instruction on Ending Preventable Child and Maternal Death (EPCMD). Internship training model: One (1) week assignments of HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff/mentor. e-Learning training model: training accessed on-line for HCW instruction in nursing topics. It may include post-tests to validate knowledge acquisition. Trainee: A health care worker who has received training under any of the six training models. Scalability: refers to the ability to scale-up a training model nationwide across Ghana Health Services. Sustainability: refers to the ability to maintain the effectiveness of the scaled-up training model OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 6 EXECUTIVE SUMMARY Introduction The United States Agency for International Development (USAID)/Ghana has made large investments in the training of health care workers (HCW) to improve their performance, with the goal of ultimately improving health outcomes. USAID/Ghana’s investment has supported a variety of models of training (e.g., traditional workshop training, on-site supportive supervision, mentoring, e-learning, internships and training of trainers) through multiple implementing partners (IPs). USAID/Ghana commissioned this study to explore the effectiveness of the various models of training funded by USAID/Ghana’s Health, Population and Nutrition Office (HPNO) in terms of their contribution to improving HCW skills and knowledge. This report presents findings and recommendations from an evaluation of optimal models of HCW training on topics related to Ending Preventable Child and Maternal Deaths (EPCMD) and malaria in Ghana. The USAID/Ghana Evaluate for Health (Evaluate) project carried out the study from October 2017 to June 2018. The evaluation’s specific objectives were to: (1) assess the degree of effectiveness of each of six training models in terms of knowledge and skills acquisition, retention and application on the job; (2) identify any external factors that influence the effectiveness of training; and (3) recommend training model(s) that appear to offer the best prospects for national-level scaling-up and sustainability. This executive summary presents the evaluation purpose, research questions, research design and methods and key findings, conclusions and recommendations from this study. Evaluation Purpose and Research Questions The purpose of this study was to evaluate the effectiveness of different USAID-funded HCW training intended for Community Health Officers (CHO) and midwives working at the Community-based Health Planning and Services (CHPS) zone level to determine optimal models that could be scaled and sustained on a national level. To achieve the study’s objectives, this evaluation sought to answer the following questions: 1. What models of training focusing on Ending Preventable Child and Maternal Deaths (EPCMD) and malaria appear to most effectively contribute to increases in HCW knowledge and skills? a. How effective are these trainings directly after training versus long-term? 2. Are the trainings conducted for HCW relevant to their needs and being utilized for improved health care delivery? a. To what extent are the knowledge and skills received at trainings being shared with other health staff at the CHPS? 3. What models of training appear to be the most scalable and sustainable across the Ghana Health Service (GHS)? a. Have the training-of-trainer (TOT) activities been effective in terms of providing long-term support to GHS? b. What are the general cost parameters of each model? 4. What other training successes have been documented in Ghana or other relevant countries? Evaluation Design and Methods The priority areas of training evaluated were EPCMD and malaria treatment. The study specifically examined traditional, training-of-trainer (TOT), supportive supervision, internship, mentoring and e-learning training models implemented with support from four USAID/Ghana implementing partners (IPs): Systems for Health, Strengthening Partnerships, Results and Innovations in Nutrition Globally (SPRING), MalariaCare and Maternal Child Survival Program (MCSP). OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 7 Types of Training Topic area 1 Target Participant Brief Description 1 Traditional training IYCF, IMNCI CHO 5 days - off-site training 2 Training of trainer (TOT) ENC, Anemia Master trainers Training GHS trainers to deliver training 3 Supportive supervision IYCF & Malaria (OTSS) CHPS staff Supervisors provided onsite support 4 Internship Malaria; malnutrition and anemia. CHO Bring CHOs to district hospitals for one week 5 Mentoring (Midwife-CHO) FP and MNCH CHO Mentoring – CHO go to health centers 6 E-learning PMTCT, IPC, breastfeeding Students Open source online The evaluation team used a mixed-methods design, collecting both quantitative and qualitative data to determine the most effective type of training supported by USAID/Ghana in the last two years. The team conducted in-depth interviews, using structured questionnaires to collect quantitative data (i.e., knowledge and environmental factor assessments) and protocols for the key informant interviews (KIIs) and focus group discussion (FGD). The team completed 109 KII and one FGD, 89 of which were with training participants and 21 of which were with public health nurses, district directors of health and e-learning facilitators. The team purposively sampled regions and districts in Ghana to include three USAID/Ghana focal regions (Northern, Volta and Greater Accra) and one non-focal region (Ashanti) in which USAID IPs have supported HCW training. The team used simple random sampling to select study participants from each of the sampled districts from lists of HCW trainees from the six training models under evaluation. Limitations Contrary to expectations, IPs were unable to provide the evaluation team with training participant pre- and post-test scores for their trainings. IPs either did not collect such data at all or could not locate the data. In addition, as the evaluation’s assessment of HCW knowledge took place from 12 to 24 months after training implementation, participants may have benefited from other types of training on the same topic before the study period. As such, training results cannot be attributed exclusively to the USAID-funded trainings. Although the evaluation team assessed knowledge acquisition based on tests that covered the key points of training, the team did not directly observe HCW performance on the job to assess skills application. The evaluation’s conclusions regarding training effectiveness in terms of knowledge acquisition are based on the HCW knowledge scores, self-evaluation and IP quarterly reports. The evaluation measured knowledge for all models except e-learning. The team excluded the e-learning module because the training was designed for students from community health worker training schools who had yet to be posted to a CHPS zone and, as such, had no on the job experience. However, the evaluation team collected qualitative data to provide insight into participants’ view of the relative value of the e-learning model as a training tool. The evaluation does report the findings and conclusions for the e-learning model in terms of training participants and trainers’ perspectives on the model. Key Findings and Conclusions Demographics: Sixty percent of the participants interviewed were between 26 and 35 years of age; 56 percent were female. About 50 percent of the HCWs interviewed were nurses and 15 percent were midwives. The rest were clinicians, public health/health promotion officers and nutrition officers. Thirty-four percent of respondents sampled were from the Northern Region, where USAID has made large investments in HCW training over the years. The largest group of interviewees participated in the TOT model (30 percent); the smallest group of interviewees participated in e-learning (2.3 percent). In addition, 59 percent of participants reported receiving training two to four times per year, while 28 percent reported receiving training once a year and 9 percent reported less than once per year. The key findings and conclusions of this study, organized by research question, follow. 1 Acronyms: IYCF – Infant and Young Child Feeding; IMNCI – Integrated Management of Neonatal and Child Illnesses ENC – Essential Newborn Care; OTSS – Outreach Training and Supportive Supervision (Malaria); FP – Family Planning; MNCH – Maternal, Neonatal and Child Health; PMTCT – Preventing Maternal-to-Child Transmission; IPC –Infection Prevention and Control. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 8 1. Models of training on EPCMD and malaria that appear to most effectively contribute to increased HCW knowledge and skills All training participants confirmed that the training they received served to increase their knowledge and skills, notably in terms of improved work output, increased professionalism and higher self-confidence. Participants cited practical demonstrations and interaction between the training participants and facilitators as the most positive attributes of effective training. Interns said their internship improved their knowledge and skills because they had the opportunity to work with experienced trainers. All OTSS participants indicated positive results from the training because it was very hands-on, carried out on-the-job and allowed for corrections to practices on the spot. Respondents’ mean knowledge scores were highest for the traditional (M=9.56 of 10) and internship (M=9 of 10) training models. Master trainers participating in TOT had the lowest score (M=8 of 10). Even though 74 percent of sampled trainees completed their training from one to two years earlier, the average knowledge score across all trainees was 8.9 out of 10, confirming strong recall. It should be noted, however, that respondents may have received additional trainings on the same topic after the USAID-supported trainings. Participants generally perceived the training programs to be too short, with too much information covered in the time allotted. Almost all participants still had their training materials, which they use for reference and share with their colleagues. Participants gave positive quality ratings (4 of 5 or above) to all models. Supportive supervision training model had the highest mean rating of 4.34. The lowest-ranked training type, the internship training model, was only slightly below that, with a mean of 4.05 of 5, still a high rating. Participants noted that the internship, mentoring and supportive supervision models had the added advantage of facilitator-supervised practice and observation of skills after they learned the theoretical content. While internship trainees were trained in higher level facilities that had better equipment and supplies to practice acquired skills than what is generally available in CHPS zones, supportive supervision trainees practiced under supervision in their own working environment and everyday clinical situations, which facilitated the application of their knowledge and skills in their CHPS zone. Participants considered traditional and internship training models as more effective for transfer of knowledge on new topics to HCWs because they provide a structured learning approach to introducing new information and skills. In response to Question 1, the study concluded that although each of the training models evaluated contributed effectively to increased HCW knowledge and skills, the best training models for transfer of new knowledge are the internship and traditional training models. The supportive supervision/OTSS training model is most useful for skills acquisition within the specific context of the trainee’s work environment. 2. Relevance and utilization of training for health care service delivery and knowledge sharing Participants reported that training topics were relevant to their needs in health care service delivery and that they do use the knowledge and skills they gained from training on the job. Participants provided the following specific examples of improved skills applications on the job: correct diagnosis of infant ear infections, testing and treatment of malaria and reductions in stillbirths and maternal deaths in their facilities. Participants used a scale (1 to 5: poor to excellent) to rate the relevance of training topics from the USAID￾supported training models. Participants rated all topics covered by the six models very relevant, based on the mean rating of each topic, with mentoring in family planning (FP) and Maternal, Neonatal and Child Health (MNCH) recording the highest average rating (4.58). Almost all participants reported that they shared the knowledge acquired from their training with colleagues after they returned to work through unit meetings, in-service training, on-the-job training and workshops. Even though the GHS training protocol requires trainees to report what they learned at training sessions to their colleagues, trainees interviewed stated they were more likely to do so when encouraged by their trainers and facility heads. In situations where other staff had already received similar training, trainees highlighted updates related to the topics. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 9 In response to Question 2, based on participant feedback, the study concluded that USAID-supported trainings are relevant to HCW work because topics align with tasks carried out daily on the job. Participants rated the mentorship on FP and MNCH most highly. Almost all trainees confirmed they shared their knowledge from training with other staff after attending trainings. 3. Most Scalable and Sustainable Training across the Ghana Health Service Although each of the six training models had certain characteristics that could contribute to scalability and sustainability across the GHS, HCW trainees, trainers and stakeholders felt that supportive supervision appeared to be the most scalable and sustainable for knowledge and skill acquisition, retention and application. Reasons cited included the fact that the model ensures that the desired target audience receives training within the specific context of their environment, that HCW activities are appraised in real life, that tailored solutions to unique challenges can be devised on site, and that supervisors provide regular follow-up visits. This assertion on scalability was also due to: (1) cost savings in per diem allowances and travel and facility/venue expenses for trainees incurred under other models; and (2) the fact that critically needed HCWs are not required to leave their health stations for days or weeks to receive training in other facilities or communities. However, participants also noted that follow-up supervisory visits must be regular and that supplies and training materials like rapid diagnostic testing (RDT) kits must be available for use during demonstrations. Stakeholders noted that the cost of the supervisor’s allowance and appropriately comfortable transportation must be appraised properly. The benefits of supportive supervision, when done correctly, should justify the associated cost. The TOT model appeared to be generally effective in delivering knowledge and skill to HCWs especially when trainers at higher GHS levels (e.g., district level) deliver training to HCWs in the subdistrict health centers and CHPS zones. Respondents also shared that although master trainers recruited for the TOT were ready to learn, the duration of training appeared inadequate for TOT trainers to fully acquire the knowledge and skills required to deliver training. Further, persistent downstream TOT roll-out challenges related to trainee accommodation, meals, transportation to and from the training venue and low facilitator allowances limit its effectiveness and its longer-term sustainability. In response to Question 3, the study concluded that the supportive supervision model appeared to be the most scalable and sustainable of the six evaluated because trainees receive knowledge and skills in their practice environment, receive immediate feedback and have access to follow-up supervision. TOT appeared to be the least sustainable, given challenges related to post-TOT training rollout. 4. Training Successes Documented in Ghana or Other Relevant Countries The literature review of training successes in Africa and Ghana indicated that strategies that contribute to increased effectiveness of community-based health care worker training include targeted training, use of appropriate teaching aids, equipment and supplies, as well as delivery of low-dose training at frequent intervals. Interactive learning methods that incorporate clinical simulations, case-based learning, practice and feedback also proved to be relatively more effective in achieving HCW learning outcomes in Ghana and Africa. In particular, the Low-Dose, High-Frequency (LDHF) training by JHPIEGO has been documented as a HCW training success in Ghana (JHPIEGO, 2013). It was associated with increased competence of HCWs and a reduction in maternal and child mortality in the health facilities where this training took place. LDHF is based on short, targeted, in-service simulation-based learning activities, which are spaced over time and reinforced with structured, ongoing practice sessions on the job. Overall, the training appears to be a cost￾effective model for HCW training (Asiedu et al., 2017). Maternal, Newborn, and Child Survival (MNCS) in-service training for frontline health workers in South Sudan is another example of effective HCW training (Nelson et al., 2012). This training incorporates targeted training, pictorial checklists and reusable equipment and commodities into their TOT program, which contributed to significant increases in the knowledge, skills and referral competencies of trainees directly after training. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 10 In response to Question 4, the study’s literature search identified JHPIEGO’s Low-Dose, High-Frequency (LDHF) learning approach as a successful model for HCW training in Ghana. This finding aligns with this study’s conclusions that supportive supervision is the most effective model for HCW skills acquisition and the most scalable and sustainable of the six models evaluated. 5. External Factors that Influence the Effectiveness of Training At USAID’s request, in addition to the four research questions, the study examined external factors that influence HCW ability to provide quality health care. The study asked questions about environmental factors such as community acceptance, safety, good roads and availability of reliable electricity, water, internet, medicines and supplies, as well as organizational factors such as incentives, protocols and independence of decision-making in delivering health care. HCW identified community acceptance as one of the most positive environmental factors related to the ability to deliver quality care. Seventy-seven percent of the CHOs/CHNs interviewed strongly agreed that they were accepted in their communities and 62 percent strongly agreed that community members sought health services from them. Eighty-nine percent of the respondents agreed that community members and leaders provided support in organizing health programs. However, 91 percent of HCWs agreed that stigma around controversial health topics (e.g., family planning) limited their ability to apply knowledge and skills acquired from training. On the organizational side, 88 percent of respondents said they receive the necessary support and supervision from their managers through management visits, feedback on their performance and suggestions on improving their work. However, 41 percent of respondents said management does not always resolve their challenges, particularly related to transportation. Forty-four percent of respondents also said they lacked adequate staff to perform their duties effectively, particularly midwives and CHNs. The lack of adequate HCW staffing at CHPS influenced access to learning, since HCWs who are alone in their facility are not allowed to leave for training or they must leave midway in their training to care for clients, especially when the training takes place within their district. Other factors cited that impede quality of care include availability of medicines and vaccines (44 percent); equipment and supplies (55 percent), especially RDT kits; unreliable electricity (66 percent); regular access to potable water (51 percent); and lack of phone service (52 percent). Of special concern were poor roads (69 percent), which participants cited as a major handicap to clients’ ability to access HCW services and to provide regular supportive supervisory visits and trainers’ follow-up visits. Key factors negatively affecting HCW training and health care provision are poor transportation and lack of adequate staffing, medicines, vaccines, equipment, supplies, electricity and potable water. Key positive factors are community acceptance and support from community members and leaders in organizing health programs. Recommendations Based on the study findings and conclusions, the evaluation team recommends the following: A. Training Models Which Effectively Contribute to Increased HCW Knowledge and Skill 1. USAID/Ghana and GHS should strategically select training models based on the objectives of planned training: a. Supportive Supervision/OTSS should be used to deliver tailored on-the-job knowledge and skills training to provide practice and feedback in trainee’s working environment. b. Internship and traditional training models should be employed in introducing new topics and delivering theoretical content. c. Mentoring and internship models should be used to deliver specific skills development through one-on-one engagement. The duration of training should be more than a week to ensure that trainees benefit fully from their mentors. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 11 2. Whenever possible, USAID/Ghana and GHS should incorporate follow-up supportive supervision of trainees after all trainings to ensure that the procedures taught are being correctly followed by trainees. B. Effectiveness of training for quality health service provision 3. The duration of traditional, TOT and mentoring training should allow for enough time to deliver the training content without overloading trainees with too much information. Trainings should be broken down into smaller focused training sessions or extended to allow enough time for new topics and skills to be introduced and taught. This is especially important for TOT because master trainers must fully absorb the content in order to train other HCWs. 4. The duration of internship training should be extended beyond a week (possibly to one month or more) to ensure that trainees benefit fully from their preceptors or mentors. This prevents trainee information overload and increases the effectiveness of training. 5. Key training reference materials like presentation slides and reference articles should be made available to trainees so that they can follow lessons during training, refer to them after training and share with colleagues in their health facilities. 6. Implementing partners and trainers should continue to select training topics and content in consultation with the GHS. Even when content is adapted from global organizations (e.g., UNICEF) they should be reviewed by GHS before delivery to HCW trainees. This measure ensures that training delivered remains relevant to Ghanaian HCW practice and application in service delivery. 7. District-level master trainers should be used for CHPS-level HCW training delivery. Participants suggested that district level master trainers appeared to have a better understanding of the CHPS environment and therefore better suited to deliver trainings. 8. Allowances offered by IPs to GHS facilitators should be uniform across all IPs to minimize situations where facilitators are biased toward activities/tasks of IPs who pay higher allowances. C. Extent to which knowledge/skills received at training were shared with other CHPS staff 9. Trainers and superiors should continue to encourage trainees to share the knowledge and skills they acquire with colleagues in their health facilities. Even though the decision to share lessons learned is standard GHS practice, facilitators’ follow-up and encouragement will motivate trainees to do so. 10. Supervisors should nominate only appropriate and interested cadre of HCWs for IPs for trainings to ensure that the training is applied and that trainees propagate the information they learn to other HCWs when they return to work. D. Most scalable and sustainable training model 11. Given the strong response from trainees and trainers on the relevance and effectiveness of SS/OTSS, USAID/Ghana and GHS should consider scaling up the model, including exploring ways to institutionalize SS/OTSS through budget allocations for supervisors whose full-time function is to rotate across facilities, rather than being pulled from their jobs. F. External factors that influence effectiveness of training and quality of care 12. Key external factors that influenced HCW training and work included the availability of medicines, vaccines and supplies; management support; good roads and reliable utilities (electricity, water, cell phone.) Trainers should appraise these factors in advance of training to determine their impact and to maximize training effectiveness and application in service delivery. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 12 13. Because of the influence of environmental factors on effective training and its application, USAID and GHS should consider requiring IPs to add a community engagement component to their HCW training. G. Training documentation and future considerations for research on HCW training 14. USAID/Ghana should require and verify that all IPs utilize pre- and post-tests and maintain accurate training records, including participant lists and contact information, training curricula, pre and post-test scores and dates of follow-up SS/OTSS visits. This will ensure that future evaluations have access to baseline data and to lists from which training participant samples can be easily drawn. 15. Given the very positive feedback from participants and trainers on the SS/OTSS model, USAID/Ghana and GHS should consider an assessment of outcomes from this training in districts where the practice has been regularly sustained, along with a review of costs and organizational options for providing supervisory support without pulling critical staff from health facilities. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 13 EVALUATION PURPOSE AND QUESTIONS Introduction The United States Agency for International Development (USAID)/Ghana has made large investments in the training of health care workers (HCWs) to improve their performance, with the goal of ultimately improving health outcomes. Health care workers, including community health nurses (CHNs), community health officers (CHOs) and midwives, are the cadre of staff in Ghana who contribute immensely to the Community-Based Health Planning and Services (CHPS) initiative implemented by the Ghana Health Service (GHS). CHPS aims at delivering primary health care at the community level. To strengthen the community￾based health program, USAID/Ghana, working with the GHS, has funded various training models to deliver both pre- and in-service training through multiple implementing partners (IPs). However, little evidence exists to suggest which training models are optimal to improve health outcomes. By evaluating the effectiveness of the various HCW training models used by USAID/Ghana health IPs (Systems for Health/Systems; Strengthening Partnerships, Results and Innovations in Nutrition Globally/SPRING; Maternal Child Survival Program/MCSP; and MalariaCare), optimal training model(s) can be identified, scaled up across GHS and sustained nationwide. This study focused on six models of training in terms of their contribution to the acquisition of knowledge and the adoption of improved skills among HCWs. This report presents the findings, conclusions and recommendations from the evaluation of optimal models of community health worker training in Ghana, undertaken from October 2017 to July 2018 by USAID’s Evaluate for Health (Evaluate) project. The evaluation’s specific objectives were to: (1) assess the degree of effectiveness of each training model in terms of knowledge and skills acquisition, retention and application on the job; (2) identify any external factors that influence the effectiveness of the training model; and (3) recommend training model(s) that appear to offer the best prospects for national-level scaling-up and sustainability. This section describes the evaluation purpose, research questions, study background and overview of USAID/Ghana training, followed by a description of the research methods and approach and evaluation findings, conclusions and recommendations. Evaluation Purpose The purpose of this study was to evaluate the effectiveness of USAID-funded health worker training models targeting CHOs and midwives (referred to collectively as HCW) to determine optimal models that could be scaled and sustained on a national level for CHPS zones for capacity building in Ghana. This study examines capacity in terms of knowledge and skills acquisition, retention and application on the job. An additional part of the study was to identify any external factors that influence the ability of HCWs to effectively practice the skills they obtain through these models of trainings and to recommend model(s) that appear to offer the best prospects for scaling-up and sustainability. The findings of this study are expected to inform USAID/Ghana HPNO, Ministry of Health (MOH), GHS and HCW trainers on the strengths and weaknesses of the training models. Additionally, the appraisal of the environmental factors that influence HCW acquisition, retention and application of required competencies will guide health system managers and policymakers regarding the critical external factors needing attention to maximize outcomes of HCW training. Evaluation Questions The research design included the following research questions that were answered through discussions with key stakeholders (e.g., USAID HPNO and its primary implementing partners) and training participants. 1. What models of training focusing on Ending Preventable Child and Maternal Deaths (EPCMD) and malaria appear to most effectively contribute to increases in HCW knowledge and skills? a. How effective are these trainings directly after training versus long-term? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 14 2. Are the trainings conducted for HCW relevant to their needs and being utilized for improved health care delivery? a. To what extent are the knowledge and skills received at trainings being shared with other health staff at the CHPS? 3. What models of training appear to be the most scalable and sustainable across the Ghana Health Service (GHS)? a. Have the training-of-trainer (TOT) activities been effective in terms of providing long-term support to GHS? b. What are the general cost parameters of each model? 4. What other training successes have been documented in Ghana or other relevant countries? EVALUATION BACKGROUND Ghana’s Health System USAID/Ghana and its IPs collaborate with the MOH and GHS to implement initiatives related to health care services. The MOH is the highest health administrative agency in Ghana and provides health-related policy direction; allocates resources; and regulates, monitors and evaluates the performance of health agencies. The GHS is an executive agency under the control of the MOH that is responsible for the management of health services delivered at the regional, district, subdistrict and community levels. To ensure that HCWs acquire and retain the relevant competencies to deliver standardized health services, the GHS provides in-service training and continuing education to its CHPS health workers. USAID/Ghana IPs provide support to GHS to deliver trainings to HCWs who deliver essential curative and preventive health care services to community members in CHPS zones. A CHPS zone is a demarcated geographical area (4 km radius) with about 4,500 to 5,000 individuals or 750 households residing in densely populated areas. Not all CHPS zones have CHPS compounds, meaning a building dedicated to health service provision. In these instances, health services are provided outdoors at designated communal locations within the community. Health services rendered at the CHPS zones include treatment of minor ailments such as fever control and first aid for cuts, burns and domestic accidents. CHPS zones also offer maternal, child and reproductive health services, with an emphasis on family planning and antenatal care, as well as health education and counselling. CHPS zones refer complicated cases to health centers at the subdistrict level for further management. HCWs in the CHPS zones typically include CHOs, community health nurses (CHNs), community health volunteers (CHVs) and sometimes midwives. The CHO is usually the primary service provider in each CHPS zone who provides reproductive, maternal and child health services, including family planning and childhood immunizations; manages diarrhea; and treats malaria, acute respiratory infections and childhood illness (World Vision International & Ghana MOH/GHS, 2015). CHOs may not perform deliveries and are expected to refer all delivery cases to the subdistrict or higher levels of care. Deliveries also can be referred to competent registered midwives operating in accredited facilities within the community. However, when required, midwives are also assigned to CHPS zones. CHVs, whose roles involve educating the community on basic health issues and serving as agents of referral services and community social mobilization, primarily through home visits, also support the CHO. Community participation, empowerment, ownership and volunteerism are key principles in the CHPS strategy. Training Models and Approaches HCW training is provided through a variety of models in Ghana. Those funded by USAID/Ghana can be categorized into traditional training, training of trainers (TOT), internship, mentoring, supportive supervision and e-learning models (Table 1). The traditional training approach is didactic classroom-based training where the instructor relays theoretical facts to the learner. It is a residential or non-residential workshop-style training usually taking place over OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 15 three to five days with instruction provided by a technical specialist in specific topics. These trainings are typically off-site, and participants do not get the benefit of practicing in real-life clinical situations. The training-of-trainers (TOT) approach involves traditional and practical training of master trainers who, in turn, deliver training to other trainers or to HCWs or volunteers. For instance, regional GHS HCW managers who are master trainers receive a five-day TOT and then they train district-level TOT teams. These district￾level trainers are expected to carry out follow-up training for target CHPS zone personnel. The TOT approach ensures that essential training trickles down through the hierarchy of GHS, from national- to community-level HCWs. The internship training model involves a one-week assignment of CHPS HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff. The mentoring training model involves clustering of CHOs/CHNs who meet as a group with a licensed midwife for advice and instruction on ending preventable child and maternal deaths (EPCMD). Each cluster comprises one midwife mentor and five to eight mentee CHO/CHNs. Mentees have their mentors’ contact numbers and can reach them by phone if they need some form of coaching in a clinical situation. Supportive supervision is when an experienced HCW or supervisor observes a CHPS HCW and provides feedback and correction on skills or application of protocols in real time. Supportive supervision offers the supervisor and the supervisee the opportunity to work as a team with the aim of delivering quality care. This approach departs from traditional supervisory functions, which aim to audit supervisees to identify faulty practices. Supportive supervision is also known as “on-the-job training and supportive supervision” (OTSS) under programs implemented by MalariaCare. E-learning is interactive training that HCWs access online. The e-learning training modules that this evaluation assessed were developed for use on web and Android-based platforms and provide content designed to give HCW students clinical decision-making practice and prepare them for client interaction through simulation and skills practice. These modules include narrated presentations that take approximately an hour to complete. The content is case-based and typically includes post-tests to validate knowledge acquisition. The web-based e-learning modules are installed on the computers of CHN training colleges and students use the programs to complement lectures they receive in class. TABLE 1. TYPES AND TOPICS OF RELEVANT IP TRAININGS FOR STUDY Types of Training Topic area Target Participant Brief Description IP 1 Traditional training IYCF, IMNCI Health staff 5 days - off-site training SPRING Systems 2 Train the trainer (TOT) ENC, Anemia Master trainers Training GHS trainers to deliver training Systems SPRING 3 Supportive supervision IYCF, & Malaria (OTSS) Health staff Supervisors provided onsite support SPRING MalariaCare 4 Internship Malaria; malnutrition and anemia. CHO Bring CHPS to district hospitals for 1 week Systems MalariaCare 5 Mentoring (Midwife-CHO) FP and MNCH CHO Mentoring – CHO go to Health Centers Systems 6 E-learning PMTCT, IPC, exclusive breastfeeding. Students - midwives, CHN Open source on-line modules MCSP Implementing Partners The major IPs that delivered the six models evaluated by this study are (1) Systems for Health (Systems); (2) Strengthening Partnerships, Results and Innovations in Nutrition Globally (SPRING); (3) the Maternal and Child Survival Program (MCSP); and (4) MalariaCare. Systems for Health (Systems) (2014 – 2019), led by University Research Corporation, LLC, aims to strengthen the vital health system building blocks (e.g., management and leadership; health information systems; health workforce; supply chain and infrastructure; and health financing) to ensure that all Ghanaians have access OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 16 to, and use, high-quality health services. Systems’ interventions promote community-based health planning and services, strategic behavior change communication and targeted demand generation to maximize service delivery coverage. To achieve program goals, GHS staff received training under modules that included Integrated Management of Neonatal and Childhood Illnesses (IMNCI) Training, Essential Newborn Care (ENC) Training of Trainers (TOT), CHO Internship on Malaria and Mentoring (midwife-CHO cluster). Strengthening Partnership, Results and Innovation in Nutrition Globally (SPRING) (2013 – 2017), led by John Snow, Inc., aimed to scale up high-impact nutrition in Ghana. Programs the SPRING project offered included anemia reduction; infant and young child nutrition (IYCN); water, sanitation and hygiene (WASH); and aflatoxin reduction. SPRING also provided cash transfers in close collaboration with the Ghana Livelihood Empowerment Against Poverty (LEAP) Project, which offered health insurance and cash to trainees. Training that SPRING offered focused on TOT programs on the topics of infant and young child feeding (IYCF) and anemia reduction. Maternal Child Support Program (MCSP) (2014 – 2018), led by JHPIEGO, aims to contribute to the improvement of health outcomes for human immunodeficiency virus (HIV), malaria, nutrition, family planning and maternal, newborn and child health services in Ghana. Strategic objectives are to ensure that a better￾prepared midwifery and nursing workforce has the knowledge and skills to effectively provide services in the aforementioned areas. MCSP also promotes the development of monitoring systems, standardization of tools, guidelines and training materials for HCWs at the pre-service and CHPS levels across Ghana. MCSP developed the e-learning training modules to support pre- and in-service training of HCWs. MalariaCare (2013 – 2017) was a PATH initiative to support the U.S. President’s Malaria Initiative (PMI) to reduce malaria mortality and morbidity. It aimed to raise both diagnosis and treatment services for malaria and other febrile diseases. MalariaCare collaborated with the Ghana National Malaria Control Program (NMCP) and other partners to implement capacity-strengthening activities in combating malaria. EVALUATION METHODS AND LIMITATIONS Methods Overview The research team used a mixed-methods approach to collect and examine both quantitative and qualitative data to determine the most effective of the six models that USAID/Ghana funded in the last two years. The team conducted in-depth interviews and used a structured questionnaire to collect quantitative data for the knowledge and environmental factor assessments. The team cleaned and validated all data prior to analysis, and systematically analyzed those data to answer the research questions. The team upheld ethical principles through the entire research process and used direct quotations from the participants to complement quantitative findings. The study collected primary data from training participants through interviews, a knowledge assessment tool based on training curriculum and an environmental assessment tool. The research team conducted focus group discussions (FGDs) and key informant interviews (KIIs) with training participants, USAID IP staff, GHS staff and trainers of HCWs. The team used a participatory approach throughout the study process, engaging a steering committee of stakeholders, which enhanced ownership of evaluation results and enabled triangulation of data across a range of stakeholders at different levels. The steering committee included representatives from USAID/Ghana-HPNO, MCSP, Systems and GHS (see Table 5 in Annex A). The research team purposively sampled regions and districts to include the four regions (Northern, Volta, Ashanti and Greater Accra) where USAID’s IPs support HCW trainings. Using simple random sampling, the evaluation team selected study participants who were HCWs from each district based on the list of trainees that the IPs provided. Trained enumerators collected data from trainees and stakeholders from February to April 2018. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 17 Data Sources 1. Secondary Data Sources The evaluation relied on both primary and secondary data to answer the evaluation questions; however secondary data was key in establishing an understanding of a starting point since these trainings had already been delivered. Without a baseline, it was not possible to directly measure knowledge acquired since the participants’ knowledge prior to the training was unknown. This evaluation relied on data collected by the IPs including: • IP reports (MCSP, Systems, MalariaCare); • IP training curricula; • GHS Service Provision Assessments; • GHS Human Resource Division’s lists of models of training and topics; • CHPS National Implementation Guidelines August 2016: Expected Skill Sets for CHOs and Midwives; and • Literature review of secondary sources on evaluations of health worker trainings in lower- and middle-income countries (emphasis on Africa and, as available, Ghana). 2. Primary Data Collection Methods In addition to the secondary data, the evaluation team developed multiple assessment tools and qualitative protocols for participants, trainers and other key stakeholders to explore a range of perceptions (see instruments in Annex E). The team shared all of the tools with USAID/Ghana and the steering committee for feedback and finalized the instruments after piloting them. The research team trained 11 experienced enumerators to collect data from training participants, trainers, GHS personnel, IP staff and community members. During the training, the team pre-tested all instruments in CHPS zones in the Ashaiman District of the Greater Accra Region. Based on the results of the pre-test, the team revised the study instruments, set up the final enumerator teams, finalized the deployment plan and prepared logistics to facilitate fieldwork. On February 5, 2018, the three enumerator teams commenced data collection simultaneously in their designated regions. Data collection took place from February 5 to 28, 2018. Ninety-seven percent (98 of 101) of the interviews were face-to-face. In three instances when interviewees were unavailable for in-person meetings, the enumerators conducted phone interviews with the participant’s consent. For more information of methods and field deployment, see Annex B. 3. Quantitative Data Initially, the evaluation team planned to rely on relevant IPs to obtain pre- and post-test data to serve as a baseline since the trainings had already been conducted. However, once the evaluation was underway, the team learned that not all IPs have records of pre- and post-training test results. Thus, to understand if increases in knowledge occurred, the team had to rely on self-reporting to determine if HCWs had learned or adopted any skills or practices. To help measure current knowledge, the team developed an assessment tool with of 10 questions based on the specific topics from the training curricula. 2 Assessments for each training are in Annex E. In addition to the knowledge assessment, the research team conducted environmental assessments to get a better understanding of the external challenges and barriers that HCW face when delivering health services. To provide a better understanding of these challenges, this assessment asked about the community and physical environment, availability of materials, CHPS infrastructure, management support and supervision. HCWs scored the factors through agreeing or disagreeing (ranging from “strongly agree” to “strongly disagree”) with the impact they had on their ability to deliver quality health services. Respondents also rated the trainings and their topics, audiences and duration on a scale of 1 (poor) to 5 (excellent). The rationale 2 Initially, post-tests from trainings were to serve to assess participants’ knowledge, but not all trainings materials included a post-test and those that did were simplistic and deemed not rigorous enough to assess participants’ knowledge. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 18 for each rating was explained to help identify the context within which the ratings were offered. All instruments are in Annex E. While this study is not designed to focus on cost-effectiveness, the study does provide a comparison of basic unit costs based on common parameters (e.g., length of training, level of effort for trainers, cost of training facility, materials, transportation, per diem, etc.) based on data collected on the training models. IPs self￾reported all budget numbers using a template with general areas of training costs. Unfortunately, not all IPs could provide costs for all areas, so the evaluation team adjusted the comparison to focus on areas where data were available. This analysis provided basic cost data to better understand the value of the training (e.g., outcomes) versus the monetary investment in the training to help inform future program planning. 4. Qualitative Data To provide more in-depth contextual information and to explore factors outside the training that influence use of skills, the evaluation team collected qualitative data using KIIs and a FGD to explore participant, trainer, IP, community and supervisor perceptions on the quality of training and the mitigating factors of expected training outcomes. In addition, the team conducted interviews with GHS to understand their roles and perspectives on the effectiveness of the training and the suitability and intended future efforts for training of HCWs. This was important since GHS delivered all IP-supported trainings. The team analyzed reasons given for respondents’ rating of each training model, training topics, audience and duration to contextualize the ratings. The qualitative data were particularly important in helping to identify possible factors that compromise or facilitate translating training to effective job performance and quality of care. In addition, the team obtained data from the CHNs and midwives’ perceptions on the quality and value of the different models of trainings they had received. Sampling The team purposively selected a total of four regions including three USAID focus regions (Northern, Greater Accra and Volta) and one non-focus region (Ashanti). The team included a non-focus region because the malaria internships were concentrated in Ashanti. The training districts sampled were also purposively selected based on the documents provided to the research team that identified different training models and number of training participants. The team randomly selected individual training participants from the training participant lists submitted by IPs. The team based the sample size for this evaluation on the assumption that 50 percent of HCWs in the regions had undertaken at least one form of in-service training. To ensure a 95 percent confidence interval and a statistical power of 80 percent, the team identified a sample size of 109 health care workers. The evaluation team used simple random sampling to select respondents from each of the districts based on the various training models that USAID/Ghana IPs supported. Table 2 details the regional distribution of participants interviewed. TABLE 2. TOTAL NUMBER OF INTERVIEWS PER REGION Region # Participant interviews Key Informant Interviews TOTAL GREATER ACCRA 17 1 (PHN) = 1 18 NORTHERN 26 5 (PHN) + 4 (DD) + 2 (e-learning facilitators) = 11 37 VOLTA 29 1 (PHN) + 1 (DD) + 1 (FGD)= 3 32 ASHANTI 16 3 (PHN) + 4 (DD) = 7 23 TOTAL 88 22 110 Data Management and Analysis Approach Quantitative Data: The evaluation team manually checked all questionnaires for completeness and logged them for easy access. The team used Statistical Package for Social Sciences (SPSS) version 23 software for independent data verification. Respondents’ socio-demographic characteristics were collated to illustrate respondent and facility profiles. The environmental (community and organizational) factors influencing HCW OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 19 work were collated and the mean score for each environmental factor was presented. Respondents’ knowledge scores on each training model were estimated and mean scores presented to identify the average knowledge level of training participants on the topics they were trained. Respondents’ training model, topic, audience and duration ratings were also collated, and mean scores estimated. Data were analyzed with SPSS using descriptive statistics and the findings presented in tables and figures. All statistically significant calculations were measured at 95 percent confidence interval (p≤0.05). Regarding missing values, analyses used pairwise exclusion of cases. This means that cases used for analyses contained the relevant data required for that specific analysis. As a result, population sizes for individual analyses varied, but to a limited degree. Qualitative Data: Using content analysis, the evaluation team assessed the qualitative data to supplement the statistical analysis across questions. Through a team approach, they coded and analyzed the transcripts. Coding of data took place, followed by manual data management to identify relevant quotations to support findings. The themes generated were presented in order to answer the research questions. This report uses verbatim quotations to provide evidence and context to the findings. In general, since trainings have already occurred, the overall findings have limitations that cannot be directly linked to the training, since no baseline/pre-test was possible. Limitations The team worked to mitigate limitations to this evaluation through the course of the evaluation. First, and contrary to expectations, IPs were unable to provide the evaluation team with training participant pre- and post-test scores for their trainings. IPs either did not collect such data at all or could not locate the data. In addition, as the evaluation’s assessment of HCW knowledge took place from 12 to 24 months after training implementation, participants may have benefited from other types of training on the same topic before the study period. Other similar training programs are conducted by UNICEF and Japan International Cooperation Agency (JICA), which could confound the knowledge gained from USAID-supported programs. As such, training results cannot be attributed exclusively to the USAID-funded trainings. Although the evaluation team assessed knowledge acquisition based on tests that covered the key points of training, the team did not directly observe HCW performance on the job to assess skills application. The evaluation’s conclusions regarding training effectiveness in terms of knowledge acquisition are based on the HCW knowledge scores, self-evaluation and IP quarterly reports. In addition, the study had limited access to cost data for the cost analysis. Because key components of the costs were not available (e.g., curriculum development, administrative support, etc.) only a general cost comparison was possible. Another limitation was the difference between the curriculum and topic areas of the training models being evaluated. Since the focus of this evaluation was training delivery type, an ideal design would have involved standardizing the training content across all models (e.g., exclusive breastfeeding, malaria in pregnancy, etc.). This would have helped ensure that the same information was being delivered to each comparison group and the delivery mode was the only difference. Although the evaluation has been narrowed to EPCMD, when examining relevant trainings across IPs, topic areas did not always align or overlap (e.g., IYCF and ENC), which made it challenging to compare across types of training. However, the study team tried to incorporate methods to mitigate this challenge by comparing general findings and comparing across alignments when possible (e.g., malaria and IYCF). The evaluation measured knowledge for all models except e-learning. The team excluded the e-learning module because the training was designed for students from community health worker training schools who had yet to be posted to a CHPS zone and, as such, had no on the job experience. However, the evaluation team collected qualitative data to provide insight into participants’ view of the relative value of the e-learning model as a training tool. The evaluation does report the findings and conclusions for the e-learning model in terms of training participants and trainers’ qualitative feedback on the model itself. The team asked participants to evaluate the six training models, topics, audience and duration. Even though participants knew what all the training models were, not all of them had undergone all the trainings so they could not compare one training model to the other. This factor might have influenced their evaluation of the training models and their associated features. The evaluation triangulates multiple types of data (e.g., OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 20 interviews, knowledge and environmental assessments) from different sources (e.g., participants, government, supervisors) to have a better understanding of the overall contributions the trainings made toward improving knowledge and skills of the HCWs. Ethical Clearance Ethical clearance for this study was obtained from the GHS ethics review committee. In keeping with institutional review committee regulations, the evaluation team maintained the confidentiality of the respondents’ information at all stages of this evaluation, generating unique identifiers for each respondent to use for recordkeeping. Prior to participating in the study, all respondents signed a written informed consent form. Copies of the consent forms were stored in a secured databank of Evaluate for Health. See Annex B for more details. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Findings 1. Respondents’ Socio-Demographic Characteristics A total of 110 community health workers were involved in the quantitative and qualitative data collection process across four regions of Ghana (Greater Accra, Volta, Northern and Ashanti Regions). Of these, 109 were individual interviews and one was a focus group discussion with mentor midwives of the mentoring cluster (see Table 3). Respondents were sampled from 20 districts across the four regions, including three of USAID/Ghana’s focal regions: Northern, Greater Accra and Volta. MalariaCare’s relevant interventions were mostly implemented outside the focal regions, and based on training participant numbers, the Ashanti Region was also selected to be included in the study. The majority of respondents sampled were from the Northern Region (33.9 percent), where USAID has made large investments in HCWs training over the years. (Table 8 in Annex C details the distribution of respondents from the various districts and regions.) Most of the participants were between the ages of 26-35 years (59.6 percent) (see Figure 1) and more than half (56 percent) of the respondents were female. Nurses comprised 50.5 percent of the HCWs involved in this study and 14.7 percent of respondents were midwives, while the rest were clinicians, public health/health promotion officers and nutrition officers (see Figure 2). The large proportion of female HCW participants could be attributed to the dominance of females in nursing and midwifery in Ghana who provide most of the preventive and curative health services available in the CHPS zone. More than half (66.1 percent) of the participants were married and 30.3 percent were single. The marital status of HCWs can play a major role in their stay at a particular health facility in a community. Married HCWs are more likely to request a transfer to other health facilities or districts for spousal reasons. These staff transfers contribute to situations where trained HCWs leave with the knowledge and skills they have acquired through training. TOT participation included 29.5 percent of the trainees, representing the highest number of trainees sampled. E-learning trainees were the smallest number of trainees sampled (2.3 percent) (see Figure 3). TABLE 3. DISTRIBUTION OF HCWs INTERVIEWED BY TRAINING TYPE Type of Interview # of Interviews Traditional 23 Training of Trainers 26 Supportive Supervision 17 Mentoring (Midwife-CHO Cluster) 10 Internship 10 Mentor Midwives’ Focus Group Discussion 1 E-Learning 2 TOTAL 89 Key Informants Interviewed # of Each Public Health Nurse 10 District Director of Health 9 E-Learning Facilitators 2 TOTAL 21 GRAND TOTAL 110 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 21 In addition, 59 percent of participants (57) said they received training two to four times a year, while 28 percent (27) reported receiving training once a year and 9 percent received training less than once a year. 2. Research Question One (Q1): What models of training focusing on ending preventable child and maternal deaths (EPCMD) appear to effectively contribute to increases in CHOs’ and midwives’ knowledge and skills? Q1 focuses on whether the training models have effectively increased the knowledge and skills of HCWs. Since no baseline data was available for in this study, the evaluation team will base its examination of the training’s effectiveness in increasing knowledge and skills on trainees’ recall of information from the training curriculum and personal reporting about the training models. The direct application of what the participants applied in their work falls under the Q2, which explored relevance and utilization of learnings. Overall Perceived Effectiveness Overall, the participants of all trainings valued the trainings and felt that they increased their knowledge and skills. Participants expressed various reasons they found the trainings to be effective, including that the trainings improved their work output (e.g. skills), professionalism and enhanced individual characteristics such as self-confidence. Almost all of the training participants described the training models as excellent and effective and allowing for practical demonstration, understanding and interaction between the training participants and the facilitators. Each participant supported the training they received and emphasized that it contributed to health care delivery. However, so the evaluators could better understand the specifics about the trainings, respondents explained why and provided examples to illustrate their training experience. Participants of the different training models also reported that the trainings increased their work output. For example, internship participants expressed that the internship training improved their knowledge and skills because they had the opportunity to work with experienced trainers (e.g., heath workers). Some participants from the traditional training reported carrying out extra responsibilities with their new FIGURE 1. DISTRIBUTION OF TRAINEES SAMPLED 29.5% 26.1% 19.3% 11.4% 11.4% 2.3% 0 5 10 15 20 25 30 35 Training of Trainer Traditional Supportive Supervision Mentoring Cluster Internship E-Learning Percentage of Trainees Sampled Training Model FIGURE 3. PARTICIPANT AGE GROUPS 3.7% 26.6% 33% 11.9% 4.6% 8.3% 11.9% 0 5 10 15 20 25 30 35 18-25 26-30 31-35 36-40 41-45 46-50 Above 51 Percentage Participant Age Groups (Years) FIGURE 2. PARTICIPANTS’ OCCUPATION PROFILE Clinicians 10.1% Nursing 50.5% Midwifery 14.7% Nutrition Officers 4.6% Public Health/ Health Promotion Officers 9.2% Others* 11% *Others: Health Administrators Laboratory Staff Pharmacists Student Nurses Tutor OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 22 knowledge and skills. For instance, a male HCW trainee revealed that after the training, he delivered babies in his facility just like a trained midwife. He added that his skills in the administration of vaccines have also improved. Participants of the traditional model said that even though they received training in their various training institutions and universities, those trainings did not always provide enough detail and the trainees did not feel fully equipped for their work until they received in-service training. “I think it has been very effective, in school though they teach you, it is not into details as they go with us in the workshop, so I think the workshop is very helpful.” (TRADITIONAL_GAR_003). “Let’s say I — for instance, lifesaving skills. In school I went through all practicals and delivery, but not as a midwife, just a staff nurse. But because of the training, in fact I do the delivering and people even appreciate me, asking if I am a midwife. The training also taught us how to give immunizations. … if you just give me any vaccines to be given, I wouldn’t even ask you what do it. I can just go ahead and do it, simply because of the training we had. Actually, it’s very excellent.” (TRADITIONAL_NR_002) Knowledge and Skill Gained Participants from four of the training types 3 (TOT, supportive supervision, internship and traditional) answered a 10-question knowledge assessment related to their specific training. Based on the data analysis conducted, looking at the average scores, traditional and internship model trainees obtained higher knowledge scores than trainees from the other training models. (See Figure 4 below and Table 15 in Annex C.) TOT had the lowest average score: 8 of 10. Without a baseline, it is hard to directly attribute these results to increases in knowledge gained from the trainings. However, the scores illustrate that participants can recall the information that was intended to be delivered to them, showing evidence of their general understanding of the specific topic. Almost all of the trainees from each training model expressed that they had aquired some knowledge. For instance, trainees from the traditional model expressed that after the INMCI training, they learned how to manage fevers better, as well as administer Paractamol correctly to children. Additionally, a midwife reported that the traditional training she received prepared her to adjust to community-based work. Prior to the training, she did not have the requisite confidence to work independently at the community level. Another midwife, who was an internship model trainee, expressed that she acquired knowledge and skills on the correct management of malaria for children under age 5 and pregnant women. TOT trainees also said that through the Helping Babies Breathe training, they aquired additional competencies in infection prevention, as well as the proper management of babies to reduce stillbirths and maternal deaths. They said this led to a decrease in birth-related complications. All participants of OTSS indicated positive results from the training. OTSS participants revealed that the infant and young child feeding training equipped HCWs with the competencies required to monitor a child’s growth rate, prevent malnutrition and also educate mothers at the child welfare and antenatal clinics about infant and child feeding practices, including lactation management. Before the OTSS training, the trainees could not plot children’s growth rates accurately to identify malnutrished childen early. The OTSS model 3 Mentoring and e-learning participants did not participate in the knowledge assessment. The evaluation team did not have curriculum for the mentoring, and e-learning participants were nursing students who had not yet received a CHPS placement. FIGURE 4. TRAINEE MEAN KNOWLEDGE SCORES PER TRAINING MODEL 9.56 9 8.78 8 7 7.5 8 8.5 9 9.5 10 Traditional Internship Supportive Supervision Training of Trainer Mean Knowledge Score Obtained Training Model OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 23 was considered the best model compared to the other models of training because it was reported to be more hands-on and on-the-job, allowing for corrections to be made on the spot during the service delivery. “It has been very effective. The training was more interactive, it was not like teacher-student interface. People were open to ask questions. They used demonstration and used training models, which were simplified to read and understand, the practical teaching materials and all that materials, so it was very interactive and friendly.” (OTSS_NR_ 002). Internship participants perceived that internships were more effective compared to other training models because of the blend of theory and practical sessions, which broadened their scope. Therefore, the higher scores in the quantitative knowledge assessment analysis for the internship model could be attributed to the increased practical exposure and theory application within the clinical site simultaneously. “I believe in practical aspect of everything. So if I’ve been taken through the theoretical aspect and I don’t know the practicals, I think in effect it’s not going to work out for me. But if I’ve done the theory and I’m into the practicals as well, I think it is going to help.” (INTERNSHIP_AR_001) Trainees of the mentoring model appeared to have aquired the requisite knowledge and skills to deliver some IMNCI services in the CHPS zones. A mentoring participant mentioned that she had aquired the skills to take medical history from mothers and correctly assess children’s ear infections, anemia and respiratory tract infections. She added that prior to the training, she missed some of the children’s health problems because their mothers did not mention or describe them accurately. The participants of the mentorship model also benefited from post-training monitoring, which they reported as helpful. “In general, I think there should be frequent follow-ups because it happens that after the training, we all go back to relax, … I think when they keep reminding us or there is a process where we are giving them quarterly or feedback about all the activities that we are implementing, I think it will help.” (MENTORING_VR_P22) Training Components Related to Effectiveness Respondents answered questions about the training duration, methods, materials and facilitators that contributed to the effectiveness of the training. Participants generally perceived the duration of the trainings to be too short. Internship participants said the internship should be extended from five days to a full month so they could gain better expertise in the skills they learned. Mentoring participants also considered the duration inadequate, since there was not enough time to cover the content areas adequately. It was recommended that mentorship training should be regular (e.g., semi-annually or annually) to help both service providers and managers be current with trends of service provision requirements. However, in general, because many teaching methods were employed, participants were still able to learn from the trainings. Most participants reported that majority of the facilitators were knowledgeable and skilled. “We are using so many methodologies you know some of the trainings they are brought to the conference room structured classroom work and we train them. Some of them we do on-the-job training, some of them we facilitate supervision we go to monitor and then when they fall short, we teach them.” (KII_DISTRICT PHN_AR) For the traditional model, most participants had the needed resources such as the course materials, teaching aids, human resources, physical facilities including accommodation and a serene environment which facilitated teaching and learning. A few of the participants of the TOT model reported that they did not receive adequate materials and the facilitators rushed through some of the topics during the training. However, some participants stated that due to the usefulness of some of the training materials, they were kept at their work places for easy referencing: “They are adequate, like I mentioned the chart booklet, there are other notes that they’ve given us that are left on the table where we consult, anytime is free you go for the books and you just go through it.” (TRADITIONAL_VR_P08) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 24 “The materials were very good because they are useful in communication. The material contains all the things that they teach them so I will say that they had good communication materials that they used for the training” (MENTORING_VR_P14) One suggestion was that the OTSS training would be more effective if the programs provided vehicles for the monitoring visits to the facilities, since the usual lack of official vehicles creates challenges. Participants also had concerns about the training venue, the accommodation as well as food and water. Participants who had their training at the southern part of the country (Ga East and Ga South districts) complained of poor accommodations and food. Some said they had to sleep in pairs and they were asked to tell colleagues who had scheduled the training not to come because of the lack of accommodations. In addition, some were taken to far-away hotels and had to be transported to the training venue daily, resulting in them being late. Concerns were about poor or inadequate food and a shortage of water. Some participants said their set targets were not met and the funding for facilitators was inadequate. These could have impacted the training negatively. On the other hand, some participants expressed that they had comfortable accommodations and good food, which they said enhanced their learning. “The USAID training have been very comfortable because they make sure you are comfortable so that you can learn well at the training grounds … the accommodation is OK, the food is OK.” (TOT_VR_DHA_P17) E-Learning E-learning training participants indicated that the training was effective because it helped its participants (students) economically by reducing the cost of buying handouts and flash drives. The evaluation of student work was also easier for tutors who used the e-learning platform. The mobile module of the e-learning platform makes information access easier, reduces paperwork and saves time. Participants believed accessing handouts online to reading them was an effective way to learn, so they took the e-learning sessions seriously. Tutors reported that the uploaded content was based on the course outline of their curriculum that was relevant for the students’ work and included questions and answers. Facilitators reportedly did their best to ensure that participants understood what they were learning. Tutors played advocacy roles and helped students understand the e-learning program. Although training participation required enrollment, some individuals ended up participating multiple times in the same training. Tutors organized training for each other to refresh their training experiences and students consulted the tutors for support after the training. Poor internet access was as a challenge of e-learning, as well as access for students without smartphones. Another challenge was large class sizes, which made training and teaching difficult. Participants said the e￾learning training should be extended to other colleges and that tutors should put adequate information onto the e-learning platform to give students access to materials that are relevant for them to read. The e￾learning module should be introduced at the early stages of the curriculum. It was also suggested that tutors should upload questions, activities and quizzes on the platform so students can try them and become more conversant with the process. Respondents said the e-learning would be better if a website hosts the platform so it is accessible across all schools and can be updated regularly. They also suggested an alert system to notify students when new information is posted on the platforms. “The scalability is very, very possible. It can be scaled. Maybe as the starting point is to make sure servers are at wherever we want to scale it to so that they can easily access it. But to make it wide, why don’t we try and go and buy website for it? Maybe the people are afraid other people will be hacking into it. I don’t know. That one is for management. But I think scaling it is highly possible.” (KII_ELEARNING_CHNTC_NR_001) Rating of Training Models Respondents rated the six training models based on their personal experience over the last two years (2016 and 2017) on a scale of 1 to 5 (1 = poor, 2 = marginal, 3 = satisfactory, 4 = good, and 5 = excellent). Even though participants knew what the various training models were, not all had attended all of the trainings. Overall, all of the trainings got positive ratings. Supportive supervision (41.4 percent) and TOT (43.6 percent) had the highest number of “excellent” scores. More than half of respondents rated the traditional, supportive supervision, mentoring, internship and e-learning training models as “good.” (See OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 25 Table 16 in Annex C.) The mean ratings of each training model were compared to identify the preferred training model. Mean scores of each training model were based on respondents’ ratings. The supportive supervision training model ranked the highest with a mean of 4.34 (SD=.614). This suggests that the average respondent preferred supportive supervision over any other training models. The lowest-ranked training type was the internship training model, with a mean of 4.05 (SD=.524) (see Figure 5). FIGURE 5: TRAINING TYPE MEAN RATING Effectiveness of Training Directly Afterward Versus Long-Term In this study, ratings for the effectiveness of training directly after and long-term were derived from participants’ responses as well as their scores from the knowledge assessment questionnaire. The HCW trainings included in this evaluation were conducted between 2014 and 2017 (see Figure 6). As such, effectiveness of the training “directly after” refers to trainees’ use or application of knowledge and skills within four weeks of the completion of training, while “long-term” effectiveness of training spans up to two years after training. Based on the responses from the HCWs, trainings were effective directly after the training because the topics were relevant to the HCWs’ work. As elaborated in the previous section, respondents revealed that they acquired new knowledge and skills that they did not have prior to the training. Additionally, training participants acquired vital experience in clinical areas, which they applied at their CHPS placement. For example, internship participants who went to the maternity unit learned delivery techniques and used this skill at the CHPS compound as necessary. It was also evident that training participants were applying the knowledge and skills they acquired in training years after the training ended, such as knowledge about managing malaria and caring for children. Even though the majority (72 percent) of all trainees sampled had completed their training in 2016 (see Figure 6), their average knowledge and skill score was 8.94 of 10. (See Table 13 in Appendix C.) Despite the possible influence of experience and subsequent training respondents may have participated in after the USAID￾supported trainings, these results suggest that the average training beneficiary could recall the training content accurately two years after their training ended. This suggests that the average beneficiary still had relatively high knowledge and skills levels to apply to their practice. Some participants still had their training materials and said they referred to them as needed at work. 4 SD= 1.241 4.34 4.31 4.25 4.22 4.13 4.05 0 1 2 3 4 5 Supportive Supervision Training of Trainer E-Learning Traditional Mentoring Internship Mean Rating of the Training Model Training Model OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 26 FIGURE 6: YEAR RESPONDENTS PARTICIPATED IN RELEVANT TRAINING The role of follow-up visits by trainers and supportive supervisors could also be highlighted as key in ensuring that trainees continuously apply the competencies they acquired in their CHPS placement years after the initial training ended. Although participants at the CHPS compounds received visits and training, some reported that the supervisory visits were not regular. Staff attrition also had an impact on the effectiveness of the trainings, since some trained staff were reshuffled, granted study leave or transferred, or they exited the service to seek other jobs. The trained HCWs left with the knowledge and skills acquired, depriving the facility/department of their enhanced competencies. 3. Research Question Two (Q2): Are the trainings conducted for HCWs relevant to their needs and being utilized for health care delivery? Q2 examines the relevance and utilization of the training models for health care delivery. Based on responses from all of the participants in all of the training models, the topics covered were directly applicable to the work of the HCWs. During the qualitative interviews, participants shared specific situations where they used the knowledge they gained, and they described the relevance of the trainings, which is discussed further below. Utilization of Skills All of the participants reported that the trainings broadened their scope of thinking, decision-making in terms of assessment, objective data-gathering and identification of patients’ problems and treatment. Participants said their ability to assess patients and treat them or prescribe medications improved after they received trainings. Examples included participants who learned how to help new mothers practice exclusive breastfeeding and who gained expertise in growth monitoring, counseling and checking mid-upper arm circumference (MUAC) among infants and children older than 6 months. “I have learned how to check the MUAC of a baby from 6 months and above; and, I have learned how to check an infant too … checking whether the child is having some palmer pallor will enable me to know whether a child is having anemia.” (TRADITIONAL_VR_P09). The midwife-CHO mentoring cluster participants said that the training built their capacity to probe in detail when mothers/caregivers come to the facility with sick children. The knowledge they gained from the training increased their confidence in caring for children and referring cases to other facilities when they were beyond the trainee’s competency. All participants reported improvement in service delivery, stating that the training was effective and relevant in that they used the knowledge and logistics they acquired during the training sessions to improve referral systems, case diagnoses and proper and prompt treatment documentation at their CHPS zone. Most staff were able to provide care outside their scope of work because of the knowledge acquired from the training. For instance, after the training, some HCW trainees could independently assess/observe children’s ear infections as well as conduct malaria testing using rapid diagnostic kits. This was important because various categories of patients (e.g., children, men, the elderly and pregnant women) visit the CHPS compounds with different ailments and health problems, which the skilled HCW can help them to resolve (Ghana Health Service, 2016). As such, the training that HCWs received prepared them to provide care to these categories of patients. Participants said they had seen improvement in perinatal care and newborn 1% 9.9% 74.3% 14.9% 0 20 40 60 80 2014 2015 2016 2017 Percentage of all Trainees Sampled Year Training Started OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 27 care, resulting in a decrease in stillbirths, maternal deaths and complications and improvement in infection prevention and control measures. Midwives confirmed that the trainings have resulted in participants saving a lot of babies. Most participants said they now do things correctly and are meticulous when rendering care. “The impact is good because when you look at their report now as compared to the time they were not trained, you will realize that the impact is good.” (TOT_GAR_P04) Internship training participants who participated in the malaria management added that they were taught how to use the rapid diagnostic test (RDT) for malaria before prescribing treatment. Before the training, even when patients tested negative for RDT, health care workers treated for malaria, but after the training, they no longer treat for malaria when the test is negative. Instead, they probe for other infections and treat accordingly. Participants added that they learned the “3Ts” method, test, treat and track, for malaria, which is helping their work. Also, the internship on malaria helped improve health education during outreaches, home visits and school health and during durbars at the community. “There was a new staff that had come … she was managing malaria cases and she hadn’t been trained so she was just messing up, so I had to train her again.” (INTERNSHIP_AR_001) According to some participants of the OTSS training model, the HCWs’ training needs assessment, training reinforcement and improved service delivery affirmed the relevance of supportive supervision. Not only were they learning, but it occurred over time, allowing trainers to have periodic check-ins to track progress. “Because when you go you identify the gaps and then coach them and then the next time you go back, you are looking to see rather those gaps are filled before you start.” (OTSS_NR_OTSS 002) Relevance of Training Topics Participants were asked to rate, on a scale of 1 to 5 (poor to excellent), how relevant they felt the topics were to their work. Based on these ratings, no one rated any training as a poor (1) or marginal (2). Overall, the topics participants found most relevant were: mentoring- FP and MNCH; internship- malaria, malnutrition and anemia; and TOT: ENC, anemia and IYCF. The highest satisfactory score was for traditional training: IYCF or IMNCI with 13.9 percent. TOT (ENC, anemia and IYCF- 62.2 percent) and internship (malaria, malnutrition and anemia- 62.5 percent) had the largest percent of excellent ratings from respondents. Figure 7 details the results. FIGURE 7: TRAINING TOPICS’ MEAN RATINGS From the mean rating of each training’s topics, mentoring topics – FP or MNCH (4.58) recorded the highest average rating. Internship training topics malaria, or malnutrition and anemia and TOT - ENC or Anemia followed closely with 4.56 and 4.54 mean ratings respectively. The lowest training topic mean rating recorded was supportive supervision - IYCF or Malaria (4.23). See Table 19 in Annex C for more details. 4.25 4.23 4.54 4.58 4.56 4.33 3 3.5 4 4.5 5 Traditional [Infant and Young Child Feeding, Integrated Management of Neonatal & Child Illnesses] Supportive Supervision [Infant and Young Child Feeding & Malaria ] Training of Trainer [Essential Newborn Care, Anemia] Mentoring [Family Planning and Maternal, Neonatal and Child Health] Internship[Malaria, Malnutrition and Anemia] E-Learning [PMTCT, IPC, Exclusive breastfeeding, Malaria, HIV] Mean Rating of Training Topic Training Model [Topics] OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 28 These findings support the consistent selection of training topics that are relevant to the work of HCWs with all having high ratings over 4. The relevance of the topics led to the direct utilization of the knowledge gained. However, the evidence shows that topics treated under supportive supervision and traditional trainings had the lowest rating. The low rating of the traditional training topics could be attributed to the high number of topics to be covered within a limited duration. This was expressed by participants; they felt there wasn’t enough time to cover all the material they were supposed to learn. Another traditional training model participant rated its training topics as satisfactory (3 out of 5) because she believed that despite their importance, the topics were too many. The low rating of the supportive supervision topics could be because malaria was an endemic condition, so several trainings may have been conducted on malaria topics. A participant revealed that she will rate the topics satisfactory (3 out of 5) because the topics were just “okay”. “Because the topics were too many. But I think the topics were all important, so I think the days should be extended.” (TRADITIONAL_GAR _P03) Overall, respondents generally found the training to be relevant to their work because the topics treated were on activities and tasks they carried out daily at their work. Participants of the traditional training model revealed that the training helped them to undertake most of the health services they provided. “The topics were very relevant to our work and will be difficult to say any of the topic treated was not good” (TRADITIONAL_NR_T002) “Yes, five because it has helped me a lot, most of the things I didn’t know, to be honest I didn’t know, and then when I went for the IMNCI workshop I was able to pick up those things” (TRADITIONAL_VR_P08) Similar opinions were expressed by the participants of the Internship training model. They found the training and topics to be based on the cases they faced on the job. Trainees were exposed to clinical situations in various departments of the health care system to enhance their knowledge and skills. “Because we, all the topics we treated were very, very beneficial. They are cases that we face in reality not outside topics.” (INTERNSHIP_AR_001). “Because the topics were very good. The topics were so good that if there had been enough time for us to treat everything, it would have helped. The topics were excellent” (INTERNSHIP_SEKYRE AFRAM PLAINS_AR_003) Participants of the supportive supervision training model found the training topics to be relevant to the HCW’s work because the topic catered for patients of all age groups. “Because I think it was relevant to almost everybody. To pregnant women, children men, and even the aged so it was good and relevant to all the groups” (OTSS_NR_003) “I think all the topics treated were relevant to the work that we were doing or are doing now. When we look at data management, data is everything in Ghana Health Service. So they take us through malaria data management and then the clinical OTSS too. The clinical aspect how malaria is being managed.” (OTSS_AR_001) TOT and Mentoring - IMNCI participants found the training to be relevant because through the topics they acquired new knowledge which they did not have prior to the training. They expressed that they acquired new knowledge which could help them in their line of duty. “I don’t have knowledge IMNCI. I never knew that when you want to attend to under-five who have malaria you have to probe in and ask of ear pains, do test and do other but with the IMNCI I now know that I don’t just listen to what the mother says but I have to probe to hear more maybe the mother may forget some of the things.” (MENTORING_VR_P08). E-learning training facilitators and participants expressed that they and the tutors contributed to the topics themselves for the IP to install onto the e-learning platform hence the topics were relevant to their work. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 29 “The main topics were also good. I will give a scale of 4 because the topics were given by ourselves. They didn’t create topics. Example they will let us give. Example if I want nutrition, environmental health. So it was from audience. And it makes understanding better. It’s not like imposition. We are bringing our own topics, to be used to teach us how the model works.” (KII_ELEARNING_CHNTC_NR_001) Shared Knowledge and Skills Received at Training with other CHPS Staff Almost all participants from all trainings reported that they shared the knowledge acquired from their training with their colleagues after they returned to work. Often participants said they shared the knowledge acquired during staff meetings. “Yes. Every time we have our Monday meeting then I will share with them or after the training, we visit the various facilities then we share with them” (MENTORING_VR_P14) According to the participants, it is a norm in their facilities to report what one learned at training sessions to colleagues who did not have the opportunity to attend. They arrange with the in-charges of the facilities who gives them the opportunity to brief other staff at the facility. Sharing of lessons learned from training by HCW trainees is part of the general practice in GHS and CHPS, and trainees are encouraged by their trainers and facility heads to share what they learn. In situations where other staff had already received similar training, updates related to the topics were highlighted. Others gave the reference/training materials to their colleagues to use as necessary: “At my former facility, anytime you go for training and come back, we have a monthly meeting we organize every month. So after the workshop, you come and first talk to the in-charge, that in charge, “I have come from when, I have come back from the workshop you asked me to go, and this is what we learned”. So at our meeting, I would like you to give me some, time, so that I would share it with my colleagues.” (TRADITIONAL_NR_001) Knowledge acquired from internship training was disseminated by participants through various means which included unit meetings, in-service training, on-the-job training, and workshops. This was similar to what participants of other trainings said. “When we came back, we called a meeting. After every workshop you attend, you have to call a meeting at your facility and brief them on what you went to learn. I taught them how to test malaria using the RDT. l”- (INTERNSHIP_AR_003) A participant mentioned that by sharing the knowledge and skills received from training, other HCWs can carry out the new skills/tasks even in her absence. “It was such that, you know I felt so good after the workshop and then I couldn’t just keep it all alone and then am not the only on at the facility working, I may not be around and they can bring a baby around. If I pass on the knowledge to the next person, when they bring a baby and am not around they are able to see to the baby as required.” (TRADITIONAL_VR_P08) However, a respondent suggested that some of the training participants did not share the knowledge acquired from training with their colleagues. In some situations, wrong or disinterested participants are nominated for training and they come back and are unable to impart whatever they have learned to their colleagues. Additionally, some participants have low confidence level and will not be able to articulate whatever they were taught. “…people …will go for workshop or clinical conference or clinical meeting…a staff will be called to disseminate the information and she is never ready to do that, which is very bad. …It’s like every training, there is some slight changes so every staff who attends a workshop and comes back should be ready to disseminate the information every time because we need it (TOT_GAR_P03). OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 30 “… some people sit there and all they are interested in…some people are very interactive; whatever is going on, they want to participate. But others, they just sit there ‘they said I should come for workshop, so I’m here.’ I go to workshops so I know what I’m talking about. People sit there and they are just there”. (KII_DPHN_AR) Some participants shared the knowledge with their colleagues because they had some post-training follow-up activities they undertake in their facilities, which their colleagues were able to learn from. These follow-up activities are usually training-related tasks carried out to practice the knowledge and skills acquired. “…sometimes there is a follow up activity they would have to do because of that they are able to share with others. As they implement it in their facility their colleagues see the skills they’ve gained and also directly and indirectly learn from them so in a way they are able to share with their other colleagues” (KIIDISTRICTDIRECTOR_VR_P10) The evidence shows that most participants of trainings share their knowledge and skills with other colleagues. This suggests that participants who attend workshops should be interested in the training and understand what is being taught so they can also share the right knowledge and skills with other colleagues. 4. Research Question Three (Q3): What models of training appear to be the most scalable and sustainable across GHS? Q3 examines the models of training (traditional, supportive supervision, mentoring, training of trainer, internship and e-learning) that appear to be most scalable and sustainable across the GHS. Traditional Among respondents, 12.7 percent, including traditional model trainees, expressed that this method was most scalable and sustainable because it is the foundation of all of the other training models. A participant reiterated that the traditional training was good because the trainees are expected by their trainers and supervisors to train others. Those who benefited from the training can also be supervised later to see if they are practicing what was taught; as done in supportive supervision. Another participant suggested that the traditional model was the most scalable because TOT trainees could not remember all of the facts they learned to train other HCWs. Some participants believed the traditional model was most scalable because instead of training individuals, several HCWs are brought together and offered the same training. They believed that this approach has a potential of saving cost when training is scaled up because trainees are gathered at a central point and training is delivered to a large audience instead of individuals. However, saving cost may be at the expense of opportunity for trainees’ hands-on practice. Moreover, the savings made may be offset by the cost of transporting, accommodating and feeding trainees in residential traditional training models. Non-residential tradition methods have been considered to reduce the cost of traditional training however, a participant suggested that trainees get tired and do not concentrate when they go home after each day of training and come the next day to continue training. Thus the residential traditional training model was encouraged. “Yeah, it’s more scalable. The OTSS as I said is complementary, so you can’t do OTSS on all issues when people haven’t had traditional training.” (OTSS_NR_OTSS001) Several participants suggested that the traditional training is mainly for theoretical facts and the practical component was limited. A trainee suggested that explanation of facts alone was not adequate for training thus a scaled-up version should include practical sessions. It seems that the trainee had benefited from the practical session incorporated into the tradition training on IMNCI. “The classroom training and the field work were good because what we learn in class we have to practice it. So we went to the field and we saw how because only explanation cannot do all but the practicals too helped.” (TRADITIONAL_NR_001) Internship It appears that trainees’ exposure to real life clinical situations, availability of equipment and practical sessions are key factors which will influence the scalability and sustainability of the internship training model across the GHS. About 24.1 percent of all participants suggested that the Internship training model was the most OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 31 scalable and sustainable model across the GHS and reported that internship trainees are exposed to various clinical situations which they had never seen. Several participants expressed the view that the internship was more scalable because it incorporates both the traditional (e.g., classroom/theoretical) training and practical sessions therefore trainees got both the information and first-hand experience. However practical sessions required adequate medical supplies, functional equipment and capable facilitators. Some participants suggested that equipment was more available for learning in internship compared to supportive supervision because they were at health centers or hospital. Other participants suggested that the internship was the most scalable model because the trainee has more access to the trainer to acquire the relevant knowledge or skills. For instance, a participant revealed that the internship trainee can be allowed to sit close to a trainer to observe how clients are treated; after which the trainee is given the opportunity to practice and acquire needed skills. “Some let’s say with the CHO internship one that I did for instance, I let her sit close to me. So with the guideline given to me, as I am doing it, I tell her to watch or observe. After, I let her also do it for me to see if she is also learning from what I’ve taught her”. (TRADITIONAL_GAR_P06) Despite these advantages for internship model to be scaled up, a few participants understood the financial implications of operating an effective GHS-wide internship program. Internship trainees are transported from their health facilities to a larger facility and their accommodation for about a week or more was paid. Also, they use hospital supplies received during the training. The practicality of large numbers of HCWs leaving their posts/clients for a week or more to undergo internship training in different communities under a scaled internship model was also highlighted by a participant as a factor to consider when scaling up the internship training model across the GHS. Mentoring 10.1 percent of the participants suggested it was more scalable and sustainable because it is a one-on-one training which offers trainees the opportunity to acquire the relevant knowledge and skills. A mentor participant added that the mentoring training program was more scalable because it was helping HCWs to manage cases which has led to a reduction in the number of referrals to bigger hospitals. In spite of this, the success of a scaled-up mentoring program depends on the availability of mentor midwives who will deliver standardized one-on-one training to their mentees. “That is what I was saying it is helping us a lot …it is even helping the biggest hospitals. It is reducing the load over them because the way we use to refer to them, we are no more referring to them”. (MENTORING_VR_P18) Not all CHPS compounds have resident midwives. In those that do, CHOs typically outnumber the midwives. Under the current Midwife –CHO cluster mentoring training model, one (1) midwife mentors about five (5) to eight (8) CHOs. This mentor-mentee ratio may have contributed to the models’ strength of delivering one-on-one knowledge and skills training to mentees. Consequently, a sustainable scaled-up mentoring program requires more trained midwives to take up the additional task of CHO mentoring. Additionally, in-service training is also required for mentors to ensure that their training is up-to-date with current practice and mentoring standards. These training have higher cost implications when the mentoring training model is scaled up nationally. The external factors which may affect the sustainability of a scaled-up mentoring program include the lack the supplies and equipment. Effective practical skills training will require stocking of training/working supplies and equipment. Even though the same set of training supplies/equipment can be reused by one mentor for several mentees/trainees, procuring them may come at an extra cost. E-Learning About 6.3 percent of participants considered the e-learning model as a scalable and sustainable model because training information is shared instantly and trainers have the ability to assess the knowledge of trainees. The lack of stable internet connectivity and cost of internet services and websites were identified as external factors that will interfere with the scalability of the e-learning training model. Only 18.5 percent of the participants sampled in this study agreed that their CHPS compound had reliable/functional Internet (See Table 10 in Annex C). But almost half (45.6 percent) agreed that their CHPS compound had reliable phone OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 32 signals/reception which was delivered mainly by mobile network service providers, MTN and Vodafone. Downloadable and sharable offline lessons could reduce the negative impact of poor internet supply. The sustainability of a scaled-up e-learning training model will be influenced by the supply of electricity which is necessary to power e-learning devices like desktop computers in air-conditioned information and communication technology (ICT) rooms. However, the negative impact of irregular electricity supply on a scaled-up e-learning training model can be reduced by incorporating portable battery-operated devices like laptops, tablets and mobile phones which do not require constant electricity or air-conditioned environments to function optimally. Portable electronic devices also present an opportunity to increase trainees’ accessibility to training materials/lesson content no matter where they are. Nonetheless the cost of procurement, repair and replacement of electronic devices/parts may influence the sustainability of scaled up e-learning training model. Moreover, the cost of operating the e-learning back-end processes like software platform development, server and data management are factors to be considered. “It is possible but it (e-learning model) requires network. I’ve been saying that throughout our conversation. It is possible. The scalability is very, very possible. It can be scaled. Maybe as the starting point is to make sure servers are at wherever we want to scale it to so that they can easily access it. But to make it wide, why don’t we try and go and buy website for it? Maybe the people are afraid other people will be hacking into it. I don’t know. That one is for management. But I think scaling it is highly possible”. (KII_ELEARNING_CHNTC_NR_001) Training of Trainers (TOT) About 13.9 percent of all participants identified the TOT model as the most scalable and sustainable because of the added advantage of developing master HCWs trainers who can train other HCWs. By this strategy, the original TOT trainers do not have to train all HCWs from the national to the CHPS levels. They train a selected few to train other levels. This strategy appears to save cost. Participants placed emphasis on the importance of a scaled-up pool of district level facilitators who will be able to organize TOT trainings themselves for HCWs at the subdistrict levels. A participant suggested that national level TOTs should only be consulted. The participant expressed the view that HCW trainees should be “in touch” with those supposed to be impacting the skill. Per the hierarchy of GHS, the CHPS level is closer to the district level than the regional and national levels. District level TOTs were more likely to understand the local circumstances of the trainee. The residential TOT model accommodates the trainees hence cost of trainee accommodation could influence the scalability and sustainability of a TOT model across GHS. “We need to have a large pool of facilitators who will be doing those trainings. I think that every district must have facilitators so that they can organize training on themselves. So if they can scale it up and get a lot more of TOTs to train more facilitators I think it will be very good” (TOT_VR_DHA_P17) Supportive Supervision 24.1 percent of the participants suggested that the supportive supervision was more scalable and sustainable because HCWs training and support is offered by supervisors in the trainee’s facility or working environment. Trainees practice the skills they are taught immediately hence knowledge and skills acquired are utilized on the spot. One participant revealed that in some situation even HCWs who are not the original targets of the training gain the opportunity to acquire new knowledge and skills by being able to observe the feedback from the supervisor. This was because OTSS training are delivered on the job and trainees can be seen by other HCWs as they go about practicing the knowledge/skills acquired under supervision. A participant suggested that the trainees do not have to leave their facility for training because they receive the training in their facility. The on-site training can eliminate cost of trainee transportation and accommodation. “…the OTSS is good you go you see what they are doing if there is any challenge you help them to do the right thing and then the”. (KII_DISTRICT PHN_AR) A few participants admitted that the supportive supervision model was expensive. The cost of suitable transportation for supervisors to reach remote health facilities was found to be expensive. Sometimes, in the OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 33 absence of functional facility vehicles or motorbikes, supervisors are forced to use/rent commercial vehicles at higher costs. But it appears to be less expensive when compared with other trainings (e.g., Traditional training) which require trainees to be transported to their centers of training and be accommodated or fed. Supervisors have to be transported from one facility to the other to render the needed coaching and supervision and their allowances must be paid. These costs if not well appraised could negatively influence the sustainability of a scaled-up supportive supervision training model. For the program to be sustainable, some participants recommended that facility-based supportive supervision facilitators should be considered so as to save cost. Thus, the supervisors who will conduct the supportive supervision can also be based within or close to the facility to eliminate travel cost and challenges. “What I want the program to do or to respond in the future, is to train the health facility …each facility should get at least 1 or 2 facilitators. …The facility focal persons the facilitator should be able to address all the challenges pertaining to the OTSS” (OTSS_AR_005) “Funds I think it wasn’t enough, because we had most of our communities in the interior that we need to also visit. And the as I said, we don’t have motorbikes or vehicle so sometimes we have to use commercial vehicles to go. So sometimes it makes it more expensive than what is being given to us”. (OTSS_AR_001) Overall Training Scalability A few participants suggested that all the training models are relevant and that none were more scalable and sustainable than the other. About 10 percent of participants suggested that all the training models are scalable. They expressed the view that the type of training employed was dependent on the area of focus of the training and the knowledge and skills it aims at impacting. A District Director of Health Services suggested that traditional training model is used for the dissemination of concepts and services after which other training model are used to “follow-up” on the trainees. Reinforcing on the same point, a public health nurse suggested that when new topics are being introduced to trainees, trainees need to be comfortable and they need time to read and learn, and therefore traditional training is effective. “That will depend on the type of training and the knowledge and skills that we want to impact. For instance the traditional, if it is dissemination of concept and message on the services that we are providing, traditional one comes to play. And you follow them up with other models.” (KII_DHD_AR) The quantitative data analysis of participants’ training model rating and rationale seems to rank supportive supervision (OTSS) higher than the others (Figure 5). Further analysis suggests that each training model has its strengths and inherent cost implications5, thus based on the focus of training, the models could be used appropriately. Nonetheless, supportive supervision possesses the strengths of the other training models (trainees are provided knowledge and skills training in their practice environment, they also receive immediate feedback and follow-up supervision) and appears to be the most scalable and sustainable training model across GHS. General Financial Costs for Trainings When considering investment in training, cost is a key factor. Although the focus of this evaluation is the effectiveness of the trainings, it is also relevant to consider the investment costs of each training. However, this study was not designed to be a cost-effectiveness study, but rather to provide general cost parameters and identify the different considerations of cost. All costs were self-reported by IPs. Because of differences in reporting and budget tracking, not all IPs could provide information for all areas of the training (e.g., curriculum development and administrative support). The cost analysis is therefore based on available data. 5 Steering Committee members reported challenges related to the cost and availability of GHS staff to deliver SS/OTSS since it is currently not part of any job position, but rather additional work. Members noted that expenses for the supervisor’s allowance and transportation must also be appraised properly to calculate the overall comparative cost. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 34 Since all five trainings used different methods for training, costs were bundled into groups6 to give the team an understanding of the different costs for different trainings. Costs examined include: • Trainer/ facilitators: Includes all costs pertaining to salaries and fringe benefits for trainers to deliver the training. Should include any labor hours for training preparation or post training responsibilities. • Per Diem and travel (Trainers/staff): Includes all per diem provided and any travel costs (Plane tickets, mileage, bus, reimbursements, etc.) for trainers or other staff or personnel. • Per Diem and travel (Trainees): Includes all per diem provided and any travel costs (Plane tickets, mileage, bus, reimbursements, etc.) for training participants. • Facility/Venue: Includes all costs related to renting a venue for the training, as well as meal packages or other rental costs (e.g., projector, microphone, chairs, taxes, etc.) Overall, the main difference between the costs of the models of training is what the money was spent on. Where one training costs more, another training cost less. Trainings ranged from $24 to $100 per participant per day. Traditional was the most expensive and Internship the least (see Table 4). Areas that consumed the large portion of the budget for most training types were per diem for trainees and the cost for a venue. However, for the internship and supportive supervision venue had no cost. Traditional, TOT and cluster trainings had the highest percent of the budget which went into per diem and travel costs for trainees. With the second highest percentage being the facility. For supportive supervision the trainers were the highest cost followed by their per diem. TABLE 4: PERCENTAGE OF BUDGET PER TRAINING Item Traditional TOT Cluster SS Internship Trainer/ facilitators 9% 5% 3% 80% 18% Per Diem and travel (Trainers/staff) 0% 2% 2% 14% 32% Per Diem and travel (Trainees) 46% 50% 57% - 40% Facility/Venue 40% 33% 33% - - * Information was missing for curriculum development and administrative support. Those elements are not included in the analysis. 5. Research Question 3A. Have the training-of-trainer (TOT) activities been effective in terms of providing long-term support to GHS? The TOT method incorporates traditional and practical training of HCWs to deliver similar training to other HCWs. The TOT approach and its activities are aimed at delivering relevant training throughout the various levels of the GHS; from the national to the community level. It is evident that TOT activities in the GHS have played a crucial role in the development of HCW competencies which when sustained could be effective in providing long term support to GHS. For instance, the TOT training on Anemia by SPRING in 2017 reached not only the 37 (26 Males, 11 Females) regional master managers but several HCWs in over 15 districts and subdistricts within the Northern and Upper East Regions. Similar successes in training distribution were observed by Systems in their ENC TOT activities under MNCH. Under the ENC training package, a total of 358 persons were trained within Year 2 (October 2015 to September 2016). They were comprised of 37 master trainers, 142 regional trainers and 179 service providers. In this study, the TOT model and its associated activities were identified by respondents as relevant to the acquisition, use and transfer of knowledge and skills for HCWs. Respondents emphasized that under the training, they had the opportunity to acquire new knowledge and skills like resuscitation of babies. This was largely because training participants witnessed demonstrations and got the opportunity to practice as well. 6 Study team requested cost information on administrative support (e.g. personnel hours for planning, etc.) and curriculum development, however most respondents left it blank. Due to missing data, this category was left out of the analysis. It is not uncommon for curriculum development to be a large portion of a training budget, however once it is developed, the curriculum can be used over many iterations, thereby reducing the overall cost of the initial investment in design. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 35 “I enjoyed it and it was more relaxed although it was intensive sometimes. The OSCE (Objective Structured Clinical Examination) made it more realistic because you are demonstrating for them to see that you have acquired something, you’ve got the knowledge.” (TOT_VR_P11) However, the efforts of the TOT implementing partners to deliver quality training was key in realizing its training objectives. Respondents placed emphasis on the facilitators who were diligent in teaching. “USAID System for Health, actually are doing a very great job. I have had more training with them as compared to others and relevant topics are always discussed and are always picked. They try to make participants also comfortable and I like the facilitators, I like everything.” (TOT_VR_P11) Respondents also shared that the participants recruited for the TOT had exhibited ambition and were ready to learn. Nevertheless, the duration of the training appeared to be inadequate for participants to fully acquire the knowledge and skills being delivered. It was suggested that the training could be extended to 2 weeks to fully achieve training objectives. “As for the average duration, that’s what I was saying it was too loaded. So I will give 3 satisfactory (rating). But later they should extend it – maybe two weeks.” (TOT_GAR_P02) From the data, the TOT training model appeared to be generally effective in delivering relevant knowledge and skills to HCWs. In the long term, TOT activities can help GHS to deliver relevant training to its cadre throughout its hierarchy. But persistent downstream TOT rollout challenges related to trainee accommodation, feeding, lateness, transportation to and from the venue and the lack of financial incentives for facilitators continue to limit its short term and long-term effectiveness, scalability and sustainability. The above-mentioned implementation challenges associated with TOT activities appeared to have limited it from achieving its full potential. Consequently, when rolled out with measure to handle the challenges stated, the TOT activities will provide long term support to GHS. 6. Research Question Four (Q4): What are other training successes that have been documented in Ghana or other relevant countries? The purpose of training health workers is to provide them with, or update, the skills and competencies that enable them to perform their designated tasks effectively and efficiently. The theory of change for health worker training interventions assumes that the knowledge learned is adopted into improved skills and competencies which translate into better quality of care and ultimately, to improvement in health outcomes. (Figure 8). However, there are many factors beyond the basic provision of training that affect health worker skills and their impact, in turn, on health outcomes. These outside factors are not only important to understand when trying to measure the effectiveness of trainings, but they also pose one of the biggest challenges of both implementing trainings and measuring their results. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 36 FIGURE 8: TRAINING EVALUATION FRAMEWORK WITH NESTED LEVELS AND SITUATIONAL FACTORS7 Training models and approaches have changed and evolved over the years. While in the past it was more common to only have theoretical, classroom-based, and off-site trainings, these have been found to contribute little to the maintenance and sustainability of services delivery (WHO, 2006). Studies have found that one-time training interventions have poor results (Rowe et al., 2009). Additionally, instruction which relies heavily on reading and lecture often results in no to low learning outcomes. However, new evidence has identified better ways to train health care providers to optimize sustained improvements in service delivery. Some of these ways include the use of interactive techniques that engage the trainee and provide opportunities for practice, such as clinical stimulation, constructive and immediate feedback, self-directed learning with interactive techniques, e-learning courses, and the provision of learning opportunities planned and delivered at an appropriate amount and frequency (JHPIEGO, 2013; Bluestone et al. 2013). For example, the simulations (e.g., role play) help HCWs tackle real-life situations more efficiently. The location of the training also matters, and to provide sufficient opportunity to practice, it is recommended training be conducted in the community or health facilities, rather than off-site (World Health Organization, 2010). This use of workplace learning has also shown to be superior for skill acquisition (JHPIEGO, 2013). Additionally, supportive supervision, a process focusing on joint problem-solving, mentoring, and two-way communication rather than simply on observation and critique, has shown to improve HCW knowledge, perception of work, and professional satisfaction and motivation (Vasan et al., 2016). As for the efficacy of computer-based e-learning, two studies in the UK and USA concluded that e￾learning facilitates not only a standardization of teaching materials and a self-directed learning environment, but also offers potential cost-reduction, allowing reorientation of budgets toward other, lagging elements of HCW training (Hadley J et al., 2010; Harrington and Walker, 2004). Repeated frequency has shown to be more effective than one-time training interventions. Some studies have shown that training delivered frequently in low doses is more effective than training delivered infrequently in high doses. This new model of learning is called low-dose high frequency (LDHF) and is being used all over the world, particularly in health care (Health Stream, 2017). LDHF is a capacity building approach that promotes maximum retention of clinical knowledge, skills, and attitudes through short, targeted in-service simulation-based learning activities, which are spaced over time and reinforced with structured, ongoing practice sessions on the job site (JHPIEGO, 2013). The eight principles of the LDHF learning approach are: a 7 O’Malley et. al, 2013 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 37 focus on competency; simulation and case-based learning; appropriately spaced and brief contents of delivery; a team-focused and facility-based oreintation; ongoing practice of skills; exposure to content after initial training; facility-based peer staff is essential to facilitating the exercise; and tracking results and changes in service delivery requires comprehensive quality improvement results. The LDHF has recently been used by GHS in partnership with Jhpiego with funding from the Bill and Melinda Gates Foundation. The Low-Dose, High-Frequency (LDHF) approach has been documented as an effective HCW training approach which yields higher trainee knowledge and skills acquisition (JHPIEGO, 2013). Between 2013 and 2017, JHPEIGO in partnership with the GHS, National Catholic Health Service, and Project Five Alive! adopted the LDHF approach in training health workers on topics aimed at reducing intrapartum stillbirths and newborn deaths within 24 hours after birth. The training attained 50 percent reduction in intrapartum stillbirth and 56% reduction in 24-hour newborn mortality in study sites within Ghana (GNA, July 6, 2017). A study on HCW experiences and opinions about the LDHF in 40 facilities across 3 regions in Ghana revealed that health providers had made significant improvements in all areas taught during the knowledge and skills￾focused training. The topics covered included quality of care at the time of labor, birth, and the immediate postnatal period (Asiedu et al., 2017). The LDHF approach is reported to be a cost-effective training which should be scaled-up in Ghana or integrated into the existing in-service training programs (Willcox et al., 2017). Their retrospective program cost analysis evaluation study found that the cost of LDHF training across 40 health facilities was $823,134, which saved approximately 544 lives a year after follow up-training. Similar training successes using the LDHF approach were recorded in Uganda. Their cluster randomized design study among midwives in 125 health facilities revealed that the percentage of fresh stillbirths and neonatal deaths within 24 hours had decreased significantly in all the study groups over a period of 6 months after training (Atukunda & Conecker, 2017). Maternal, Newborn, and Child Survival (MNCS) in-service training for frontline health workers (FHWs) in South Sudan is another example of a training which was found to be effective in increasing the competency of community-based health workers (Nelson et al., 2012). The program, which trained 72 trainers and 708 FHWs in 7 South Sudan states, yielded significant improvements in knowledge from 62.7 percent to 92 percent. Additionally, average FHW scores on maternal OSCEs increased from 21.1 percent pre‐training to 83.4 percent post‐training, and to 61.5 percent following 2–3 months after the training. Even though the trainers used the training-of-trainer (TOT) model, incorporating targeted training, pictorial checklists, and reusable equipment and commodities contributed to the significant increase in the knowledge, skills, and referral competencies of trainees directly after training. The approximately 20 percent drop in FHW maternal OSCE scores 3 months after training highlight the importance of frequent follow-up training programs to reinforce lessons learned. Under the Making Every Baby Count Initiative, health care provider training programs were carried out by PATH, in collaboration with the GHS, to strengthen health care provider capacity and skills in order to improve both mother and baby’s quality of care during delivery. What made this training successful was that it built health care providers’ capacity and equipped them with the basic devices to carry out tasks like resuscitation. The training was rolled out across Ghana, which ensured that 65 percent of health care providers had received the initial training by December 2016. Follow-up visits were conducted on training beneficiaries to ensure that they received full training, which will enhance their knowledge and skill for service provision (Children’s Investment Fund Foundation, 2017). In conclusion, didactic training methods which involve passive learning were found to yield limited learning outcomes, whereas clinical simulations and case-based learning, practice, and feedback were relatively more effective HCW educational techniques (Bluestone et al., 2013). 7. External Factors That Influence Effectiveness of Training To recognize that knowledge and skill are not the only aspects that contribute to the ability of the HCWs to deliver quality health services, this chapter describes the community, physical and organization related factors which influence the effectiveness of HCW work in the CHPs zones. For each item, respondents chose from a range of strongly agree to strongly disagree to factors on a questionnaire and the provided reasons for their responses, which were analyzed qualitatively. Examining the external factors which influencing HCW ability to provide quality health care give an important insight into the environment the HCW live and work in within the community. External factors influencing the work of HCWs include OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 38 community acceptance, safety, good roads, and availability of reliable electricity, water, internet, medicines and supplies. Organizational support in the form of motivation, protocols and independence to practice also influence the use of knowledge and skills acquired during training at work. Community Environment Situation Community acceptance of the HCW was found to influence trainees’ in a positive way. Majority (76.9 percent) of the CHOs/CHNs strongly agreed that they were accepted in the communities they worked in and 62.4 percent strongly agreed that community members sought health services from them (See Table 11 in Annex C). HCWs shared that health seeking behavior of community members was based on health workers’ accessibility, relationship building, and acceptance by the community members. When the HCWs are introduced to the community appropriately, it enhanced acceptance. “It’s the way you introduce them (HCWs) to the community, normally when a community is identified and we are sending a community health officer or worker, we do our underground work, we meet the opinion leaders, queen mothers, traditional rulers; in fact opinion leaders, herbalist, traditional birth attendants, all those people who matter, we meet with them and do the jaw-to-jaw, we sit with them, discuss issues, they come out with their concern and all that, we try to address everything. When everything is settled down, that is the time that we go” (KII_AR_MHD) Although the risks of armed attacks (e.g., Fulani herdsmen) and animals (e.g., snakes) were present in some areas, majority (88.2 percent) of the respondents felt safe in their community. This was because CHNs felt they were accepted as members of the community and some communities had volunteers who protected CHNs at night. It was found that majority (89.4 percent) of the respondents agreed that community members/leaders provided support in organizing health programs. Most respondents (75.9 percent) also suggested that the community's political/chieftaincy stability was conducive for their work (Figure 9). They stated that even in situations where there are some chieftaincy/political problems, their HCW was generally not affected. In addition, respondents were asked whether there was stigma toward controversial health topics (e.g., family planning), which 91.3 percent of HCWs agreed with. This was related to religious influence and misconceptions about controversial topics like family planning. The misconceptions would lead to low patronage of family planning services. “Because yesterday my durbar I talked about family planning and it was not easy for me at all. They really didn’t understand it, they have a whole lot of misconceptions about the family planning. Some were saying that we’re letting them limit the number of children God told them to deliver.” (TRADITIONAL_NR_T003) FIGURE 9: COMMUNITY ENVIRONMENT FACTORS CONTRIBUTING TO SERVICE DELIVERY Organizational Situation A supportive, safe, and positive work environment helps to support HCWs in their ability to effectively deliver health services. Majority of the respondents (88.4 percent), felt they receive the necessary support/supervision for their managers through management visits and receiving feedback on their 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Community members seek health services Accepted in the community Stigma towards controversial health topics, e.g. FP Community leaders provide support I always feel safe within the community and at work Community's political/chieftaincy stability is conducive Women need spousal permission to access health services Agree Disagree N/A OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 39 performance and suggestions were made on how to improve their work. However, about 41.4 percent of respondents reported that management does not always resolve their challenges. The reasons cited by respondents were related to managements’ inconsistency in resolving their challenges, particularly challenges related to transportation. About 43.7 percent of respondents did not feel they had adequate staff to perform their duties effectively. The reasons cited by respondents were related to the availability of adequate midwives and CHNs in the health facility. Some respondents explained that few midwives were available to manage labor cases brought to the health centers. In some circumstances, HCWs did not close from duty because there was no extra midwife to handover to. “I’m the only person there, for the RCH side, …there’s nobody in the facility to perform my duties because there’s no midwife, and I have to do my work in addition to the midwife field” (TRADITIONAL_NR_T003) Almost all respondents (94.1 percent) felt they were able to make decisions on their own regarding their work and that the chain of command helped to enhance the work they do (93.1 percent). Respondents were related to their ability to make independent decisions as well as take decision pre-determined decisions by supervisors. However, a few respondents expressed that there were pre-determined or scheduled decisions made by their superiors who might have been in the national, regional or district health directorate, so they did not feel like they could make independent decisions. Others felt having pre-determined roles based on their job description made the chain of command very clear to understand. “I agree, because, we have clearly spelled out levels of engagement and of authority, so there are levels of authority. From the facility level, subdistrict level, unit level, district level, regional level. So it’s clearly, is spelled. So depending on how and what issue or matter it is, the different levels handle them.” (OTSS_NR_OTSS001) About 44.2 percent of respondents reported they did not always have the required medicines and vaccines available. In addition, 54.9 percent did not have the equipment and supplies required to do their work effectively (Figure 10). Some respondents mentioned that they occasionally run out of vaccines like yellow fever and measles. “Because some of them are there and some too as just I said, the measles rubella and yellow fever vaccines, most of the times they’re not there. There’s a shortage” (TRADITIONAL_NR_T003) Some respondents, however, admitted that they sometimes are credited from the regional medical stores hence they did not run short on vaccines. However, some respondents mentioned that equipment and supplies like syringes, needles, weighing scales were unavailable or inadequate. “For example, the weighing scale. We have them but most of them are not functioning properly.” (OTSS_ASHANTI_001) Majority of respondents (70.9 percent) reported that the record system was adequate for effective work. Reasons cited by the respondents were related to the use of manual and electronic records. Several respondents stated that they used manual processes to manage their patients’ records, such as notebooks and registration books to record patient data. A few identified that the manual system had synchronization challenges that sometimes when patients died, they were unable to manually reconcile their very old record books which may have been stored away. “In the sense that whatever we do we keep records, our notebooks, our registration books, we even have our registration books for especially all pregnant women in the catchment area and we are able to trace defaulters” (TOT_GREATER ACCRA_P03) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 40 FIGURE 10: ORGANIZATIONAL ENVIRONMENT FACTORS CONTRIBUTING TO SERVICE DELIVERY Some respondents recognized the importance of computerized record keeping and the ease associated with it, this was especially true for HCWs using the District health information management system (DHIMS2) software. “The record system is very good and now with our reporting software, ‘DHIMS2’ that we are using it is very good. DHIMS2 is a District health information management system, is a reporting system software that you key in our data and if you key it here, the regional director can see it and the director general can see the work that the Offinso-north district are doing and that was what I went on monitoring on because tomorrow is the deadline for us to enter all the data set for January. So we have to go to them or call for them to do it. So when it comes to the ‘DHIMS2’ software I think it is really helping.” (KII_DHD_ASHANTI) Respondents were asked whether the referral system was suitable for quality care. Majority (57.7 percent) of respondents reported that they felt it was suitable. However, for the 39.4 percent that disagreed, reasons were related to the unavailability of transportation for referred cases. In the absence of ambulances, HCWs depended on other drivers and cars to carry patients to their referral points. Clients like pregnant women in labor had to use unsuitable means of transportation like motor bikes to assess health care. “…ambulance service is not working. Sometimes you refer the clients and they have to go look for their way and means to the referral point.” (MENTORING_VOLTA_P08) “Because there are no ambulance so it makes referral difficult. Sometimes you have to arrange with public transport or arrange for a car to come and by the time the person gets to the referral center whatever would have gone on” (TOT_VOLTA_P19) A majority (83.5 percent) of the respondents agreed that the treatment guidelines for practice were available to facilitate work. Although some respondents admitted that they relied mainly on their own skills, several respondents confirmed the availability of protocols like standard treatment manuals which guide their practice. Some respondents suggested that trainers provided all the protocols in the health facilities hence it was always available to the HCWs. Some respondents disagreed that the treatment guidelines for practice were available to facilitate their work because they did not have some of the protocols like standard treatment guideline within their health facility. Thus they relied on their skills to get task done. “Because I don’t have any treatment protocol that I follow. It’s based on the skills we got from the thing that we are using.” (INTERNSHIP_ASHANTI_001) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Chain of command at workenhances the work I do Able to make work decisions on own Duties are well-defined Provided support/supervision/mentor from management I have adequate time to attend to my clients Treatment guidelines for practice are available Records systems are adequate Challenges are resolved by management Referral system is suitable Adequate staff to perform duties effectively Remunerations/Salaries/Allowances are adequate and timely Agree Disagree N/A OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 41 Physical Environmental Situation In assessing the influence of physical environment on the work of HCWs, respondents were asked whether the community's roads were always accessible / passable. About 69.2 percent did not think the roads were suitable and cited the main reason being related to tarred or damaged community roads, including potholes, rain/flood damage (Figure 11). Potholes, rain/flood damaged community roads were not passable for HCWs especially in the rainy season. This sometimes cut off communities from the HCWs and impeded their work. However, respondents stated in some areas there had been improvements. “Now we have tarred roads. Only few of them are untarred …the main roads are now being tarred.” (TRADITIONAL_GREATER ACCRA_P06) “We use foot paths as our routes. Especially during the rainy season, you go and all the place is bushy.” (INTERNSHIP_ASHANTI_002) Respondents suggested that the inadequate number of motorbikes made access to transportation more difficult for HCWs. This negatively influenced their ability to deliver health services to the community. “We use motor bikes and one motor bike is being used by five different CHP zones. So sometimes you will feel like going to your community and the other person too wants to. (INTERNSHIP_ASHANTI_002) In relation to infrastructure, many of the respondents disagreed when asked whether they had potable water, reliable phone signals/internet, and buildings. This was because they believed that their health facilities were dilapidated and required renovation. Majority (52.4 percent) of respondents did not agree their CHPS compound was in good condition and reported that their facility had issues including leaking roof, infestation with bats and bees, and broken faulty doors and locks. “Because most of the CHPs compounds when you go there the ceilings are falling off and then the doors have no locks and our staff are there like that” (TOT_NR_TOT003) Only 33.7 percent of respondents reported that there was reliable electricity at their CHPS compound. Respondents that reported they did have reliable electricity often had generators and could continue their work despite power fluctuations. When there was none, electricity supply to the health facility became irregular which negatively influenced the work of HCWs. In addition, 51 percent of respondents reported they did not have regular access to potable water and some would have to buy water to bring to the CHPS compound. The most common source of water was bore holes. “Because about half of them have borehole on site. Two of them are even mechanized. The others, the community and the staff fetch water from borehole in town to the facility. None of them use surface water” (KII_DHD_ASHANTI) Respondents were asked whether the CHPS compound/health facility they worked in had reliable phone and internet signals/reception. About 52.4 percent of respondents reported they had unreliable phone signal, while a majority (78.6 percent) of CHO/CHN reported that they did not have reliable or functional internet. Responses suggested that the internet services providers were typically the cell network providers in Ghana such as MTN and Vodafone. Thus, the reliability of their internet was influenced by their cell phone signal/reception. The lack of reliable phone signal/receptions sometimes made HCWs unavailable to persons who wished to contact them readily on their phones and limits their ability to access up-to-date information. “No, no we have CHPS compounds, six CHPS compounds and only two have phone signals, the rest…Even the two at times you have to stand somewhere before you can get it, so I don’t agree.” (KII_ASHANTI_MHD) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 42 FIGURE 11: PHYSICAL ENVIRONMENT FACTORS CONTRIBUTING TO SERVICE DELIVERY Other Health Worker Trainings in Ghana Some participants in this study have benefited from other trainings for HCWs organized by GHS with the support of different partners. This presupposes that the knowledge and skills of HCWs exhibited at work could be attributed to all the trainings they have benefited from and not only USAID funded trainings. Notable trainings mentioned as beneficial include: 1. Making Every Baby Count Initiative by Program for Appropriate Technology in Health (PATH) 2. Baby friendly and management of acute malnutrition delivered by World Vision International. 3. Family Planning by Korea International Cooperation Agency (KOICA) 4. Newborn Care by UNICEF 5. The World Bank MNCHPS - Maternal Nutrition Child Health Planning Service. 6. Malaria, Tuberculosis, and HIV Training by GLOBAL FUND In view of the successful trainings implemented over the years, the participants called for more training. “I’ve seen that the trainings helped and we need to do trainings more often. … It’s good” (KI_NR_DD001) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% CHPS has reliable electricity CHPS has potable water Equipment and supplies/logistics are always available Access to transportation is always available. Reliable/functional Internet Agree Disagree N/A OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 43 CONCLUSIONS 1. What models of training (e.g., on-site, e-learning, etc.) focusing on Ending Preventable Child and Maternal Deaths (EPCMD) appear to most effectively contribute to increases in CHO and midwives knowledge and skills? a. How effective are these trainings directly after vs. long-term? Overall, each training had its strengths and weaknesses. However, the study revealed that participants of the internship and traditional training models recorded higher scores from the knowledge assessment compared to participants of the other training models (i.e., supportive supervision and TOT) (See Figure 4 and Table 14). The availability of learning resources such as the course materials, teaching aids, human resource, physical facilities including accommodation and a serene environment appeared to have facilitated teaching and learning for participants of the internship and traditional training models. The practice component of the HCW training appeared to be more effective for skill transfer (e.g., insertion of implants and assessment/management of anemia or febrile illnesses like malaria). Internship, mentoring and supportive supervision trainees had the added advantage of facilitator-supervised practice and observation of skills after theories were taught. All of these trainings included exposure to real life experiences. Since they were hands-on approaches to learning, skill deficiencies/errors could be identified and corrected by mentors/supervisor. Even though internship model trainees were more likely to have the equipment and supplies to practice acquired skills in the health facilities, they were sent to learn, supportive supervision trainees practiced under supervision in their own working environment and every-day-clinical situations. This appeared to have facilitated the effectiveness of knowledge and skill acquisition by supportive supervision trainees, the opportunity for challenges to be identified and tailored solutions being delivered within the specific context of the trainee’s working environment. And since the focus of supportive supervision is also on the trainee’s health facility, other HCWs can also benefit from each visit The OTSS should be used in post-training/follow-up visits of trainees to determine the uptake and use of competencies acquired from training. The traditional and internship training models were identified as being more effective for the transfer of knowledge on new topics to HCWs. These models provide a structured learning approach for introducing new information and/or skills. It is critical that the duration of training should be adequate to deliver the training content without overloading trainees with information. In addition, training materials like presentation slides and literature provide resources that can be referenced by the trainee at a later date. Since the e-learning training model did not require active supervision by a tutor and could be applied at the trainees’ own pace, it had the highest accessibility factor. The content on the e-learning platform is however limited hence it can best be used as complimentary material for other training. It is however worth noting that HCWs’ environmental challenges associated with electricity, phone signal and internet reception in the community could reduce HCWs’ accessibility to content or updates. The results from this evaluation do not necessarily indicate participants’ level of skill performance of training related tasks since they were not placed under observation (i.e., demonstrated skills). An observation of participants as they perform tasks related to the training will provide a deeper insight into their skill levels. Nonetheless, the evaluation found that according to the participants, the training models were effective directly after training as evidenced by consistent use of knowledge and skills acquired from training. Long term effectiveness of training can best be assessed with follow-up visits; however, HCWs suggested that they still demonstrated the knowledge and skills they had acquired even 2 years after the training. It was reinforced that even though majority (72 percent) of the participants’ training ended in 2016, trainees still obtained significantly high knowledge and skill scores in this study. In summary, each of the training models contributed effectively to the increase in HCWs’ knowledge and skill because they were relevant to the work of trainees. However, the results suggest that if the objective for a HCW training session is to transfer knowledge, the best training models to employ are the internship and traditional training models. Participants of both training models obtained the highest quantitative knowledge scores and qualitative analysis suggested that they were effective in increasing trainees’ knowledge on topics delivered. Nonetheless, the supportive supervision (OTSS) training model was identified by HCWs as highly effective in identifying trainee skill challenges as well as delivering tailored solutions and follow-up supervision within the specific context of the trainee’s working environment. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 44 2. Are the trainings conducted for CHO and midwives relevant to their needs and being utilized for health care delivery? a. To what extent are the knowledge and skills received at trainings being shared with other health staff at the CHPS? HCWs generally asserted that the training was relevant to their needs in health care delivery since they used the knowledge and skills daily at work. Some HCWs still referred to their training materials as required in their daily work. The high relevance of the training to HCWs’ health care delivery could be related to the fact that training topics and content were selected in consultation with the GHS. Furthermore, most of the training content are directly based on GHS’s training curriculum. Only a few were adapted from global organizations but vetted by GHS (e.g., UNICEF IYCF curriculum). Consequently, the HCW’s practice is based on training which is relevant to HCWs working within the GHS service. It was also evident that HCW trainees utilized the skills they had acquired from training at work which appeared to have improved treatment outcomes. Various participants reported how their newly acquired knowledge and skills improved their service delivery in the form of correct diagnosis of infants’ ear infections as well as testing and treatment of malaria. Others reported reduced abuse of Paracetamol as well as incidents of stillbirths and maternal deaths in their facilities as result of training skill utilization. The evaluation found that the HCWs typically shared the knowledge and skills received at training with other CHPS staff. The decision to share was a standard GHS practice however, trainees were encourage to share by their trainers and supervisors. Acquired knowledge and skills were shared by writing reports to their superiors, presenting the content of what they learned from the training to their colleagues in staff seminars, unit meetings, in-service training, on-the-job training, and workshops. In some situations, they shared the training materials with their colleagues. HCWs suggested that the knowledge and skills received were shared to ensure that when participants are absent or transferred, the best practice can continue in the facility. 3. What models of training appear to be the most scalable and sustainable across GHS? a. Have the training of trainer (TOT) activities been effective in the terms of providing long-term support to GHS? The evidence from this study indicated that the most suitable training model for knowledge acquisition was Traditional training. The most suitable training model for skill acquisition were internship and mentoring. The most widely accessible training model was e-learning, but it is best used as complimentary training for the other models. The TOT model was suitable for trickling relevant training down the hierarchy of the GHS. However, the supportive supervision model was identified as the most effective, scalable and sustainable training model for knowledge and skill acquisition, retention and application. The model ensures that the desired target audience are trained within the specific context of their environment. HCW activities can be appraised in real-life, tailored solutions to unique challenges could be preferred in real time. Additionally, the supportive supervision model is the most scalable across GHS because HCWs receive the monitoring and supportive visits they expect to from their managers in higher hierarchy. Thus cost is saved on separate monitoring visits. Effective supportive supervision however requires ample contact time between trainees and their supervisors. Follow-up supervisory visits must be regular. Supplies and training materials like RDT kits must also be made available for use during demonstration. Cost of supervisors’ allowance and appropriately comfortable transportation must be appraised properly. Nonetheless, the benefits of supportive supervision if done correctly could potentially justify the cost associated. 4. What are other training successes that have been documented in Ghana or other relevant countries? Training which targeted the right HCW cadre for specific training was found to contribute the success of training. Additionally, the appropriate use of teaching aids like pictorial checklist and learning equipment/supplies facilitated the effectiveness of HCW training. Despite the evidence of knowledge and skill acquisition immediately after training, frequent follow up on training is required to maintain trainees’ knowledge levels. Low Dose High Frequency (LDHF) by JHPIEGO was identified as a documented HCW OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 45 training success in Ghana. It was associated with increased competence of HCWs and a reduction in maternal and child mortality in the health facilities they were implemented in. Some training successes documented in Africa and Ghana were found in literature. Strategies which appeared to have contributed to the increased effectiveness of various community-based health worker trainings included targeted training, use of appropriate teaching aids (equipment and supplies) as well as delivery of low dose training at frequent intervals. Invariably, interactive learning methods which incorporate clinical simulations, case-based learning, practice and feedback were found to be relatively more effective in achieving HCW learning outcomes as well as the success of HCW trainings in Ghana and Africa. External Factors That Influence Effectiveness of Training From the findings, the environmental factors which significantly influenced effectiveness of HCW’s training included the community’s stigma associated with controversial health topics like family planning, availability of essential medical supplies, constant supply of utilities, availability of protocols and the nature of community roads. The availability of essential medical supplies (e.g., vaccines and rapid diagnostic kits) and equipment for trainees to practice appeared to influence the effectiveness of training. Trainees may have to learn and practice with models in place of the real supplies and equipment. For instance, when diagnostic test kits run out, trainees may be forced to proceed to treat without testing. Treatment guidelines guided trainees’ practice hence they formed a strong foundation to build on trainees’ knowledge and skills during training. Trainees who have not been exposed to the protocols and guidelines may require more training to familiarize them with new training concepts. Utilities like electricity, potable water, phone signal reception and internet services were identified as factors which influenced HCWs’ ability to delivery health services in the community. The reliability of electricity was largely dependent on the availability of generators with consistent supply of fuel. In the absence of electricity, e-learning devices (computers) cannot be used effectively. The application of learning and teaching aids like projectors to show slides as well as videos to illustrate concepts becomes limited. The inadequacy of HCWs in the health facility was found to possibly influence the effectiveness of learning since sometimes HCWs who are alone in their facility are not allowed to leave for trainings. In some cases, trainees leave midway in their training sessions to go and care for clients especially when the training is being held within their district. It was evident that the stigma associated with controversial health topics like family planning limited HCWs’ application of competencies acquired during training. For instance, an HCW trained in family planning may not receive so much patronage from a predominantly Catholic or Muslim community to practice such services. However, the supposed stigma was largely grounded on misconceptions and the community’s religious preferences. Community members and opinion leaders must therefore be educated further on these controversial health topics to enhance the patronage and benefits of health services such as family planning. The nature of roads greatly influenced transportation and clients’ ability to access HCWs’ health services. Tarred roads were more accessible to motor bikes which were the main source of transportation for HCWs and clients. Bad roads were also a threat to the frequency and duration of Supportive Supervisory visits. From the findings it was also clear that HCWs’ were guided by their job description in the performance of their duties but they needed consistent support from management in the form of frequent visits, timely allowances, logistics and protocols like standard treatment guides. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 46 RECOMMENDATIONS Based on the study findings and conclusions, the evaluation team recommends the following: A. Training Models Which Effectively Contribute to Increased HCW Knowledge and Skill 1. USAID/Ghana and GHS should strategically select training models based on the objectives of planned training: a. Supportive Supervision/OTSS should be used to deliver tailored on-the-job knowledge and skills training to provide practice and feedback in trainee’s working environment. b. Internship and traditional training models should be employed in introducing new topics and delivering theoretical content. c. Mentoring and internship models should be used to deliver specific skills development through one-on-one engagement. The duration of training should be more than a week to ensure that trainees benefit fully from their mentors. 2. Whenever possible, USAID/Ghana and GHS should incorporate follow-up supportive supervision of trainees after all trainings to ensure that the procedures taught are being correctly followed by trainees. B. Effectiveness of training for quality health service provision 3. The duration of traditional, TOT and mentoring training should allow for enough time to deliver the training content without overloading trainees with too much information. Trainings should be broken down into smaller focused training sessions or extended to allow enough time for new topics and skills to be introduced and taught. This is especially important for TOT because master trainers must fully absorb the content in order to train other HCWs. 4. The duration of internship training should be extended beyond a week (possibly to one month or more) to ensure that trainees benefit fully from their preceptors or mentors. This prevents trainee information overload and increases the effectiveness of training. 5. Key training reference materials like presentation slides and reference articles should be made available to trainees so that they can follow lessons during training, refer to them after training and share with colleagues in their health facilities. 6. Implementing partners and trainers should continue to select training topics and content in consultation with the GHS. Even when content is adapted from global organizations (e.g., UNICEF) they should be reviewed by GHS before delivery to HCW trainees. This measure ensures that training delivered remains relevant to Ghanaian HCW practice and application in service delivery. 7. District-level master trainers should be used for CHPS-level HCW training delivery. Participants suggested that district level master trainers appeared to have a better understanding of the CHPS environment and therefore better suited to deliver trainings. 8. Allowances offered by IPs to GHS facilitators should be uniform across all IPs to minimize situations where facilitators are biased toward activities/tasks of IPs who pay higher allowances. C. Extent to which knowledge/skills received at training were shared with other CHPS staff 9. Trainers and superiors should continue to encourage trainees to share the knowledge and skills they acquire with colleagues in their health facilities. Even though the decision to share lessons learned is standard GHS practice, facilitators’ follow-up and encouragement will motivate trainees to do so. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 47 10. Supervisors should nominate only appropriate and interested cadre of HCWs for IPs for trainings to ensure that the training is applied and that trainees propagate the information they learn to other HCWs when they return to work. D. Most scalable and sustainable training model 11. Given the strong response from trainees and trainers on the relevance and effectiveness of SS/OTSS, USAID/Ghana and GHS should consider scaling up the model, including exploring ways to institutionalize SS/OTSS through budget allocations for supervisors whose full-time function is to rotate across facilities, rather than being pulled from their jobs. F. External factors that influence effectiveness of training and quality of care 12. Key external factors that influenced HCW training and work included the availability of medicines, vaccines and supplies; management support; good roads and reliable utilities (electricity, water, cell phone.) Trainers should appraise these factors in advance of training to determine their impact and to maximize training effectiveness and application in service delivery. 13. Because of the influence of environmental factors on effective training and its application, USAID and GHS should consider requiring IPs to add a community engagement component to their HCW training. G. Training documentation and future considerations for research on HCW training 14. USAID/Ghana should require and verify that all IPs utilize pre- and post-tests and maintain accurate training records, including participant lists and contact information, training curricula, pre and post-test scores and dates of follow-up SS/OTSS visits. This will ensure that future evaluations have access to baseline data and to lists from which training participant samples can be easily drawn. 15. Given the very positive feedback from participants and trainers on the SS/OTSS model, USAID/Ghana and GHS should consider conducting an assessment of outcomes from this training in districts where the practice has been regularly sustained, along with a review of costs and organizational options for providing supervisory support that would not require pulling critical staff from health facilities for the SS/OTSS visits. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 48 ANNEXES Annex A: Evaluation Statement of Work Purpose The purpose of this study was to evaluate the effectiveness of USAID-funded health worker training for CHOs and midwives to determine optimal methods that could be scaled and sustained on a national level for CHPS capacity building in Ghana. To achieve this, specific evaluation objectives and questions were developed, and answers sought for. The specific objectives were to assess the degree of effectiveness of each model in terms of knowledge and skills acquisition, retention and application on the job; to identify any external factors that influence the effectiveness of the model; and to recommend model(s) that appear to offer the best prospects for national-level scaling up and sustainability. Evaluation Questions To explore these different areas of interest this evaluation will seek to answer the following questions: 1. What models of training (e.g. on-site, e-learning, etc.) focusing on Ending Preventable Child and Maternal Deaths (EPCMD) appear to most effectively contribute to increases in CHO and midwives knowledge and skills? a. How effective are these trainings directly after vs. long-term? 2. Are the trainings conducted for CHO and midwives relevant to their needs and being utilized for health care delivery? a. To what extent are the knowledge and skills received at trainings being shared with other health staff at the CHPS? 3. What models of training appear to be the most scalable and sustainable across GHS? a. Have the training-of-trainer (TOT) activities been effective in the terms of providing long-term support to GHS? b. What are the general cost parameters of each model? 4. What are other training successes that have been documented in Ghana or other relevant countries? A. Scope of Evaluation USAID/Ghana invests in a variety of types, levels (e.g. district, regional, etc.) and areas of training, however to ensure meaningful findings, the scope of the evaluation was narrowed to key trainings. Based on discussions with USAID/Ghana and the relevant implementing partners (e.g. SYSTEMS, MalariaCare, MCSP, SPRING), it was determined that this evaluation will focus on trainings at the community level (CHPS) with CHO, CHN and midwives. In addition, because USAID/Ghana has a diverse portfolio and to be able to provide some comparable data between training types, all trainings were related to Ending Preventable Child and Maternal Deaths (EPCMD). With these parameters, the study has narrowed down the specific relevant trainings for each IP that will be included in the study (See Table 1). Methods overview A mixed methods approach was used for this study to determine the most effective type of training, (e.g. traditional, training of trainer, supportive supervision, mentoring, internships and e-learning). This approach included an analysis of training participants’ knowledge and skills after the training using IPs pre- and post-test data, follow-up assessments and qualitative data collection with training participants and key informants. All quantitative and qualitative data were analyzed together to understand any changes in participants’ knowledge and skills, as well as its application to their job. The study also did direct data collection with training participants to determine knowledge and skills through an assessment tool. Group interviews and key informant interviews were also conducted with training participants, IP staff, GHS staff and trainers. From September to December 2017, the team gathered secondary data from the Implementing partners (IPs) and stakeholders on the health worker training curricula used, trainee details and test records. The team OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 49 conducted desk reviews of the secondary data to facilitate the preparation of the study protocol, sampling, and field work plan. The team also developed the respondent sampling plan, various qualitative and quantitative data collection tools and submitted the final protocol to the Ghana Health Service Ethics Review Committee (GHS-ERC) for ethical clearance. The research team worked closely with the study’s steering committee whose mandate was to provide technical feedback and suggestions at key points during the study. Meetings were held quarterly during the period of the study to update them on progress whiles seeking further support as and when required. The Committee’s Terms of Reference were as follows: • Review and make recommendations on the study’s scope, design and approach. • Provide technical input to the draft data collection instruments. • Facilitate access to data where needed. • Receive and review progress reports. • Provide technical support to the Lead Researcher (e.g. answering questions). • Review preliminary findings, conclusions and recommendations. • Review and comment on the final study report. B. Membership of Steering Committee SC participants represent key USAID/Ghana and GHS stakeholders and Implementing Partners from the study’s selected training programs: MCSP, Systems for Health, MalariaCare and SPRING projects. The following are tentative steering committee participants: Table 5: Steering Committee Membership No Name Institution 1. Rubama Ahmed USAID HPNO 2. Felix Osei-Sarpong USAID HPNO 3 Juliana Pwamang USAID HPNO 4. Karen Caldwell MCSP – COP 5. Dr. McDamien Dedzo SYSTEMS – DCOP 6. Dr. Samuel Kaba GHS, ICD 7. Mrs. Eva Mensah CNO, GHS, ICD 8. Mr. James Avoka Asamani GHS, Human Resource Division 9. Dr. Frank Nyonator Evaluate for Health 10. Dr. Lydia Aziato Evaluate for Health 11. Mr. Emmanuel Mahama Evaluate for Health On 20th October 2017, The Team presented its progress made and fieldwork plans to the study's Steering Committee (SC) chaired by Project Director USAID/Ghana Evaluate for Health, Dr. Frank Nyonator in the Evaluate for Health Offices, Accra. This was the first meeting of the SC. The protocol and fieldwork plans were discussed by the SC members who gave suggestions and recommendations to guide the team's progress. The team swapped Eastern region for Ashanti region as suggested by some members of the SC to ensure that the sampling process is more representative. On 9th January 2018, the team presented a study update and deployment plan to the steering committee during its second meeting at the Evaluate for Health Ghana Office in Accra. The SC members, led by the chairman Dr. Samuel Kaba (Director of GHS Institutional Care Division) discussed the team's progress and deployment plan and made the following suggestions: • The research team should interview GHS Human Resource Division (HR) because they will have to validate findings from the study when it is sent to the GHS. • The team should contact the National and Regional GHS HR and the IPs again with a simplified list of gaps for more information which will facilitate sampling prior to field entry. • The research team should be cautious in recruiting other cadres as replacement participants. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 50 • The study should consider asking CHOs, CHNs, and Midwives what benefits they derived from being supervised under Outreach Training and Supportive Supervision (OTSS). The S.C's suggestions and recommendations were taken up for implementation by the research team. C. GHS-ERC Ethical Review and Clearance The protocol and its associated documents were finalized by the team and submitted to the GHS-ERC for ethical clearance on 9th November 2017. On 30th November 2017, the GHS-ERC offered a conditional approval, subject to modifications to the consent forms and currency of the study budget. The team utilized the suggestions made by the GHS-ERC and re-submitted for full ethical clearance in December 2017. The GHS-ERC gave approval for the implementation of the study in a letter dated 2nd January 2018. D. Field Work Planning and Development The team developed the field work plan for the study. On 5th December 2017, the lead investigator presented the schedule of steps for enumerator selection, training and field work to Technical Director Evaluate for Health, Deborah Orsini and her team. Based on discussions and recommendations after the presentation, the team decided to consolidate the knowledge assessment questions with the other qualitative instruments. A target of 90 minutes was also set for each interview for the training participants across the 4 regions. E. Background Key Informant Interviews Conducted In collaboration with Evaluate for Health, the Key Informant Interview qualitative instruments were finalized for data collection among the key informants located within the Greater Accra Region. In December 2017, A total of nine (9) KII interviews were conducted with IPs and other collaborators to collect background data for the study. The respondents interviewed were from USAID/HPNO, MalariaCare(IP), SYSTEMS(IP), MCSP(IP), SPRING(IP), GHS Institutional Care Division (ICD) and GHS Family Health Division (FHD). These interviews were transcribed, validated, coded and analyzed by the research team to guide this evaluation. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 51 Figure 12. Ethical Clearance from GHS-Ethics Review Committee Table 6: Study Deliverables DELIVERABLE DESCRIPTION PERSON RESPONSIBLE ESTIMATED DEADLINE Research plan and protocols Proposed research plan and options presented to HPNO and SC Lead Researcher Oct. 2017 Data sources and collection tools Full list of data sources and data collection tools (e.g. interview guides) Lead Researcher Oct 2017 ERB Approval Protocol to go through the Ghana Health Service ethical review board. Lead Researcher + research assistant. Nov/Dec 2017 Enumerator Training Training on data collection instrument and pilot instruments Lead Researcher + enumerator team January 2018 Data collection (Post-training testing) Field data collection in focus region CHPS compounds - knowledge and observation Research assistant and enumerators Feb – April 2018 Draft research report First draft of report presented to HPNO, GHS and IPs Lead Researcher + research assistant. June 2018 Final research report and dissemination Full report prepared after feedback from committee; Dissemination to key stakeholders Lead Researcher August 2018 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 52 Annex B: Evaluation Methods and Limitations A. Ethical Considerations The study was dependent on primary and secondary data. The study therefore sought ethical approval from the Ghana Health Service Ethical Review Committee. For secondary, anonymized data, the terms and conditions under which the data was primarily obtained were explored with the view of ensuring strict respect for its provisions. Confidentiality of study participants was protected based on the data that was extracted (e.g. data for participants was anonymized by assigning them with unique IDs during the data analysis). In addition, for any primary data collected, consent was established for all respondents. No participant will be forced to take part in the study. A participant could withdraw from the study at any time with no repercussions. B. Gender Considerations Gender was integrated at the design phase to ensure appropriate consideration during data collection tool design, enumerator selection and training, data analysis and reporting. The evaluation disaggregated knowledge and skills acquisition and post-training performance based on sex and age. Gender-related contextual factors that could impact post-training performance were also identified and analyzed (e.g. performance as a factor of location, housing, community engagement, etc.). Group discussions were also done to address gender considerations in service provision. C. Management of the Evaluation A senior health worker capacity building consultant led this study, working with a research assistant and a team of enumerators/interviewers who were experienced health workers capable of observing health care service provision and evaluating with respect to protocols, record keeping and case management. The led evaluation expert served as the principal investigator of the study, working in consultation with the Evaluate TO2 research manager, the Evaluate project director and MSI’s home office technical director. A research assistant was recruited independently and was responsible for support for the design of survey instruments and a literature review. The research team consulted regularly with HPNO on the research design and provided monthly progress reports. D. Enumerator Selection and Training From January to February 2018, the Research Team trained 11 enumerators who were shortlisted after a review of their credentials and experience. During training, the team pre-tested the study instruments in CHPs zones located in the Ashaiman District of the Greater Accra Region. Based on the results of the pre￾test, the team revised the study instruments, set up the final enumerator teams, finalized the deployment plan and prepared logistics to facilitate field work. Three enumerator teams were formed for data collection. E. Field Entry, Data Collection and Quality Control Visits To facilitate data collection, the USAID Evaluate Office sent letters to the GHS Regional Health Directorates in the Northern, Ashanti, Volta and Greater Accra regions, introducing the enumerator teams and stating their purpose in the region. The letters also sought the support of the regional heads in facilitating data collection processes within the various CHPs zones in their jurisdiction. On February 5, 2018, the three enumerator teams commenced data collection simultaneously in their designated regions. Data collection spanned from February 5-28, 2018. Ninety-seven percent (98 of 101) of the interviews conducted were face￾to-face. In three instances when interviewees were unavailable for in-person meetings, phone interviews were conducted with the consent of the participants. F. Quality Control Visit I & II On February 7, 2018 (data collection Day 2), the Lead Researcher, Dr. Lydia Aziato joined the Enumerator Team II (A) in the Ada East District of the Greater Accra region. The purpose of the field visit was to supervise enumerators' data collection processes as well as offer them support and guidance as required. From the monitoring visit conducted, Dr. Aziato observed that all three enumerators were healthy, in their designated district and were carrying out their tasks as assigned by the Team Lead. The team dynamics appeared suitable OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 53 for optimal work. The Team Lead had adequate supplies and functional equipment to execute the remaining data collection. The daily reports submitted on the team’s progress corresponded to the actual situation in the field. The participants recruited by the evaluation met all the inclusion criteria. Because of the close collaboration with Evaluate for Health senior evaluation advisor (Emmanuel Mahama), the district's participant sample size was met despite situations where some of the trainees were either unavailable or had been transferred. The enumerator team had received a backup list of additional participants to select from to ensure that sample sizes were met. From the lead researcher's inspection, the correct data collection tools were being used for each category of participants interviewed and the enumerators were probing relevant items as required for accurate data analysis. Dr. Lydia Aziato commended the enumerators and encouraged them to maintain the high standards they had used in the data collection process so far. The lead researcher also met with the District Public Health nurse to thank her for the assistance she had offered to the team. On February 14, 2018 (Day 9), Dr. Lydia Aziato met the Enumerator Team II (B) in the Ketu South District of the Volta region. The Team members were having some challenges recruiting participants from the Midwife￾CHO Learning Cluster. The Team liaised with Emmanuel Mahama and the District Director of Health for the participants to be identified, recruited and interviewed. While in the field, the lead researcher received assurances that arrangements would be made for the trainees to be available to the enumerators so that the remaining interviews could be conducted. Emmanuel Mahama, Evaluate’s Senior Evaluation Advisor, carried out quality control visits in the Northern Region during the first week of field work. He made spot checks and observations on selected questions from the training beneficiary key informant interview guide to check respondents’ answers in certain CHPS zones (Kasulyili CHPS) in the Tolon district of the Northern region, Kapbya Health Centre in the Mion district which was formally a CHPS, Jimli CHPS, Yendi Municipal Health Directorate, Kuni CHPS and Gushegu District Health Directorate. His checks indicated that KIIs were actually conducted and sampled questions from KIIs were consistent with what was captured on the corresponding audio recordings for those respondents. Key observations made were that most health care workers who had benefited from USAID supported trainings in the district for the past two years had either been transferred to other health facilities within the same district or taken their study leave to further their education. Some facilities that were CHPS two years ago had been upgraded into health centers. Travel time to some districts due to bad road networks affected the expected number of interviews to be conducted by enumerators for the day especially for the Mion and Sambu districts. G. Data Analysis Approach Available primary and secondary data were triangulated to complement the quantitative data analysis. Quantitative Data: Respondents’ socio-demographic characteristics were collated to illustrate respondent and facility profiles. The environmental (community and organizational) factors influencing HCW’s work were collated and the mean score for each environmental factor was presented. Respondents’ knowledge scores on each training model were summed and mean scores presented to identify the average knowledge level of training participants on the topics they were trained. Data were analyzed with Statistical Package for Social Sciences version 23 software (SPSS) using descriptive statistics and the findings presented in tables and figures. Kruskal Wallis’ tests were conducted to identify statistically significant group differences between groups. All statistically significant calculations were measured at 95% confidence interval (p≤0.05). Regarding missing values, pairwise exclusion of cases was used for analyses. This means that cases were used for analyses if they had the relevant data required for that specific analysis. As a result, population sizes for individual analyses varied, but to a limited degree. Qualitative Data: Content analysis was used to analyze the qualitative data to supplement the statistical analysis across questions. A team approach was used to code and analyze the transcripts. Coding of data was done and the data was managed manually to identify relevant quotations to support findings. The themes generated were presented in order to answer the research questions. The qualitative data provided the necessary context to the quantitative data and generated a better understanding of the outcomes of the trainings. Verbatim quotations were used in the report to provide evidence and context to the findings. In general, since trainings have already been delivered, there were limitations to the overall findings which cannot be directly linked to the training since no baseline/pre-test was possible. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 54 Table 7: GETTING TO ANSWERS Evaluation Question Data Source Data Collection Methods Data Analysis methods 1. What models of training (e.g. on-site, off-site, e￾learning, etc.) focusing on EPCMD appear to most effectively contribute to increases in CHO and midwives knowledge and skills? a. How effective are these trainings directly after vs. long-term? Secondary data: - IP participant lists - IP training pre- and post-tests - Supportive supervision documentation - DHIMS2 and CHPS data - IP curriculum Primary data: - Training participants; Trainers; GHS personnel; Community members; Project staff Secondary data: -Desk review Primary data: -KII -GD - Survey -Knowledge/skill assessment - Comparisons - Pattern / Content Analysis - Descriptive/ Inferential Analysis 2. Are the trainings conducted for CHO and midwives relevant to their needs and being utilized for health care delivery? a. To what extent are the knowledge and skills received at training being shared with other health staff at the CHPS? Secondary data: - IP participant lists - Supportive supervision documentation - DHIMS2 and CHPS data Primary data: - Training participants; Participant coworkers; Trainers; GHS personnel; Community members Secondary data: -Desk review (disease incidences) Primary data: - KII - GD - Survey - Pattern / Content Analysis - Descriptive/ Inferential Analysis 3. What models of training appear to be the most scalable and sustainable across GHS? a. Have the training of trainer (TOT) activities been effective in the terms of providing long￾term support to GHS? Secondary data: - DHIMS2 and CHPS data - IP Budgets/cost - IP reports - Literature review Primary data: - Trainers; GHS personnel; Community members; Project staff Secondary data: -Desk review Primary data: -KII -GD -Comparisons -Pattern / Content Analysis -Descriptive/ Inferential Analysis 4. What are other training successes that have been documented in Ghana or other relevant countries? a. What is the return on investment for these trainings? Secondary data: - Relevant studies and reports Secondary data: -literature review - Pattern / Content Analysis OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 55 Annex C: Additional Tables Table 8. District and Regional Distribution of Health Facilities Sampled REGION DISTRICT NUMBER OF HEALTH FACILITIES SAMPLED Northern Mion 8 Yendi 5 Gushegu 8 Tolon 6 Tamale 1 Total 28 Ashanti Bosome Freho 3 Offinso-North 4 Offinso Municipal 2 Ejisu-Juabeng 3 Sekyere-Afram Plains 3 Total 15 Greater Accra Ada East 8 Ga South 5 Total 13 Volta Nkwanta North 3 Krachi West 3 Hohoe 2 Biakoye (Worawora) 1 Keta 6 Ketu South 8 Ho 2 Central Tongu 1 Total 26 GRAND TOTAL 82 Table 9. Distribution of Respondents According To Their Occupation DESCRIPTION OCCUPATION CATEGORY FREQUENCY PERCENT Clinician Doctor Medical Doctor Medical Practitioner Physician (Clinician) Physician Assistant (Medical) Clinicians 11 10.1 Nurse, Nurse (Staff) Community Health Nurse Enrolled Nurse Medical Nurse Practitioner Nurse Practitioner Nursing Public Health Nursing Nursing 55 50.5 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 56 Midwife Midwifery Midwifery 16 14.7 Nutrition Technical Officer, Nutrition Officer Nutrition Officer (Public Health Nutritionist) Nutrition Officers 5 4.6 Disease Control Officer, Health Promotion Officer, Health Promotion Specialist, Health Information Manager Public Health Officer Technical Officer (Health Information Officer) Public Health/ Health Promotion Officers 10 9.2 District Director of Health Services Medical Director Health Administrators 2 1.8 Medical Laboratory Science Medical Laboratory Scientist Laboratory Staff 2 1.8 Pharmacist Pharmacists 2 1.8 Health Worker Health Worker 2 1.8 Student Nurse Student Nurse 2 1.8 Tutor Tutor 2 1.8 TOTAL 109 100 Table 10. How Often Respondent Attends Training How often respondents attend training Frequency Percent 3 times in 2016 1 1 3-4 times in 2016 1 1 As and when called 1 1 As and when training in organized and was called to attend 1 1 At least four times 1 1 Cannot remember 1 1 Couple of times 1 1 Couple of times a year 35 36.5 Couple of times a year (4 Times) 1 1 Currently less than one a year 1 1 Every 2 to 3 month, 6-7 times a year 1 1 Four times a year 1 1 Irregularly 1 1 Just once in 4 years 1 1 Less than once a year 1 1 Once 1 1 Once a month 1 1 Once a year 25 26 Once a year at times 1 1 Once in two years 1 1 Once or twice a year 1 1 Only once 1 1 Thrice a year 7 7.3 Twice a year 8 8.3 Very Often, Malaria CHO training 1 1 Total 96 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 57 Table 11. Respondents’ Responses to Environmental Factors Influencing the Work of HCWs Environmental Factor Frequency Percentage M(SD) 1 Community members seek health services from the CHOs/CHNs Strongly Agree 63 62.4 1.38(.487) Agree 38 37.6 Disagree 0 0 Strongly Disagree 0 0 Not Applicable 0 0 Total 101 100 2 I am accepted as a CHO/CHN working in the community Strongly Agree 80 76.9 1.2(.451) Agree 23 22.1 Disagree 1 1 Strongly Disagree 0 0 Not Applicable 0 0 Total 104 100 3 The community leaders provide support to the CHOs/CHNs or health program (e.g. durbar) Strongly Agree 41 39.4 1.78(.836) Agree 52 50 Disagree 6 5.8 Strongly Disagree 3 2.9 Not Applicable 2 1.9 Total 104 100 4 Women need spousal permission to access health services Strongly Agree 27 26 2.31(1.025) Agree 33 31.7 Disagree 30 28.8 Strongly Disagree 13 12.5 Not Applicable 1 1 Total 104 100 5 There is stigma toward controversial health topics, e.g. family planning Strongly Agree 52 50.5 1.61(.731) Agree 42 40.8 Disagree 6 5.8 Strongly Disagree 3 2.9 Not Applicable 0 0 Total 103 100 6 I always feel safe within the community and at work Strongly Agree 50 49 1.65(.74) Agree 40 39.2 Disagree 10 9.8 Strongly Disagree 2 2 Not Applicable 0 0 Total 102 100 7 The community's roads are always accessible / passable Strongly Agree 7 6.7 2.95(.918) Agree 24 23.1 Disagree 41 39.4 Strongly Disagree 31 29.8 Not Applicable 1 1 Total 104 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 58 Table 10. Respondents’ Responses to Environmental Factors Influencing the Work of HCWs (cont’d) Environmental Factor Frequency Percentage M(SD) 8 Access to transportation is always available. Strongly Agree 9 8.7 2.92(.947) Agree 22 21.4 Disagree 41 39.8 Strongly Disagree 30 29.1 Not Applicable 1 1 Total 103 100 9 The transportation system is always suitable for clients who seek health services Strongly Agree 4 3.8 3.12(.780) Agree 13 12.5 Disagree 55 52.9 Strongly Disagree 31 29.8 Not Applicable 1 1 Total 104 100 10 The community's political/chieftaincy stability is conducive for work Strongly Agree 30 28.8 2.09(1.006) Agree 49 47.1 Disagree 15 14.4 Strongly Disagree 6 5.8 Not Applicable 4 3.8 Total 104 100 11 The CHPS compound has reliable electricity Strongly Agree 29 27.9 2.29(1.094) Agree 35 33.7 Disagree 24 23.1 Strongly Disagree 13 12.5 Not Applicable 3 2.9 Total 104 100 12 The CHPS compound has potable water Strongly Agree 24 23.1 2.64(1.184) Agree 24 23.1 Disagree 26 25 Strongly Disagree 27 26 Not Applicable 3 2.9 Total 104 100 13 The CHPS compound has reliable phone signals/reception Strongly Agree 14 13.6 2.63(1) Agree 33 32 Disagree 35 34 Strongly Disagree 19 18.4 Not Applicable 2 1.9 Total 103 100 14 The CHPS compound has reliable/functional Internet Strongly Agree 4 3.9 3.40(1.023) Agree 15 14.6 Disagree 34 33 Strongly Disagree 36 35 Not Applicable 14 13.6 Total 103 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 59 Table 10. Respondents’ Responses to Environmental Factors Influencing the Work of HCWs (cont’d) Environmental Factor Frequency Percent M(SD) 15 The CHPS buildings are in good condition Strongly Agree 15 14.6 2.72(1.088) Agree 30 29.1 Disagree 31 30.1 Strongly Disagree 23 22.3 Not Applicable 4 3.9 Total 103 100 16 I always get the requisite support/supervision/motivati on/mentoring from management Strongly Agree 28 26.9 1.88(.692) Agree 64 61.5 Disagree 10 9.6 Strongly Disagree 1 1 Not Applicable 1 1 Total 104 100 17 My challenges are always resolved by management when required Strongly Agree 10 9.6 2.38(.74) Agree 50 48.1 Disagree 40 38.5 Strongly Disagree 3 2.9 Not Applicable 1 1 Total 104 100 18 I have adequate staff to perform my duties effectively Strongly Agree 9 8.7 2.50(.862) Agree 48 46.6 Disagree 32 31.1 Strongly Disagree 13 12.6 Not Applicable 1 1 Total 103 100 19 My Remunerations/Salaries/ Allowances are adequate and timely Strongly Agree 12 11.7 2.61(.983) Agree 38 36.9 Disagree 34 33 Strongly Disagree 16 15.5 Not Applicable 3 2.9 Total 103 100 20 My duties as a CHO/CHN are well-defined in relation to other health care providers in the community Strongly Agree 54 51.9 1.63(.872) Agree 43 41.3 Disagree 3 2.9 Strongly Disagree 0 0 Not Applicable 4 3.8 Total 104 100 21 I am able to make decisions as required for work on my own Strongly Agree 40 38.8 1.72(.733) Agree 57 55.3 Disagree 2 1.9 Strongly Disagree 3 2.9 Not Applicable 1 1 Total 103 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 60 Table 10. Respondents’ Responses to Environmental Factors Influencing the Work of HCWs (cont’d) Environmental Factor Frequency Percentage M(SD) 22 The chain of command in my work environment enhances the work I do Strongly Agree 34 33 1.73(.613) Agree 65 63.1 Disagree 3 2.9 Strongly Disagree 0 0 Not Applicable 1 1 Total 103 100 23 The medicines and vaccines required for effective work are always available Strongly Agree 14 13.5 2.43(.879) Agree 43 41.3 Disagree 36 34.6 Strongly Disagree 10 9.6 Not Applicable 1 1 Total 104 100 24 The equipment and supplies/logistics required for effective work are always available Strongly Agree 7 6.9 2.62(.809) Agree 39 38.2 Disagree 42 41.2 Strongly Disagree 14 13.7 Not Applicable 0 0 Total 102 100 25 The records systems are adequate for effective work Strongly Agree 25 24.3 2.16(.947) Agree 48 46.6 Disagree 22 21.4 Strongly Disagree 5 4.9 Not Applicable 3 2.9 Total 103 100 26 I have adequate time to attend to my clients Strongly Agree 39 37.5 Agree 50 48.1 Disagree 9 8.7 Strongly Disagree 3 2.9 Not Applicable 3 2.9 Total 104 100 27 The referral system is suitable for effective work Strongly Agree 23 22.1 2.36(1.023) Agree 37 35.6 Disagree 31 29.8 Strongly Disagree 10 9.6 Not Applicable 3 2.9 Total 104 100 28 The treatment guidelines for practice are available to facilitate my work Strongly Agree 36 35 1.89(.896) Agree 50 48.5 Disagree 12 11.7 Strongly Disagree 2 1.9 Not Applicable 3 2.9 Total 103 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 61 Table 12. Knowledge Scores Obtained Per Knowledge Questionnaire Answered Traditional IYCF n (%) Traditional IMNCI n(%) TOT Anemia n(%) TOT IYCF n(%) Internship Anemia, Malnutrition, Malaria n(%) Internship Malaria n(%) S.S Malaria n(%) SS IYCF n(%) Total n(%) Knowledge Question_1 Wrong 0 3(14.3) 0 13(92.9) 0 0 1(10) 0 17(27) Correct 4(100) 18(85.7) 3(100) 1(7.1) 2(100) 5(100) 9(90) 4(100) 46(73) Total 4(100) 21(100) 3(100) 14(100) 2(100) 5(100) 10(100) 4(100) 63(100) Knowledge Question_2 Wrong 0 2(9.5) 1(33.3) 14(100) 2(100) 0 2(20) 1(25) 22(35.5) Correct 4(100) 19(90.5) 2(66.7) 0 0 4(100) 8(80) 3(75) 40(64.5) Total 4(100) 21(100) 3(100) 14(100) 2(100) 4(100) 10(100) 4(100) 62(100) Knowledge Question_3 Wrong 0 0 0 0 0 1(20) 0 0 1(1.6) Correct 4(100) 22(100) 3(100) 13(100) 2(100) 4(80) 10(100) 4(100) 62(98.4) Total 4(100) 22(100) 3(100) 13(100) 2(100) 5(100) 10(100) 4(100) 63(100) Knowledge Question_4 Wrong 0 0 0 1(8.3) 0 0 1(11.1) 0 2(3.3) Correct 4(100) 22(100) 3(100) 11(91.7) 2(100) 4(100) 8(88.9) 4(100) 58(96.7) Total 4(100) 22(100) 3(100) 12(100) 2(100) 4(100) 9(100) 4(100) 60(100) Knowledge Question_5 Wrong 0 4(18.2) 0 1(7.1) 0 0 0 0 5(7.9) Correct 4(100) 18(81.8) 3(100) 13(92.9) 2(100) 4(100) 10(100) 4(100) 58(92.1) Total 4(100) 22(100) 3(100) 14(100) 2(100) 4(100) 10(100) 4(100) 63(100) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT JUNE 2018 62 TABLE 11. Knowledge Scores Obtained Per Knowledge Questionnaire Answered (cont’d) Traditional IYCF n(%) Traditional IMNCI n(%) TOT (SPRING) Anemia n(%) TOT IYCF n(%) Internship Anemia, Malnutrition, Malaria n(%) Internship Malaria n(%) S.S Malaria n(%) SS IYCF n(%) Total n(%) Knowledge Question_6 Wrong 0 1(4.5) 0 1(7.1) 1(100) 0 0 1(25) 4(6.5) Correct 4(100) 21(95.5) 3(100) 13(92.9) 0 4(100) 10(100) 3(75) 58(93.5) Total 4(100) 22(100) 3(100) 14(100) 1(100) 4(100) 10(100) 4(100) 62(100) Knowledge Question_7 Wrong 0 2(9.1) 0 0 0 0 2(22.2) 2(50) 6(10) Correct 4(100) 20(90.9) 3(100) 13(100) 2(100) 3(100) 7(77.8) 2(50) 54(90) Total 4(100) 22(100) 3(100) 13(100) 2(100) 3(100) 9(100) 4(100) 60(100) Knowledge Question_8 Wrong 0 1(4.5) 0 0 0 0 3(42.9) 0 4(6.9) Correct 4(100) 21(95.5) 3(100) 13(100) 1(100) 4(100) 4(57.1) 4(100) 54(93.1) Total 4(100) 22(100) 3(100) 13(100) 1(100) 4(100) 7(100) 4(100) 58(100) Knowledge Question_9 Wrong 0 0 0 3(23.1) 0 0 0 0 3(4.9) Correct 4(100) 22(100) 3(100) 10(76.9) 2(100) 4(100) 9(100) 4(100) 58(95.1) Total 4(100) 22(100) 3(100) 13(100) 2(100) 4(100) 9(100) 4(100) 61(100) Knowledge Question_10 Wrong 0 0 0 0 0 0 2(28.6) 0 2(3.4) Correct 4(100) 22(100) 3(100) 13(100) 1(100) 4(100) 5(71.4) 4(100) 56(96.6) Total 4(100) 22(100) 3(100) 13(100) 1(100) 4(100) 7(100) 4(100) 58(100) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 63 Table 13: Total Knowledge Score Obtained By Respondents Variable N Minimum Maximum M SD Total Knowledge Score 41 6 10 8.9 1.241 Table 14. Distribution of Trainee Knowledge Mean Scores Based on Questionnaire Answered Questionnaire Answered N Mean SD Min Max Traditional IYCF 4 10 0 10 10 Traditional IMNCI 14 9.43 0.852 7 10 TOT Anemia 3 9.67 0.577 9 10 TOT IYCF 9 7.44 0.882 6 8 Internship Anemia, Malnutrition, Malaria 1 8 0 8 8 Internship Malaria 1 10 0 10 10 S.S Malaria 5 8.6 1.673 6 10 SS IYCF 4 9 0.816 8 10 Table 15. Distribution of Trainee Knowledge Mean Score Based On Training Models Training Model N Mean SD Min Max Traditional 18 9.56 0.784 7 10 Internship 2 9 1.414 8 10 Supportive Supervision 9 8.78 1.302 6 10 Training of Trainer 12 8 1.279 6 10 Table 16. Respondents’ Rating of Training Models Training model Rating Frequency Percent Mean(SD) Traditional Poor 0 0 4.22(.681) Marginal 1 2.8 Satisfactory 2 5.6 Good 21 58.3 Excellent 12 33.3 Total 36 100 Supportive Supervision Poor 0 0 4.34(.614) Marginal 0 0 Satisfactory 2 6.9 Good 15 51.7 Excellent 12 41.4 Total 29 100 Training of Trainer Poor 0 0 4.31(.731) Marginal 1 2.6 Satisfactory 3 7.7 Good 18 46.2 Excellent 17 43.6 Total 39 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 64 Mentoring Poor 0 0 4.13(.640) Marginal 0 0 Satisfactory 2 13.3 Good 9 60 Excellent 4 26.7 Total 15 100 Internship Poor 0 0 4.05(.524) Marginal 0 0 Satisfactory 2 10.5 Good 14 73.7 Excellent 3 15.8 Total 19 100 E-Learning Poor 0 0 4.25(.500) Marginal 0 0 Satisfactory 0 0 Good 3 75 Excellent 1 25 Total 4 100 Table 17. Respondents’ Rating of Training Implementing Partner Training model Rating Frequency Percent Mean(SD) Traditional Poor 0 0 4.29 (.611) Marginal 0 0 Satisfactory 3 7.9 Good 21 55.3 Excellent 14 36.8 Total 38 100 Supportive Supervision Poor 0 0 4.27 (.640) Marginal 0 0 Satisfactory 3 10 Good 16 53.3 Excellent 11 36.7 Total 30 100 Training of Trainer Poor 0 0 4.38(.594) Marginal 0 0 Satisfactory 2 5.4 Good 19 51.4 Excellent 16 43.2 Total 37 100 Mentoring Poor 0 0 4.29(.726) Marginal 0 0 Satisfactory 2 14.3 Good 6 42.9 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 65 Excellent 6 42.9 Total 14 100 Internship Poor 0 0 4.18(.809) Marginal 1 5.9 Satisfactory 1 5.9 Good 9 52.9 Excellent 6 35.3 Total 17 100 E-Learning Poor 0 0 4.33(.577) Marginal 0 0 Satisfactory 0 0 Good 2 66.7 Excellent 1 33.3 Total 3 100 Table 18. Respondents’ Rating of Training Audience Training model Rating Frequency Percent Mean(SD) Traditional Poor 0 0 4.05(.733) Marginal 0 0 Satisfactory 9 23.7 Good 18 47.4 Excellent 11 28.9 Total 38 100 Supportive Supervision Poor 0 0 4.36(.678) Marginal 0 0 Satisfactory 3 10.7 Good 12 42.9 Excellent 13 46.4 Total 28 100 Training of Trainer Poor 0 0 4.32(.525) Marginal 0 0 Satisfactory 1 2.6 Good 24 63.2 Excellent 13 34.2 Total 38 100 Mentoring Poor 0 0 4.27(.786) Marginal 0 0 Satisfactory 2 18.2 Good 4 36.4 Excellent 5 45.5 Total 11 100 Internship Poor 0 0 4.12(.697) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 66 Marginal 0 0 Satisfactory 3 17.6 Good 9 52.9 Excellent 5 29.4 Total 17 100 E-Learning Poor 0 0 4.33(.577) Marginal 0 0 Satisfactory 0 0 Good 2 66.7 Excellent 1 33.3 Total 3 100 Table 19. Respondents’ Rating of Training Topics Training model Rating Frequency Percent Mean(SD) Traditional Poor 0 0 4.25(6.92) Marginal 0 0 Satisfactory 5 13.9 Good 17 47.2 Excellent 14 38.9 Total 36 100 Supportive Supervision Poor 0 0 4.23(.626) Marginal 0 0 Satisfactory 3 10 Good 17 56.7 Excellent 10 33.3 Total 30 100 Training of Trainer Poor 0 0 4.54(.650) Marginal 0 0 Satisfactory 3 8.1 Good 11 29.7 Excellent 23 62.2 Total 37 100 Mentoring Poor 0 0 4.58(.515) Marginal 0 0 Satisfactory 0 0 Good 5 41.7 Excellent 7 58.3 Total 12 100 Internship Poor 0 0 4.56(.629) Marginal 0 0 Satisfactory 1 6.3 Good 5 31.3 Excellent 10 62.5 Total 16 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 67 E-Learning Poor 0 0 4.33(.577) Marginal 0 0 Satisfactory 0 0 Good 2 66.7 Excellent 1 33.3 Total 3 100 Table 20 : Respondents’ Rating of Training Duration Training model Rating Frequency Percent Mean(SD) Traditional Poor 0 0 3.37(.913) Marginal 8 21.1 Satisfactory 11 28.9 Good 16 42.1 Excellent 3 7.9 Total 38 100 Supportive Supervision Poor 0 0 3.73(.583) Marginal 0 0 Satisfactory 10 33.3 Good 18 60 Excellent 2 6.7 Total 30 100 Training of Trainer Poor 0 0 3.92(.850) Marginal 0 0 Satisfactory 15 39.5 Good 11 28.9 Excellent 12 31.6 Total 38 100 Mentoring Poor 0 0 3.33(.985) Marginal 3 25 Satisfactory 3 25 Good 5 41.7 Excellent 1 8.3 Total 12 100 Internship Poor 0 0 3.76(.831) Marginal 1 5.9 Satisfactory 5 29.4 Good 8 47.1 Excellent 3 17.6 Total 17 100 E-Learning Poor 0 0 4.33(.577) Marginal 0 0 Satisfactory 0 0 Good 2 66.7 Excellent 1 33.3 Total 3 100 OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 68 Annex D: Quantitative Tests Conducted Each training model’s participants were assessed with knowledge and skills assessment questionnaires based on the topics they were trained on. Total Knowledge Score (n = 41, M = 8.9, SD = 1.241) is a summation of all correct scores obtained by respondents who answered the 10-item Knowledge assessment questionnaire. Each respondent answered only 1 of the 8 knowledge assessment questionnaires based on the topics they were trained on as well as the training model used in training them. Every correct answer was scored 1 for all the knowledge questionnaire and a wrong answer was scored 0, the Total Knowledge Scores ranged from a possible minimum of 0 to a maximum score of 10. Higher Total Knowledge scores indicated a higher level of Knowledge on the topics respondent were trained. The average score obtained by respondents was 8.9 out of 10. Mentoring and E-Learning trainees did not answer knowledge assessment questionnaires. Annex E: Data Collection Instruments DEMOGRAPHIC DATA OF PARTICIPANTS (ALL) 1. Participant's ID: ……………………………Name:…………………………………….... 2. Participant's Health Facility…………………………………………………………………… 3. Region…………………………………… District…………………………………………… 4. Gender (a) Male (b) Female 5. Age: 18 – 25 26 – 30 31 -35 36 – 40 41 – 45 46 – 50 Above 51 6. Marital status (a) Married (b) Co-habiting/Living together (c) Single (d) Divorced (e) Separated (f) Widowed (g) Never Married 7. Place of residence ………………………………………………………………………...……… 8. Occupation ……………………………………………………………………………………….. 9. Job Title:…………………………………………………………………………………………… 10. How long respondent has been working in his/her current position ……………. 11. Date of relevant USAID training program (TOT, internship, SS, Traditional, etc.) started: …….…………………………………………………… 12. Date USAID training ended (if applicable): ………………………………………………… 13. What other relevant training have you participated in (either USAID or not) in the last 2 years? _________________________________________________________ 14. About how often do you attend trainings? (E.g. once a year, couple times a year, etc.) ___________________________________________________________ 15. Interviewer: ____________________________________________________ 16. Interview Date: ________________________________ 17. Time started: __________ Time ended: ________________ General Instructions: Most of the interview questions below have probes that are follow-up questions to ask if the interviewee has not provided the information requested in the basic question. If the answer was provided, please skip that probe and go to the next one. Please probe for a deeper understanding when the interviewee provides information that is not clear. Please also probe for additional detail when you find an answer especially interesting in the context of the study. In this event, please ask the respondent to expand on his/her answer. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 69 Overview of Study for introductory communications to respondents pre-interview (KII only): The focus of this study is to identify the most effective, scalable and sustainable training models for CHPS-zone staff (CHOs, CHNs and, to a more limited degree, midwives, referred to collectively as community-based health workers- HCWs) among six models of training currently being implemented by GHS with support from USAID/Ghana. The priority areas of training evaluated are Ending Preventable Child and Maternal Deaths (EPCMD) and malaria treatment. The study examines training implemented with support from four USAID/Ghana implementing partners: Systems for Health, Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING); MalariaCare; and Maternal Child Survival Program (MCSP) over the last two years. The six models include: • Traditional training: residential or non-residential workshop-style training delivered usually over 3-5 days by a technical specialist in specific topics related to the functions of the target CHPS zone personnel • TOT: training of trainers on the district or subdistrict level who are expected to carry out follow-up training for the target CHPS zone personnel • Supportive supervision- on site supervision from an experienced HCW or supervisor to provide feedback and correction on skills/protocols application (known as OTSS – on the job training and supportive supervision- for MalariaCare) • Internships- 1 week assignments of HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff • Mentoring- clustering of CHO/CHNs who meet as a group with a licensed midwife for advice and instruction on EPCMD • e-learning- training modules accessed on-line for instruction in nursing topics, including post￾tests to validate knowledge acquisition At the interview- read the following to the participants (all types): Thank you very much for agreeing to participate in this interview. My name is ______. I work for Evaluate for Health, a USAID/Ghana project. We are studying the effectiveness of six models of community-based health worker (HCW) training and the environmental factors that contribute to HCW performance in training and on the job. The categories of trainees included in this study are Community Health Officers (CHOs), Community Health Nurses (CHNs) and midwives assigned to CHPS zones. Our questions are specific to training carried out in the last two years, 2016 and 2017. The study is intended to assist in informing training improvements for HCWs in Ghana. We are very grateful for you for agreeing to participate in the study. As someone who has organized or participated in one of these training, we would like to ask your views on your experience with the selected training models, specifically what works and doesn’t work, and what improvements could be made in HCW training to ensure effective and sustainable results in terms of knowledge and skills acquisition, retention and application on the job. We would also appreciate your views on the external factors that influence skills acquisition and application. The interview will probably take between 45-60 minutes. We will digitally-record the session, but your responses will remain anonymous and no names will be mentioned in any reports. We hope that you will speak openly and honestly about your experiences and viewpoints. Please give us as much detail as you are willing and able to provide, and feel free to ask any questions, or tell us if you do not understand a question being asked. I will be taking notes during the interview to flag your key points. This will help us later when we go back and organize all the information that we received today. Do you have any questions or concerns before we begin? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 70 A. Qualitative Interview Guides GENERAL INTRODUCTION FOR ALL INDIVIDUAL KIIs Basic Information Interviewer: _________________________________________________________________ Interview Date: ________________________________ Organization/Location: __________________________ Region: ______________________________________ District: __________ (if applicable) Time started: __________ Time ended: ____________ Participant ID_____________ General Instructions: Most of the interview questions below have probes that are follow-up questions to ask if the interviewee has not provided the information requested in the basic question. If the answer was provided, please skip that probe and go to the next one. Please probe for a deeper understanding when the interviewee provides information that is not clear. Please also probe for additional detail when you find an answer especially interesting in the context of the study. In this event, please ask the respondent to expand on his/her answer. Overview of Study for respondents (to be sent to interviewees by email in advance of the interview): The focus of this study is to identify the most effective, scalable and sustainable training models for CHPS-zone staff (CHOs, CHNs and, to a more limited degree, midwives, referred to collectively as community-based health workers- HCWs) among six models of training currently being implemented by GHS with support from USAID/Ghana. The priority areas of training evaluated are Ending Preventable Child and Maternal Deaths (EPCMD) and malaria treatment. The study examines training implemented with support from four USAID/Ghana implementing partners: Systems for Health, Strengthening Partnerships, Results, and Innovations in Nutrition Globally (SPRING); MalariaCare; and Maternal Child Survival Program (MCSP). The six models include: • Traditional training: residential or non-residential workshop-style training delivered usually over 3-5 days by a technical specialist in specific topics related to the functions of the target CHPS zone personnel • TOT: training of trainers on the district or subdistrict level who are expected to carry out follow-up training for the target CHPS zone personnel • Supportive supervision- on site supervision from an experienced HCW or supervisor to provide feedback and correction on skills/protocols application (known as OTSS – on the job training and supportive supervision- for MalariaCare) • Internships- 1 week assignments of HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff • Mentoring- clustering of CHO/CHNs who meet as a group with a licensed midwife for advice and instruction on EPCMD • e-learning- training modules accessed on-line for instruction in nursing topics, including post￾tests to validate knowledge acquisition The study also examines environmental factors that contribute to HCWs’ performance in training and on the job. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 71 As a stakeholder who has funded, implemented and/or managed HCW training, we would like to ask your views on your experience with the selected training models, specifically what works and doesn’t work, and what improvements could be made in HCW training to ensure effective and sustainable results in terms of knowledge and skills acquisition, retention and application on the job. We would also appreciate your views on the external factors that influence skills acquisition and application. At the interview- read the following to the participants: Thank you very much for agreeing to participate in this interview. My name is ______. I work for Evaluate for Health, a USAID/Ghana project. As explained in the introductory email addressed to you, we are studying the effectiveness of six models of community-based health care worker (HCW) training and the environmental factors that contribute to HCW performance in training and on the job. The categories of trainees included in this study are Community Health Officers (CHOs), Community Health Nurses (CHNs) and midwives assigned to CHPS zones. Our questions are specific to training carried out in the last two years, 2016 and 2017. The study is intended to assist in informing training improvements for HCWs in Ghana. We are very grateful to (name the specific organization) for agreeing to participate in the study. The interview will probably take between 45-60 minutes. We will digitally-record the session, but your responses will remain anonymous and no names will be mentioned in any reports. We hope that you will speak openly and honestly about your experiences and viewpoints. Please give us as much detail as you are willing and able to provide, and feel free to ask any questions, or tell us if you do not understand a question being asked. I will be taking notes during the interview to flag your key points. This will help us later when we go back and organize all the information that we received today. Do you have any questions or concerns before we begin? KII INTERVIEW GUIDE FOR USAID/Ghana HEALTH POPULATION AND NUTRITION (HPNO) PERSONNEL General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training programs in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of HCW training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs’ ability to deliver community-based health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. How does USAID evaluate the training programs they fund? • Is this process used systematically by IP/GHS? If not, why not? • What actions are taken if expected results are not achieved? 6. What do you view as the biggest challenge for implementing quality USAID-funded HCW training? 7. In the last two years, has USAID/Ghana made any significant changes to its training strategy or implementation? If yes, please describe. 8. What lessons has USAID/Ghana learned from its HCW training over the past 2 years? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 72 9. What are some examples of best practices in HCW training from your experience? Training models: 10. Per the list below, what models of training and related topics, audiences and duration are you personally familiar with over the last two years (2016-2017)? Please rate each training model on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating Type I P Training Audience Topic Duration Rating/Why Training classroom training TOT Supportive Supervision Internships Mentoring e-learning 11. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 12. What do you think about the scalability of the different models of training? Are certain types more scalable than others? Please explain why or why not. 13. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR GHS DIRECTORS (ICD AND FHD RECOMMENDED) PLUS TRAINING SUPERVISOR, HUMAN RESOURCES DIVISION General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training programs in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of HCW training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to deliver community-level health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. What GHS guidelines/protocols apply to HCW training in your division? Please describe and indicate how often these are updated. 6. How does GHS evaluate its training programs? • Is this process used systematically by GHS trainers? • What actions are taken if expected results are not achieved? 7. What do you think is the biggest challenge for GHS when it comes to delivering training? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 73 8. In the last two years, has GHS made any significant changes to its training strategy or implementation? If yes, please describe. 9. What lessons has GHS learned from its training over the past 2 years? 10. What are examples of best practices in GHS training from your experience? Training models: 11. Per the list below, what models of training and the related topics, audiences and duration are you personally familiar with over the last two years (2016-2017)? Please rate each training model on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating Type Training Audience Main Topics Average Duration Rating/Why Training classroom training TOT Supportive Supervision Internships Mentoring e-learning 12. Do you have any observations regarding the relative costs of these different training models￾e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 13. What do you think about the scalability of the different models of training? Are certain types more scalable than others? Please explain why or why not? 14. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR DISTRICT PUBLIC HEALTH NURSES AND DISTRICT DIRECTORS General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in your District? 2. What is your overall impression of the effectiveness of the current HCW training in your district? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. Which donors other than USAID/Ghana provide training support in your district? • On what topics? 5. What external factors seem to impact HCWs ability to deliver health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation • How do the people in the community relate to your HCWs? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 74 i. Describe any cultural practices that might conflict with training content. ii. What support do HCWs receive from the community members or leaders for training and for their day to day jobs? iii. How well are HCWs accepted as Health workers in the community? • How does the physical environment of the community influence the work of the HCWs￾e.g. roads, transportation, electricity, water supply, phone/internet, 6. How do you evaluate training programs? • What actions do you take or recommend be taken if expected results are not achieved? 7. What do you think is the biggest challenge in implementing quality HCW training in your district? 8. In the last two years, has the district made any significant changes to its training strategy or implementation? If yes, please describe. 9. What lessons have you learned from the training in your district over the past 2 years? 10. What are examples of best practices in GHS training from your experience? Training models 11. Per the list below, what models of training and the related topics, audiences and duration are you personally familiar with over the last two years (2016-2017)? Please rate each training model on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating Type USAID/ Ghana IP Training Audience Main Topics Average Duration Rating/Why Training classroom training TOT Supportive Supervision Internships Mentoring e-learning 12. What role did you play to facilitate the trainings offered by the USAID/Ghana Implementing Partners you engaged with? 13. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 14. What do you think about the scalability of the different models of training? Are different types more scalable than others? Please explain why or why not. 15. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR IMPLEMENTING PARTNER - STRENGTHENING PARTNERSHIPS, RESULTS, AND INNOVATIONS IN NUTRITION GLOBALLY (SPRING) Relevant Training for SPRING: • Traditional training (residential or non-residential workshop-style training delivered by a technical specialist in specific topics related to the functions of the target CHPS zone personnel) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 75 • TOT (training of trainers on the district or subdistrict level who carry out follow-up training for the target CHPS zone personnel) • Supportive supervision- on site supervision from an experienced HCW or supervisor to provide feedback and correction on skills/protocols application (known as OTSS – on the job training and supportive supervision- for MalariaCare) General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to be able to delivery health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference):: community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. How did SPRING evaluate the training programs it funded? • Were the evaluation tools used systematically by SPRING? If not, why? • If yes, what if any actions were taken if expected results were not achieved? 6. What do you view as the biggest challenge in delivering quality training to HCWs? 7. How were trainees selected for participation in your training? Were the right trainees selected? 8. In general, what post-training follow-up was provided after training? 9. What lessons did you learn from SPRING training over the past 2 years? 10. What are examples of best practices in training from your experience? Training models: 11. Per the list below, please describe the SPRING-implemented programs and the related topics, audiences and duration over the last two years (2016-2017)? Please rate each training model you utilized on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating. We will then ask specific questions on each training type. Type IP Training Audience Topic Duration Rating/Why Training classroom training- IYCF TOT-Anemia Supportive Supervision￾IYCF 12. For each program, answer the following questions (IYCF Traditional Training/TOT Anemia and IYCF Supportive Supervision): a. How was the content/method selected? b. How well adapted was this content to the needs of the trainees in terms of their duties? c. How was the content taught/delivered to trainees in a typical training session? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 76 d. What guidelines do you use in designing the training program/supportive supervision? e. Was the program duration adequate? If not, why? f. How skilled were the instructors/supervisors? g. Did participant skills increase as a result of training? • How do you know? (give examples if possible) h. How was the program monitored? i. What use was made of monitoring data? j. Were the resources provided for the training adequate? k. Describe the support offered by GHS, the district and subdistrict for training and supervision. l. Overall, did the training program meet its objectives? Why or why not? m. What could make implementation of the training/supervision more effective? 13. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 14. What do you think about the scalability of the different models of training? Are different types more scalable than others? Please explain why or why not? 15. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR IMPLEMENTING PARTNER - SYSTEMS FOR HEALTH Relevant Training for SYSTEMS: • Traditional training (residential or non-residential workshop-style training delivered by a technical specialist in specific topics related to the functions of the target CHPS zone personnel) • TOT (training of trainers on the district or subdistrict level who carry out follow-up training for the target CHPS zone personnel) • Internships- 1 week assignments of HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff • Mentoring- clustering of CHO/CHNs who meet with a licensed midwife for advice and instruction on EPCMD General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to be able to delivery health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference):: community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. How does Systems for Health evaluate the training programs it funds? • Are any evaluation tools used systematically by Systems for Health? If not, why? • If yes, what if any actions were taken if expected results were not achieved? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 77 6. What do you view as the biggest challenge in delivering quality training to HCWs? 7. How were trainees selected for participation in your training? Were the right trainees selected? 8. In general, what post-training follow-up was provided after training? 9. What lessons did you learn from Systems or Health training over the past 2 years? 10. What are examples of best practices in training from your experience? Training models: 11. Per the list below, please describe the Systems or Health -implemented programs and the related topics, audiences and duration over the last two years (2016-2017)? Please rate each training model you utilized on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating. We will then ask specific questions on each training type. Type IP Training Audience Topic Duration Rating/Why Training classroom training- IMNCI TOT- ENC Internship Mentoring 12. For each program, answer the following questions (IMNCI Traditional Training, TOT ENC, inte4rnship and mentoring): a. How was the content/method selected? b. How well adapted was this content to the needs of the trainees in terms of their duties? c. How was the content taught/delivered to trainees in a typical training session? d. What guidelines do you include in designing the training program/supportive supervision? e. Was the program duration adequate? If not, why? f. How skilled were the instructors/supervisors? g. Did participant skills increase as a result of training? • How do you know? (give examples if possible) h. What can make the implementation of the training/supervision more effective? i. How was the program monitored? • What use was made of monitoring data? j. Were the resources provided for the training adequate? k. How relevant do you think the training was for the participants in their work? l. Describe the support offered by GHS, the district and subdistrict for training/supervision. m. Overall, did the training program meet its objectives? Why or why not? 13. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 14. What do you think about the scalability of the different models of training? Are different types more scalable than others? a. Please explain why or why not? 15. Do you have anything else you would like to add regarding the training you offer? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 78 KII GUIDE FOR IMPLEMENTING PARTNER - MATERNAL CHILD SURVIVAL PROGRAM (MCSP) Relevant Training for MCSP: • E-Learning- training modules accessed on-line for instruction in nursing topics, including post￾tests to validate knowledge acquisition General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to be able to delivery health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference):: community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. How did MCSP evaluate the training programs it funded? • Were the evaluation tools used systematically by MCSP? If not, why? • If yes, what if any actions were taken if expected results were not achieved? 6. What do you view as the biggest challenge in delivering quality training to HCWs? 7. How were trainees selected for participation in your training? Were the right trainees selected? 8. In general, what post-training follow-up was provided after training? 9. What lessons did you learn from MCSP training over the past 2 years? 10. What are examples of best practices in training from your experience? Training models: 11. Per the list below, please describe the MCSP-implemented programs and the related topics, audiences and duration over the last two years (2016-2017)? Please rate each training model you utilized on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating. Type Training Audience Topics Duration Rating/Why E-Learning 12. Please answer the following questions for the E-learning a. How was the content/method selected? b. How well adapted was this content to the needs of the trainees in terms of their duties? c. How was the content taught/delivered to trainees in a typical training session? d. What guidelines do you include in designing the training program/supportive supervision? e. Was the program duration adequate? If not, why? f. Did participant skills increase as a result of training? • How do you know? (give examples if possible) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 79 g. What can make the implementation of the training/supervision more effective? h. How was the program monitored? • What use was made of monitoring data? i. Were the resources provided for the training adequate? j. Describe the support offered by GHS, the district and subdistrict for training/supervision. k. Overall, did the training program meet its objectives? Why or why not? 13. How will you compare the E-learning training modules with other USAID-funded training in terms of the content and effectiveness? 14. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 15. What do you think about the scalability of the different models of training? Are different types more scalable than others? b. Please explain why or why not? 16. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR IMPLEMENTING PARTNER - MALARIACARE Relevant Training for MalariaCare: • Supportive supervision- on site supervision from an experienced HCW or supervisor to provide feedback and correction on skills/protocols application (known as OTSS – on the job training and supportive supervision- for MalariaCare) • Internships- 1 week assignments of HCWs to work at larger facilities, including district hospitals and subdistrict health centers, to observe and practice specific skills under the supervision of senior staff General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to be able to delivery health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference):: community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. How did MalariaCare evaluate the training programs it funded? • Were the evaluation tools used systematically by MalariaCare? If not, why? • If yes, what if any actions were taken if expected results were not achieved? 6. What do you view as the biggest challenge in delivering quality training to HCWs? 7. How were trainees selected for participation in your training? 8. In general, what post-training follow-up was provided after training? 9. What lessons did you learn from MalariaCare training over the past 2 years? 10. What are examples of best practices in training from your experience? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 80 Training models: 11. Per the list below, please describe the MalariaCare -implemented programs and the related topics, audiences and duration over the last two years (2016-2017)? Please rate each training model you utilized on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating. We will then ask specific questions on each training type. Type IP Training Audience Topic Duration Rating/Why Supportive Supervision/OTSS Internships - CHO 12. For each program, answer the following questions (Supportive Supervision and Internship): a. How was the content/method selected? b. How well adapted was this content to the needs of the trainees in terms of their duties? c. How was the content taught/delivered to trainees in a typical training session? d. What guidelines do you include in designing the training program/supportive supervision? e. Was the program duration adequate? If not, why? f. How skilled were the instructors/supervisors? g. Did participant skills increase as a result of training? • How do you know? (give examples if possible) h. How was the program monitored? • What use was made of monitoring data? i. Were the resources provided for the training adequate? j. How relevant do you think the training was for the participants in their work? k. Describe the support offered by GHS, the district and subdistrict for training/supervision. l. Overall, did the training program meet its objectives? Why or why not? m. What can make the implementation of the training/supervision more effective? 13. Do you have any observations regarding the relative costs of these different training models-e.g. are certain models more expensive to implement than others and, if so, is the additional investment justified? 14. What do you think about the scalability of the different models of training? Are different types more scalable than others? Please explain why or why not. 15. Do you have anything else you would like to add regarding the training you offer? KII GUIDE FOR PRINCIPALS OF SCHOOLS OF E-LEARNING General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact CHNs ability to be able to delivery health services in Ghana? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 81 • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation, community support. 5. What do you view as the biggest challenge in delivering quality training to HCWs? 6. How were trainees selected for participation in your training? 7. In general, what post-training follow-up was provided after training? 8. What lessons, if any, did you learn from the e-learning training over the past 2 years? 9. What are examples of best practices in training from your experience? 10. Do you think that e-learning is an effective model for HCW training in Ghana? Why or why not? Training models: 11. Please describe the MCSP-funded program (e-learning) and the related topics, audiences and duration over the last two years (2016-2017)? Please rate the training model you utilized on a scale of 1-5 (poor, marginal, satisfactory, good, or excellent) and explain briefly your rating. Type Training Audience Topics Duration Rating/Why E-Learning 12. Please answer the following questions for the E-learning a. What role(s) did you and your training institution play in the E-learning training? l. How was the content/method selected? m. How well adapted was this content to the needs of the trainees in terms of their duties? n. How was the content taught/delivered to trainees in a typical training session? a. Probe: was it required, set due dates, on own time, etc. o. What guidelines do you include in designing training program and any follow-on supportive supervision? p. Was the program duration adequate? If not, why? q. Did participant skills increase as a result of training? • How do you know? (give examples if possible) r. How relevant do you think the training was to participants’ work? s. What can make the implementation of the e-learning more effective? t. How was the program monitored? • What use was made of monitoring data? u. Were the funds/materials provided for the training adequate? v. Overall, did the training program meet its objectives? Why or why not? w. What do you think could have made the E-Learning training module better? a. What do you think can make the implementation of the E-Learning training in other areas easier? x. Which aspect(s) of the e-learning module do you think is (are) most useful for students to become more effective in their work? 13. How would you compare the e-learning training modules with other training types in terms of the content and effectiveness? 14. Do you have any observations regarding the relative costs of the e-learning training models versus other models of training teaching the same or similar content? a. Probe for issues: Quality (curriculum, instructors, and materials)? 15. What do you think about the scalability of e-learning training? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 82 c. Please explain why or why not? d. Should e- learning program be expanded to other schools? Why or why not? 16. Do you have anything else you would like to add regarding the e-learning training? KII GUIDE FOR GHS TRAINERS (TOT) General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to deliver health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. What do you think is the biggest challenge in implementing quality HCW training in your district? 6. In the last two years, has the district made any significant changes to its training strategy or implementation? If yes, please describe. 7. What lessons have you learned from training in your district over the past 2 years? 8. What are examples of best practices in GHS training from your experience? Training models 9. Did the TOT from USAID, provide you with the knowledge and skills to effectively deliver training on Armenia or ENC? Please explain. a. How was the content taught/delivered to trainees in a typical training session? a. If anything, what can make the implementation of the training more effective? Please explain. b. How was the program monitored? (E.g. pre/posttests, number of hours trained, etc.) b. Have you received other similar training? If so, how do these trainings compare to the USAID training? Please explain. 10. How long have you been conducting trainings for GHS? c. Approximately how many trainings have you delivered in Anemia or ENC since you were trained by USAID? 11. For the delivered Anemia or ENC training, rate on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating Type USAID/Ghana IP Training Audience Main Topics Average Duration Rating/Why Anemia or ENC 12. Please answer the following questions for the Anemia or ENC training that you were trained to deliver. c. How was the content taught/delivered to trainees in a typical training session? d. How relevant do you think the training was for the participants in their work? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 83 a. Are there ways to make it more relevant? Please explain. e. Was the program duration adequate? If not, why? f. Did participants’ skills increase as a result of training? • How do you know? (PROBE for examples) g. What can make the implementation of the training more effective? h. How was the program monitored? (E.g. pre/posttests, etc.) i. Were the resources (e.g. materials, funds, etc.) provided for the training adequate? j. Overall, did the training program meet its objectives? Why or why not? 13. What do you think about the scalability of this training? a. Are different models of training more scalable? Please explain why or why not. 14. Do you have anything else you would like to add regarding the trainings? KII GUIDE FOR GHS SUPPORTIVE SUPERVISION General questions with probes: 1. What is your general impression of the knowledge and skills of the HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to deliver health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. What do you think is the biggest challenge in implementing quality HCW training in your district? 6. In the last two years, has the district made any significant changes to its training strategy or implementation? If yes, please describe. 7. What lessons have you learned from training in your district over the past 2 years? 8. What are examples of best practices in GHS training from your experience? Training models 9. Did the OTSS training from USAID, provide you with the knowledge and skills to effectively provide supportive supervision? Please explain. a. How was the content taught/delivered to trainees in a typical training session? b. If anything, what can make the implementation of the training more effective? Please explain. c. How was the program monitored? (E.g. pre/posttests, number of hours trained, etc.) d. Have you received other similar training? If so, how do these trainings compare to the USAID training? Please explain. 10. How long have you been conducting trainings for supportive supervision? a. Approximately how many supportive supervision have you delivered in malaria since you were trained by USAID? 11. For the supportive supervision you delivered at CHPS, rate on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 84 Type USAID/Ghana IP Training Audience Main Topics Average Duration Rating/Why Malaria 12. Please answer the following questions for OTSS provided at the CHPS level. b. How was the content taught/delivered to trainees in a typical training session? c. How relevant do you think the supportive supervision was for the participants in their work? a. Are there ways to make it more relevant? Please explain. d. Was length of the visit adequate? If not, why? e. Did participants’ skills increase as a result of the visit? • How do you know? (PROBE for examples) f. What can make the implementation of the supportive supervision more effective? g. How was the program monitored? (E.g. pre/posttests, etc.) h. Were the resources (e.g. materials, funds, etc.) provided the visit adequate? i. Overall, did the OTSS visits meet the objectives? Why or why not? 13. What do you think about the scalability of the different models of training (e.g. OTSS, Traditional training, internships, etc.)? a. Are different types more scalable than others? Please explain why or why not. 14. Do you have anything else you would like to add regarding the trainings? KII GUIDE FOR CHN, MIDWIVES & CHO – RECIPIENT OF TRAININGS (FOR ALL TRAINING PARTICIPANTS) General questions with probes: 1. What is your general impression of the knowledge and skills of HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What do you think is the biggest challenge in implementing quality HCW training in Ghana? 4. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? • What changes, if any, should be made to the current trainings? 5. What external factors seem to impact your ability to deliver health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation. • How could the effects of these factors be reduced? Training models 6. Please rate training model on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 85 Type USAID/Ghana Implementing Partner Training Audience Main Topics Average Duration Rating/Why 7. Please answer the following questions about the _______ training. a. How was the content taught/delivered to trainees in a typical training session? a. Was this an effective way to deliver the training? b. What did you learn from the training? (PROBE for examples) b. Was the training duration adequate? If not, why? c. How relevant do you think the training was for your work? a. Do you use the skills you learn? i. How do you use these skills? Probe for examples. b. How often do you use the skills you learned at work? Please give examples. c. Do you have any suggestions on how they could have made this training more relevant to the work you do? Please explain. d. Were the materials provided for the training adequate? e. What was the best thing about the training? f. What can make the implementation of the training more effective? a. Which factors facilitated the training you were provided? b. Which factors inhibited the training you were provided? 8. After the training, did you share what you had learned with your co-workers? a. If yes, how did you do this? Please give examples. b. If no, why not? 9. Have you received other similar training? a. If so, how do these trainings compare to the USAID training? Please explain. b. Which training would you recommend? 10. Have you ever received a supportive supervision visit (e.g. been visited by district or regional level)? a. If so, how many times? Or how often? b. What area/topic was the visit for (e.g. malaria)? c. Did you find this visit useful? Why or why not? d. Did you learn anything from this visit? i. If yes, please explain. 11. Do you have anything else you would like to add regarding the USAID training you received? SPECIFIC QUESTIONS FOR E-LEARNING PARTICIPANTS 12. Please answer the following questions as a recipient of E-learning training. a. How relevant do you think the training was to your work? b. Which aspect of the training was irrelevant? Explain. c. How well adapted was the content to your needs as a trainee in terms of the duties you will carry out? d. Was the program duration adequate? If not, why? e. Did your knowledge increase as a result of the E-learning training? Explain how. f. Did your skills increase as a result of the E-learning training? Explain how. 13. What do you think could have made the E-Learning training module better? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 86 a. What do you think can make the implementation of the E-Learning training in other areas easier for trainees like you? b. Should E-learning training be made available in other training colleges? Explain why. c. What specific challenges did you experience while undergoing the E-Learning training? d. How can these challenges be resolved for trainees? 14. Do you have anything else you would like to add regarding the e-learning training? FOCUS GROUP DISCUSSION GUIDE FOR MIDWIVES - MENTORS Focus Group ID code ____________Focus group facilitator: _________________________ Note taker: ______________________________ Region_______________ District: ____________ Interview Date: ______________ Specific location of interview: _______________________ Time discussion started: _________ Time ended: ________________ Number of Participants: ____________ Sign in Sheet: Name Gender Age Facility # of years as midwife # of rounds of mentoring 1. 2. 3. 4. 5. 6. 7. General Instructions: Most of the interview questions below have probes that are follow-up questions to ask if the interviewee has not provided the information requested in the basic question. If the answer was provided, please skip that probe and go to the next one. Please probe for a deeper understanding when the interviewee provides information that is not clear. Please also probe for additional detail when you find an answer especially interesting in the context of the study. In this event, please ask the respondent to expand on his/her answer. Read the following to the participants: Thank you very much for agreeing to participate in this interview. My name is ______. I work for Evaluate for Health, a USAID/Ghana project. We are studying the effectiveness of six models of community-based health worker (HCW) training and the environmental factors that contribute to HCW performance in training and on the job. The categories of trainees included in this study are Community Health Officers (CHOs), Community Health Nurses (CHNs) and midwives assigned to CHPS zones. Our questions are specific to training carried out in the last two years, 2016 and 2017. The study is intended to assist in informing training improvements for HCWs in Ghana. We are very grateful for you for agreeing to participate in the study. As someone who has participated in the Midwife-CHO Cluster (Mentoring Cluster), we would like to ask your views on your experience, specifically what works and doesn’t work, and what improvements could be made in HCW training to ensure effective and sustainable results in terms of knowledge and skills acquisition, retention and application on the job. We would also appreciate your views on the external factors that influence skills acquisition and application. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 87 The entire session will probably take between 60-90 minutes. We will digitally-record the session but your responses will remain anonymous and no names will be mentioned in any reports. We hope that you feel comfortable to speak openly and honestly about your experiences and viewpoints. Please give us as much detail as you are willing and able to provide, and feel free to ask any questions or tell us if you do not understand a question being asked. There are no right or wrong answers. We want to hear many different viewpoints and would like to hear from everyone so kindly speak out when you are sharing your views. We hope you can be honest even when your responses may not be in agreement with the rest of the group. In respect for each other, we ask that only one individual speak at a time in the group and to protect everyone’s confidentiality, we want to remind you not to share what you hear during today’s group discussion outside this group. Also, remember that you can refuse to answer any question during the session if you do not feel comfortable sharing. Do you have any questions or concerns before we begin? Could we kindly switch our phones off or put it on silent mode so we do not disrupt our discussion? General questions with probes: 1. What is your general impression of the knowledge and skills of HCWs in Ghana? 2. What is your overall impression of the effectiveness of the current HCW training in Ghana? 3. What are the primary issues, if any, impacting the effectiveness of the training? • Probe for issues: Quality (curriculum, instructors, materials), Quantity (numbers of sessions and length and availability of staff to attend), Frequency (how often and what follow-up, refresher training)? 4. What external factors seem to impact HCWs ability to deliver health services in Ghana? • Probe for the following factors (have copy of the environmental assessment with you for reference): community-related factors, GHS support and supervision, facility supplies, equipment and systems, protocols, facility infrastructure, transportation 5. What do you think is the biggest challenge in implementing quality HCW training in your district? Training model Questions 6. How were you selected to participate in this training? a. What was your role as a mentor? b. Approximately, how many CHN/CHOs did you mentor? 7. Did you receive any training or guidance to become a “mentor”? a. Did you feel you were provided with the knowledge, skills and support to be an effective mentor? b. Why or why not? 8. For the delivered of the mentoring program, rate on a scale of 1-5 (poor, marginal, satisfactory, good, excellent) and explain briefly your rating Type USAID/Ghana IP Training Audience Main Topics Average Duration Rating/Why Mentoring SYSTEMS 9. Please answer the following questions for mentoring. a. How was the content taught/delivered to trainees in a typical training session? • Was this an effective way to deliver the training? b. Did you provide any practical opportunities for the trainees during the training? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 88 • If yes, please explain how you provided practical opportunities during training. c. How relevant do you think the training was for the participants in their work? d. Was the program duration adequate? If not, why? e. Did participants’ skills increase as a result of training? • How do you know? • How do the use what they learned? (PROBE for examples) f. What can make the implementation of the training more effective? g. How was the program monitored? (E.g. pre/posttests, etc.) h. Were the resources provided for the training adequate? i. What was the best thing about the mentoring? j. What can make the implementation of the training/supervision more effective? • Which factors facilitated the training you were provided? • Which factors inhibited the training you were provided? 10. During or after the training, is it common for participants to share what they have learned with their co-workers? a. Please explain and give examples. 11. What do you think about the scalability of the mentoring program? Would it be worth expanding the program? j. Please explain why or why not. 12. Have you been involved in other similar training? a. If so, how do these trainings compare to the USAID training? Please explain. 13. Do you have anything else you would like to add regarding the training? B. Quantitative Data Collection Tools In this health worker training evaluation, quantitative data will be gathered using the following questionnaires; 1. Knowledge and Skill Assessment Questionnaires 2. Questionnaire For Environmental Factors Influencing The Work Of Community Health Officers And Nurses (CHO & CHN) QUESTIONNAIRE FOR ENVIRONMENTAL FACTORS INFLUENCING THE WORK OF COMMUNITY HEALTH OFFICERS AND NURSES (CHO & CHN) Name:……………………………………………………. Age: ……………… Facility/Community: ……………………………………. Date……………….. Position:………………………... Gender: Male Female Prefer not to say This checklist is to assess the external factors which may influence my work as a Health Care Worker (CHO, CHN, CHV, Midwife etc.). Please indicate to what extent you agree or disagree with each factor. SA - Strongly Agree SD - Strongly Disagree A - Agree D - Disagree For each question, please provide some Explanation (How, Why, Why Not) regarding your choice of answer and how it affects your care delivery. Thank You. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 89 A. COMMUNITY RELATED FACTORS No. ITEM SA A D SD N/A, Don’t know 1 Community members visit the CHPS zone when they are sick 2 I am accepted as a CHO/CHN working in the community 3 The community Leaders provide support to the CHPS compound or health programs (e.g. durbar) 4 Do women need spousal permission to access health services? 5 There is stigma toward controversial health topics, e.g. family planning 6 I always feel safe within the community and at work PHYSCIAL ENIVRNOMENT 7 The community's roads are always accessible / passable 8 Access to transportation is always available. 9 The transportation system is always suitable for clients who seek health services 10 The community's political/chieftaincy stability is conducive for work LOCAL ECONOMY 11 The CHPS compound has reliable electricity 12 The CHPS compound has potable water 13 The CHPS compound has reliable phone signals/reception 14 The CHPS compound has reliable Internet 15 The CHPS buildings are in good condition B. ORGANIZATION RELATED FACTORS No. ITEM SA A D SD N/A, Don’t know MANAGEMENT SUPPORT & SUPERVISION 16 I always get the requisite support/supervision/motivation/mentoring from management 17 My challenges are always resolved by management when required 18 Are you happy with your conditions of service? 19 I have adequate staff to perform my duties effectively 20 My Remunerations/Salaries/Allowances are adequate and timely 21 My duties as a CHO/CHN are well-defined in relation to other health care providers in the community 22 I am able to make decisions as required for work on my own 23 The hierarchy of staff(chain of command) in my work environment facilitates the effective performance of my duties AVAILABILITY OF MATERIAL 24 The drugs required for effective work are always available OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 90 No. ITEM SA A D SD N/A, Don’t know 25 The equipment and supplies required for effective work are always available FACILITY SYSTEMS 26 The records systems are adequate for effective work 27 The rate of flow of clients accessing health care is reasonable (client waiting times) 28 The referral system is suitable for effective work PROTOCOLS 29 The management protocols and treatment algorithms for practice are available to facilitate my work 30 National and local policies support my work KNOWLEDGE AND SKILLS ASSESSMENT QUESTIONNAIRES KNOWLEDGE AND SKILL ASSESSMENT - TRADITIONAL (SPRING) ON IYCF Participant ID……………………… Date…………………………… Facility………………………………… This questionnaire is to assess your knowledge and skills on Infant and Young Child Feeding (IYCF). Please offer the best answer to each question. Questions 1. What is Anemia? • Low Blood level • Low Hb • Hb less than 13.5g/dl in Men & 12g/dl in Women 2. Name One (1) sign of anemia in Children • Easy Fatigue/tiredness • Pallor • Dizziness 3. What food groups must be part of children's meals to make it a balanced diet? • Proteins • Carbohydrates • Vitamins/Mineral/Fats and Oils 4. What items are used to prepare Oral Rehydration Therapy (ORS/ORT) in the home? • Salt, Sugar and Boiled Water 5. Deficiency of what food group causes Kwashiokor? • Protein OR Foods containing protein 6. Mention One (1) sign of malnutrition in children • Poor/Stunted Growth • Underweight/ low weight • Anemia 7. Exclusive breastfeeding should be done from birth till baby is how many months old? OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 91 • 6 months - Half a year 8. In which month can complementary feeding in infants begin? • In month 7 • After 6 months • When exclusive breast feeding ends 9.How can infection be prevented whiles feeding infants? • Hand washing • Using clean utensils • Clean environment 10. What is the best position to breastfeed infants? • Sitting up position • Whiles mother is leaning back in bed propped by pillows KNOWLEDGE ASSESSMENT - TRADITIONAL (SYSTEMS) ON IMNCI Participant ID…………………………… Date……………… Facility………………………………………… This questionnaire is to assess your knowledge and skills on Integrated Management of Neonatal and Childhood Illnesses (IMNCI). Please offer the best answer to each question. Questions 1.What is Malnutrition? • When one lacks the nutrients they need to grow (e.g. Proteins, Carbohydrates, Vitamin, Minerals, Fats and Oils) 2.How can severe malnutrition in infants and children be treated? • By breastfeeding infants regularly. • By serving children balanced diet (meals with Proteins, Vitamins, Minerals, Carbohydrates) • By offering fortified formula to correct deficiencies 3.How can illnesses in children be prevented? • Proper personal hygiene by caregivers • Good nutrition (e.g. breastfeeding, balanced diet) • Immunization 4.In which situations must the mother or caregiver wash his/her hands whiles caring for the baby? • Before feeding • After contact with any source of infection (e.g. diapers, toilet, other persons) • After visiting the washroom 5.How often should an infant be breastfed to ensure proper nutrition? • On baby's demand (e.g. Crying, rooting reflex) • At least 8 times a day 6.Name one (1) typical sign of anemia in children • Pallor • Easy Fatigue/tiredness/ dizziness • Poor Growth 7.What food will you serve an infant as complementary feed? • Fruits and vegetables • Cereal Puddings (e.g. Koko, Corn/Millet/Sorghum Porridge) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 92 • Mashed foods 8.Why must the meals served children be warm? • To reduce risk of infection in children • To prevent illnesses like diarrhea. 9.Mention 2 vaccine preventable illnesses in children • Tuberculosis (BCG), Polio, Rotavirus • Hepatitis B, Diphtheria, Pneumococcal • Pertussis (Whooping cough), Tetanus • Haemophilus Influenza Type B, Measles (Rubella), 10. How does breastfeeding prevent infant and childhood illness? • It contains antibodies • It is not contaminated when suckled directly from the breast KNOWLEDGE ASSESSMENT - TOT (SPRING) ON ANEMIA Participant ID…………………… Date……………… Facility……………………………………… This questionnaire is to assess your knowledge and skills on Anemia, its management how community health volunteers (CHVs) can deliver anemia messages. Please offer the best answer to each question. Questions 1.What is Anemia? • Low Blood level • Low Hb • Hb less than 13.5g/dl in Men & 12g/dl in Women 2.What is the function of Hemoglobin? • It carries oxygen around the body 3.Name One (1) sign of anemia • Easy Fatigue/tiredness • Pallor • Dizziness 4.Why is it dangerous for a woman to deliver her baby when she has low Hb? • A mother can die during childbirth if her Hb is low. • A baby can also die as a result 5.Apart from pregnant women, what other groups are at risk of anaemia? • Infants • Pre-School and School-age Children • Adolescents 6.How does a health worker assess for anemia? • Through signs and symptoms of anemia (Pallor, Fatigue, Dizziness) • Checking conjunctiva(eyes), palms and nail beds for pallor • Laboratory investigations: Check Hb, HemoCue, Hematocrit, Tallquist, Full blood count. 7.What are the major causes of anemia in Ghana? • Life cycle changes (Pregnancy) • Insufficient iron intake and iron-absorption losses. • Non-iron related causes like Malnutrition/Malaria 8.Mention 3 local foods that are high in Iron • Animal-Source Foods: Meat, chicken, fish, liver; eggs; and milk and milk products. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 93 • Staples: Grains, roots, tubers. • Legumes: Beans, lentils, peas, groundnuts, agushie, were, neri, seeds. • Fruits and Vegetables: Especially vitamin A–rich fruits (e.g., pawpaw, mango, and passion fruit) and vitamin A–rich vegetables (e.g., dark green leafy vegetables, carrots, pumpkins, orange-fleshed sweet potato). 9.Which teaching aids can a community health volunteer (CHV) use to counsel community members on anemia? • Flip charts • Pictures • Charts 10. How does Malaria contribute to anemia? • It breaks down the red blood cells (hemolysis) • It causes loss of appetite hence poor feeding and nutrition KNOWLEDGE ASSESSMENT - TOT (SYSTEMS) ON IYCF (ENC) Participant ID…………………………… Date……………………… Facility………………………………. This questionnaire is to assess your knowledge and skills on Infant and Young Child Feeding (IYCF)/Essential Newborn Care (ENC). Please offer the best answer to each question Questions 1.How long should initial skin-to-skin care be provided by healthy mothers of well babies? • Until the placenta is delivered 2.During the first hour after birth, how often should babies be observed for breathing problems? • Every 5 minutes • Continuously 3.Why is it important to begin breastfeeding within the first hour after birth? • It helps babies breastfeed more successfully. 4.When should a baby be given liquids other than breast milk? • Never • Not until baby is more than 6 months old 5.What is an early sign that a baby is ready to breast feed? • Opening the mouth and licking 6.What do you do for the baby in the first minute after birth? • Help the baby breathe • Wrap baby to keep it warm 7.What are the routine care given to a healthy baby at birth • Drying the baby • Removing the wet cloth • Positioning the baby skin-to-skin 8.How do you prepare for safe delivery • You identify a helper and review the emergency • You gather all requirements for delivery (e.g. Instruments, Injections, Supplies) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 94 9.When should the umbilical cord be clamped or tied and cut during routine care? • Immediately after the baby is born • After placing baby on the mother's abdomen 10. How can infections be reduced in the health facility. • Hand washing by staff and relatives • Protective instruments like gloves and goggles • Use of sterile instruments KNOWLEDGE ASSESSMENT- MIDWIFE-CHO CLUSTERS (SYSTEMS) ON FP AND MNCH Participant ID……………………………….. Date…………………………… Facility………………………………………………………………………………………. This questionnaire is to assess your knowledge and skills on Family Planning (FP) and Maternal, Neonatal and Child Health (MNCH). Please offer the best answer to each question Questions 1.Mention 2 long-acting Reversible Contraceptives • Intrauterine devices (IUDs) • Implants • Contraceptive injectable (e. g. Depo Provera) 2.Which anesthesia is administered to the patient when inserting implants? • Local anesthesia (e.g. Lidocaine, Xylocaine) 3.In which circumstances should implants be removed? • When adverse effects occur/persist (e.g. Fainting, Excessive menstrual bleeding) • When the woman wishes to get pregnant • If the woman gets pregnant 4.Why should implants be removed after the recommended 3 or 5 years of use? • The implant becomes less effective in preventing pregnancy • The woman may get pregnant 5.Which parts of the body are implants usually inserted under the skin? Mention One (1) • Inner parts of the arm • Inside the non-dominant arm 6.Which contraceptive can prevent both pregnancy and HIV infection? • Condom 7.Mention One (1) permanent method of contraceptives • Vasectomy • Tubal Ligation/Female Sterilization 8.During the first 4 hours after a male baby's circumcision, which assessments should be made on the wound site? • Hemorrhage (Bleeding) 9.During the post-natal clinic, you notice a week old baby's skin and sclera (eye) looking yellowish. What should you suspect? • Jaundice OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 95 10. Sunken fontanelles, Dry mucus membranes (e.g. dry lips, skin) and Poor skin turgor in a baby are signs of? • Dehydration KNOWLEDGE ASSESSMENT - INTERNSHIPS (MALARIACARE) ON ANEMIA, MALNUTRITION AND MALARIA Participant ID………………………… Date………………… Facility………………………… This questionnaire is to assess your knowledge and skills on Malaria and its management. Please offer the best answer to each question. Questions 1.What is Anemia? • Low Blood level • Low Hb • Hb less than 13.5g/dl in Men & 12g/dl in Women 2.What is the function of Hemoglobin? • It carries oxygen around the body 3.Name Two (2) sign of anemia • Easy Fatigue/tiredness • Pallor • Dizziness 4.How can anemia be prevented? • Eat foods rich in Iron e.g. Green leafy vegetables • Administer heamatenics • Eating balanced diet 5.Deficiency of which food group causes Kwashiokor • Proteins 6.Mr. Kwame tested positive for Malaria. Why is folic acid not added to his treatment? • Folic acid reverses the inhibition by antifolate drugs of plasmodial growth or survival in vitro • Folic acid may increase the failure rate of treatment 7.When triaging for Severe Malaria in children, which signs do you look out for? Mention Two (2) signs. • Convulsion • Fast breathing • Inability to interact with caregivers/Weakness 8.A child with a temperature of 39.6°C is rushed to your clinic. What should be the first intervention? • Control/Reduce the temperature by sponging, • Administer antipyretics e.g. Acetamenophen (Paracetamol) 9.How can Malaria be prevented in the community? • Residual spraying • Use of insecticide treated mosquito nets • Destroy breeding sites e.g. Stagnant water 10. Which vector spreads the malaria parasite • Female anopheles mosquito OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 96 KNOWLEDGE ASSESSMENT - INTERNSHIPS (SYSTEMS) ON MALARIA Participant ID…………………………… Date……………………Facility……………………………… This questionnaire is to assess your knowledge and skills on Malaria and its management. Please offer the best answer to each question Questions 1.Which pathogen causes Malaria? • Plasmodium parasite • Protozoa 2.Malaria is transmitted through the bites of infected • Infected Female Anopheles mosquito 3.Mention 2 typical signs and symptoms of Malaria • Fever • Headache • Chills 4.Why is a confirmatory test for Malaria required prior to treatment? • To ensure that only Malaria infections are treated 5.How can Malaria be prevented in the community? • Residual spraying • Use of insecticide treated mosquito nets • Destroy breeding sites e.g. Stagnant water 6.When is the pregnant women given Sulphadoxinepyrimethamine? • After 16 weeks • After quickening has occurred 7.Why are malaria medication generally administered after meals? • To prevent hypoglycemia (low blood glucose/sugar) • To reduce neurologic side effects of the anti-malaria medication 8.Why is it important to complete anti-malaria treatment courses even if the patient feels better? • To reduce risk of anti-malarial drug resistance 9.How does Malaria contribute to cause Anemia? • It causes/contributes to the breakdown (hemolysis) of Red blood cells 10. How can fever (high body temperature) accompanying Malaria be managed? • By sponging (bathing) patient with tepid (luke warm) water. • By administering anti-pyretics (e.g. Acetamenophen/ Paracetamol) OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 97 KNOWLEDGE ASSESSMENT - SUPPORTIVE SUPERVISION (MALARIACARE) ON MALARIA (OTSS) Participant ID…………………… Date…………………………… Facility………………………………………… This questionnaire is to assess your knowledge and skills on Malaria and its management. Please offer the best answer to each question Questions 1.A child with Malaria presents with fever, where do you start sponging from? Answers: • From the feet • Lower part of the body before the head 2.In sponging down a patient with Malaria what temperature of water is used? • Tepid water • Luke warm 3.Mention 2 typical signs and symptoms of Malaria. • Fever • Headache • Chills 4.Why is necessary to confirm for Malaria infection before anti-malarials are administered? • To ensure that only Malaria infections are treated • To prevent anti-malarial drug resistance 5.How can Malaria be prevented in the community? • Residual spraying • Use of insecticide treated mosquito nets • Destroy breeding sites e.g. Stagnant water 6.To prevent eye and skin irritation, what should one do before using a new insecticide treated mosquito net? • Air Dry the new net in a shaded area 7.When is the pregnant women given Sulphadoxinepyrimethamine? • After 16 weeks • After quickening has occurred 8.Why are malaria medication generally administered after meals? • To prevent hypoglycemia (low blood glucose/sugar) • To reduce neurologic side effects of the anti-malaria medication 9.Why is it important to complete anti-malaria treatment courses even if the patient feels better? • To reduce risk of anti-malarial drug resistance 10. How does Malaria contribute to cause Anemia? • It causes/contributes to the breakdown (hemolysis) of Red blood cells OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 98 KNOWLEDGE ASSESSMENT -SUPPORTIVE SUPERVISION (SPRING) ON IYCF Participant ID……………………………. Date…………………… Facility……………………………… This questionnaire is to assess your knowledge and skills on Infant and Young Child Feeding (IYCF). Please offer the best answer to each question. Questions 1.Within what time frame should breastfeeding be initiated after delivery? • Within 1 hour • Immediately 2.Why should breast feeding be initiated early? • It ensures that infant receives the colostrum, or “first milk”, which is rich in protective factors • It facilitates bonding between mother and baby 3.How frequently should a mother breastfeed a 1week old baby? • Each time the baby demands 4.Why should Colostrum (thick yellowish milk the mother produces during the first few days after delivery) be given to the baby? • It contains a lot of nutrients (e.g. protein, fats and oils) and antibodies the baby needs. 5.At which age can a child be weaned/taken off breast milk completely? • 2 years 6.How can a breastfeeding mother's nipple sore or fissure be treated? • By applying some of the breast milk around the nipple • By applying warm compresses 7.Marasmus in children is associated with deficiency of which nutrients? • Carbohydrate, Proteins and Fats 8.How can a breastfeeding mother's breast engorgement be managed? • Allow baby to continue breastfeeding • Express some milk until breast is soft enough for the baby to suckle • Apply warm compresses 9.Why must the mother wash her and the child hands before feeding? • To clean contaminated hands • To reduce the risk of infection • To prevent illnesses 10. In which month can water be introduced into the meals of infants? • In month 7 • After month 6 • After completing exclusive breastfeeding OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 99 x Annex F: Disclosure of Any Conflicts of Interest Name Dr. Lydia Aziato Title Principal Investigator Organization USAID/Ghana Evaluate for Health Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-641-Q-14-00001/AID-641-TO-17-00002 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 100 Annex G: References Asiedu, A., Nelson, A., Gomez, P., & Effah, F. (2017). QUALITATIVE EVALUATION OF LOW-DOSE HIGH-FREQUENCY LEARNING EXPERIENCES TO IMPROVE NEWBORN OUTCOMES AND QUALITY OF CARE, GHANA. International Journal for Quality in Health Care, 29(suppl_1), 21–21. https://doi.org/10.1093/intqhc/mzx125.30 Atukunda, I. T., & Conecker, G. A. (2017). Effect of a low-dose, high-frequency training approach on stillbirths and early neonatal deaths: a before-and-after study in 12 districts of Uganda. The Lancet Global Health, 5, S12. https://doi.org/10.1016/S2214-109X(17)30119-5 Bluestone, J., Johnson, P., Fullerton, J., Carr, C., Alderman, J., & BonTempo, J. (2013). Effective in-service training design and delivery: Evidence from an integrative literature review. Human Resources for Health, 11(1), 51. https://doi.org/10.1186/1478-4491-11-51 Business Dictionary. (2018). What is training? definition and meaning - BusinessDictionary.com. Retrieved June 8, 2018, from http://www.businessdictionary.com/definition/training.html Children’s Investment Fund Foundation. (2017). Making Every Baby Count Initiative. Retrieved June 3, 2018, from https://ciff.org/grant-portfolio/making-every-baby-count-initiative/ Crigler, L., Gergen, J., & Perry, H. (2013). Supervision of Community Health Workers, (September). Funes, R., Hausman, V., Rastegar, A., & Bhatia, P. (2012). Preparing the next generation of community health workers: the power of technology for training. Iheed Institute. Ghana Health Service. (2002). Community-based Health Planning and Services (CHPS) The Strategy for Bridging the Equity Gaps in Access to Quality Health Services. Ghana Health Service. (2016). About CHPS - Ghana Health Service. Retrieved June 3, 2018, from http://www.ghanahealthservice.org/chps/category.php?chpscid=98 Ghana Health Service. (2017). Organisational Structure | Ghana Health Service. Retrieved June 6, 2018, from http://www.ghanahealthservice.org/ghs-subcategory.php?cid=2&scid=43 Government of Ghana: National Community Health Worker Program. (2014). Ghana Roadmap: One Million Community Health Workers Campaign. Retrieved from http://www.moh.gov.bw/ Jefferson, B., Informatics, D., & Group, F. (n.d.). Community Health Worker ( CHW ) Training - Total Cost of Ownership ( TCO ). JHPIEGO. (2013). Low Dose , High Frequency : A Learning Approach to Improve Health Workforce Competence , Confidence , and Performance. Mobula, L. M., Okoye, M. T., Boulware, L. E., Carson, K. A., Marsteller, J. A., & Cooper, L. A. (2015). Cultural Competence and Perceptions of Community Health Workers’ Effectiveness for Reducing Health Care Disparities. Journal of Primary Care & Community Health, 6(1), 10–15. https://doi.org/10.1177/2150131914540917 Nelson, B. D., Ahn, R., Fehling, M., Eckardt, M. J., Conn, K. L., El-Bashir, A., … Burke, T. F. (2012). Evaluation of a novel training package among frontline maternal, newborn, and child health workers in South Sudan. International Journal of Gynecology & Obstetrics, 119(2), 130–135. https://doi.org/10.1016/j.ijgo.2012.05.019 Rowe, A. K., Onikpo, F., Lama, M., Osterholt, D. M., Rowe, S. Y., & Deming, M. S. (2009). A multifaceted intervention to improve health worker adherence to integrated management of childhood illness OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 101 guidelines in Benin. American Journal of Public Health, 99(5), 837–846. https://doi.org/10.2105/AJPH.2008.134411 Stream, H. (2017). Trends in Healthcare Learning. Studies, C. C. (n.d.). Global Experience of Community Health Workers for Delivery of Health Related Millennium Development Goals : WHO. (2006). Working together for health. the world report 2006.Policy briefs. Trends in Plant Science, 5(1), 32. Willcox, M., Harrison, H., Asiedu, A., Nelson, A., Gomez, P., & LeFevre, A. (2017). Incremental cost and cost-effectiveness of low-dose, high-frequency training in basic emergency obstetric and newborn care as compared to status quo: part of a cluster-randomized training intervention evaluation in Ghana. Globalization and Health, 13(1), 88. https://doi.org/10.1186/s12992-017-0313-x World Health Organization. (2010). Global Experience of Community Health Workers for Delivery of Health Related Millennium Development Goals. Community Health, 377. World Vision International, & Ghana Ministry of Health/Ghana Health Service. (2015). Ghana National Community Health Worker Training Manual: Module 1 : Community Health Basics. OPTIMAL HEALTH WORKER TRAININGS STUDY REPORT 2018 102 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523