End-of-Project Evaluation of the Capacity Project | i END-OF-PROJECT EVALUATION OF THE CAPACITY PROJECT FINAL EVALUATION REPORT March 2010 This document was produced for review by the United States Agency for International Development (USAID/Uganda). It was prepared by The Mitchell Group, Inc. for the Uganda Monitoring and Evaluation Management Services, herein referred to as UMEMS, Contract Number 617-C-00-08-00012-00 End-of-Project Evaluation of the Capacity Project | ii End-of-Project Evaluation of the Capacity Project Prepared for United States Agency for International Development USAID/Uganda Team Members: Paul Freeman, Dr PH, MBBS. MHP (ED) MPH (TH) Grace Kwiyucwiny MBA, PGD. Dip Man, BA (SS) Nathan Nshakira, M.D, MPH Denis Muhangi BA (SWSA), MA, PhD James Tapera, BSc, ACCA, ACPA. The Mitchell Group, Inc. 1816 11th Street N.W. Washington, D.C. 20001 DUNS#: 17528-5121 TIN#52-1467119 Telephone: (202) 745-1919 Facsimile: (202) 234-1697 Internet: the-mitchellgroup.com Kampala March 2010 End-of-Project Evaluation of the Capacity Project | iii Executive Summary Introduction: The USAID-supported Capacity Project in Uganda was a 5-year project to improve human capacity to implement quality health programs. It was implemented by a consortium led by IntraHealth International Inc., as part of a multi-country initiative that reached 48 countries between October 2004 and September 2008. The project had three main objectives: to enhance capacity for human resources for health (HRH) policy and planning at the central and district levels; to strengthen systems for effective performance-based health workforce development; and to identify and promote health workforce management practices for improved performance and retention. This end-of-project evaluation was commissioned by USAID/Uganda in February 2010 to determine whether the project achieved the expected specific results; and to examine the factors that facilitated and/or hindered its achievement of planned results. The evaluation was also to assess the extent to which the assistance provided by the project contributed to overall performance in delivery of health services in Uganda. It aimed at identifying the remaining gaps/weaknesses in HRH systems in the country to inform implementation of a new Capacity Program to be supported by USAID/Uganda for five years. The evaluation adopted a qualitative methodology, which included review of more than 80 documents, and interviews with over 80 respondents in Kampala and six out of 12 districts reached by focused project activities in northern and western Uganda. Data analysis and conclusions were based on triangulation of qualitative data from respondents and quantitative information extracted from reviewed documents. This report presents the key evaluation findings along the three project objectives and a synthesis of general findings focused on the remaining HRH gaps. It makes recommendations for continuation of specific elements of the project, and new interventions to address some of the issues not covered in the completed project. Key findings: The general evaluation finding is that the project was successful in initiating critical interventions to improve HRH in Uganda. The most outstanding finding was an increase in health staff power positions filled (by an average of 25 percentage points). There was some evidence of improved service delivery in the six districts studied, in improved rankings on the annual District League table, and in increased coverage of HIV services. Findings specific to the three project objectives are summarized below. Objective 1: Enhanced capacity for HRH policy and planning • An integrated human resource information system (iHRIS) was introduced, including iHRIS Qualify for the four health professionals’ councils, and iHRIS Manage at the Ministry of Health (MOH) and in nine districts. A knowledge management portal was developed at the MOH, and electronic reference hard-drives (e-granaries) were installed at the Makerere University medical library and the health staff power development centre in Mbale. • Data from the established HRIS and other studies supported by the project informed the refining of the health workforce projections in the HRH Strategic Plan 2005-2020, and accelerated recruitment of health staff in the supported districts. • The project linked multiple actors at central level (in the health, public service, local government and finance sectors) to enhance strategic planning for health workforce growth; End-of-Project Evaluation of the Capacity Project | iv and supported district health and personnel offices, together with district service commissions to manage recruitment of health workers. • All districts were supported to develop annual HRH action plans as an integral element in district development strategies and budgets. • Opportunities for building strategic alliances and collaboration for HRH were provided at the international level (inherent in the multi-country design for the project) and within the country, including HRH forums for PEPFAR implementers in Uganda, and inter-district collaboration in northern Uganda (in partnership with NUMAT). Objective 2: Strengthened systems for health workforce development • Performance improvement was supported through training, district-based assessments, and development of tools to guide performance improvement planning and monitoring. The MOH was supported to develop a performance management strategy. • Three health professionals’ councils were supported to improve registration, licensure and promoting adherence to professional standards for the respective professionals (through the iHRIS). The Pharmacy Council is still in the early stages of establishment and not ready for such support. • Provided institutional development support to the health professionals unions—the Uganda National Association of Nurses and Midwives (UNANM)—to enhance mobilization, mutual support, and welfare and protection advocacy for members. Objective 3: Improved health workforce performance and retention • The MOH was supported to map HRH management processes, and to facilitate application of the results oriented management (ROM)—a pre-existing commitment of government for all levels of operation—in district health services. • Support to districts to assess and conduct rapid recruitment to meet the existing critical health workforce shortages resulted in large increases in staffing levels. • Health workforce satisfaction and retention was enhanced through studies on retention and turnover, improving payroll management, and development of a staff motivation and retention strategy. • Community participation in health service improvement was increased through facilitating field training for medical students at Mbarara University, which focused on training and support to village health team (VHT) operations. • The MOH was supported to develop workplace safety policy and guidelines, and to roll-out their application in selected districts. Gaps, Challenges, and Constraints • Roll-out and application of data from the HRIS was still in the early stages, constrained by weak information technology (IT) application and maintenance capacity at the MOH and in the districts. • Inclusion and focus on specific HRH issues in private not for profit (PNFP) and private for profit (PFP) sectors was limited. • District roll-out of performance improvement was not fully achieved. The tools were difficult to understand and use, and many districts lacked funds to support this as part of routine work. • Support to the Health Manpower Development Center (HMDC) in Mbale to harmonize pre￾and in-service training was constrained by delayed institutional restructuring. End-of-Project Evaluation of the Capacity Project | v • Adoption of ROM as a performance improvement approach was constrained by inadequate follow-up and support to lower level health facilities, and lack of management skills among In-Charges of health centers (HCs). Other General Findings • The project design, based on the human capacity development cycle (Plan, Develop, and Support) was appropriate, and the consultative approach to implementation very well appreciated. • There is still a major gap in availability of key mid-level management staff, especially doctors at health sub-district level (HCIV); turnover at this level is high, largely because staffs are over-burdened with non-medical duties, have limited equipment and supplies, and are disillusioned. • Services at the health centers visited are still severely constrained by prolonged medicine stock-outs, unmotivated and overworked staff, limited local accommodation, limited transport, no planned in-service training and career development, and poor pay. • The approach to project monitoring focused on activities and their outputs (with occasional review and evaluation studies). Measurement of quality of project outputs and life-change results and outcomes was limited. • Achievement of project results was enhanced by the appropriate design and implementation strategy adopted, adequate funding, good project leadership, competent technical advisors, and committed counterpart leaders (in central and district governments). Recommendations: The evaluation recommends continuation of key project interventions to complete planned cycles of intervention development and trial implementation, and to institutionalize successful innovations into government and private sector health systems. Examples of specific areas for further support include HRIS roll-out and application in the MOH and at the district level; institutional and operational support to health professionals’ councils and associations; performance improvement in district health services; and in implementation of the MOH staff motivation and retention strategy. Sustained deployment of appropriate technical assistance to districts (senior nurses experienced in district health services management) is recommended to provide mentoring and hands-on support to health facility officer’s in-charge. This approach will ensure diffusion of district level capacity building to lower level health centers; development of learning organizations/team at the health centre level, building peer learning and support, and strengthening linkage and support between health centers and the communities they serve. End-of-Project Evaluation of the Capacity Project | vi Table of Contents EXECUTIVE SUMMARY ........................................................................................................ III ABBREVIATIONS AND ACRONYMS ................................................................................... IX 1.0 INTRODUCTION................................................................................................................ 1 1.1 Global focus of the Capacity Project ................................................................................. 1 1.2 Overall Technical Approach .............................................................................................. 1 2.0 THE CAPACITY PROJECT IN UGANDA ..................................................................... 2 2.1 The Capacity Project Approach ......................................................................................... 2 2.2 The HRH Situation in Uganda ........................................................................................... 4 2.3 Rationale for the Capacity Project ..................................................................................... 6 2.4 Key Questions for the Evaluation ...................................................................................... 7 2.5 Methodology ...................................................................................................................... 8 3.0 EVALUATION FINDINGS .............................................................................................. 10 3.1 Introduction ...................................................................................................................... 10 3.2 Enhancing Capacity for Human Resources for Health Policy and Planning ................... 12 3.2.1 Strengthening the human resource information system (HRIS) ................................................................ 12 3.2.2 HRH policy review, analysis, and reform ................................................................................................. 16 3.2.3 Health workforce strategic planning ......................................................................................................... 18 3.2.4 Strategic alliances for HRH ...................................................................................................................... 20 3.3 Strengthening Systems for Performance-Based Health Workforce Development .......... 20 3.3.1 Harmonizing pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs and priorities ..................................................................................................... 20 3.3.2 Activities to develop and promote approaches for effective performance improvement at the central and district levels .............................................................................................................................................. 20 3.3.3 Professional registration, licensure, and continuing professional development ....................................... 21 3.4 Identification and Promotion of Health Workforce Management Practices for Improved Performance and Retention ....................................................................................................... 22 3.4.1 Strengthening systems for health workforce performance management ............................................... 22 3.4.2 Developing strategies and systems for improved recruitment and deployment ..................................... 24 3.4.3 Developing tools and initiatives for improved job satisfaction and retention ....................................... 27 3.4.4 Enhancing systems for community participation in health services .......................................................... 30 3.4.5 Strengthening systems for workplace safety, protection, and care of the health workforce .................. 31 3.5 Project Design and Implementation Approaches ................................................................ 31 3.6 Other Findings .................................................................................................................... 32 3.7 Other Major Donors ............................................................................................................ 34 3.8 The For-Profit Health Sector in Uganda in the Area of HRD ............................................ 36 4.0 CONCLUSIONS AND LESSONS LEARNT .................................................................. 36 4.1 Overall ................................................................................................................................ 36 End-of-Project Evaluation of the Capacity Project | vii 4.2 Appropriateness of the Capacity Project Approach ............................................................ 36 4.3 Achievement of Intended Results ....................................................................................... 37 4.4 Enhancing Capacity for Policy and Planning ..................................................................... 37 4.5 Strengthened Health Workforce Systems and Improved Performance .............................. 37 4.6 Key Project Successes and Failures .................................................................................... 37 4.7 Unintended Results ............................................................................................................. 38 4.8 Factors Hindering/Enabling Success .................................................................................. 38 4.9 Remaining Gaps .................................................................................................................. 38 4.10 Lessons Learned ............................................................................................................... 39 5.0 RECOMMENDATIONS ...................................................................................................... 40 5.1 Introduction ......................................................................................................................... 40 5.2 Enhancing Capacity for Human Resources for Health Policy and Planning ...................... 40 5.3 Strengthening Systems for Effective Performance-Based Health Workforce Development ........................................................................................................... 42 5.4 Identification and Promotion of Health Workforce Management Practices for Improved Performance and Retention ...................................................................................... 45 5.5 Other Recommendations .................................................................................................. 47 APPENDICES ............................................................................................................................. 49 Appendix 1: Map of Uganda Showing Districts Visited .......................................................... 49 Appendix 2: Capacity Project District Intervention Matrix, 2009 ............................................ 50 Appendix 3: Evaluation Itinerary ............................................................................................. 51 Appendix 4: Documents Reviewed .......................................................................................... 52 Appendix 5: List of Key Persons Interviewed at the National and District Levels ................. 56 Appendix 6: Guidelines for In-Depth Interviews ..................................................................... 59 Appendix 7: Standardized Instrument to document Components of a Capacity-Building Organization ............................................................................................... 71 Appendix 8: Strategic Objectives for Uganda Human Resources for Heath Strategic Plan 2005 -2020 .............................................................................................. 74 Appendix 9: Capacity Evaluation—USAID/Uganda Questionnaire May 12, 2008 ................ 75 Appendix 10: Achievements against Planned Activities per Objectives .................................. 78 Appendix 11: Sample Output Data from HRIS Qualify Databases .......................................... 84 Appendix 12: Performance Improvement Assessment Report ................................................. 86 Appendix 13(A): List of Consultants Available For Work on HRH ...................................... 107 End-of-Project Evaluation of the Capacity Project | viii Appendix 13 (B): Potential Private Consultants Available at the School of Public Health, University of Makerere ................................................................... 108 LIST OF TABLES Table 1: Illustrative list of Actions within each element of the HCD Cycle .................................. 2 Table 2: Focus of Capacity Project Implementation in Uganda ..................................................... 3 Table 3: Number of Health Staff Employed and Percent of Positions Filled ............................... 10 Table 4: District League Table showing ranking of Districts according to level of Health Service Performance in the Sample Capacity Project Districts ................................................................. 11 Table 5: District League Table showing Service Provision Performance for HIV/AIDS in the Sample Capacity Project Districts ................................................................................................ 11 Table 6:Summary of Health Workers Establishment Status as at January 2010 .......................... 15 Table 7: Projected growth in HRH between 2008 and 2020 – BU and HP Scenarios ................. 18 LIST OF FIGURES Figure 1: Flow Chart of the HRIS Strengthening Process ............................................................ 12 Figure 2: Health Workforce Information Domain Map ................................................................ 15 End-of-Project Evaluation of the Capacity Project | ix Abbreviations and Acronyms AHPC Allied Health Professionals Council AIDS Acquired Immune-Deficiency Syndrome AMREF African Medical and Research Foundation ANC Antenatal Care ART Anti-Retroviral Therapy COP Chief of Party CORP Community-Owned Resource Person CP Capacity Project CPD Continuing Professional Development CRS Christian Rural Service DANIDA Danish International Development Agency DFID Department for International Development DHMT District Health Management Team DHO District Health Officer DPO District Personnel Officer DSC District Service Commission EU European Union FBOs Faith-Based Organizations FP Family Planning HAF Health workforce Action Framework HC Health Centre HCD Human Capacity Development HIV Human Immune-deficiency Virus HMDC Health Manpower Development Center HR Human Resource HRDD Human Resource Development Division HRDTCG Human Resource Donor and Technical Consultative Group HRH Human Resources for Health HRHD Human Resource for Health Development iHRIS Human Resource Information System Software HRIS Human Resources Information System HRWG Human Resource Working Group HSS Health Systems and Strengthening HU Health Unit HW Health Worker HWAB Health Workforce Advisory Board IFMS Integrated Financial Management System IPPS Integrated Personal and Payroll System IST In-Service Training KMRC Knowledge Management Resource centre LAN Local Area Network MGT Management MOES Ministry of Education and Sports End-of-Project Evaluation of the Capacity Project | x MOFPED Ministry of Finance, Planning, and Economic Development MOGLSD Ministry of Gender, Labor, and Social Development MOH Ministry of Health MOPS Ministry of Public Service MoU Memorandum of Understanding MUST Mbarara University of Science and Technology NGOs Non-Governmental Organizations OIC Officer in Charge PEPFAR Presidential Emergency Program for AIDS Relief PFP Private for Profit PHC Primary Health Care PI Performance Improvement PM Performance Management PMPs Performance Management Plans PMTCT Prevention of Mother to Child Transmission of HIV PNFP Private Not for Profit RC Resource Centre RFA Request for Applications RHD Reproductive Health Division ROM Results-Oriented Management TASO The AIDS Support Organization TRT Technical Resource Team UMDPC Uganda Medical and Dental Practitioners Council UNANM Uganda National Association of Nurses and Midwives UNMC Uganda Nurses and Midwives Council UPMB Ugandan Protestant Medical Board USAID U.S. Agency for International Development VHT Village Health Team WHO World Health Organization End-of-Project Evaluation of the Capacity Project | 1 1.0 INTRODUCTION 1.1 Global focus of the Capacity Project The goal of the 5-year Capacity Project was to improve human capacity to implement quality health programs and focused on three intermediate results: • Improved workforce planning and leadership to ensure that the right type and number of health workers are deployed where they are needed; • Better education and training programs so that health workers have the skills and knowledge to meet the needs of the communities they serve; and • Strengthened systems to support quality workforce performance and encourage workers to remain on the job. The project design placed emphasis on human capacity development with a program focus, within the five strategic objective areas of the USAID Bureau for Global Health.1 The project also aimed to address the strategic objective of the USAID Office of Population and Reproductive Health to advance and support voluntary family planning and reproductive health. In addition, the Capacity Project aimed at responding to the human capacity implications of HIV and AIDS as a key element in accomplishment of the PEPFAR goals, and a major stress factor in the fragile health systems.2 Project funding was derived from two main sources: core funding from USAID/Washington, and field support funding from the USAID missions in the respective countries of focus. In general, core funding was dedicated to project elements with multi-country relevance; while field support funding focused more on specific country priorities in Human Resources for Health (HRH), and the demands of each USAID mission. A total of US$103,409,066 was invested by USAID in the five years of Capacity Project implementation: 74.2 percent as field support funding, and 25.8 percent as core funding. 1.2 Overall Technical Approach The technical approach adopted in the Capacity Project was based on the Human Capacity Development (HCD) Cycle; with three fundamental elements of HCD: planning, development and supporting human capacity.3 Table 1 below presents the key actions in each element of the HCD cycle. 1 WHO (2006) Working Together for Health: World Health Report 2006 2 Ibid 3 Ibid End-of-Project Evaluation of the Capacity Project | 2 Table 1: Illustrative List of Actions within Each Element of the Human Capacity Development (HCD) Cycle Plan Develop Support • Understand the HCD environment • Set objectives and strategies • Conduct policy audits • Develop supportive policies • Use information in decision-making • Advocate for HCD policies and systems • Allocate resources • Plan for workforce needs • Deploy workforce rationally • Set quality standards • Establish clear, rational compensation and reward systems • Link health system demand with educational system supply • Recruit future health workers into pre-service programs • Ensure correct educational and training content • Develop quality faculty • Establish appropriate practicum and training sites • Ensure use of best education and training methodology and innovative approaches • Strengthen linkages between Ministries and between the public and private sector, as needed • Harmonize pre-service, in￾service, and continuing education • Foster good performance • Ensure adequate and supportive supervision • Achieve high job satisfaction and motivation • Provide the needed work environment and infrastructure • Increase teamwork and problem solving skills • Maintain strong accreditation, licensing and certification systems • Ensure functioning human resources management and quality assurance systems • Increase dialogue between health sector leadership and educational system practitioners 2.0 THE CAPACITY PROJECT IN UGANDA Implementation of the Capacity Project in Uganda started in June 2005, initially as a field operation through an in-country consulting firm, PS Consulting. A country project office was established in 2006, and operated until the end of the project in September 2009. In addition to the project consortium partners, implementation in Uganda included the following in-country partners: • Government Ministries – Health, Public Service, Education, and Finance • Other Capacity Building Projects – EU/Developing Human Resources for Health • Health Training Institutions – Aga Khan University • Faith-Based Health Networks – Uganda Protestant Medical Bureau (UPMB) • Consulting Groups – Human Resources Development Partner Group, Health Resources and Services Administration, PS Consulting 2.1 The Capacity Project Approach According to the Capacity Project Chief of Party in Uganda, the approach adopted in the Uganda implementation emphasized four main aspects including 1) the imperative of human resources in delivery of all health services; 2) direct partnership with and support to the MOH as the primary End-of-Project Evaluation of the Capacity Project | 3 mover and owner of the HRH agenda; 3) a program orientation in all HRH support to enable stakeholders appreciate the problem and achievement of the program results; and 4) a decentralized focus on specific units within the MOH and the district health services, in line with government decentralization policy. Capacity Project strategy was to identify the HRH issues, formulate plans, develop the systems, and support the Health Sector at Ministry of Health and District levels. It also had a strong link with other USAID funded projects; they met periodically to discuss about the challenges of HRH and the need to address these challenges in respective areas of implementation. The Northern Uganda Malaria, HIV/AIDS and TB Program (NUMAT) undertook a specific intervention on HRH and supported recruitment of 242 health workers in Gulu, Kitgum, and Pader Districts and supported the attachment of 157 medical students from Makerere and Gulu Universities to peripheral sites in the country. Project implementation in Uganda addressed all three intermediate result areas, with focus and illustrative outputs as presented in Table 2 below.4 Table 2: Focus of Capacity Project Implementation in Uganda Objective 1: To enhance capacity for HRH policy and planning at the central and district levels to ensure adequate health workforce for integrated HIV/AIDS and reproductive health services • Strengthening the human resource information system (HRIS) at the central level • Enhancing the MOH capacity for HRH policy review, analysis, and reform • Enhancing the MOH capacity for long-term strategic health workforce planning, roll-out, and reviews • Strengthening district capacity to translate HRH policy and strategic plan into action plans to address HRH priorities at the district level • Strengthening networking and strategic alliances for HRH Objective 2: To strengthen systems for effective performance-based health workforce development • Harmonizing pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs and priorities • Developing and promoting approaches for effective performance improvement (mentoring, performance improvement, support supervision, action learning, etc). • Developing strategies to motivate continuing professional development including standards for accreditation, certification, licensure • Strengthening the role of the health workforce professionals’ councils and associations in in-service training and continuing professional development Objective 3: To identify and promote health workforce management practices for improved performance and retention • Strengthening systems for health workforce performance management • Developing strategies and systems for improved recruitment and deployment • Developing tools and initiatives for improved job satisfaction and retention • Enhancing systems for community participation in health services • Strengthening systems for workplace safety, protection and care of the health workforce 4 Capacity Project (2008) The Capacity Project in Uganda, Capacity Project Knowledge Sharing, Country Brief No. 4 Nov 2008 End-of-Project Evaluation of the Capacity Project | 4 The Capacity Project was implemented in 13 districts but with more in-depth support provided for Kabarole, Oyam, and Amolatar districts (see Map, Appendix 1). Specific activities included training in Human Resources Information System (HRIS), HRH action planning, performance improvement, performance management, support to the recruitment process, payroll management, workplace safety, and village health team (VHT) development. While other activities were implemented in only 13 districts, the workplace safety was disseminated in all the districts in the country. The details of what activities were implemented where, is presented in Appendix 2. 2.2 The HRH Situation in Uganda A total of 85 documents were reviewed as in the attached literature. This section highlights the HRH situation that prevailed before the Capacity Project intervention. The Capacity Project and Ministry of Health conducted a number of studies from 2006 to 2008 to understand the HRH situation in Uganda and define possible strategies and specific indicators to address the inherent human resources (HR) issues. Also, the study would provide the Ministry of Health data on health workforce turnover trends that could assist in planning and deployment of the health workforce in Uganda,5 enhance effective management of health workforce, and minimize waste of the scarce human resource. Management Sciences for Health (MSH) and the African Medical and Research Foundation (AMREF), with support from the Office of HIV/AIDS of the U.S. Agency for International Development (USAID), also undertook an exploratory study of managers with significant responsibility for human resource management in health institutions in the four East African countries: Uganda, Ethiopia, Tanzania, and Kenya.6 The study was designed to document the role and experience of health professionals with significant responsibility for HRM; identify the challenges that these health professionals face, identify additional skills and knowledge needed by these health professionals to address human resources management (HRM) challenges, and solicit recommendations for changes in pre￾service and in-service HRM training. These studies revealed significant problems with working conditions in all health facilities.7 Overall, there is less satisfaction especially with salaries. Doctors are significantly unhappy and ready to leave (57 percent of the doctors interviewed) as compared to nurses (20 percent of the nurses interviewed). The working conditions of health staff are often difficult, characterized by poor infrastructure, lack of staff accommodation, inadequate equipment and supplies, work overload, and inadequate remuneration. The poor working condition is aggravated by weak HRH management. The managers of human resources are not adequately prepared for the responsibility and lack basic requirements to perform their functions. 5 Uganda Health Workforce Turnover study, December 2008 6 Competency Gaps in Human Resource Management in the Health Sector: An Exploratory Study of Ethiopia, Kenya, Tanzania, and Uganda, Prepared by the African Medical and Research Foundation (AMREF) and Management Sciences for Health (MSH) July 2009 7 Capacity Project Knowledge sharing: The Capacity Project in Uganda, November 2008 End-of-Project Evaluation of the Capacity Project | 5 Performance management, regulatory, and disciplinary mechanisms are ineffective. These poor working conditions do not attract staff nor motivate them to stay. As a result the staff turnover is high, particularly in remote rural districts generally regarded as difficult to reach and difficult to stay in. The morale of health workers is low, which in part results into poor attitude towards clients, absenteeism, and low productivity. The public image of health staff has been eroded, the quality of care provided is perceived as poor, and the utilization of health services is not optimal. The studies asserted that despite rising attention to the acute shortage of health care workers, little attention has been paid to the role of those who have human resource management responsibilities and whose job it is to transform health workers into a productive, motivated, and supported workforce capable of improving health and saving lives. The ministries were responsible for essential HR functions—recruitment, deployment, HR planning, personnel policy, performance management, training, HR data systems, strategy development, and HR leadership and management. Lower level managers, mainly at the district levels, lacked the knowledge and skills to carry out many of their HR functions and address the many challenges in their work, including understaffing, lack of employee satisfaction, lack of skilled clinical staff, poor working conditions, and inadequate mechanisms for dealing with staff grievances. During the implementation of the Capacity Project in Uganda, a HRH assessment was carried out in nine districts and a dissemination workshop was done in Acholi and Lango sub-regions.8 The workshop report highlights some of the key HRH issues as; • An expensive recruitment process, especially the cost of advertisements in the local media, which the districts cannot afford regularly • Poor living conditions, especially of staff living up-country and in internally displaced camps in the north • The long process by health workers to access payroll due to delays in confirmation of their Tax Identification Numbers • Inadequate definition and guidelines for qualifications making recruitment of comprehensive nurses difficult • Lack of performance appraisal upon which confirmation in service and promotions would be made • Poor performance management arising from lack of a written job description, performance appraisal reports, and skills in managing the appraisal process • Poor HRH information management resulting in inaccurate and outdated staff records like the payroll • High staffing gaps in many health facilities, especially the lower levels, due to difficulties in attraction and retention of staff • Incomplete composition and poor facilitation and funding of the District Service Commission activities • Failure to induct newly recruited staff due to lack of orientation plans and insufficient funding • Lack of personnel officers in both numbers and skills in the districts • Lack of promotion of staff due to inadequate awareness on promotion process and plans 8 Dissemination Workshop Report on HRH Assessment Report and follow-up visit in nine Districts in Northern Uganda (Acholi and Lango Sub Regions) 2008 End-of-Project Evaluation of the Capacity Project | 6 As specified in the Capacity Project’s Performance Plan (PMP), the technical capacity interventions in each of the result areas and the four cross cutting areas were expected to result in the following: • Quality assessment of needs for changes in the workforce planning process implemented • Workforce data systems developed or strengthened • Country specific workforce aligned to better meet the priority health objectives • Pre-service education for specific cadre strengthened • In-service education systems strengthened • Health professional councils increased • HRH management capacity demonstrated • Health sector human resource management systems strengthened • Worker retention systems strengthened • Global partnering on HRH developed • HR planners and managers of Human Capacity Development (HCD) initiatives developed under the Capacity Project demonstrated • Increased awareness of the value of workforce gender equity in planning and policy development promoted Increased capacity of faith-based organizations (FBOs) to respond to HRH issues developed 2.3 Rationale for the Capacity Project Human resource management and capacity are increasingly acknowledged as components of health systems that directly impact the success or failure of USAID and Ministry of Health investments in the health sector. Uganda, like many developing countries, is experiencing a human resources crisis in health. The health workforce is lacking both the numbers and the skill mix required to respond effectively to Uganda’s health needs. Meanwhile, the HIV/AIDS epidemic placed an additional demand on these already stretched human resources because of both the special skills required for HIV/AIDS prevention and treatment, and health workers themselves being affected by the disease. Besides, the institutional capacity for human resources for health (HRH) policy and planning within Uganda historically has been weak. Capacity to develop, regularly monitor and review HRH policy and plans either at national or district level is limited. Although significant steps have been taken in the development of a human resources (HR) policy and strategic plan, HRH development, deployment, and utilization are still not rigorously directed in a sustainable manner. This results in a mismatch between service requirements and training, both in numbers and skills, and inequities in the distribution of the available human resources. There was an urgent need therefore, to professionalize the HRH role and develop a cadre of well￾trained HR managers, especially in large public sector and private sector health institutions. This would include expanding both the number of HR managers and the organizational view of their role, as well as updating their skills. These changes would also enable HR managers to be more effective in leading and implementing positive solutions that in turn would improve the performance and retention of staff. End-of-Project Evaluation of the Capacity Project | 7 2.4 Key Questions for the Evaluation The purpose of this evaluation is to provide USAID/Uganda with an independent assessment of field support-funded work under the Bureau for Global Health’s Capacity Project.9 Key findings and recommendations should provide an overview of program results to date and inform strategies and approaches for USAID/Uganda’s new Capacity Project Associate Award, awarded in September 2009, and future human resources for health initiatives. The primary purpose of this review was to critically examine the overall project to: • Determine whether the Capacity Project achieved the expected specific results identified for each objective and examine what factors facilitated and/or hindered its achievement of planned results • Determine to what extent the human resource development assistance provided by Capacity Project contributed to the MOH and district overall performance in delivery of health and HIV/AIDS programs • Identify remaining gaps/weaknesses in HRH systems • Determine if the Capacity Project approach for capacity building was cost-effective • Identify lessons learned and best practices, particularly health workforce management practices for improved performance and retention, that will inform USAID/Uganda’s Capacity Project Associate Award and other USAID, Government of Uganda, and development partners’ future capacity building programs Key questions that the evaluation addressed include the following: • How appropriate was the Capacity Project’s approach for HRH policy development and health systems strengthening? • How has the Capacity Project enhanced capacity for HRH policy and planning at the central and district levels to ensure an adequate health workforce for the provision of integrated HIV/AIDS and reproductive health services? • To what extent has Capacity Project achieved its intended results for both the central and district levels? • What factors facilitated and/or hindered its achievement of planned results? • What are the remaining gaps/weaknesses in Uganda’s human resource systems? • To what extent has the Capacity Project’s assistance contributed to strengthened health work force systems and improved performance in the provision of quality service delivery? 9 Scope of Work for End-of-Project of the Capacity project (CP) – Dec 7, 2009 End-of-Project Evaluation of the Capacity Project | 8 • How cost-effective were the Capacity Project interventions? • What are the key successes of the Capacity Project? What are the key failures? • What are the key lessons learned and best practices from the work of the Capacity Project in Uganda? • Has the Capacity Project had any positive or negative unintended results? To what factors can such unintended results be attributed? 2.5 Methodology To answer the above questions, the evaluation not only assessed project performance but also looked for other areas of deficit in capacity building that go beyond the project that could be addressed in future interventions. Consequently the evaluation focused on two themes in deriving the methodology, analysis, and recommendations, namely contribution of the Capacity Project and gaps and needs for future consideration. The evaluation itinerary is presented in Appendix 3. Data collection was conducted using a combination of the following activities and techniques: • Review of relevant program documents. So far, 85 documents relevant to this project were reviewed (Appendix 4). At this stage, key documents describing program work plans, quarterly and annual reports, and end-of-project review reports, monitoring system, and performance management plans (PMPs) have been reviewed to identify documentation of project achievements so far and inform the further collection of data through interviews and capacity tool assessment. Remaining reports listed will be reviewed in an ongoing manner as the evaluation progresses. • Site Visits at which data was collected included the major institutions and organizations involved in the project at the national level and district level in six of the 13 partner districts involved in the project. These districts were chosen because they are representative of the areas of the country involved in the project and the number of project interventions per district. The districts included Kabarole and Kamwenge (both in the South-West with one receiving extensive support and the other receiving significant support); Oyam and Amolatar (both in the north and receiving extensive support); and Apac and Lira, also both in the North but receiving only a significant level of support. In each of the districts, one of two categories of primary health facilities were visited; Health Center level IV (catchment population of 100,000) or Health Center level III (catchment population of 20,000) was visited to understand the actual relationship between the Capacity Project support provided and the health services provided. End-of-Project Evaluation of the Capacity Project | 9 A Regional Health Planning Workshop for northern Uganda in Gulu was attended to understand the context of planning process, the prevailing HRH issues, and to take the opportunity to interview some persons otherwise not reachable. The School of Clinical Officers was also visited to gauge the supply and demand of health professional for the health facilities. • Key informant interviews were held with the key stakeholders including the Chief of Party of the Capacity Project and the Monitoring and Evaluation expert. Other interviews were done with the NUMAT Deputy Chief of Party and Capacity Building Manager in Gulu. Details are in the list of persons interviewed. • In-Depth Interviews were held with all those listed in Appendix 5 representing the Ministry of Health, Ministry of Public Service, Ministry of Finance, Planning and Economic Development, Ugandan Protestant Medical Board (UPMB), Uganda Catholic Medical Bureau (UCMB), and other key persons. Within each of the six districts, those interviewed include the District Health Officer, the District Personnel Officer, the District HMIS focal person, Assistant District Health Officer for Maternal and Child Health, HSD in-charges Hospital in-charges, Assistant District Health Officer for Environmental Health, District HIV/AIDS focal persons and Chairpersons/Secretaries of District Service Commissions. A total number of 77 officials at national and district levels were interviewed over a period of 16 days. The full list of those interviewed is attached in Appendix 5. General and specific guideline questions were developed and used in the interviews at the national and district levels according to responsibilities of respective persons interviewed. These general guidelines are included in Appendix 6. • A standardized capacity assessment tool was developed and used to ascertain capacity status in the supported institutions and identify remaining gaps/weaknesses (MOH and districts). This tool was used to collect data in Kabarole, Kamwenge, Oyam, Amolatar, Apac, and Lira Districts. While this tool contained standardized questions to allow summation of data, allowance was made to include further qualitative data to provide elaboration of the answers to each question. This tool is included in Appendix 7. • Data Analysis, Conclusions, and Recommendations Focusing on the scope of the evaluation, analysis of data was based on triangulation of qualitative and quantitative data from the above sources, content analysis, tabulation of frequencies from the standardized instrument and interpretation of qualitative analysis. Conclusions from the analysis and recommendation of the key findings from document review and relevant articles concerning capacity building and AIDS programs in Uganda and input from USAID have all been considered in this report. In addition, issues raised from other interviews from key stakeholders at the national and other district were also included for discussion and confirmation as relevant. This report is written up following the format presented in The Mitchell Group proposal document. End-of-Project Evaluation of the Capacity Project | 10 3.0 EVALUATION FINDINGS 3.1 Introduction The Project’s strategies and activities were found to be consistent with those of the Uganda Human Resources For Heath Strategic Plan 2005-2020, (Appendix 8), those of the Global Health Workforce Alliance10 and the wishes of the USAID Mission expressed during the evaluation of the multi-country Capacity Project in 2008 (see Appendix 9). A summary table presenting achievements against planned objectives is presented in Appendix 10. At the end of this appendix a table summarizes the Chief of Party’s (COP) explanations for why activities were not completed as scheduled. Where further elaboration is needed, this is presented under the relevant Objective in Findings. As illustrated in the table below, for the six districts studied, the most outstanding achievement of the project was the increase in the number of health staff positions that were filled. Table 3: Number of Health Staff Employed and Percent of Positions Filled District Actual Number at 2004 Percent Staff Level 2004 Actual Number at 2009 Percent Staff Level 2009 Remarks Amolatar 102 68 128 85 Recent recruitment data not included Apac 231 45 350 67.5 Kabarole 224 38 376 62.5 Kamwenge 174 48 290 80 Lira 392 76 443 86 Oyam 135 50 204 76 The above table illustrates that the average number of health positions filled in these districts increased by 25 percent to average over 76 percent as a result of project support. Most findings about projects dealing with the issues covered in this Capacity Project are qualitative in nature. Any link with changes in HIV and reproductive health outcomes is not seen directly. Similarly, increases in services provided in relationship to these areas may not be evident in the short term. Also the Capacity Project is not the only project whose activities may increase the delivery of HIV/AIDS and reproductive service. For example, in Lira there were five other donor agencies supporting the provision of HIV/AIDS-related services. Findings from the District League table are presented below as an indicator of the improvements in health services that might be seen from a general increase in staff numbers. 10 See http://www.who.int/workforcealliance End-of-Project Evaluation of the Capacity Project | 11 Table 4: District League Table showing ranking of Districts according to level of Health Service Performance in the Sample Capacity Project Districts 2007/8 2008/9 District Total Score National Ranking Total Score National Ranking Kabarole 68 12 62.6 41 Lira 68 13 76.9 6 Amolatar 58 41 69.6 26 Oyam 55 57 60 48 Apac 53 65 52 69 Kamwenge 51 72 62.7 39 National Average 62 65.9 Key: Bold = Above national average Italics = Improvement from previous year The above table shows that all the districts studied except Kabarole had increased their District League table National Ranking—a ranking based on the scores of a composite of many indicators covering the full range of primary health care service—during the time of project support when average number of health positions filled in these districts increased by 25 percent to average over 76 percent. Within the composite District League table score is an indictor of the rate of provision of HIV/AIDS services. These results are presented below for the same period. Table 5: District League Table showing Service Provision Performance for HIV/AIDS in the Sample Capacity Project Districts 2007/8 2008/9 District HIV/AIDS Service Provision Score National Ranking HIV/AIDS Service Provision Score National Ranking Kabarole 72% 7 12 100% 10 41 Lira 84% 8 13 98% 10 6 Amolatar 65% 7 41 100% 10 26 Oyam 83% 8 57 70% 7* 48 Apac 91% 9 65 98% 10 69 Kamwenge 74% 7 72 100% 10 39 Key: Score 10: Best performance and improvement from previous year Bold Black = Performance above national average Italics = Below national average Blue = Improved but below national average Red = Regression from previous national position Red with Star = Regression from previous HIV/AIDS service performance The above table shows that HIV/AIDS service provision had increased in all districts studied, even when the national ranking of some decreased. These improvements may be attributed in part to the project—remembering other projects active in the area—but the evidence is not strong. In Amolatar, Oyam, and Kabarole, the project also supported the promotion of attendance for antenatal care and PMTCT at local health centers. End-of-Project Evaluation of the Capacity Project | 12 3.2 Enhancing Capacity for Human Resources for Health Policy and Planning This section presents findings with respect to the first project objective: to enhance capacity for HRH policy and planning at the central and district levels to ensure adequate health workforce for integrated HIV/AIDS and reproductive health services. Project activities to achieve this objective focused on two main HRH problems: shortage of health workforce and inadequate HRH data. The project support under this objective addressed four main areas: • Strengthening human resource information system (HRIS); • Enhancing capacity for HRH policy review, analysis, and reform; • Enhancing capacity for long-term strategic health workforce planning, roll-out, and reviews; and • Strengthening networking and strategic alliances for HRH. 3.2.1 Strengthening the human resource information system (HRIS) The project introduced HRIS to the MOH and districts as a platform for managing information on the trained, registered, and licensed HRH in the different disciplines (HRIS Qualify), and the health workforce deployed in the public sector at different levels (HRIS Manage). The third component in this software package, for projecting the future HRH requirements (HRIS Plan), was still under development by the end of project implementation. HRIS Qualify was provided to the four health professionals’ councils in the MOH: Nursing and Midwifery, Medical and Dental Practitioners, Allied Health Professionals, and Pharmacy. The need to provide HRIS Qualify to the MOH Human Resource Development Division (to track training of all health professionals in the country) was recognized during project implementation, and included in the activities for the follow-on Capacity Program. HRIS Manage was provided to the MOH Human Resource Management (HRM) division and 9 districts. The system development process started in 2006 with the Nursing and Midwifery Council, and evolved progressively through the other professional councils, the MOH Resource Centre, and finally to the districts. Most roll-out in the districts was done in 2009, and ran until the end of the project implementation period in September 2009. The project design outlined the HRIS roll-out process in four main steps, as shown in Figure 1 below. Figure 1: Flow Chart of the HRIS Strengthening Process Source: Capacity Project Knowledge Sharing Legacy Series No. 6 – September 2009 End-of-Project Evaluation of the Capacity Project | 13 A highly consultative and participatory approach was used in rolling out the HRIS in Uganda, which resulted in the formation of a Health Workforce Advisory Board (HWAB) as a structured and sustained HRH reference group. This experience was hailed as an exemplary “good practice” in a multi-country evaluation of the HRIS development component of Capacity Project, conducted in Rwanda, Swaziland, and Uganda in July 2009. The excerpt from the evaluation report presented in the text box below illustrates some of the factors that contributed to this success. Source: HRIS evaluation report, July 2009 The other phases in the HRIS establishment process that were most felt and appreciated by evaluation respondents include: • The participatory approach used in assessment of the data available, the critical information needs at each point, and the IT capacity in place (hardware, software, connectivity, staff skills, etc.). • Provision of IT hardware and software, and enhancing local area network connectivity in some settings. • Training of existing staff in data entry, and in actual use and management of the information system. • Providing on-going support in use and management of the system, including refining of the data entry and analysis capabilities, and basic hardware and software maintenance. In general, the HRIS system was established up to basic utilization level at all 14 points covered by the project. Appendix 11 shows sample outputs from the HRIS Qualify databases on registered health professionals at the time of evaluation. A key observation at evaluation was on lack of standardization of entries with regard to professional categorization, the primary basis for all HRIS Qualify databases. This limits the accuracy of HRH estimates based on the database. The stakeholder engagement process in Uganda was very successful, as the Health Workforce Advisory Board (HWAB) met more or less monthly during its initial phase, remained stable and made significant development contributions, without being seen as duplicating efforts made by an existing HR Working Group. The following factors contributed to the success of the SLG in Uganda: • Members were highly motivated and committed to participate in meetings; even though no reimbursements were provided. • Some members had pre-existing working relationships focused on HRH, including joint analysis of HRH data and recognition of gaps that needed to be addressed. • Dedicated and diplomatic leadership by the group chair and close support and participation by senior in-country Capacity Project leaders (Chief of Party and Senior Workforce Planning and Policy Advisor). • The HWAB “Guiding Principles” that largely focused on the HWAB members’ relationships and organizations and stressed respect, equality, mutual support and efficiency. End-of-Project Evaluation of the Capacity Project | 14 Evaluation findings from documents reviewed and interviews with system users at the different levels indicate factors that constrained the pace and effectiveness of the HRIS roll-out process. Examples include: • Inadequate numbers and limited skills of the data entrants available for training. • Busy schedules of data entrants and systems users trained (reduced concentration during training, and application of skills gained to actual system development and use). • Absences of IT personnel at all levels—a critical cadre of staff in operation and maintenance of such an integrated information system. • Inadequate resources to support demands for hands-on follow-up support in the districts, which turned out to be more than initially planned for as a result of limited basic IT skills of most people trained. Respondents interviewed for the evaluation at the national and district levels appreciated the value and benefits from computerized management of HRH data through the project-supported HRIS development. Examples quoted include improved efficiency in processing data, greater accuracy in data records, identification and weeding out of personnel with forged academic papers, and improved data accessibility for in-house and external use. The system was found operational and in use at all sites visited at central level and in one out of four districts visited. One other district had a functional system that was not in active use because the biostatistician in the DHO’s office had shifted the equipment to the District Personnel Office. “After the training, I entered all the data about health staff into the system, and generated some reports, which we sent to MOH. The system is very powerful, with many functions; it is too big for our office. I thought it would fit well under the personnel office, since they are in the same building with the Registry and more directly linked to each other in function. The system should be used for other sectors and not just health, in which case the personnel department is best suited to manage it. I discussed these views with the HRIS technical support person, and I then moved the computer with the software to personnel office.” (Key Informant Interview, Biostatistician) Since nobody from the personnel office had been trained in using the software, the system was not in use at the time of this evaluation. In the other two districts, the system visited was not functional. One had experienced computer breakdown problems and returned the computer central processing unit to Capacity Project for repair. The system in other district had stopped functioning in December 2009, when payroll management software from the Ministry of Finance was installed on the same computer. Although the fault was reported to Capacity Project, no support had been received to reactivate the system by the time of evaluation visit. Instead, an alternative system (in Microsoft Access) had been developed by the district Chief Administrative Officer, and was in use to manage data on the entire workforce of the district (including health workers). A summary on health workers extracted from this system is presented in Table 6 below. End-of-Project Evaluation of the Capacity Project | 15 Table 6: Summary of Health Workers Establishment Status as at January 2010 Category Approved Positions Positions Filled Positions Vacant Staff Excess* District Health Office 6 3 3 0 Medical Officers 4 2 2 0 Nursing 253 200 58 5 Clinical Officers 36 28 8 0 Laboratory 34 11 23 0 Environmental Health 44 16 28 0 Dental 2 1 1 0 Anesthesia/Theatre 8 2 6 0 Asst. Health Educator 2 0 2 0 Cold Chain 3 2 1 0 Pharmacy/Dispenser 2 1 2 0 Health Information 17 12 5 0 Admin staff 168 84 87 3 Total 579 362 226 8 Source: Access Database of all district personnel; Kabarole district Users interviewed at the national level were satisfied with the level training and follow-up support received to ensure optimum utilization of the system. However, respondents in all districts visited indicated that the level of training received (category and number of people trained, duration of training, provision of reference materials, etc.) and the follow-up support from the project were not adequate. Figure 2: Health Workforce Information Domain Map Data on HRH based in the private sector health facilities (both PNFP and PFP), and in the health programs of non-governmental and community-based organizations (NGOs and CBOs) is not yet included in the current HRIS at district or central level. Data on HRH training in the country is also not yet included in the current HRIS. The health workforce information domain map utilized in the Project (as shown in Figure 2) recognizes all these among the critical data necessary to create a complete picture of the health workforce. Source: Capacity Project; Legacy Series No. 6 – Sept 2009 An integrated knowledge management portal was established at the MOH Resource Centre, at the Capacity Project global office in United States, based on the model established at the Global HRH Resource Centre in Chapel Hill, NC. The web-based portal is linked through the MHO local area network to three of the four HRIS Qualify databases at the professional councils, and End-of-Project Evaluation of the Capacity Project | 16 to the HRIS Manage database at MOH. The database of the Nursing and Midwifery Council, which is housed in premises separate from the MOH, is connected to the portal through the internet. None of the nine HRIS Manage databases at district level has an internet connection to the portal; data from the districts is reentered/imported into the portal system at the MOH Resource Centre. As part of the support to establish the knowledge management portal, two e-granaries (hard-drive digital libraries containing over 10 million resources on a variety of educational topics downloaded from the web) were procured and installed. One, installed at the Makerere University College of Health Sciences, was set up as an integral element in the on-line information at the main medical library. Staff interviewed in the evaluation indicated that a number of academic and research users at the library had utilized the resource and commended its extensive content. Its set up did not provide for monitoring of use, or for generating feedback on how to improve the resource. However, its adapter connection to the electricity mains had blown out, so it was not in use at the time of the evaluation. One of the challenges about the operation and maintenance of the device is that it was installed in a program-specific office (Uganda Chartered Healthnet) but was neither formally integrated as a program-owned resource, nor clearly allocated as a resource for the medical library that needed to be managed at its “remote” location. The other e-granary, installed at the Health Manpower Development Centre (HMDC) in Mbale, was acknowledged as a useful addition to the resource materials pool at the centre. However, it had not been used much by the time of evaluation because of limited activities in the entire centre as it undergoes restructuring. Data generated from the HRIS system established was used in revising the HRH projections for Uganda, and in periodic HRH updates published with support from the Capacity Project, as discussed further in sections 3.2.2 and 3.2.3 below. As a result of valuable HRH data generated from the district HRIS and applied in planning for necessary recruitment, the WHO Country Office supported roll-out of the system to 10 more districts, as part contribution to the Capacity Program. 3.2.2 HRH policy review, analysis, and reform The project was implemented in an evolving national policy context, especially with respect to poverty eradication and long-term development focused on the millennium development goals (MDGs); review of the second Health Sector Strategic Plan (HSSP II) and formulation of HSSP III; and revision of the national health policy. There was concern about the slow progress in maternal and child health indicators towards the MDGs and country targets for poverty eradication. HRH is a specific focus in the current national health policy, with focus on: • Addressing the major constraint of inadequate numbers and inappropriate distribution of trained health personnel; • Developing guidelines for optimal development of health personnel; and • Ensuring increased productivity in accordance with the results oriented management (ROM) approach. End-of-Project Evaluation of the Capacity Project | 17 The specific commitments of the health policy on HRH are shown in the box below. Source: HRH Strategic Plan 2005-2020 The core HRH problem in the 2005-2020 HRH strategy for Uganda is the health workforce as a critical constraint to delivering the Uganda National Minimum Health Care Package equitably to all. The underlying causes described in the strategy include: • Inadequate numbers and poor skills mix for service delivery; • Deficient knowledge and information base to guide HRH management; • Mismatch between health service requirements and HRH training output; • Weak HRH management (inequitable distribution, poor retention, and low productivity); and • Weak capacities for maintaining professional standards. These contextual realities influenced the project approach and focus in supporting HRH policy review, analysis, and reform. The project worked closely with the MOH and districts to produce, disseminate, and apply critical analysis of HRH issues (employment, deployment, and gaps; skills audit; retention and intention to stay in service; etc.) that informed key health policy processes that were concurrent with the project period. Key among these policy processes were refining and initial implementation of the HRH Strategic Plan 2005-2020; review of HSSP II and development of HSSP III; and revision of the National Health Policy. The project deployed a consultant to support the MOH in producing an HRH biannual report in April 2009 based on data generated from the HRIS. The report provided a basis for health sector budgeting to include projected HRH recruitments; its timing for April aimed to have input in the budget for the next financial year, which starts on July 1. The MOH intention is to produce the second report in September in order to inform district planning and budgeting, which takes place in October-November. The September 2009 report was still in draft form at the time of the evaluation. Its development was supported by the same consultant. Review of the April and September reports at the evaluation found substantial overlap in the HRH data presented in both reports. Investment in production of one report each year may be adequate to inform both district and national level budgeting. Health Policy Commitments on HRH • Strengthen human resources management at all levels. • Promote equal opportunities for both men and women in the health professions. • Develop and promote incentive schemes for equitable deployment and retention of health workers, especially to the rural areas. • Establish national guidelines for staffing standards (based on expected workload) for all categories of health workers and institute effective mechanisms for implementing them. • Establish and maintain mechanisms for assuring relevant continuing education for and supportive supervision of all health personnel. • Decentralize in-service training including its funding to district level. • Develop and update training standards in collaboration with the Ministry of Education. • Promote the training and support of Community Resource Persons. End-of-Project Evaluation of the Capacity Project | 18 One of the key elements in the published supplement to the HRH strategy is a refined projection of the anticipated growth in HRH deployed in public, PNFP, and PFP services, based two scenarios: continuing at the current pace of “natural” growth (business as usual); or the accelerated growth as proposed in the “Health for People” option in the HRH strategy. Table 7 below presents the projected trend between 2008 and 2020 in both scenarios. Table 7: Projected growth in HRH between 2008 and 2020 – BU and HP Scenarios Year Public Sector PNFP PFP Total BU HP BU HP BU HP BU HP 2008 25.9 25.9 10.1 10.1 10.7 10.7 46.7 46.7 2009 27.3 30.4 10.6 11.8 11.1 10.8 49.0 53.0 2010 28.8 35.7 11.1 13.8 11.4 12.2 51.3 61.7 2011 30.5 41.9 11.7 16.2 11.8 13.7 54.0 71.8 2012 32.3 49.2 12.3 18.9 12.2 15.5 56.8 83.6 2013 34.4 57.7 13.1 22.1 12.5 17.5 60.0 97.3 2014 36.8 67.7 13.9 25.8 12.9 19.7 63.6 113.2 2015 39.5 79.4 14.8 30.2 13.3 22.2 67.6 131.8 2016 42.7 93.2 15.9 35.3 13.6 25.1 72.2 153.6 2017 46.2 109.4 17.0 41.3 14.0 28.3 77.2 179.0 2018 50.4 128.4 18.4 48.3 14.4 31.9 83.2 208.6 2019 55.3 150.6 19.8 56.4 14.9 36.0 90.0 243.0 2020 61.1 176.7 22.8 66.0 15.3 39.3 99.2 282.0 Ratios 2008 57% 57% 22% 22% 21% 21% 100% 100% 2020 63% 63% 23% 23% 14% 14% 100% 100% BU: Business as usual option HP Health for People option In the above projections, the PNFP and PFP sectors will continue to play a major role as settings for HRH deployment and health service delivery. The HRIS so far developed through Capacity Project support does not include complete capture of HRH deployed in PNFP health facilities, health programs operated by NGOs and CBOs, and the PFP health care points (including established private hospitals and nursing homes, and the less structured clinics and doctor’s offices). Since many of the NGO and CBO health programs often operate at the community level and in settings other than recognized health facilities, systems to capture a complete picture of HRH deployment should include such levels of care and service delivery. Staff at health professional councils, the MOH, and districts (health, personnel, and planning departments) were trained to take on relevant policy analysis and implementation action (e.g., on ethical standards and code of conduct for the different health professionals, harmonizing and standardizing in-service training as an integral element in continuing professional development, etc.). 3.2.3 Health workforce strategic planning The project supported central actors (MOH/HSC, MOPS/PSC, MOFEP, and MOLG) to integrate HRH recruitment and deployment into the national annual development planning and budgeting End-of-Project Evaluation of the Capacity Project | 19 process. The Capacity Project played a key role as an “independent broker” to link the different government ministries, departments, and agencies with a role to play in HRH improvement. Provision was made for step-wise recruitment of health personnel at different levels, integrated with necessary adjustments in the wage budget provisions, and ensuring provision of other requirements for necessary service growth (health facilities, equipment, drugs and other supplies, etc.). The project initiated partnership with the Uganda National Association of Nurses and Midwives (UNANM), as an example of a health workers union, to foster health workforce mobilization and mutual support. It supported research to review and recommend strategies to improve the relevance of the association in meeting the felt needs of members. The project also supported training of association leaders at the national level and in four out of six regional centers in communication skills to enhance effective engagement with media and the general public as a mechanism for positive image building for the nursing profession. Association leaders interviewed in the evaluation indicated that the communication skills gained were also useful in advocacy engagements with government and other employers to improve the conditions of service and workplace safety for nurses and midwives. In addition, the skills were put to use in improving internal communication between the association and members. “The communication consultant did a very good job in training the executive members, and supporting us to train leaders and members in four regions. We have worked with many radio stations and the government television station to discuss issues faced by nurses and midwives at work, and to explain why our services may occasional fall below the expected standard. I have collected at least 1,500 e-mail addresses of members (out of a total membership of 8,000) and I am now in regular communication with them.” (Key Informant Interview, UNANM General Secretary) Interviews in the districts visited at evaluation found the Uganda Medical Workers Union as the most active grouping that brings together health workers across the different cadres to address common welfare issues. Most of the health workers who are members in the union have a standing commitment for a 2 percent deduction from their monthly salary towards the operations of the union. However, concern was expressed that although the central office was supposed to disburse 20 percent of the total contribution to district union branches to support their operations, this had not been received for a long time. All districts in the country were supported to develop HRH action plans. More than half of the districts received follow-up support on integrating the HRH action plans into the district health plans and budgets to ensure sustained implementation. Based on the national level provisions, supported districts were able to recruit, orient, and deploy the necessary health workforce (e.g., 54 people in Apac district, and 27 in Lira district). Districts were supported to orient recruited staff on the different aspects of district operations and services. Challenges experienced in adapting the HRH improvement plans to existing government plans at national and district levels include delays in negotiating the necessary adjustments in wage bill ceilings to accommodate new staff; failure to attract some cadres of health workers (especially medical officers); and difficulties in accommodating some health cadres in the current workforce End-of-Project Evaluation of the Capacity Project | 20 structure (e.g., comprehensive nurses, Nursing Officers in midwifery). Although the private sector (both PNFP and PFP) is a recognized player in development and deployment of the health workforce in the country, its inclusion and involvement in HRH strategic planning was limited. 3.2.4 Strategic alliances for HRH The project supported national and district-level actors to build and nurture networking and collaboration relationships at different levels. It sponsored MOH and PNFP personnel to participate in international workshops, conferences, and learning attachment visits. The project also organized national and sub-national consultations on HRH (e.g., PEPFAR implementers meetings on HRH, HRH planning for northern Uganda districts, etc.). The project worked closely with the WHO at global and country levels to promote application of the HRH action framework, and in application of HRIS at national and district levels. However, efforts at collaboration and joint working were constrained at some points by differences in strategic orientation and working approaches. Many of the MOH actors and the development partners operating within the Paris framework for harmonization of aid preferred a more integrated and sector-wide approach. This contrasted in many ways with the USAID￾supported and project-focused approach in the Capacity Project (e.g., in locus of resource control responsibility, differences in payment rates for different activities, etc.). 3.3 Strengthening Systems for Performance-Based Health Workforce Development The second objective of the Capacity Project focused on strengthening systems for performance￾based health workforce development. Three main activities were implemented by the project under this objective, namely 1) harmonize pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs and priorities; 2) develop and promote approaches for effective performance improvement at the central and district levels; and 3) develop strategies for continuing professional development, including issues of accreditation, certification, licensure, in-service training, and strengthening the role of health workforce professional councils. 3.3.1 Harmonizing pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs and priorities To implement this activity, the project had planned to work with the Health Manpower Development Center in Mbale. An e-granary digital library was set up with materials on clinical, public health, and management aspects of health. However, with the restructuring of educational entities under the Ministry of Education and Sport, the center lost its budgetary support and so there was no longer anyone with whom the project could continue development. 3.3.2 Activities to develop and promote approaches for effective performance improvement at the central and district levels A central-level technical resource team was trained in a performance improvement approach based on staff performance in each facility against standards and with their aid and the aid of local sensitized district health team (DHT) members and officers in charge (OICs) of local HCs, performance improvement assessments were performed at all levels in nine districts. These End-of-Project Evaluation of the Capacity Project | 21 assessments were then meant to be used to develop health unit intervention plans as a basis for supportive supervision and regular follow-up. An example of these assessments is included in Appendix 12. However, project reports stated that only a few districts developed these health unit intervention plans. The development of these intervention plans had been left to the districts. It was also expected that this process would be supported by other actors (e.g. NUMAT). The Capacity Project supported this development to lower levels in only three core districts. According to interviews with DHT staff members, in Kabarole district apparently there was already a reasonably well functioning system of performance improvement and supportive supervision in place, as staff had received previous training, including additional training in quality improvement at the hospital and Health Center IV level, and staff continued to use this. A common finding from district health officers (DHOs) and HCs in which these assessments had taken place in four of the districts visited was that although they could be understood and followed at the district level, they were too difficult for local lower level HC staff to understand. These assessments were also seen as too long and time consuming in several HCs. In several of the districts, follow-up supervision did not take place apparently due to lack of funds. HCs in all four districts visited did not follow up on these assessments other than in relationship to initial visits from the higher levels. National level nursing trainers were oriented to the Learning for Performance Approach and trainers were trained in this method.11 Similarly, district trainers in Kabarole were supported to carry out mentoring training on infection control in 21 health facilities. While this training was apparently initially effective, further follow-up evaluation is needed to assess the degree of success of these approaches in the early stages of the new project. Apparently this has not been done at this stage because these activities were initiated late in the project. With the Ugandan Protestant Medical Board (UPMB), a regular HMIS to report disease statistics and a parallel HMIS to record performance improvement in use of the regular HMIS were developed and tested at 72 health facilities in the UPMB system. The HMIS assessing performance improvement used observations based on performance standards for assessment to be used in supportive supervision. The tool used in this assessment of performance improvement measured good practices that local teams should follow in registering, compiling, analyzing, reporting, and using local HMIS information in their new HMIS for proper decision making in the management of health units. Follow-up impact assessment of this tool found marked improvement almost uniformly against all standards. For example the standard use of data in decision-making increased from 28 percent at baseline to 72 percent one year later. These improvements in performance were noted uniformly across all levels of health facilities involved—hospitals, HCIVs, HCIIIs, and HCIIs. 3.3.3 Professional registration, licensure, and continuing professional development Computerized databases for professional registration were established and are functioning successfully at all professional councils. Most recently standards for continuing professional 11 Gaye. PA, Nelson D. Effective scale-up: avoiding the same old traps. Human Resources for Health 2009,7:2. End-of-Project Evaluation of the Capacity Project | 22 development (CPD) were developed successfully with the professional councils and published in the book Standards for Accreditation of Continuing Professional Development for Health Workers. Development in this area has now proceeded to the stage that the current approach is to accredit institutions that have satisfactory courses and equipment as CPD centers. Professionals will then attend courses at these centers until they complete their CPD requirements for renewal of registration. This process was not completed at the end of the project due to delays in dissemination of the guidelines. The project carried out several activities to strengthen the Uganda Nurses and Midwives Association (UNANM). An assessment to determine nurses’ and midwives’ satisfaction with services from UNANM was performed, and a communication strategy for UNANM produced and implemented, including workshops, production of a UNANM handbook and brochures, and the development of a UNANM website. Evaluations of the communication workshop found that people who participated in the communication workshops were using better communication approaches, were positive about improved communication with patients, and UNANM leaders said they were better able to communicate with policy makers and with the media.12 Nurses interviewed in these evaluations attributed increases in UNANM enrollments to the newly acquired skills in communication and leadership facilitated by the Capacity Project. Codes of ethical professional conduct have been developed and published in booklet form. These codes have been well received, widely distributed, and used in orientation proceedings. The only complaint has been that most districts wanted additional copies of these booklets. 3.4 Identification and Promotion of Health Workforce Management Practices for Improved Performance and Retention The project activities under Objective 3 of the project broadly focused on five groups of activities: 1) strengthening systems for health workforce performance management, 2) developing strategies and systems for improved recruitment and deployment, 3) developing tools and initiatives for improved job satisfaction and retention, 4) enhancing systems for community participation in health services, and 5) strengthening systems for workplace safety, protection, and care of the health workforce. The presentation of findings in this section follows these five groups of activities. 3.4.1 Strengthening systems for health workforce performance management Interventions for improved workforce performance management were based on a study supported by the project entitled “Mapping HRH Management Processes,” Following this, the project supported introduction of output-based performance management practices based on the Result Oriented Management (ROM) approach. To further build local capacity in leadership and management, MOH staffs from HRM and HMDC were sponsored to attend an HR leadership and management development program in Nairobi. The team thereafter drafted and shared a concept paper on developing leadership in Uganda. 12 Yinger.N, McQuide P, Neema S, Olenja J, Zuyderduin, Mayer D. Strengthening Health Professional Associations in Kenya and Uganda: A Project Assessment Draft report 9/28/09. Capacity Project End-of-Project Evaluation of the Capacity Project | 23 The ROM approach supported by the project had already been adopted by the Government of Uganda through the Ministry of Public Service, although its implementation had not been effectively realized. The project support towards the implementation of this approach entailed: • Developing a strategy and tools for ROM and • Testing the approach in three districts, including training of managers and staff in the processes of staff appraisal using ROM principles. District level staff interviewed during this evaluation appreciates the value of performance-based appraisals supported by the project. It was argued that the process enables staff to be conscious of their job descriptions, and to focus their energies on fulfilling them. The process also enables staff to set performance goals, identify indicators of performance, set targets, and make commitments to achieve them. A number of challenges with this process were identified however. In most of the districts visited, the appraisal process was still not embraced by staff as an important practice. In Kabarole district for instance, it was reported that some staff do not remember to complete appraisal forms in time as scheduled and wait to be reminded and pushed to do so. Very few people (staff and managers) take appraisals as an important responsibility. Most times they have to be forced to do it—by circumstances such as need for confirmation, promotion or recommendation for training (Personnel Officer, Kabarole) Problems of supplies of appraisal forms were also reported, with the result that often, staff have to buy photocopies of the forms themselves. In addition, little progress was being made to achieve the targets due to a multiplicity of challenges including lack of drugs, equipment, funding, and support supervision. The other problem is that while appraisals may document some achievements and commitments to improve, this is often not related to the reality on the ground. There is little follow-up to verify and support such improvements. In a number of districts including Apac and Kabarole, staff in charge of the personnel office decried lack of resources to monitor the performance of staff in health centers. They argued that the approach used by the DHT to provide support supervision cannot bring out the real truth about the presence and performance of staff because they follow scheduled visits, and health centre staff make sure that they are present and working when the supervision team is expected. The personnel offices wish to conduct impromptu, unannounced visits and interaction with the consumers of services to understand the true situation on the ground. Another challenge identified was lack of leadership and management skills among health workers entrusted to manage others at the different levels of the health system. This is coupled with little time available for officers in charge to attend to staff issues. Failure to address staff concerns was thought to be leading to low morale that may ultimately result in poor performance, absenteeism, and staff turnover. End-of-Project Evaluation of the Capacity Project | 24 In addition, uncoordinated trainings were found to be frequently taking several health workers away from their work stations, thus affecting the delivery of services. 3.4.2 Developing strategies and systems for improved recruitment and deployment In order to improve recruitment and deployment of health workers for district health services provision, the project supported districts to advertise identified vacancies for health workers. This was however preceded by supporting processes to identify vacant posts, done in two ways: 1) through the iHRIS systems of the districts that show established posts, those filled and those that are vacant; and 2) through nation-wide staff audits that show the existing staff and the shortages in all health centers. Advertising vacant positions in districts was done through central advertising by the MOH, instead of the adverts by individual districts, which has often been costly for districts as it involves paying newspapers to run several full pages of adverts containing vacancy details, job descriptions, and application procedures. Centralized newspaper adverts were supplemented through adverts on notice boards and radio announcements directing the public to where the adverts can be found. A review of one of the adverts placed by the MOH advertising for health workers in The Monitor of June 27, 2007 shows that the advert focused on Northern Uganda, including an “inducement” of 30 percent of 6 months basic pay – given as a one-off allowance (pledged to be paid by UN agencies). The objective as stated in the advert was to increase staffing levels from 33 percent to 55 percent (HSSP target). The number of positions advertised in the project districts included: • 27 in Lira • 46 in Apac • 9 in Docile • 11 in Oyam • 8 in Amolatar • 38 in Amuru The project also paid sitting allowances for members of the DSC during the process of interviews and the technical persons that were co-opted to participate in interview panels. The project also supported the introduction of software for short-listing applicants. The system for e-short-listing was expected to save time, reduce costs, and promote objectivity by enhancing adherence to merit in the short-listing process. This software was installed in Lira, Amolatar, Kabarole, Dokolo, and Amuru districts (the latter two not in the study sample). District staff were trained to use the software. In some districts, the short-listing software was successfully used, resulting in time-savings and easing work. A good example of such districts is Amolatar; where staff interviewed indicated that the short-listing process was handled by only 3 people and completed in 4 days – usually it takes the entire District Service Commission 2 weeks of sitting. In others, the system did not work. In Lira for instance, the system was never used, the problem being attributed to the caliber of staff that had been trained to use the system. Within the Lira DSC, which is responsible for short-listing, a clerical officer had been trained, together with a records assistant from health. Key informants at the district thought that more competent people such as the statistician are the ones who should have been trained to use the system. End-of-Project Evaluation of the Capacity Project | 25 In districts where the system was put to use, such as Amolatar, the users were pleased with the help afforded by the system. As the acting secretary to the DSC in Amolatar observes: …We were trained on how to shortlist electronically, which was very good. It made things easy for us. It would be good to use the same system to shortlist candidates applying to other sectors. (Acting Secretary, DSC, Amolatar) Before the project, there was simply no money to recruit staff. For example, in Lira, the DSC’s budget for FY 2009/2010 was only enough for the DSC to sit for only 21 days. The project helped the district to advertise vacant posts, which was done centrally. This saved the district a lot of money. If the district was to advertise on its own, probably it would not afford it. The district receives only Ug Shs 29 million (US$13,875) for a whole year, which may not even be released at once. (Secretary, DSC – Amolatar) Staff in Kamwenge reported that before support from the Capacity Project, the district had not done any recruitment for three years because of the cost. It was reported that the advertisement alone costs Ug Shs 13 million (US$6,220), while one sitting by the DSC costs over Ug Shs 2 million (US$957). In several districts, it was reported that the staffing for health had improved as a result of the project support. As previously observed, staffing levels had increased for the districts supported by the project to undertake recruitment of health workers. It must however be cautioned that not all the observed increases can be attributed to the Capacity Project, as other interventions were also going on, e.g. the 30 percent salary incentive paid in the north supported by UNICEF/DFID in 2007. Nevertheless, the fact that districts were enabled to recruit additional staff means that the project made a significant contribution. Districts were further supported to orient newly recruited staff. A total of 371 newly recruited staff for Lira, Amolatar, Kabarole and Kamwenge were inducted. In Lira, the DHO reported that for the first time in a long time, the district was able to induct newly recruited staff. Before this, any new staff would be posted to work stations without any orientation. This could mean that they would not understand their job description, their immediate work environments, and other aspects important for their integration into the workforce of the specific health system of the district. However, it was observed that the orientation carried out covered general issues and left out the technical aspects related to individual job requirements. In Kamwenge, orientation of health staff was conducted and the topics covered included terms and conditions of service, record keeping, payroll management, and HR issues. The orientation did not include technical issues that relate to the facilities and services because this should be done at the department level. District informants in several districts reported that it was preferable to include an attachment of each new staff to an officer already in post for at least one or two months before one can go and take his or her post. Although this can be viewed as purely a management issue that district and health facility managers should have been able to handle on their own, many such managers reported that they needed support to also be able to do technical orientation. End-of-Project Evaluation of the Capacity Project | 26 Apac district reported that support for recruitment of staff had translated to better service provision in the following ways: • The critical vacant posts were filled and this has improved the availability of staff at the facilities and increased coverage, improved OPD attendance and immunization coverage. • The quality of the services provided improved because of the presence of qualified staff. Despite project support, some districts reported persistent problems in attracting staff and filling vacant posts. In Apac, the district had advertised four posts of medical officers but had not received any applicant. Informants from the district attributed this to lack of incentives to attract staff to these “hard-to-reach” districts. They compared themselves to districts such as Moroto (in northeastern Uganda) and Nebbi (West Nile region), which pay special allowances to attract staff. In all districts, particular cadres of staff were reported to be difficult to attract. These include medical officers, cold chain technicians, and laboratory technicians. Some of the problems/challenges affecting recruitment were found to include • There is a ceiling set at central level that determines the total salary bill. Recruitments have to fit all health staff within that ceiling and this therefore determines the total number that districts can recruit and the specific cadres that can be included. In Apac district for instance, it was reported that they cannot recruit the second Assistant DHO (for Environmental Health) because of the ceiling. District informants further reported that even the agreed wage bill often gets cut by the Ministry of Finance, meaning that they do not have adequate funds to pay all their staff. In Apac, it was reported that the money for wages had been cut by Ug Shs 400 million (US$191,387). In Kabarole, two assistant DHOs recruited in July 2009 had never been appointed due to inadequate funds for wages. This evaluation however found that part of the problem with wage bill management is partly attributable to lack of information to districts about the actual ceilings on the wage bill. The Ministry of Finance releases 65 percent of the funds approved for the wage bill, the rest being accessible upon request by the districts. Many districts are not aware of this and they only operate within the 65 percent allocation. • In Oyam district where a PNFP agency set up seven health centers expecting that the district would provide staff to work there, the district could not provide such staff due to the limits on the number of staff that can be hired. • The short-listing system was set up to be used only for health staff. It would have been good if the system could also shortlist for other sectors. • In some of the districts such as Amolatar, the training for using the short-listing software came after job applications had been received. This meant that the applications had to be entered into the system and this delayed things a bit. Had the training taken place earlier, every application would have been entered as soon as it came in and this would have moved things much faster. End-of-Project Evaluation of the Capacity Project | 27 • For most districts, not all the required staff were recruited. Some positions did not attract any qualified applicants. Part of this problem arose because of the centralized advertising. When vacancies are centrally advertised, a jobseeker can see that the same vacancy exists in several districts. S/he can therefore choose to apply to work in a better located or a well-resourced district, and will not apply to hard-to-reach districts. Similarly, because of central advertising, some staff that are already in post in certain districts opt to apply to go to better off districts. Apac reported that they lost a number of nurses and clinical officers who opted to go and work elsewhere. While this might have a positive side in terms of enabling health workers to exercise some choice about where they should work, enabling them to realize their career growth, and probably getting them better motivated, it definitely has serious implications for the attraction and retention of staff to remote, hard-to-reach districts. • The project was only willing to pay allowances for DSC members at lower rates than what they normally earn as per government guidelines. DSC members normally earn a total of 290,000 Ug Shs (US$138) per day (i.e.140, 000 or US$67 for night allowance and 150,000 or US$71 for each day’s sitting). The project only paid them a lump-sum of Ug Shs 900,000 (US$430) for the whole process from short-listing to conducting interviews. DSC members first had to be encouraged before they were wiling to work on those terms. • For some districts such as Amolatar, the information about vacant posts needed for purposes of central advertising was obtained by the project directly from the DHO. Under normal circumstances, they should have gotten them from the DSC. The DHO should have forwarded them to CAO, who would then have forwarded them to DSC, and the DSC would advise the project. Because of the procedure used, some positions that do not exist in the staffing structure for Amolatar were advertised and recruited for. It was discovered later that some of the positions that had been recruited for did not exist. The DSC had to renegotiate with the candidates to accept lower positions or to cancel their appointments. 3.4.3 Developing tools and initiatives for improved job satisfaction and retention The project undertook studies whose findings provide essential information about the status and challenges of HRH in Uganda. One was the retention study carried out in July 2006 entitled “Uganda Health Workforce Retention Study: Satisfaction and Intent to Stay among Current Health Workers.” This study focused on health workers job satisfaction, motivation, and intent to stay or leave the service. The findings of this study showed that levels of satisfaction with their jobs, salaries, and work environments were generally low, and about 24 percent of the health workers planned to leave their jobs soon. The second study focused on documenting the rate of turnover among health workers and was conducted in 2008. Entitled “Uganda Health Workforce Turnover Study,” this study showed that average annual attrition rates were 1.2 percent and 13.0 percent in public and PNFP sectors respectively. Absconding was one of the major causes of staff turnover. Based on the findings generated by the above studies, the project supported the development of a “Motivation and Retention Strategy for Human Resources for Health” for Uganda’s health sector (dated October 2008) to be implemented over five years. The goal of the strategy is to strengthen the capacity of the health system to improve the attraction, retention, equitable distribution, and End-of-Project Evaluation of the Capacity Project | 28 performance of health workers. The strategy elaborates the actions needed in order to institutionalize and maintain sustainable schemes for salaries and benefits to attract and retain health workers; strengthen leadership and management capacities to mange health workers; strengthen mechanisms for planned career progression; and provide and maintain flexible, conducive, and safe working environments for all heath workers. The Motivation and Retention Strategy was presented at the Annual Health Sector Performance Review Meeting in 2009, at which the MOH and other stakeholders adopted the strategy, and agreed that it should be costed and implemented. Some development partners, including the World Bank have already agreed to support some elements of the strategy. In addition, the project supported districts in payroll management. Regional trainings in payroll management were conducted for all districts and regional hospitals in the country, followed by on-site, hands-on support in 13 districts. These trainings targeted payroll managers in the districts and regional hospitals. In addition, copies of the payroll management manual were distributed to districts. The evaluation found evidence of possession of these copies in the districts. Interviews with the Ministry of Public Service (MOPS) indicate that the national average access to the payroll for health workers increased from 50 percent to 90 percent following these trainings. Interviews with districts also show that following this training, access to the payroll increased. In Oyam, no health staff were not on the payroll by the time of this study. In Amolatar, only one staff member was known not to be on the payroll by the time of this study. In other districts, however, such as Kamwenge, problems in accessing the payroll had persisted. With project facilitation, personnel from the UPMB and UCMB attended a conference in Nairobi in 2006 where they learned about a program designed to strengthen leadership and management skills—the Virtual Leadership Development Program (VLDP) developed by MSH. The UPMB and UCMB personnel formed a team and chose, as their leadership challenge, to reduce the attrition rate at their health facilities from 18 percent when they started the VLDP to 10 percent within a year. The team developed a joint position paper to present to the government, seeking input from hospital managers and using concepts from the VLDP’s approach to facing challenges. This paper communicated the current situation the organizations face as well as their contribution to the health system of the country. The two bureaus also had joint planning workshops with health facilities to address their human resource challenges. The UPMB/UCMB team completed and implemented its strategy to reduce attrition among its health centers. The final evaluation completed by all the VLDP participants at the conclusion of the program documented that 78 percent of the participants have brought about changes in their organizations as a direct result of the VLDP, with 98 percent of participants stating that they would recommend the VLDP to others (downloaded from http://capacityproject.org, February 26 2010). The current HRH technical advisor for UCMB informed the evaluation team that the strategy did indeed reduce attrition to 10 percent but with subsequent changes, attrition in UCMB facilities is now 20 percent in hospitals and 18 percent in lower level facilities although they are still applying this strategy. End-of-Project Evaluation of the Capacity Project | 29 Some of the districts have, on their own, taken measures to attract and retain key staff such as medical doctors. In Kamwenge, it was reported that the district council had approved a top-up allowance of Ug Shs 700,000 as an attraction package for medical doctors. Discussions with MOPS officials indicated that a software system “Integrated Pay Roll and Personnel System” (IPPS) for payroll management, supported by World Bank funding, has been introduced by the Ministry of Public Service and will be piloted in Lira and Jinja districts starting in July 2010. It will cover information such as payroll status, staff details, qualifications, probation, confirmation, vacancies, short-listing, trainings, enrollment for training, and deployment of staff. A back-up system has also been provided for at-source (districts), the MOPS, Jinja and Nairobi. The system has training modules designed for the staff, and the Integrated Financial Management System (IFMS) infrastructure has already been installed in all the local governments with support from the World Bank. Some of the existing challenges to staff retention were reported to include the following: • The turnover of health workers is high in the districts mainly because the promotional outlets are limited. When people attain higher qualifications and they cannot be accommodated in the existing district staffing structure, they look for opportunities for better career growth elsewhere. Because of the limitations imposed by the current staff establishment, health workers who get an additional qualification and are eligible for promotion may not be promoted immediately because there is no vacancy to which they wish to be promoted. Consequently, such staff members decide to leave their current district of service to go and work elsewhere where they can be appointed to a higher post. This appears to be a key challenge to staff retention. • Poor retention was also thought to be arising from delayed confirmation and promotion of staff. Promotion and confirmation are not effectively done by the DSC and this has left many staff waiting and static in one position or probation level for as many as 12 years. • Inadequate motivation is another cause for poor retention. Poor motivation was attributed to lack of housing accommodation at work stations, low salaries, poor allowances, and lack of training opportunities for staff. In addition, some districts reported that they no longer provide uniforms and shoes to nursing staff, yet they expect them to appear in uniform every day. Such a situation was reported in Apac. • Poor retention is also arising from few organized opportunities for further training. Health workers who struggle on their own to find study opportunities, including those who sponsor themselves for further study, have no commitment to their districts of service and often do not return after the training. The system of bonding staff members that are offered study leave was reported not to be effectively enforced to retain staff. • Some districts reported the persistence of problems in getting some staff members onto the payroll. In Apac district, it was reported that six of the staff recruited in 2007 had not accessed the payroll up to February 2010. In Kabarole, the evaluation team found that one of the medical officers managing a health sub-district and two of his colleagues, End-of-Project Evaluation of the Capacity Project | 30 along with six other staff members, all recruited in September 2009, had not accessed the payroll by the time of this study. While the district had committed itself to supporting the doctors with Ug Shs 300,000 per month, they had received this money only once. Kamwenge also reported persisting problems with the payroll, although the situation had improved (e.g. the DHE recruited in September 2009 was not on the payroll by the time of this study). Problems in payroll management are still related to inaccurate filling in of forms and code sheets, as well as lack of communication from MOPS as to any errors or problems in the forms that need to be corrected by the districts. 3.4.4 Enhancing systems for community participation in health services The project worked with Mbarara University of Science and Technology (MUST) to implement a village health team (VHT) strategy in the districts of Kabarole and Kasese and to attach students in communities for practical training. Project documents indicate that the project supported the placement of Mbarara University students in health centers in Kabarole and Kasese districts where they worked closely with VHTs. The focus was on getting the students to work at the community level to support VHT functioning and address community needs in the areas of PMTCT, HCT, and family planning. The students participated in VHT training and development in Kabarole and Kasese districts, and a total of 63 new VHTs were supported in providing PMTCT, HCT and family planning services. The project distributed VHT development guidelines from the MOH to the university and the districts. Students also worked with health centre staff to provide outreach services. Interviews with some stakeholders indicated that the process had helped to expose medical students to work in remote rural settings, making them more disposed towards working in similar settings in the future. The support that the students offered to VHTs was reported to have enabled VHTs to respond to community needs, and stepped up community mobilization and resulted into increased turn up for PMTCT, HCT, and immunization services. Although project documents indicate that the students worked with the district VHT facilitators and trainers as well as VHTs themselves, interviews with district staff revealed that the actual processes of student placement, day-to-day work, and supervision were managed by the university with little involvement of the district managers. The districts were only informed that the placements were taking place. In addition, a major weakness in the development of the VHTs acknowledged both in project documents and from the interviews carried out for this evaluation, was the lack of system for regular supervision and ongoing support to the VHT once they had been trained. It should be noted, however, that in most districts, VHTs had been trained in the past before the advent of the Capacity Project. VHTs had worked well under the Home-Based Management of Fever (HBMF) Strategy, distributing HOMAPAK (the prepackaged anti-malarial consisting of Chloroquine and Fansidar [Sulfadoxine Pyrimethamine]). However, they slackened because of poor motivation, including poor facilitation and lack of medicine supplies after the change in the treatment regimen from Fansidar SP to Coartem. Other programs have utilized them in recent years (since 2008), including for distribution of family planning products, ITNs, and End-of-Project Evaluation of the Capacity Project | 31 mobilization for in-door residual spraying of mosquitoes. In Northern Uganda, VHTs have been variously trained by different agencies such as THETA, UNICEF, and NUMAT. In many places, VHTs’ management of information did not take off well, with the use of the Village Health Register not being closely followed up, and the value of the register not well appreciated. The general challenge with VHT work is lack of follow-up and support. In some districts, such as Apac, it was reported that they wished to hold annual health assemblies at HSD and district levels, where all health workers would have the opportunity to participate. These had however not been held because of lack of resources. 3.4.5 Strengthening systems for workplace safety, protection, and care of the health workforce The project supported the MOH to develop a Workplace Safety Policy and Guidelines. The policy and guidelines were printed, distributed, and disseminated at national and district levels. It also facilitated the formation of workplace health and safety committees at district level. Such committees were found to have been formed in the districts of Oyam and Kabarole. By the time of this study, the committees in Oyam and Kabarole had not met and had not undertaken any other activities. In Oyam, the district committee was awaiting the formation of lower level committees at health facilities, while in Kabarole, it was reported that the committee was awaiting formal appointment by the district authorities. At the health unit level, it was found that implementation of workplace safety procedures was hampered by a number of constraints including lack of basic supplies such as soap, detergents, and gloves. Such constraints were reported in Apac hospital. Another problem reported was under-staffing, which means that overworked and hasty health workers may fail to observe all necessary safety precautions and procedures. In all districts, it was found that districts/health centers no longer provided staff uniforms for health workers, and the heath workers are supposed to buy their own uniforms. Many work without protective gear like gumboots and gloves, which puts them at risk of infectious diseases like HIV/AIDS. District informants in the north reported that besides the interventions of the Capacity Project, their main actions to improve workplace safety in recent years had focused on making medical injections safer; a process that has been supported by USAID through JSI. The process included training staff and improving waste management for all health staff in the district, even those in NGO and private facilities. Most recently, NUMAT has supported training staff involved in HIV care and PMTCT on AIDS service delivery. Districts also reported that they had received training on hepatitis prevention, but they did not have the vaccines to immunize staff at risk. They reported that they had auto-disposable syringes in all facilities, to prevent disease spread through injection reuse. 3.5 Project Design and Implementation Approaches End-of-Project Evaluation of the Capacity Project | 32 The approach of the project to design and implementation was widely praised. Initial consultation with stakeholders, most commonly at the MOH, began from the point of view of asking what are your problems and needs and therefore what the project can offer to meet these needs. These needs were then met through the activities in the project design. Only in a few districts did DHT staff feel that implementation strategies were being imposed on them. The fact that all project consultants were based in Kampala is important to note from the point of view of ability to engage in ongoing activities such as mentoring often needed by weaker partners. For many system-focused interventions this is not important, as arguably DHT members should be providing this mentoring, but observations demonstrated that this was certainly not the case for health center-level staff. In the short term such mentoring is necessary for weaker stakeholders until they achieve basic competency that can then be maintained locally. The approach of the project to monitoring and evaluation used indicators that monitored only completion of activities rather than including measures of quality of outcomes. However, quite often independent evaluation of the quality and impact of key project activities did take place and could be found in reports published online at http://www.capacityproject.org. 3.6 Other Findings In order to meet the scope of this evaluation—to identify other areas of capacity building not currently addressed—the evaluators probed for factors that affected human resource development and capacity in addition to those not covered by the project. While problems in recruiting medical officers for HCIV level were found everywhere, some district hospitals were also short of doctors and clinical officers were filling in for them. The few doctors found in place at HCIV level were obviously overburdened with non-medical duties, disillusioned with not being able to practice their craft because few patients needed their advanced knowledge, and supplies and equipment including electricity were inadequate to perform operative procedures. The doctors interviewed felt that they were being deskilled by their current work and were planning to move on for further training in clinical specialties within a year. They were the personification of the young rural medical officer the Medical Council Registrar interviewed described when he said, “A major reason doctors leave, overriding no pay, no housing, and poor general conditions, is that they cannot practice their skill….with a lack of equipment, they do not get much useful experience in developing their clinical skills and are not doing the work they were trained to do.” Similarly wherever the evaluators went at the health center level, the same problems were clearly evident as many reviews have documented: lack of drug supplies for months, staff absent or physically appearing obviously overworked, no local accommodation or adequate transport for staff, no planned in-service training, poor pay with few prospects of increase or promotion, no staff development, cheerless empty-feeling buildings at the HCIII level or buildings crowded with patients and few staff at the HCIV level late in the day. However, in several places staff were well motivated and some were keen on furthering their careers themselves. At one HCIV center, four staff were away for up to five years furthering their careers in achieving Masters degrees or becoming doctors. Of note is that while this center officially had four staff positions vacant, two of them medical officers, functionally eight staff members were absent. Many of the End-of-Project Evaluation of the Capacity Project | 33 OICs interviewed in HCs said that they needed further training in management specifically in the areas of leadership, staff management, planning, and logistics. An interview with the principal of the Gulu School of Clinical Officers and his deputy was in stark contrast. He stated that the school produced 90 graduates per year and that competition for places in their training program was fierce. However, there were never enough places available to employ all his graduates in Uganda. Some have left the country for Sudan and Kenya. In 2005, a draft curriculum was developed that would enable a clinical officer to become a doctor after three years further training instead of the current five years. This was supported by AMREF and the MOH. However, nothing has been heard of this draft since then. A clinical officer costs many times less than a doctor to be trained, is internationally less marketable and therefore more likely to remain in Uganda if employed, and could be encouraged to stay working in rural health centers for a number of qualifying years before s/he is allowed to undertake a shorter course to become a doctor, if the course mentioned above was mobilized. Unprompted, the principal and his deputy also identified addressing all the general factors listed in the previous paragraph, as well as provision of access to soft loans, as needed to improve clinical officer performance. A recent study of AIDS treatment in Uganda found that 64 percent of the people who prescribe antiretroviral treatment were not doctors. 13 Findings like these suggest that there is need to redefine the role of doctors limiting them to only those clinical services for which their expertise is needed so that the few present, with a transport arrangement, could cover several HCIV facilities, if standards of clinical care are to be maintained. This, along with equipping selected rather than all HCIVs with adequate equipment to do common operations may be one way to rationalize the use of doctors. The HCIVs that are chosen for adequate equipment could be chosen on the basis of mapping all health facilities in each district and seeing that these equipped facilities are well-spaced through the district. Making sure there is an adequate emergency transport system from centers not so well-equipped would be an essential part of the system. The evaluators also assessed the need for human resource and management support through the use of the standardized tool presented in Appendix 7. The detailed findings of the use of this tool are also presented in that Appendix. In summary health manpower planning was good in all six districts, and the DHT has a reasonable knowledge of current staffing gaps. Review and day-to￾day planning meetings generally take place on an approximately weekly basis and DHT members participate in planning and problem solving. HCII staff had no role in district planning. No analysis of workforce data takes place, and while records do include gender of staff, no considerations with regards to gender take place. All districts had established procedures for recruitment and orientation of new staff but these were inadequate in those districts that had not been supported directly by the project. None gave any technical orientation. All said they had job descriptions but no one could produce theirs and examination of summary documents, where possible, showed that job descriptions did not exist for all positions. Regular performance appraisal takes place for all staff but only one district had clear performance standards that included ethical standards and standards concerning data confidentiality, although these could not be produced. 13 Lutalo IM, Schneider G, Weaver MR et al. Training needs assessment for clinicians at antiretroviral therapy clinics: evidence from a national survey in Uganda. Human Resources for Health 2009,7:76. End-of-Project Evaluation of the Capacity Project | 34 Generally there is no fully operational system that records staff level of training and in-service and continuing education received. There is no formal curriculum for in-service training. Individual staff normally identify continuing education opportunities themselves and then seek OIC and DHO approval to train. The village health teams are supervised by the health centers directly but also by the senior or an assistant health educator from the district level in some districts. There are checklists and reporting systems used. Monthly supervision is supposed to be the norm but this may not be followed due to lack of resources. Generally there are minimal or no training facilities for health at the DHO. Each training is planned for including all its consumables, which are purchased at the time of training. Training materials, when supplied, are those supported by whatever project is operational in the area. The data the evaluators obtained was incomplete but is consistent with a general increase in recruitment to over 85 percent of positions available and prompt payment of pay as a result of Project activities. Payroll access in districts increased on average from 50 percent to above 85 percent. 3.7 Other Major Donors As requested by USAID/Uganda the evaluators interviewed other major donors who planned to have activities in the next year in the area of HRD in health. World Bank The World Bank (WB) has extended to the Government of Uganda a US$130 million health sector loan, which has four components of health infrastructure development, leadership & management systems, human resources development and management, and project implementation and management. The details of the new project that the WB is funding can be seen in overview from the concept paper for the World Bank project document.14 The Capacity Project facilitated the development of this document for the MOH. Within the human resources development and management component, the loan will focus on technical assistance and training. With regards to HRD, the World Bank project will generally focus on 44 districts, while the Capacity Project and the WHO will cover the remaining districts between them. Within this component, the major areas are consolidating central level HRH functions; strengthening personnel management functions; improving staff retention in remote and hard-to-reach areas; and improving pre-service and in-service education. The leadership and management systems component covers performance-based management approaches; hospital regulatory framework; professionalizing management of health facilities; logistics and supply chain management and strengthening delivery of health services at the district level. Within hospitals, an emphasis will be placed on strengthening teams of dedicated experts dealing with clinical management of newborns and mothers. Most support will be through the central level but some support will be given to some districts directly with specific needs. 14 Uganda Health Systems Strengthening Project Concept Paper Prepared for World Bank Funding Republic of Uganda Ministry of Health, August 2009 End-of-Project Evaluation of the Capacity Project | 35 The World Health Organization (WHO) The WHO works to provide technical support to countries in its key areas of intervention, such as policy guidance, development of guidelines, and health research. The WHO’s focus, therefore, is on improving HR planning capacities using evidence from HR information systems. The second concern is improving health workforce productivity. A strategy for incentives for health workers in hard-to reach areas was drafted years back but there is still need for dialogue and agreement between the MOH, MOPS, and MFPED and PNFPs. This issue has been on the table for debate for about 10 years, and is not just about salaries, but broadly about equity issues. A pilot intervention was implemented in 2007/2008 with funding from DFID channeled through UNICEF, but initiated by the UN system under its humanitarian response initiative. Health workers in the Northern Districts were paid a top-up equivalent to 30 percent of their monthly salaries for six months. This initiative had a positive impact—in terms of attracting staff but only for low cadre staff. It didn’t work for doctors. Adverts for vacancies were put out, and it was mentioned that the top-up would apply to existing as well as new staff, and those in PNFPs. There was an overwhelming response—but again not from doctors. Such an incentive system is at the top of the WHO and other partners’ agenda, but it cannot be operational zed until there is agreement. PNFPs, for instance, are concerned that if it is implemented only in government facilities, their staff will run away to work in government. Danish International Development Agency (DANIDA) DANIDA is working closely with the Belgian Technical Cooperation, UNFPA, and the Italian Cooperation in the sector-wide framework. Their focus on system support is on improving skills at the lower level, in the local governments where service delivery takes place. They support two main areas: relevant training and technical support supervision. They have an elaborate framework for technical support supervision based on area teams composed of central level and regional staff (from regional referral hospitals) providing regular back-stopping support to districts. Their approach to training support is to provide scholarships for staff to upgrade, with a current focus on nurses and mid-wives. The informant, Peter Ogwang of DANIDA, saw management capacity of health managers, as a direct and deliberate investment. In the past all people posted in management positions had to spend 2 weeks at the Uganda Management Institute to get grounding in the specifics of management in public service. Belgian Technical Cooperation The Belgian Technical Cooperation (BTC) supports the Government of Uganda in the health sector through two mechanisms. Each year they give €5 million through sector support to the GoU as they participate in the 10 technical working groups supporting health one of which focuses on human resource development. They also have a €6.5 million project running over four years that supports capacity building working centrally and in two regional referral hospitals, a few general hospitals, districts, and health sub-districts in areas nearby. They informed evaluators that it is worthwhile for USAID and the Capacity Project to familiarize themselves with the three-year priorities of the Joint Budgetary Support Framework managed through the Office of the Prime Minister that works with the MOH. Budget support donors donate through this framework using a common frame with four sets—water and sanitation and health are two of End-of-Project Evaluation of the Capacity Project | 36 these sets. Priorities are set year by year over three years. Indicators at least on deliveries, immunization, drug supplies, and human resources monitor progress. The Swedish International Development Agency is also giving project support to the tune of €1.2 million in similar areas of activities to the BTC. From a technical document available through the UMEMS office, the evaluators were able to get an overview of their project.15 The project plans to create two formal training centers for training of HSD management teams located at HCIV level, and work with the MOH to develop a coordination system for donor capacity building efforts. Close collaboration will take place with the Institute of Tropical Health Antwerp, and the School of Public Health at Makerere University. Synergies and coordination will be sought with USAID, the World Bank, and Japan International Cooperation Agency (JICA). 3.8 The For-Profit Health Sector in Uganda in the Area of HRD The evaluators discussed the for profit health sector with the COP of the Capacity Project, MOH personnel, and staff at the School of Public Health at Makerere University. Generally no formal system exists. Individuals are usually approached because of personal connections. Newspaper advertisements are generally not fruitful. The COP of the Capacity Project provided us with a list of Ugandan private consultants in the area (Appendix 13(a)). The MOH often approaches staff members at the University of Makerere for consultancy work. Potential consultants from the School of Public Health at Makerere University are usually contacted by MOH staff or NGOs directly on a one-to-one basis. A list of the teaching staff at the School of Public Health, their area of specialization, and qualifications is in Appendix 13(b). The Martyrs’ University also has a reputation for teaching health management topics and has an agreement with the MOH to do so. 4.0 CONCLUSIONS AND LESSONS LEARNT 4.1 Overall The project has been successful in stimulating and supporting initial implementation of specific interventions at the central level and in a few districts to improve HRH. It has contributed to increased acknowledgement and operational focus on HRH issues as priorities in health service improvement, as evident in generation and application of HRH data in health service planning, and in recruitment of additional health personnel to rapidly fill critical gaps in district health services. Although the project approach adopted was useful in linking multiple government sectors with a stake in HRH improvement, it also resulted in innovations and successes that are not fully integrated into the routine operations of districts and central government. Sustained implementation of such interventions and their scale up to all districts in the country will require further support and necessary adaptation in the approaches used to empower and support government staff as the main drivers of interventions. 4.2 Appropriateness of the Capacity Project Approach 15 Technical & Financial File. Institutional Capacity Building in Planning, Leadership and Management in the Ugandan Health Sector Project. Uganda DGDC Code NN 3008322 BTC Code UGA 017011. End-of-Project Evaluation of the Capacity Project | 37 The HCD cycle (plan, develop, and support) was appropriate as a framework to define and respond to specific issues in planning, developing, and supporting HRH improvement. Project interventions strengthened data-based decision making on HRH deployment, in enhancing mechanisms to support continued professional development of health personnel, and in reinforcing support systems to increase health worker motivation, productivity, and efficiency. However, other challenges in the broader health system context (e.g., inadequate equipment, drugs and other supplies) were key mitigating factors that prevented optimal benefit from the HRH improvement interventions. The projects approach to implementation of first asking stakeholders what assistance they wanted and moving from there to show how the project implementation met their expressed need was widely appreciated, especially by the MOH counterparts. 4.3 Achievement of Intended Results The project was successful in realizing most of the stated outputs in the project plan. Where the project did not achieve planned results was reasonable and usually beyond their control (See Table A10 in Appendix 10). However, the project design and monitoring indicators did not include explicit outcome and impact results, although these were often available in separate follow-up evaluation reports and publications. 4.4 Enhancing Capacity for Policy and Planning The project strengthened capacity for policy and planning by strengthening systems to generate dependable data, supporting analysis and publication of key information to inform HRH decisions, and facilitating revision, dissemination, and application of key policies and guidelines relevant to HRH improvement. 4.5 Strengthened Health Workforce Systems and Improved Performance The Project strengthened health workforce systems with regards to much easier determination of health workforce needs, advertising for new staff, short-listing of applicants, and their orientation. However, further follow-up facilitation is needed to make wide use of the iHRIS more sustainable through more careful selection of who is going to be in charge of the system, the number of operators to train at each site, and sustainable hardware and software support. The performance improvement assessment system linked to follow on planning and implementation of supervision had limited success because many DHTs did not do the follow-on planning and implementation that they were meant to do. The procedures used were also too complicated for health personnel in health centers at the lower levels. Modification and further development of these systems needs to be undertaken to encourage their wider use. 4.6 Key Project Successes and Failures The notable successes of the project include the roll-out of HRIS at the central level and in nine districts, support to districts to recruit health personnel, and support to the MOH to produce a number of key documents to inform HRH support and improvement decisions. The project played a critical role in development and functioning of the HWAB as a functional reference End-of-Project Evaluation of the Capacity Project | 38 group on HRH, and in coordinated planning for necessary adjustments in the HRH payroll and its management. However the project successes at the level of national and district strategic decision-making were not matched at operational level, especially when interventions were meant to be carried down from the DHT level to lower-level health centers. Challenges such as inadequate operational funding, and medicine stock-outs were a key constraint at this level. 4.7 Unintended Results Payroll management was not initially planned for and a big impact was not anticipated but it proved to be very popular and a major motivator for staff retention. The general accessibility to the payroll increased from about 40 percent to 85 percent for health personnel in the districts, and from 90 percent to 95 percent at the national level. The payroll intervention had spillover effects that were beneficial to all public service employees. Similarly, the iHRIS in Lira was found to be beneficial to all public service district staff, not just those in the health sector. This was a deliberate intervention whose impact was far more than expected because helping health staff also benefited all employees who were paid through the public service system. 4.8 Factors Hindering/Enabling Success Effective implementation of project interventions was a result of an appropriate design and implementation strategy, existing momentum and basic commitment to HRH improvement globally and in Uganda, adequate funding to support all project activities, and the appropriate people deployed in project leadership as technical assistance to government and as counterpart leaders in the government offices that worked with the project. Weak government IT infrastructure and support systems, inadequate funding for routine health systems operations and inadequate equipment and supplies in health facilities were important constraints to project implementation and sustained benefit from its interventions. Staff mobility and absenteeism, known to often be over 48 percent, were common impediments. Staff may continue in a similar role but move to a different district rather than resign. 4.9 Remaining Gaps The HRH interventions introduced or strengthened by the project have worked well through a project approach. There remains a critical need to sustain this experience as an integral element in routine government functioning at the central level and in the districts reached. The project successes most evident in the three districts of focus need to be taken to scale in the other seven districts where partial interventions were supported, and indeed across the country. Task shifting, especially in relation to the roles of the medical officer versus that of clinical officers and nurses, is an important area to be addressed. Indeed the important role that VHT members and community members can have in promoting attendance for antenatal care, follow up of PMTCT, promotion of HIV prevention and reproductive health at low cost needs to addressed, though not necessarily through this project. To quote the WHO “From 70 to 90 percent of all sickness care takes place in the home. Household members, especially mothers, make the primary diagnoses of illnesses, assess the severity and likely outcomes, select among available providers and treatment End-of-Project Evaluation of the Capacity Project | 39 options, and procure and administer treatments.”16 Interventions, including those for HIV have been demonstrated to be relatively inexpensive in Africa.17 4.10 Lessons Learned The constraints of working in the Ugandan health system—high staff mobility, high absenteeism rates, and low morale in some workers—mean that capacity building is problematic. Sustainability and scalability of activities to strengthen systems should therefore be taken into account at the planning stage. For example, in establishing iHRIS systems at the district or indeed the central level, not just the proffered users should be trained but also several other personnel who have a high chance of rotating into the role of current designated users. How the system is likely to be used in the real world needs to be examined at time of planning and setup on the basis of what happens already needs to be considered. Two backup systems or, as practical and affordable, distant or on-line backup solutions need to be considered. If there is only one backup system, then surely it will fail. Perhaps the CPO should have an emergency backup disc in his/her safe. Systems that rely on local planning and trickle down of learning at the district level to lower levels in the health system often did not work. What is already happening in a particular location before training is a good indicator of what is likely to happen after training. Lack or resources is a common, perhaps legitimate excuse, but even supplying resources for supervision does not mean that they will be used appropriately. Capabilities of local actors need to be considered. In some locations, mentoring of particular individuals may be needed. Motivations that would encourage DHT members to do follow-up planning and supervision need to researched and tested. Addressing HRH issues necessitates addressing other non-HR aspects if health workers are to translate their competences into improved services. The required resources, supplies, and a conducive working environment need to be considered also. In this respect, the efforts by the current Capacity Program to work closely with other USAID implementing projects that may provide these other necessary inputs is commendable. The involvement of local actors from an early stage in planning, implementation, and evaluation creates ownership and commitment towards sustainability. For example, this was the case in the establishment of the iHRIS system as expressed by Dr. Edward: “Everybody felt they were able to contribute and were part of the process of developing a human resources database for Uganda that would be functional. That approach ensures that the program that’s being designed fully meets the local requirements and promotes use of information from the human resources team for policy decisions and allocations. The three key words here are ownership, sustainability and capacity-building.” (Dr. Edward Mukooyo, Assistant Commissioner, Resource Center, Uganda MOH)18 16 WHO, World Health Report 2002. Reducing Risks, Promoting Healthy Life. WHO Geneva 2001 17 Community Directed Interventions for Major Health Problems in Africa. A multi-country study final report. Special Program for Research & Training in Tropical Diseases UNICEF/UNDP/WHO/World Bank ISBN 9789241596602 2008. End-of-Project Evaluation of the Capacity Project | 40 Mapping and analysis of complex processes can produce key unexpected results. This was the case when the project analyzed reasons for staff satisfaction and raised morale of many health staff greatly by increasing the rate of timely access to payroll from 45 percent to 85 percent. 5.0 RECOMMENDATIONS 5.1 Introduction The recommendations below are based on the findings described in the previous section, the project related documents viewed, and the reading of the current relevant literature and experience in similar circumstances. It is important to remember that the Capacity Project has been implemented in Uganda for just over three years under the many constraints have listed in section 3.6. Many of the activities only began in 2009 and so there has been limited time for promising activities to reach their full potential. Therefore many of the recommendations are basically concerned with continuing activities that are showing promise while others are new. The latter will be elaborated upon in more detail. It can be assumed that the recommendation is for the project to follow the same processes as previously stated unless there are comments otherwise. Recommendations are grouped according to the relevant Objective. Other recommendations in relation to broader issues raised in Section 3.6 are grouped here. 5.2 Enhancing Capacity for Human Resources for Health Policy and Planning The Capacity Program should provide further support to the health professional councils and MOH Resource Centre/HRD/HRM to consolidate HRIS operation. This is a follow-on from previous project activities and would deal with such issues as keeping data input up-to-date, conduct regular analysis and produce reports as needed, and apply data in operational and strategic decisions and actions. Using the previously successful approach, support to the professional councils should include development of comprehensive strategic plans and resource mobilization strategies to enable them to fulfill their mandate as defined in law. The MOH Human Resource Development Division should be supported to develop and utilize an HRIS Qualify that captures data on all HRH trained in the country, including nationals who may go abroad for health-related training. District health offices, with necessary support from MOH and the Capacity Program, should lead the process to refine the district HRIS roll-out strategy. This should include building the district HRIS as a shared resource, useable in the district health office, district personnel department, and the district planning unit. Effort should be made to learn from and integrate local innovations in database development (e.g., the Access-based system in Kabarole district), to inform HRIS adaptations that would enhance sustained utilization in the districts. Training for district users of the HRIS should be structured to meet the needs of basic or starter IT users, and should include appropriate reading 18 Capacity Project Knowledge Sharing: A National Impact: Local Ownership of Health Workforce Initiatives in Uganda, November 2008 End-of-Project Evaluation of the Capacity Project | 41 materials as handouts. Follow-up support to districts after training in HRIS use should include a minimum of monthly remote contact (e.g., phone discussion or e-mail exchange) and quarterly visits that provide for adequate time to review and address issues on system utilization, and generation of analysis reports and using them to discuss specific HRH issues in the district. Districts should be supported by the Capacity Project and the appropriate organs in central government (e.g., Ministry of ICT, Uganda Communication Commission, MOH-Resource Centre) to develop basic IT troubleshooting and maintenance capacity. The Capacity Program, the MOH, and district local governments should include clear mechanisms and adequate provision for operation and maintenance of new technologies introduced for data and information management to enhance their sustained utilization. Consideration should be made to integrate operation and maintenance of the new technologies into existing mechanisms for equipment management. Necessary adaptations in existing systems (e.g., training of personnel supplementing toolkits and expanding connections to expert support through referrals) should be provided for, and monitored as necessary to ensure performance and sustainability. The Capacity Program should provide further support to the Health Workforce Advisory Board (HWAB) to consolidate its role as an HRH reference group that cuts across the different government sectors, and includes representation from the other stakeholders. Other stakeholders suggested to be involved include training and research institutions, faith￾based agencies operating PNFP health facilities, and NGOs involved in health care, etc. This broader involvement would raise the influence and profile of the HWAB and enable it to promote good quality HRH. Support should be provided to refine the terms of reference for the HWAB and its mandate to coordinate and guide HRH issues The Capacity Program should expand engagement with and support to associations for health professionals, including the medical workers union, as a mechanism for strengthening workforce organization, mobilization, and motivation. The project should reinforce institutional strengthening and strategic planning support to UNANM. Dialogue should be established with associations for Allied Health Professionals, Medical and Dental Practitioners, and Pharmacy professionals to explore ways in which their role in mobilizing and motivating health professionals can be strengthened. Specific partnership should be developed with associations of private practitioners (e.g., the Private Midwives Association, and Private Medical Practitioners Association), as an entry point for capturing HRH and service data from this important domain of the health workforce. The Capacity Program should build on the experience of partnership with UPMB to enhance support and inclusion of the broader range of PNFP health service providers as key players in training and deployment of HRH. The project should provide further support to UPMB for HMIS improvement, solidifying and extending their previous success in this area, and apply relevant lessons to strengthening HMIS End-of-Project Evaluation of the Capacity Project | 42 practice in the other PNFP medical bureaus, UCMB and UMMB. The bureaus should also be supported to develop HRIS to capture their training and deployment of health workers. Their contribution to continuing medical education and to management of district health services (e.g., as HSD lead health facilities) should be specifically strengthened. 5.3 Strengthening Systems for Effective Performance-Based Health Workforce Development The Capacity Program should support the reorganized Health Manpower Development Center (HMDC) to harmonize pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs. HMDC is now in the process of being moved to part of HRDD. With this movement it should have a budget again. Before restructuring, this center was effective in communicating distance￾education for health professionals with up to 1,500 enrolled students per year. With adequate support it can become so again. The process of disseminating the standards for continuing education and the accreditation of institutes for delivering CPD not finished in this project should be supported in the new project. This process was proceeding successfully in the previous project. All the relationships and facilitation from the previous project are still in place. In order to develop effective performance improvement procedures, the processes used in this project should be reexamined and developed with local MOH input at all the levels in the health system that they are to meant to be used to make them more user-friendly. This process of local consultation and revision could be followed in a few trial districts so that new procedures can be developed. Experience of successful application of performance improvement procedures, as represented in the current instrument, by members of the Capacity Project overseas suggest that it is worthwhile giving them a further trial in the new project after local adaptation with appropriate stakeholder input. As part of this redevelopment of performance improvement procedures feedback loops should be established: (a) on supervisory assessments performed at the HC levels so that supervisors at higher levels can see that assessments are actually taking place at lower levels. (b) from higher to lower levels so that lower levels know what is being done with the data sent to higher levels. Such feedback loops improve these procedures as in (a) they let supervisors at higher levels know that programs area being followed. In the case of (b), feedback helps motivate those at lower levels to collect the desired data since they come to appreciate how it is being used and End-of-Project Evaluation of the Capacity Project | 43 analyzed. These loops can be established by establishing regular report flows for this purpose that are also discussed by supervisors when they visit. Once innovations in effective performance improvement (and other aspects of health program management) are established, their sustainability and ongoing development should be established by linking them with appropriate established institutions. Established institutions with expertise in health systems management include Martyrs University, which already has an agreement with the MOH for the training of health staff in management, and the School of Public Health at Makerere University. With such linkages, successful innovations can be maintained, further developed, and evaluated, despite the constant change of both MOH staff and supporting health projects. These linkages could be established by involving the selected institutions in the ongoing development of these tools from the early stages of the new project. Two experienced nurses should be recruited as consultants and based at the district level in two different regions to act as local workshop leaders, facilitators, and mentors of OICs in local HCs. Their key functions would be: 1) To follow up local project workshops at the district level, concentrating on those OICs identified as needing mentoring, to see that information and skills learned at this level are passed down to the HC levels. As indicated, with appropriate support, they can help with adaptation of materials used in workshops that are understandable at HCIII and HCII. 2) As a trial to support HCs IV (where there is no doctor present), III and II to aid development at these levels of one local learning center in HCs at each level in one location. The HC chosen for these centers will be determined by the presence of a positive OIC and a local community open to working closely with their local HC. At and through these centers: a. Performance appraisal assessment linked with supportive supervision will be established in association with that happening at higher levels in the district. b. The building of local positive learning organizations and teams will be established to facilitate the motivation and delivery of good quality primary health care services, especially for reproductive health and HIV. c. Once these HC learning centers are functioning well, other OICs and staff from surrounding HCs can attend for peer mentoring, discussion, and skill-building so that they can return to their own facilities and apply what they have learned there. d. The development of local consultative networks can be facilitated between local OICs so that they can give one another mutual support and help in solving common problems. End-of-Project Evaluation of the Capacity Project | 44 e. If as recommended below, training in management is undertaken for health center OICs, these consultants could follow-up practice of learning at workshops, through local mentoring or indeed lead such workshops themselves. The process to establish these centers would include: a. Careful selection of the nurse facilitators. These nurses would already have qualifications and experience in management, be relatively experienced and dedicated, selected with some input from nurses in the area in which they will work and already orientated to living in rural areas.19 b. Careful selection of positive OICs and associated communities where these centers are to be established so that the chance of initial success can be maximized and positive learning processes established. c. Training of the consultant nurses in the group processes necessary to facilitate the building of learning organizations. To build local positive learning organizations, a basic approach to team building, development of common goals, group problem solving and appreciative enquiry techniques would be built through a facilitated group process. These techniques are now used in USAID-funded child survival programs. An appropriately skilled external consultant would need to be recruited to facilitate this process through recurrent visits over a one-year period. In addition, community support and appreciation of their local health workers would be built through locally appropriate processes such as political support. (The project Chief of Party is experienced in what may work best in Uganda to build community support). d. Establish these centers through the application of the facilitation processes learnt above gradually through a plan agreed by consultation between the local HC members and their community over an initial six months trial period. e. Once a few centers are successfully functioning, the consultant nurses could move on to other health centers where the nurses invite them. Through this on-site learning and peer learning and support this approaches would gradually go to scale. It is proposed that these local learning centers be NOT resourced at any more than the basic level required for the delivery of good basic health care. At most, accommodation for a few visiting staff at a time should be added to these centers. The establishment of such centers is proposed because, appropriately facilitated, they have potential to address many facets of poor motivation, absenteeism, and poor esteem contributing to poor delivery of health services in rural Uganda. Although it is common knowledge that attitudes of health professionals at lower levels in rural health services in Uganda towards their work and of community members towards them is generally negative, this does not mean that 19 Apparently such nurses would be relatively easy to find—indeed an interviewee from the WHO already can think of some suitable candidates from her involvement in health system management courses at the Martyrs University. End-of-Project Evaluation of the Capacity Project | 45 such a change is not possible. Change often begins with “early adopters” who are later joined by others as these see change is possible. Such an approach, properly facilitated, concentrates on individual and group strengths and appreciation of one another rather than problems, builds group cohesion, and mobilizes group-think towards solving manageable problems.20 A study should be carried out of those who are regularly practicing supportive supervision in demanding areas in rural Uganda to ascertain their characteristics. A current area of management development is the use of Positive Deviance analysis in identifying those managers who are practicing well despite demanding circumstances that defeat many others. Such a study may identify some characteristic that are worth including in training programs. Indeed identifying these positive deviant practitioners as models and leaders for training would be useful. a) Follow-up evaluation of the application of the training of trainer’s courses in the Learning for Performance Approach method should be performed early in the new project. b) Follow-up evaluation of the application of the training of trainer’s courses for District trainers in Kabarole on mentoring training on infection control should be performed early in the new project. These activities were performed in 2009 and so apparently there has not been time for follow-up evaluation of the application of the skills learned in these trainings. Such a evaluative report would, of course, include examination of the course reports documenting the skills acquired by students trained by these trainers. 5.4 Identification and Promotion of Health Workforce Management Practices for Improved Performance and Retention Management capacity building training should be provided for the health facility IV, III, and II levels with an emphasis on planning and budgeting, supervision of staff, and relationship with the communities. The basis for this recommendation is that it was commonly observed by the evaluators that OIC Health Centers often lacked basic management skills. This recommendation can be implemented by a) on-the-job management training courses for the OICs of health facilities designed in modular form to enable them work and practice at the same time, and b) posting management mentors at some selected health center IVs who will support the planning and management functions for a defined period of time. The management mentor will also support the HCIII and II. The two nurse consultants mentioned above could also support these processes. Since the Martyrs University already undertakes this type of training on behalf of the MOH, the possibility of them undertaking this role for the project should be explored. Staff members at the School of Public Health at Makerere University have also lead short training courses in management at the district level for NGOs. 20 Hammond, S.A. (1998). The Thin Book of Appreciative Inquiry. Plano, TX: Thin Book Publishing Company. End-of-Project Evaluation of the Capacity Project | 46 The orientation of new staff should be comprehensive and include both the general and the technical aspects to ensure a solid foundation for service provision. The orientation should be done through attachment of staff to an officer of a particular title and job description (internship) for at least one or two months. It should be the responsibility of the DHO/DHT that this orientation is carried out. An on-the-job training curriculum should be developed that covers critical health issues and challenges, roles and standard procedures, the health worker’s role as part of a mutually supporting health team, relationships with the community, and workplace safety. As above, for sustainability, it may be desirable that this activity be facilitated by an established institution. A system for staff development should be reestablished and those with responsibilities in this area should receive formal training in planning and supporting staff development. An inclusive and ongoing consultation process including key stakeholders, likely students, and those with responsibilities in this area, should take place so that a practical and sustainable system is developed and adopted. This system should encourage staff to pursue their own professional development while at the same time encouraging them to develop or strengthen the skills needed to deal with local health needs. The Capacity Program should continue supporting DSCs to undertake recruiting, confirming, promoting, and disciplining district staff. Running the business of DSCs was found to be very expensive for districts. Some districts reported a big backlog of staff that are overdue for confirmation and promotion, which indirectly affects their motivation and retention. Continuing support to DSCs, with which many good relationships have been established, is desirable until improvements in the above areas become institutionalized and so more likely to be maintained. The Capacity Program should continue to support payroll management reform and help develop a feedback mechanism from MOPS to districts. During the previous phase of the project, district personnel were trained in filling in payroll forms. However, some districts still face challenges in getting their staff onto the payroll. They attribute this to unexplained problems at the MOPS. More hands-on training by MOPS staff should be facilitated until the level of error by all parties involved is reduced to an acceptable level. The Capacity Program should facilitate the implementation of the health sector staff motivation and retention strategy. The motivation and retention strategy has elements on staff remuneration and benefits, opportunities for career progression, institutional accommodation for health workers, support supervision, safety at workplaces, continuing professional development and in-service training, and other job and work environment elements. All these elements require financial and technical End-of-Project Evaluation of the Capacity Project | 47 support if they are to be realized. A key aspect of this strategy is to work out special incentives to attract and retain health workers in the hard-to-reach districts. Working with other health sector partners, the project can facilitate the implementation of this strategy as practical. Given the project’s other commitments, this support may not be financial. The Capacity Program should follow-up on the earlier successful retention initiatives carried out by the UCMB and UPMB. Initiatives undertaken by these bodies with Capacity Project facilitation were successful in increasing staff retention in 2007. Although these initiatives are still in place, staff retention is down to its old level. Further support is needed to help them correct this situation. The short-listing software should be made usable for short-listing staff in other sectors. Since this software is used by the DSC, which manages all recruitments for the whole district, training DSC staff to use this software for all district recruitment will lead to the software becoming part of their normal procedures and so increase its sustainability, especially given the high rate of staff turnover of DSC staff themselves. 5.5 Other Recommendations Primary health care services budgets should be justified on the basis of lives saved by services delivered. There is no doubt that, even with improved efficiency and effectiveness, the current national health budget needs to be increased to improve health workforce and resources levels. Several interviewees have volunteered that a method is needed to justify increased funding in terms of results. The Bellagio (Lives Saved) calculator is an established method to quantify the health benefits in terms of lives saved of common primary health care interventions. Using a reasonably rigorous measurement of, usually, under-5 mortality (DHS Survey), an established measure of the efficacy of a particular intervention, and a change in coverage of the intervention, the benefit of the change in coverage in terms of lives saved can be measured. These calculations can, in turn, be used to justify the increased budget necessary to provide the resources, including manpower, to bring about this increase in coverage. This is a recently established valid practice. Further information about the Bellagio calculator can be obtained from http://www.jhsph.edu/dept/ih/IIP/list/index.html To implement this approach, a short-term consultant experienced in the use of the Bellagio calculator and budgeting needs to be recruited to train appropriate MOH staff members in the development of increased budgets for HIV/AIDS, reproductive health and primary health care services justified on the basis of measurable health benefits. The Capacity Program should continue its activity in promoting dialogue about task shifting. The role of doctors, nurses, and clinical officers in the rural Ugandan health system is clearly a End-of-Project Evaluation of the Capacity Project | 48 long-standing area of contention in which many agendas abound. The role of none of these groups can be considered in isolation from the others. Rightfully, doctors are concerned with standards of practice in relationship to complex areas such as the management of ARV therapy, but what is the evidence of what those few doctors present are actually doing? To forward this dialogue, it is proposed that a national level workshop be supported that helps key stakeholders build a common practical vision of what Ugandans want their rural health care system to be based on—the best evidence available. This workshop needs to be informed by actual current practice of doctors, nurses, and clinical officers in rural Uganda. Their representatives must testify and the current state of the literature, reasonable estimates of the costs of different scenarios, and reasonable representations of the current manpower system, present in many documents that the project has helped develop, must be examined. Informed by this background information, key stakeholders could be guided towards this common practical vision. A working committee towards such a national event would need to be formed to gather all the necessary evidence and present it in an accessible form for all stakeholders. For actual facilitation of the workshop, relevant costs need to be estimated beforehand and an expert in the field be available at the time of workshop to estimate any scenarios envisioned. Further, if wanted, the common vision could be widened to include the general roles of other health workers and that of the community. Again, an evidence-based background should be used to inform this vision. While this may be too big a step in one workshop, it may be an important step to include at a later date. Alternatively, with careful facilitation and political support, a momentum could be created that would lead to real change. End-of-Project Evaluation of the Capacity Project | 49 APPENDICES Appendix 1: Map of Uganda Showing Districts Visited Districts colored red are those visited by the evaluation team. Districts colored purple are the other districts in the Capacity Project. End-of-Project Evaluation of the Capacity Project | 50 Appendix 2: Capacity Project District Intervention Matrix, 2009 This matrix shows the activities implemented by the Capacity Project and the districts covered. However, the Workplace Safety training was done in all the 80 districts in Uganda. CAPACITY PROJECT DISTRICT INTERVENTION MATRIX 2009 DISTRICT HRIS ACTION PLAN PI PM RECRUIT MENT PAYROLL MGT WORK PLACE SAFETY VHT DEVELOP MENT Gulu * * * * Amuru * * * * * Kitgum * * * * Pader * * * * Lira * * * * * * Apac * * * * Dokolo * * * * * Amolatar * * * * * * * Oyam * * * * * * * Kabarole * * * * * * * * Hoima * * Kamwenge * * * Bulisa * * KEY HRIS Human Resources Information System PI Performance Improvement PM Performance Management VHT Village Health Team MGT Management End-of-Project Evaluation of the Capacity Project | 51 Appendix 3: Evaluation Itinerary Period of Evaluation The end-of-project evaluation for Capacity Project was done from 1 February to 10 March 2010. Details of Activity (including field travel work plan) Days and Dates Team Activity Wednesday 9 Feb Whole team Travel to North Thursday to Saturday 10-13 Feb Whole team Interviews in Lira Monday to Tuesday 15-16 Feb Group 1 Group 2 Interviews in Amolatar District Interviews in Apac District Tuesday 15 Feb Group 1 Interviews with NUMAT and Gulu School of Clinical Officer Wednesday 16 Feb Group 1 Return to Kampala Wednesday to Thursday 16-17 Feb Group 2 Interviews in Oyam District Thursday to Friday 18-19 Feb Group 1 Interviews in Kampala Friday 19 Feb Return to Kampala Monday to Friday 22-26 Feb Group 1 Interviews in Kampala Monday to Thursday 22-25 Feb Group 2 Interviews in Kabarole and Kamwenge Friday Group 2 Return to Kampala Monday to Friday 1-6 March Whole Team Interviews in Kampala Report writing/analysis of draft report Debrief to USAID Mission Monday to Wednesday 8-10 March Report writing Debrief to USAID Mission Submit Final Draft Report to USAID UMEMS End of Evaluation End-of-Project Evaluation of the Capacity Project | 52 Appendix 4: Documents Reviewed Capacity Project Documents 1. Request for applications (RFA) Number M/OP/GH/POP-04-961 Human Capacity Development Project, April-May 2004 2. Original Proposal for Human Capacity Development Project (HCDP) Number GPO-A￾00-04-00026-00, September 2004 3. USAID/Uganda UMEMS Technical Proposal for the Evaluation of the Capacity Project: Volume 1, January 2010 4. Scope of Work for End-of-Project Evaluation of the Capacity Project (CP), Dec 7, 2009 5. UMEMS Evaluation, Special Studies and New Activity Design Protocol, June 2009 6. Capacity Project Monitoring and Plan, September 2005 7. Evaluation of the Capacity Project: Assessing Progress on HRH Issues, August 2008 8. Annexes: Evaluation of the Capacity Project: Assessing Progress on HRH Issues, August 2008 9. Human Resource for Health Bi-Annual Report, April 2009 10. Human Resource for Health Bi-Annual Report, September 2009 11. Ministry of Health Sector Strategic Plan II, 2005/06 – 2009/2010, Volume 1 12. Capacity Project Quarterly Report, July – September 2009 13. Capacity Project Quarterly Report, April – June 2009 14. Capacity Project Quarterly Report, January – March 2009 15. Capacity Project Quarterly Report, April – June 2008 16. Capacity Project Quarterly Report, January – March 2008 17. Capacity Project Quarterly Report, October – December 2007 18. Capacity Project Quarterly Report, June 2007 19. Capacity Project Annual Report, October 2006 – September 2007 20. Revised Final Draft Capacity Project Work Plan, 2007 – 2008 21. Final Draft Capacity Project Work Plan, 2008 – 2009 22. Capacity Project Annual Report, October 2007 – September 2009 23. Payroll Management Manual for the Traditional Public Service 24. Strengthening Health Workforce Systems in Uganda; Review Report December 2009 25. Uganda Human Resource for Health Strategic Plan 2005 – 2020: Responding to Health Sector Strategic Plan and Operationalising the HRH Policy, June 2007 26. Uganda Health Workforce Retention Study: Satisfaction and Intent to Stay Among Current Health Workers, Ministry of Health, 2006 27. Uganda Health Workforce: Satisfaction and Intent to Stay Among Current Health Workers, April 2009 28. Competency Gaps in Human Resource Management in the Health Sector: An Exploratory Study of Ethiopia, Kenya, Tanzania, and Uganda. Prepared by the African Medical and Research Foundation (AMREF) and Management Sciences for Health (MSH), July 2009 29. Uganda Health Workforce Turnover Study, December 2008 30. Standards for Accreditation of Continuing Professional Development for Health Workers End-of-Project Evaluation of the Capacity Project | 53 31. Strengthening the Uganda National Association of Nurses and Midwives (UNANM) Communication Capacity, February – June 2007 32. Policy for Mainstreaming Occupational Health and Safety the Health Services Sector, February 2008 33. Mapping the Human Resources Management Processes in Uganda, March 2008 34. Health Workforce Action Framework: Integrative Analysis Reports, June 2008 35. The Payroll Management Manual for the Traditional Public Service, December 2008 36. Guidelines for Occupational Safety and Health: Including HIV in the Health Sector Services, February 2008 37. Motivation and Retention Strategy for Human Resources for Health, October 2008 38. Professional Code of Conduct and Ethics for Pharmacists in Uganda, September 2009 39. Professional Code of Conduct and Ethics for Nurses and Midwives, September 2009 40. Code of Conduct and Ethics for Allied Health Professionals, September 2009 41. Human Resources for Health Audit Report, May 2009 42. Leadership and Management Development, February 2009 43. Ministry of Health. Quality of Care Improvements Tools for MCH/HIV Services, October 2006 44. The Human Resources for Health Strategic Plan 2005 – 2020, A Health Workforce Crisis in Uganda: Time for Real Action – Supplement, 2009 45. Annual Health Sector Reform Report, Financial Year 2009/2010 46. An Overview of the Human Resource for Health Audit Report for 2008, Ministry of Public Service 47. Performance Improvement for Health Management Systems of the Uganda Protestants Medical Bureau, July 2009 48. Implementation of Performance Support Approaches in Central America and Uganda 49. Assessment of Capacity for Provision of PMTCT and Family Planning Services in Kabarole District Report, March 2008 50. Assessment Report on Action Points on Capacity to Implement PMTCT and Family Planning in Kabarole District, June 2008 51. Dissemination Workshop Report on HRH Assessment Report and follow-up visit in nine Districts in Northern Uganda (Acholi and Lango Sub Regions), 2008 52. Capacity Project Team-Building Retreat Report. Jinja Resort Hotel, 11-12 September 2008 53. Study of Nurses’ and Midwives’ Perception of the Roles and Functions of the Uganda National Association of Nurses and Midwives, 2008 54. Report on the Performance Improvement Assessment, Kabarole District, April 2009 55. Assessment Report and Action Points on Capacity to Implement PMTCT and Family Planning in Kabarole District, June 2008 56. Assessment of Capacity for Provision of PMTCT and Family Planning Services in Kabarole District, March 2008 57. Kioga Health Sub Draft Work Plan 2009 and 2010 (Revised) 58. Amolatar Health District Development Plan 2009/10 to 2010/2011 (Revised) 59. Performance Improvement Assessment Report for Dokolo District Carried Out in January 2009. Report of Performance Improvement Assessment in Gulu District 60. Project No. IAEHAR/2006/02 Support to Human Resource for Health (HRH) Component: 30 Percent Top-Up Final Report to DFID, 2007 End-of-Project Evaluation of the Capacity Project | 54 61. Oyam District Health Sector Integrated Annual Work Plan, Financial Year, 2009/10 62. HMIS Report 2008/2009, Oyam District 63. Erute North Health Sub-District: Ogur Health Center IV Annual Work Plan for Financial Year 2009/2010. 64. Kamwenge District Health Annual Development Plan, 2009/2010 65. Capacity Project Knowledge Sharing: The Impact of Human Resources Information Systems (HRIS) Strengthening. Daniel H. de Vries, Dykki Settle and Pamela A. McQuide, IntraHealth International, September 2009 66. Capacity Project Knowledge Sharing: Task Shifting—Field Officers in Uganda, April 2007 67. Supporting Health Worker Performance with Effective Supervision. Bruno M. Benavides, Jhpiego, September 2009 68. Strengthening Professional Associations for Health Workers. Nancy Yinger and Pamela A. McQuide, IntraHealth International, September 2009 69. Health Workforce: “Innovative Approaches and Promising Practices” Study: Incorporating Lay Human Resources to Increase Accessibility to Antiretroviral Therapy: A Home- Based Approach in Uganda, July 2006 70. The President’s Emergency Plan for AIDS Relief: Next Generation Indicators Reference Guide, August 2009 71. Evaluation of the Capacity Project’s Human Resources Information Systems (HRIS) Strengthening Process in Swaziland, Uganda, and Rwanda, July 2009 72. Global Health Workforce Alliance: Working Group on Tools and Guidelines 73. Capacity Project Knowledge Sharing: From Nightmare to Awakening—Performance Improvement for Health Management in Uganda 74. Planning, Developing, and Supporting the Health Workforce: Results and Lessons Learned from the Capacity Project, 2004-2009 75. Capacity Project Knowledge Sharing: The Capacity Project in Uganda, November 2008 76. Capacity Project Knowledge Sharing: Building HR Information Systems—Leading the Way Together in Uganda, February 2007 77. Capacity Project Knowledge Sharing: A National Impact—Local Ownership of Health Workforce Initiatives in Uganda, November 2008 78. Performance Improvement Toolkit. IntraHealth/PRIME II Project (implemented in partnership with in collaboration with partners Abt Associates, EngenderHealth, Program for Appropriate Technology in Health and Training Resources Group, and supporting institutions, the American College of Nurse-Midwives and Save the Children. http://www.intrahealth.org/sst/intro.html (Accessed 22 February, 2010) 79. Capacity Project Knowledge Sharing: Sharing Knowledge—From Nightmare to Awakening Performance Improvement in Health Management in Uganda, April 2009 80. Capacity Project Knowledge Sharing: Supporting Health Workers Performance with Effective Supervision, September 2009 81. Capacity Project Knowledge Sharing: Knowledge Management and Human Resources for Health Using Quality Information to Make Better Decisions, September 2009. 82. Uganda Health Systems Strengthening Project Concept Paper Prepared for World Bank Funding, August 2009 83. The Uganda In-Service Training Strategy – 2000 End-of-Project Evaluation of the Capacity Project | 55 84. A Review of the In-Service Training Strategy of the Ministry of Health Uganda, September 2007 85. Health Sector Strategic Plan III 2010/11-2014/15 End-of-Project Evaluation of the Capacity Project | 56 Appendix 5: List of Key Persons Interviewed at the National and District Levels Serial No. Name Title Contact Ministry of Health and other institutions in Kampala 1 Dr. Ndiku John Medical and Dental Practitioners Council 2 Dr. Juliet Bataringaya Wavamunno National Professional Officer, Health Systems Development – WHO Uganda 3 Ruth Nabaggala UPMB Monitoring & Evaluation Officer 0414271776/0772415 886 4 Isaac Kagimu UCMB HRH Technical Advisor 5 Dr. George Bagambisa, Asst. Commissioner, HRD – MOH 6 Okello Patrick OIC Payroll Management in the Ministry of Public Service 7 Kamya Mugalu Assistant Commissioner HRM, Ministry of Public Service 8 Charles Isabirye Senior Health Training Officer, HRD Division 9 Dr. Michael Igune Director HMDC Mbale 0772565494 10 George Pariyo School of Public Health, Makerere University 11 Dr. Joshua Kiwanuka Uganda Medical Practioners Board 0772418281 12 Dr. Luc Geysels Belgian Technical Corporation 0414230543 13 Rose Kato Belgian Technical Corporation Uganda Nurses and Midwives Council (UNMC) 14 Janet Obuni President 0712845181 15 Patrick Baleganya General Secretary 14 Juliet Ezaga Senior Nursing Officer, OIC Training 0772452604 15 Eunice Tushemereirwe Data Manager/Records Officer 0772513194 16 Rebbeca Kwemogera Senior Records Assistant 0772928547 17 Joyce Muwanga Senior Nursing Officer, OIC Administration 0772369888 World Bank 18 Dr. Peter Okwero World Bank Danish Embassy 19 Peter Ogwang Danish Embassy Swedish International Development Agency (SIDA) 20 Ulrika Hertel SIDA OIC Health System 0772707103 Amolatar District 21 Dr. Quinto Okello District Health Officer (DHO ) 22 Harriet Miriam Atim Senior Nursing Officer (SNO) End-of-Project Evaluation of the Capacity Project | 57 23 Ambrose Ocen Acting Secretary to the District Service Commission/Town Clerk at Amolatar Town Council 24 Sharon Atimango Enrolled Nurse 25 Henry Oddi Health Assistant, Aputi Health Centre III 26 Ogwal Alex Principal Health Inspector/HIV/AIDS focal person 0772676408 0392887985 Apac District 27 Dr. Emer Mathew District Health Officer 28 Sabiti Mbabazi Atenyi District Health Inspector, Environment 29 Opio A. Peter.Chrisostom. Principal Dispenser, Apac Hospital; also Chairperson Apac District Medical Workers’ Union 30 Sr. Anne Grace Auma In-Charge Pediatric Ward (and Member, Infection Control Committee), Apac Hospital 31 Sr. Consolata Akulu Assistant DHO 32 Francis Odap Principal Personnel Officer 33 Dr Semugenyi OIC Aduku HCIV Kabarole District 34 Dr. Richard Mugahi OIC Rukuunyu HCIV 35 Tonny Mugisa OIC Ruteete HCIII 36 Fulgensio Ntegyereize Acting DHO/Assistant District Health Officer, Environment 0775401191 37 Rita Ahaisibwe Records Officer 0782902562 38 Kamuhanda Gideon Senior Personnel Officer 39 Mpuuga Hosea District Health Educator 0772654376 mpuugahosea@yahoo.co.uk 40 Maanimake Elizabeth Senior Nursing Officer 0772665338 bethmake@yahoo.com 41 Baguma Joseph Acting DHDI 0772563907 bagwojoseph@gmail.com 42 Tumwebaze Mathias DHT Member 0772916618 mathias@yahoo.com Kamwenge District 43 Magezi Kisembo Acting OIC Ntara HCIV 44 Katembo Mwesigwa Acting OIC Rukunyu HCIV 45 Kabatoro Pussy Nursing Assistant Bigodi HCIII 46 Habassa Elia Nursing Assistant Bigodi HCIII 47 Katirimo Gad HMIS 48 Ssozi Chris G. Secretary, District Service Commission 49 Mpaka A. John Principal Health Inspector 50 Boaz Katuramu Principal Personnel Officer 0772662466 Lira District 51 Bernard Ogwang Principal Personnel Officer/Secretary District Service Commission End-of-Project Evaluation of the Capacity Project | 58 52 Lillian Eyal District Principal Personnel Officer 0772663258 53 Henry Omoo HMIS Statistician 54 Okao Ben District HIV Focal Person/TB and Leprosy Supervisor 55 Ojok Milton Personnel Officer 56 Dr. Tom Odongo OIC Ogul HCIV 57 Thomas Bell Akaki Senior Health Educator 58 (Name not given) Enrolled Midwife from Olilim HCIII 59 John Nelson Opio Assistant DHO, Environmental Health 60 Paul Omoko Town Clerk, Lira Municipality 61 Dr. Peter Kusolo District Health Officer (DHO) 62 Leo Okol Medical Records Assistant 63 Fred Ogwang Medical Workers Union, Lira District Branch 0772398613 64 Sr. Christine Atyam Senior Nursing Officer/Acting Deputy RH Oyam District 65 Acan Margaret Ogomarach Senior Nursing Officer, DHO’s Office 66 Francis Leone Ocheng DHI (also focal person for performance improvement) 67 Orech John Bosco Senior Health Educator 0772831744 68 Michael Mulindwa Principal Personnel Officer 69 Sarah Awor Biostatistician 0702227668/0774470 761 70 Richard Atepo Assistant Health Educator 0772951367 Other Districts 71 Dr. Samuel Ojok District Health Officer, Dokolo District 72 Kiboko Olobo James Principal Head, Gulu School of Clinical Officers 0782440375 kikek2james@yahoo.com 73 Martin Onek Ocaya Deputy Principal, Gulu School of Clinical Officers 0772594290 martin.onek@gmail.com 74 James Otim Deputy Chief of Party, NUMAT 75 William Oloya Program Officer, NUMAT Capacity Program (Former Capacity Project) 76 Dr. Vincent Okectho COP Capacity Program 77 Fred Kakaire Chief Executive Officer, Uganda Chartered Healthnet – Makerere University College of Health Sciences 78 Ernest Mwebaze Former Employee – Healthnet Uganda End-of-Project Evaluation of the Capacity Project | 59 Appendix 6: Guidelines for In-Depth Interviews 6A: General Key Informant Interview Guide for National-Level Issues for discussion 1. The strategic focus of the Capacity Project (CP); how it fits with the priorities for Uganda Please describe/explain to me your understanding of the Capacity Project and its intentions Use prompts where necessary: its goal and objectives; the key elements in its design; the results it intends to achieve at different levels; any differences in its intentions at global, country, and sub-national levels. How do the intentions of CP fit with/contribute to the priorities for Uganda? Prompts: in health service development and support, in HRH development, in USAID support objectives. 2. The specific value that CP adds to health and development in Uganda What do you consider to be the main ‘value addition’ from CP to health and development in Uganda? Prompts: value from the project design and outputs; value with respect to HRH development in Uganda; value with regard to health service delivery (Especially in AIDS, RH) value with respect to an integrated approach to health service planning and development. 3. Gaps, weaknesses in CP design, implementation What gaps or weaknesses (if any) do you see in the design and implementation of CP in Uganda? Prompts: aspects of HRH that CP did not recognize/address; priority health problems and interventions that the project did not recognize/support; important stakeholders in HRH that CP did not recognize/include as necessary. 4. Specific CP activities implemented in each organization What activities did the CP undertake in relationship to your HRD responsibilities? Was the activity well or not well implemented? Explain? What hindered or facilitated the activity? Was the CP approach used appropriate? Suggestions to improve it? When was last CP support given in relation to this activity? Is this activity being maintained now? Why/why not? Where any activities that you know about planned but not implemented? Why? 5. Key lessons, good practice experiences from CP Please give examples of key lessons or good practice experiences (if any) you know about from CP implementation in Uganda. Prompts: aspects of the project that performed particularly well; what other countries have come to learn from in CP implementation. End-of-Project Evaluation of the Capacity Project | 60 6. Suggestions for future HRH support/work Please make suggestions on what you consider to be important things for the new Capacity Project in Uganda to focus on and address. Prompts: what design modifications to do from CP I; changes in the implementation approaches and processes; changes in focus at national level, changes in focus at district level; changes in sector inclusion and involvement. End-of-Project Evaluation of the Capacity Project | 61 6A1: Key Informant Interview Guide for National-Level—Respondent Category 1: Strategic Leaders (MOH, CP-COP, and USAID) Introduction: Introduce yourself to the respondent, and mention the focus of this evaluation is: • Our understanding of Capacity Project in Uganda – part of the global whole (active in 48 countries); a three-fold focus (plan, develop, support HRH) • This evaluation – commissioned by USAID/Uganda, to focus on CP results and lessons from the Uganda work, to inform priorities and methods in a new Capacity Program started in October 2009 • Evaluation approach – both strategic and operational, cover national and district-level work, use qualitative methods (Documents Review, Key Informant Interviews) Issues for discussion 1. The strategic focus of CP; how it fits with the priorities for Uganda Please describe/explain to me your understanding of the Capacity Project and its intentions Use prompts where necessary: its goal and objectives; the key elements in its design, the results it intends to achieve at different levels; any differences in its intentions at global, country, and sub-national levels How do the intentions of CP fit with/contribute to the priorities for Uganda? Prompts: in health service development and support, in HRH development, in USAID support objectives, etc. 2. The specific value that CP adds to health and development in Uganda What do you consider to be the main ‘value addition’ from CP to health and development in Uganda? Prompts: value from the project design and outputs; value with respect to HRH development in Uganda; value with regard to health service delivery (in AIDS, RH, etc.) value with respect to an integrated approach to health service planning and development, etc. What are some of the specific examples of CP interventions and their specific outputs that you are personally familiar with? Prompts: activities that you personally participated in/witnessed; activities that have achieved results most relevant to your work, activities that you consider to have made most difference, etc. Give details on amounts of output, what show that it was effective, efficient; is it a sustainable result; etc. 3. Gaps, weaknesses in CP design, implementation What gaps or weaknesses (if any) do you see in the design and implementation of CP in Uganda? Prompts: aspects of HRH that CP did not recognize/address; priority health problems and interventions that the project did not recognize/support; important stakeholders in HRH that CP did not recognize/include as necessary, etc. 4. Key lessons, good practice experiences from CP Please give examples of key lessons or good practice experiences (if any) you know about from CP implementation in Uganda. Prompts: aspects of the project that performed particularly well; what other countries have come to learn from in CP implementation, etc. End-of-Project Evaluation of the Capacity Project | 62 5. Suggestions for future HRH support/work Please make suggestions on what you consider to be important things for the new Capacity Program in Uganda to focus on and address. Prompts: what design modifications to do from CP I; changes in the implementation approaches and processes; action research priorities in HRH; changes in focus at the national level; changes in focus at district level; changes in sector inclusion and involvement, etc. End-of-Project Evaluation of the Capacity Project | 63 6A2: Issues for Discussion with Other National-Level Strategic Leaders These issues will be discussed with leaders in other agencies and institutions with interest in health systems strengthening (HSS) and human resources for health development (HRHD). Examples of agencies and institutions to target include development partners (World Bank, EU, DFID, DANIDA, Irish Aid, WHO etc.); Academic/Research Institutions (Makerere University￾College of Health Sciences/School of Public Health, Mbarara University of Science and Technology, Gulu University Medical School, Uganda Christian University-Department of Health Sciences, Uganda Martyrs University – Faculty of Health Sciences, etc.) and Project Management Offices for specific HSS and HRHD Projects. 1. The priorities in HRH and HSS in Uganda Please describe/explain to me your understanding of the priority issues for HRHD and HSS for Uganda Use prompts where necessary: what the main challenges are, what is currently addressed in health sector strategies, what is addressed by the agency/institution you work for 2. The specific HRH and HSS interventions of the agency/institution you work for What HRH and/or HSS interventions has your agency supported or implemented over the recent past? Prompts: projects supported research studies undertaken, training courses sponsored/supported, etc. Ask for documents that may provide details on such interventions What HRH/HSS interventions are included in the current and near future plans of you agency/institution? Prompts: Ask for documents that may provide details on such interventions 3. Gaps, weaknesses and challenges in HRH and HSS in Uganda What gaps, weaknesses or challenges (if any) do you see in the design and implementation of HRHD and/or HSS interventions in Uganda? Prompts: aspects of HRHD/HSS that are not adequately recognized/addressed; factors that make HRHD/HSS interventions ineffective, inefficient, unsustainable, etc. 4. Key lessons, good practice experiences from HRHD/HSS Interventions Please give examples of key lessons or good practice experiences (if any) you know about in HRHD and HSS interventions that could be of value for Uganda. Prompts: examples from Uganda and other developing countries; examples from developed countries; aspects from such experiences that need to be modified for application in Uganda, etc. 5. Suggestions for future HRHD and HSS support/work in Uganda Please make suggestions on what you consider to be important considerations for HRHD and HSS interventions in Uganda. Prompts: what priority interventions to focus on for national, district, and community levels; what priority action research topics to focus on in HRHD and HSS; the key sectors that need to be involved and their respective roles (government, FBOs, NGOs, private sector, etc.); the potential sources of funding for such interventions, etc. End-of-Project Evaluation of the Capacity Project | 64 6A3: Key Informant Interview Guide for National-Level - Respondent Category 2: Operational Leaders (MOH, CP-Country Office, NUMAT, MOPS, MOGLSD, MOFPEP, MOES, UPMB, UCMB, UNANM, etc.) Introduction: Introduce yourself to the respondent, and mention the focus of this evaluation is: • Our understanding of Capacity Project in Uganda – part of the global whole (active in 48 countries); a three-fold focus (plan, develop, support HRH) • This evaluation – commissioned by USAID Uganda, to focus on CP results and lessons from the Uganda work, to inform priorities and methods in a new Capacity Program started in October 2009 • Evaluation approach – both strategic and operational, cover national and district-level work, use qualitative methods (Documents Review, Key Informant Interviews) Issues for discussion 1. Project design/intentions Please describe/explain to me your understanding of the Capacity Project and its design intentions. Use prompts where necessary: its goal and objectives; the key elements in its design and how they fit together (the model), the results it intends to achieve at different levels; any differences in its intentions at global, country, and sub-national levels 2. The CP implementation process (as you know it; as you experienced it; as specific/relevant to your work) Please describe/explain to me the different activities that CP supported in your organization/agency/ institution/department/office. Prompts: Ask the respondent to provide all relevant details (including copies of plans, reports, etc., that can help us appreciate all the contribution made by CP. Ask for sample products from information systems developed by CP. What was your main role in implementation of the project? Who else in your department/section participated in implementation, monitoring, review, supervision, etc.? Would you say that the activity was well or not well implemented? Explain? Prompts: What hindered or facilitated the activity; were the approaches used in CP appropriate; was the activity implemented to completion (if not, why); etc. 3. Trickle-down and scale-up Our understanding is that CP support in Uganda’s decentralized context also included the intention to “trickle down” HRH planning, development, and support to the district and lower levels. What is your experience and opinion about the progress in this aspect of CP work? Prompts: How have the district and lower levels participated in and/or benefited from the CP supported activities and outputs; which districts and other lower levels have participated/benefited most and why; what plans were made/implemented for taking such participation/benefit to other settings; what scale-up/trickle-down opportunities are present in the country but were not fully used in CP? End-of-Project Evaluation of the Capacity Project | 65 4. Gaps, weaknesses in CP design, implementation What gaps or weaknesses (if any; as relevant to your work) do you see in the design and implementation of CP in Uganda? Prompts: aspects of HRH that CP did not recognize/address; priority health problems and interventions that the project did not recognize/support; important stakeholders in HRH that CP did not recognize/include as necessary, etc. 5. Key lessons, good practice experiences from CP Please give examples of key lessons or good practice experiences (if any) you know about from CP support/implementation in your work, and in other settings in Uganda. Prompts: aspects of the project that performed particularly well; what has been documented and disseminated as a good practice example; what other countries have come to learn from in CP implementation in Uganda, etc. Are there any results from the CP (positive or negative) that were not anticipated or planned for? 6. Suggestions for future HRH support/work Please make suggestions on what you consider to be important things for the new Capacity Program in Uganda to focus on and address. Prompts: what design modifications to do from CP I; changes in the implementation approaches and processes; changes in focus at national level, changes in focus at district level; changes in sector inclusion and involvement, etc. Approaches for adaptation of the discussions for different categories of respondents All or most of the above issues will be relevant for respondents in the MOH. Respondents in other government or non-government settings may have had more targeted involvement in CP. In such cases, the approach will be to understand and focus on the relevant areas of experience, rather than go through the whole set of questions and prompts. The issues of focus for each category, as presented in Table 1 below, will be useful reference in deciding the approach to focused discussions. End-of-Project Evaluation of the Capacity Project | 66 Table 1: Categories and key elements of interest for national level KI Category Approximate number of KI Key issues to discuss Project Lead Team In-country Global Office 2-4 Country Team 2-3 Global Team All elements in project Strategic issues (as for MOH leaders) Operational details (as for MOH Op. Team) Cost-effectiveness USAID Uganda Country Office Supported Projects (NUMAT, UHMG, JCRC/TREAT, etc.) 2-4 People 2 People in each project All elements in project – Strategic level Operational details (as for MOH Op. Team) Other development Partners European Union; DFID; DANIDA; Irish Aid; WHO; World Bank 1 or 2 people each General elements on HRH and HSS in Uganda A: MOH Leaders: 1. Director General; Commissioner Planning; 2. Assistant Commissioners: (Human Resource Dev., HMIS￾Resource Centre’ Human Resource Dev.) B: Health Services Commission Chairman; Secretary; Commissioners C: Health Workforce Advisory Board D: Health Resources Working Group 5 People (1 person each office) Strategic level; all 3 Intermediate Results The strategic focus of CP; how it fits with national priorities Value addition from CP – in design, in outputs (Generic to HRH, specific to programs of focus – AIDS/RH, in integrated health service delivery) Gaps, weaknesses in CP design, implementation Un-met HRH needs, priorities Suggestions for future HRH support/work MOH Operational Teams 1. Resource Centre; HRD; HRM; ACP 2. Professional Councils (4) 3. HMDC 12 people (1 person each Council; 2 people each unit) Operational details Project design/intentions Implementation – milestones, achievements, good practices, why Gaps, weaknesses in implementation; and reasons why Trickle down intentions and experiences (to districts; to program/ service delivery level) Scale up opportunities, possibilities, plans Lessons learnt from implementation process Other government ministries MOPS; MOFEP; MOES; MOGLSD 5 people (2 MOPS; 1 each for all others) Linkage issues Capacity Project and its connections to the ministry (results achieved, good practices, lessons learned; gaps, issues not addressed Other HRH issues of relevance to the ministry Suggestions for future HRH development Other stakeholders 1. FBO Medical Bureaux UPMB; UCMB 2. Health Prof. Associations UNANM; UMA 3. Private Practice Associations UPMA; Uganda Private Doctors Assoc. 4. Universities MU-CHS (Principal); MUST (VC); Gulu Univ (VC); UCU (Head; Health Sciences); UMU (Head; Health Sciences) 5 people (2 each for UPMB and UNANM; 1 UCMB) Participation, impact issues Capacity project and the opportunities it represented for you Experiences of connection and benefit from CP Un-met expectations from CP, areas of HRH need in your work Suggestions for future HRH improvement – in your work; nationally End-of-Project Evaluation of the Capacity Project | 67 6B1: General Question Guide for Key Informants at the District Level (DHO & other DHT Members) 1. What activities were implemented in this district under the Capacity Project? 2. What are the major changes in how you do things here as a result of the project? 3. Tell us more about each of the activities that were implemented: What did each entail (tackle them one by one)? Typical activities include: a. The HRIS computerized system at the district level b. Supporting the health department to identify its HR needs and gaps c. Supporting the health department to formulate annual HR action plans d. Supporting the district to integrate health HR action plans into district plans e. Training and support in performance improvement f. Improving supportive supervision of health workers in the district g. Supporting the district to recruit health staff h. Support in orienting recruited staff i. Training in payroll management j. Supporting VHT training, supervision, and facilitation k. Dissemination of workplace safety guidelines l. Training of health workers in workplace safety and health m. Support to form occupational and safety committees Probes 1. Did this activity take place in this district? 2. Was the activity well or not well implemented? Explain? 3. What hindered or facilitated the activity? 4. Was the CP approach used appropriate? Suggestions to improve it? 5. When was last CP support given in relation to this activity? 6. Is this activity being maintained now? Why/why not? 7. Where any activities that you know about planned but not implemented? Why? 4. In total about how many members of staff in your district benefited from the trainings/technical assistance provided by the project? 5. To what extent would you say the project support contributed to better delivery of HIV/AIDS and family planning/reproductive health services in the district? Any concrete examples of this contribution? 6. Overall, would you say the project was a success? What are your reasons for this assessment? 7. What are the remaining problems/gaps in human capacity that the project did not address? 8. If the project were to continue, what changes would you like to see in the design/implementation? 9. Which of the activities implemented here do you think can be/should be rolled out to other districts? The remaining questions are general questions not directly related to the CP project. 10. Tell us about the processes you use to develop local action and work plans? 11. What role do staff members have in developing these plans? 12. What processes do you have to solve work-related problems in your organization? 13. What do you do to encourage participation of staff and community representatives in local planning and decision making? End-of-Project Evaluation of the Capacity Project | 68 6B2: Question Guide for Key Informants at the District Level (Personnel Officer and the Chair/Secretary of District Service Commission) 1. What activities implemented in this district under the Capacity Project were you involved in? 2. How were you involved? 3. What are the major changes in how you do things here as a result of the project? 4. Tell us more about each of the activities that you were involved in: What did each entail (tackle them one by one)? Typical activities include: a. The HRIS computerized system at district level b. Supporting the health department to identify its HR needs and gaps c. Supporting the health department to formulate annual HR action plans d. Supporting the district to integrate health HR action plans into district plans e. Supporting the district to recruit health staff f. Support in orienting recruited staff g. Training in payroll management Probes 1. Did this activity take place in this district? 2. Was the activity well or not well implemented? Explain? 3. What hindered or facilitated the activity? 4. Was the CP approach used appropriate? Suggestions to improve it? 5. When was last CP support given in relation to this activity? 6. Is this activity being maintained now? Why/why not? 7. Where any activities that you know about planned but not implemented? Why? 5. What is your overall assessment of the activities that the project supported in which you were involved? Probe: Would you say the project helped you to improve your work? Did the project activities help to improve the HR management practices?) What are your reasons for this assessment? 6. What are the remaining problems/gaps in human capacity that the project did not address? 7. If the project were to continue, what changes would you like to see in the design/implementation? 8. Which of the activities implemented here do you think can be/should be rolled out to other districts? 9. What other actors (donors, projects, NGOs, etc) have been involved in supporting district HRD activities? 10. What have these actors done? What activities have they supported? 11. Comment on what they did; what was good? What did not go well? 12. Did they support any similar activities as the Capacity Project? Which ones? End-of-Project Evaluation of the Capacity Project | 69 6B3: Question Guide for Key Informants at the District Level (Hospitals) 1. What activities implemented under the Capacity Project was your organization (hospital) involved in? 2. How were you involved? 3. What are the major changes in how you do things here as a result of the project? 4. Tell us more about each of the activities that were implemented: What did each entail (tackle them one by one)? Typical activities include: a. The HRIS computerized system at district level b. Supporting the health department to identify its HR needs and gaps c. Supporting the health department to formulate annual HR action plans d. Supporting the district to integrate health HR action plans into district plans e. Training and support in performance improvement f. Improving supportive supervision of health workers in the district g. Supporting the district to recruit health staff h. Support in orienting recruited staff i. Training in payroll management j. Supporting VHT training, supervision, and facilitation k. Dissemination of workplace safety guidelines l. Training of health workers in workplace safety and health m. Support to form occupational and safety committees Probes 1. Did this activity take place in this district? 2. Was the activity well or not well implemented? Explain? 3. What hindered or facilitated the activity? 4. Was the CP approach used appropriate? Suggestions to improve it? 5. When was last CP support given in relation to this activity? 6. Is this activity being maintained now? Why/why not? 7. Where any activities that you know about planned but not implemented? Why? 5. In total about how many members of staff in your organization benefited from the trainings/technical assistance provided by the project? 6. To what extent would you say the project support contributed to better delivery of HIV/AIDS and family planning/reproductive health services by this organization? Any concrete examples of this contribution? 7. Overall, would you say the project was a success? What are your reasons for this assessment? 8. What are the remaining problems/gaps in human capacity that the project did not address? 9. If the project were to continue, what changes would you like to see in the design/implementation? 10. Which of the activities implemented here do you think can be/should be rolled out to other similar organizations? The remaining questions are general questions not directly related to the CP project. 11. What processes do you have to solve work-related problems in your organization? 12. What do you do to encourage participation of staff and community representatives in local planning and decision making? 13. What other actors (donors, projects, NGOs, etc) have been involved in HRD activities in your organization? End-of-Project Evaluation of the Capacity Project | 70 14. What have these actors done? What activities have they supported? 15. Comment on what they did; what was good? What did not go well? 16. Did they support any similar activities as the Capacity Project? Which ones? End-of-Project Evaluation of the Capacity Project | 71 6B4: General Question Guide for Key Informants at the District Level (HMIS Focal Person) 1. What activities were implemented in this district under the Capacity Project? 2. What are the major changes in how you do things here as a result of the project? 3. Tell us more about each of the activities that were implemented: What did each entail (tackle them one by one)? Typical activities include: a. The HRIS computerized system at district level b. Supporting the health department to identify its HR needs and gaps c. Supporting the health department to formulate annual HR action plans d. Supporting the district to integrate health HR action plans into district plans e. Training and support in performance improvement f. Improving supportive supervision of health workers in the district g. Supporting the district to recruit health staff h. Support in orienting recruited staff i. Training in payroll management j. Supporting VHT training, supervision, and facilitation k. Dissemination of workplace safety guidelines l. Training of health workers in workplace safety and health m. Support to form occupational and safety committees Probes 1. Did this activity take place in this district? 2. Was the activity well or not well implemented? Explain? 3. What hindered or facilitated the activity? 4. Was the CP approach used appropriate? Suggestions to improve it? 5. When was last CP support given in relation to this activity? 6. Is this activity being maintained now? Why/why not? 7. Where any activities that you know about planned but not implemented? Why? 4. In total about how many members of staff in your district benefited from the trainings/technical assistance provided by the project? 5. To what extent would you say the project support contributed to better delivery of HIV/AIDS and Family Planning/Reproductive Health services in the district? Any concrete examples of this contribution? 6. Overall, would you say the project was a success? What are your reasons for this assessment? 7. What are the remaining problems/gaps in human capacity that the project did not address? 8. If the project were to continue, what changes would you like to see in the design/implementation? 9. Which of the activities implemented here do you think can be/should be rolled out to other districts? End of Project Evaluation of the Capacity Project | 71 Appendix 7: Standardized Instrument to document Components of a Capacity-Building Organization This tool is a sample that was used to ascertain the management capacity of the District Health Team/Organization and showing evidence of practice as per the third column. Key information drawn from the filled forms in all the six districts visited is summarized herein. The column “Evidence” will be filled in if the evaluation team members see documents that give objective supportive evidence. Serial Number Standardized Instrument to Document Components of a Capacity Building Organization Yes No Evidence Management Systems 1 Does the organization have a mission statement or equivalent statement of common goals? Comment on what this is if present 2 Strategic planning takes place for HRD at national/district level 3 HRD needs of districts incorporated into national-level strategic planning 4 Technical aspects of strategic planning for HRD at national level incorporated into district-level plans 5 Human resource planning incorporated into local action plans 6 Annual action planning including monitoring and evaluation of HRD needs takes place 7 Monitoring plan-based review meetings take place regularly (at least three monthly) 8 Weekly work plan and review meetings take place 9 Staff members other than managers have an active role in planning their work Comment on in which meetings staff have a decision-making role 10 Does group problem solving take place involving management and staff? Comment Workforce Planning and Policy 11 Does the organization have up-to-date/accurate data on its workforce? 12 Does the organization keep gender disaggregated data on staffing? 13 Does the organization know the actual gaps in staffing (or have data on staffing gaps?) 14 Is the HRIS supplied by the CP project in current use? Give examples or recent use End of Project Evaluation of the Capacity Project | 72 Serial Number Standardized Instrument to Document Components of a Capacity Building Organization Yes No Evidence Systems for Recruitment and Performance-Based Health Workforce Development 15 Does the organization have an established procedure for recruitment of staff? 16 Does the organization have an established system for orientating new staff? 17 Does the organization have clear job descriptions for all health staff? 18 Do staff interviewed know or have a copy of their job descriptions? 19 Does staff know standards for performance, data confidentiality, and ethics? 20 Does staff receive at least monthly feedback on their work, including reports? 21 Is there a system for regular performance review using objective performance standards? 22 Is there a system for recognition, promotion, or reward based on this performance review? Comment 23 Is there a system that records staff level of training, in-service, and continuing education received? 24 Is there a system of appeal to address grievances of staff that staff members know about? 25 Is there a system of staff discipline and firing that staff state is fair? Comment 26 Is there a training curriculum and plan for in-service training? 27 Do district staff perform regular supervision of VHTs using a systematic approach (checklists or similar) 28 How often does this supervision occur? (state frequency) frequency Health Workforce Management Practices for Improved Performance and Retention 29 Is there a system that allows staff to have input into decisions that affect their working environment? 30 Describe. 31 Is there a workplace committee for maintaining workplace safety standards? 32 Is there a system for regular maintenance of facility equipment used for training? 33 Is there a system for timely ordering of consumables used for training? Payroll and Posts Filled 34 Percent of staff accessing payment: before (B) vs. after (A) capacity project B A 35 Percent of posts filled in the organization (district) before versus after the capacity project B A End of Project Evaluation of the Capacity Project | 71 Appendix 7B: Summarized Information from the Standardized Instrument on Components of a Capacity-Building Organization Planning is taking place at the district and even sub-district levels. The strategic district health planning takes place annually using the guidelines for district action plans developed by CP. The HRH needs are accordingly identified and integrated into the national level through the guide on planning indicators and wage bill ceilings. Health facilities generate their own plans that is harmonized at the HCIV, integrated into the District Health Plans, and then into District Development Plan. However, the challenges identified are seldom responded to. These include housing, transport, and power, especially for HCIV and III. The DHT undertakes quarterly integrated monitoring and evaluation using the PHC funds. Monitoring plan and review meetings and weekly meetings are not regular. This is inadequate for mentoring and backstopping (monthly is preferred). Recruitment is planned by DHT, using the existing staff and gaps by category identified. This is submitted to the local council, who then recommends the plan to the MOH for approval. Recruitment was accordingly done in all the districts. Orientation training for the new staff was also done in all the districts for a period of one to two days and covered general issues. However, the orientation is inadequate since it left out the crucial technical work-related part regarding health issues and challenges, scope and state of facilities, the health team as a resource, and relationship with the community. It was preferable to include an attachment to an officer of a particular title and job description (internship) for at least one or two months before one can go and take his/her post. Staff Levels in the Districts District Actual Number at 2004 Percent Staff Level 2004 Actual Number at 2009 Percent Staff Level 2009 Remarks Amolatar 102 68 128 85 Recent recruitment data not included Apac 231 45 350 67.5 Kabarole 224 38 376 62.5 Kamwenge 174 48 290 80 Lira 392 76 443 86 Oyam 135 50 204 76 The capacity to manage and for one to have management skills at the health facilities is more critical. The management committees are weak. The clinical officer in charge of the health facility therefore has to: • Ensure the facility is running; • Plan for the health facility and manage their budgets (for drugs, human resources, and finances); End of Project Evaluation of the Capacity Project | 72 • Lobby for staff and ensure that the staff members are available at the facility; • Hold routine monthly meetings with the staff; • Supervise the staff through all the departments; • Appraise the performance of staff and recommend them for promotion, award, etc; • Receive and solve all HR complaints including salaries, payroll issues, accommodation, transport, meals, uniforms and gadgets, and drugs; • Select and forward staff members for training; and • Coordinate with the community on management of the facility. Workplace Safety Policy and Guidelines Medical waste management and safety equipments are expensive to manage because of a lack of funds and logistics at the districts. The policy has brought additional duties, which some of the staff do not want to take on. Yet there are more cost implications that go with the duties. It is good to compensate the workers. However, the districts need to be helped by the Ministry of Health because they cannot afford the compensations.21 A health workforce management committee is in place in Apac Hospital called the “Infectious Control Committee.” They look into health risks and infections between health workers, health workers and patients, and patients and patients. Most of the health facilities are managed by clinical officers. As far as training is concerned, there is no formal curriculum for in-service training. The staff normally identifies training opportunities and then seeks approval to train. • A health service training card was introduced by the MOH four years ago. The training provider signs on a staff card at every training attended • In Lira a training attendance book is kept (innovation) • The Capacity Building training program sponsors District Local Government (DLG) staff for training • No formal management training or on job training The main performance management/improvement tools are the performance appraisal forms (for personnel) and the performance improvement assessment form (for services). The two tools should be managed hand-in-hand to improve the quality of both the personnel and services provided. The appraisal forms must be objectively filled in and information used for strengthening performance rather than use it as a routine. Meanwhile the assessment tools are too long and time consuming, and it is important to harmonize them with existing assessment tools to make them user friendly. The Village Health Teams supported by other programs, including CRS, NUMAT, and UNICEF, exist and they are supposed to be supervised by the health facilities. There are checklists and an elaborate reporting system in Kamwenge. The VHTs need more supervision. The HRIS supplied by the CP are not functional due to challenges ranging from viruses, lack of back up, centralized data (computer and software only in the personnel officer’s office), and poor linkage between the health and district personnel. Data is not analyzed or used for decision￾making. In Apac, the HRIS has crashed and are still waiting for a backup. Kabarole has resorted 21 Capacity Project: Strengthening Health Workforce Systems in Uganda, review report, December 2009 End of Project Evaluation of the Capacity Project | 73 to using own systems and in Kamwenge the system was not introduced. There was also the challenge of installing an internet connection in the Ministry of Health to different districts due to unreliable electricity, especially in the rural areas. By the end of December 2009, the following had not been done: • Off site back up for the MOH HRIS • Linking the district HRIS database to the resource centre • Establishment of the HRIS in six out of the 12 planned districts • Evaluation of the HWAB • The planned development of district HRH plans in 10 districts While the records of staff members show Gender Disaggregated data (the female and male categories), it does not highlight challenges and deliberate interventions for support to the women or men. End of Project Evaluation of the Capacity Project | 74 Appendix 8: Strategic Objectives for Uganda Human Resources for Heath Strategic Plan 2005 -2020 SOB 1: Attain the right HRH numbers and skills mix to populate the health sector taking into account recurrent budget constraints. SOB 2: Create, maintain, and use a strong knowledge and information base for evidence based and effective HRH functions. SOB 3: Build capacity for HRH training and development to ensure constant supply of adequate, relevant, well-mixed, and competent health workforce. SOB 4: Manage HRH efficiently and effectively, with the aim of attracting and maintaining a sufficient, equitably distributed, well-motivated, empowered, and productive health workforce. SOB 5: Maintain standards and promote the transformation of present practices into an ethical, professional organizational culture. SOB 6: Build sustainable partnerships and strengthen coordination among HRH stakeholders, including the community. SOB 7: Manage scarce financial resources in support of HRH in a cost-effective and efficient manner. SOB 8: Develop capacities for HRH policy stewardship and evidence-based planning, monitoring, analysis, and review in line with a changing environment. End of Project Evaluation of the Capacity Project | 75 Appendix 9: Capacity Evaluation—USAID/Uganda Questionnaire May 12, 2008 1. What did you want the Capacity Project to accomplish in your country? Provide technical assistance (TA) for addressing HRH gaps for delivery of HIV/AIDS services. 2. Were your expectations fulfilled? If not, why not? To a great extent, our expectations were met. However, CP spent a lot of time developing the HRIS for nurses and midwives and only focused on the other aspects of HRH in the third year. 3. What do you consider the most effective activities or aspects of the assistance you have received from the Capacity Project? TA for HMIS, retention study, establishment of a coordination mechanism for all stakeholders in HRH. 4. What do you consider the least effective activities or aspects of the assistance you have received from the Capacity Project? How might you structure these activities or use Capacity differently if you were able to do them over? Moving from TA to actual implementation of activities that would address the gaps in HRH like recruitment, deployment, retention, availability, appraisal, motivation. 5. How would you describe the Capacity Project’s performance in terms of providing timely assistance that demonstrated general responsiveness and technical proficiency, appropriate to the country setting? CP was very responsive to the Mission and Ministry of Health requests and needs. The technical competence improved with time and was able to hire additional staff that would address the HRH needs beyond HRIS and the nursing profession. 6. How has the Capacity Project managed its personnel and resources in your country? Excellent. 7. How did the work of the Capacity Project complement the efforts of bilateral projects in your country? The CP is a counterpart to the head of the Human Resource and Personnel Departments of the Ministry of Health. The CP worked through the Ministry of Health multi-sectoral Human Resource Working Group and was Secretariat to the Health Workforce Advisory Board (HWAB), which has both Government of Uganda and Development Partner representation. 8. If the Capacity Project implemented core-funded activities in your country, did they complement or detract from your field-supported activities? Please provide a brief explanation. The Mission activities focused only on HIV/AIDS. In general, core and field support activities complemented each other, although at first it was difficult to for CP to address the HRH gaps in HIV/AIDS. By design, core funds focused on TA, yet the Mission wanted a balance of TA and service delivery. CP found it difficult to respond to this requirement because they are used to providing TA alone. End of Project Evaluation of the Capacity Project | 76 9. What do you see as the added value of using a centrally-managed project instead of a bilateral project to address human resource issues in your country? It is easier and quick to access the mechanism if use centrally-managed projects, especially if they are responsive to the Mission SOW. 10. Any other comments or feedback? FUTURE DIRECTIONS 11. If there were to be another centrally-managed mechanism with a focus on human resources for health (HRH), would you be interested in using it? If not, why not? Yes 12. If you would, which of the following areas would be of interest/use to you (place an X in front of those items)? X Human resource information systems (HRIS) X Human resources management Human resources policy X Workforce planning, including data-driven decision-making X Leadership for human resources Pre-service education systems In-service/continuing education systems Innovative curriculum design and delivery (team-based learning, community-based learning, integrated pre- and in-service programs) Information and communication technologies for pre- and in-service education (distance learning, telemedicine, e-learning, web-based learning, etc.) X Strengthening/development of professional associations X Task shifting/task alignment/task allocation X Incentive/retention programs X Performance-based financing X Productivity interventions Supportive supervision/quality assurance Other—please specify (list as many areas as you need) 13.If there were to be a centrally-managed mechanism for HRH, it could be structured as either a Leader with Associate Award Cooperative Agreement (LWA) or an Indefinite Quantity Contract (IQC). To help us decide which mechanism to choose, please indicate your order of preference for how you would use your field support funds to access the central project. Use 1 to indicate your top preference, 2 your second choice, 3 your third choice and 4 your least favorable option. 4 Field support to the Leader Award (LWA) 1 Develop your own Associate Award (LWA) 3 Field support to a central Task Order (IQC) 2 Develop your own Task Order (IQC) End of Project Evaluation of the Capacity Project | 77 14. Please share with us why you made these choices. The preference is to develop a Mission LWA or IQC because this is the only way the Mission can have more control on the implementation of the program and balance TA with service delivery. End of Project Evaluation of the Capacity Project | 78 Appendix 10: Achievements against Planned Activities per Objectives The table below shows the status of activities planned for each objective and Result Area. By the end of the project in September 2009, a number of activities were implemented. Those that were not were no implemented for a number of reasons including: • Concentration in a few districts and activities to ensure efficient and effective functionality of the systems; • Inadequate budget; • Technical delay; and • Internal delays in some local governments. Objective 1: To enhance capacity for HRH policy and planning at the central and district levels to ensure adequate health workforce for integrated HIV/AIDS and reproductive Health services Planned Activities Achievements by September 2009 Comments/ Shortfalls 1.1 Strengthen HRIS at the central and district levels • HRIS established in the four health professional councils and the HRM Division of the MOH • Able to generate reports automatically from the database • Regular updating of data possible • Hardware and software installed at these institutions • Key users of the data trained • Data from the professional councils’ databases linked to the HRM databases at the MOH via the Knowledge Management Portal Planned off-site back-up not yet established by June 2009 • Rolled out HRIS establishment in nine out of 12 planned districts, including Kabarole, Amolatar, Gulu, Mukono, Wakiso, Oyam, Lira, Kitgum, and Pader • Average of three users trained in each district Roll-out of HRIS happened in 9 out of 12 planned districts • Data from HRIS databases analyzed and used to produce HRH bi-annual reports • Knowledge Management Portal established as an online resource at MOH. By Sept 2009, over 700 materials uploaded onto the portal • One fulltime MOH/Library staff trained to manage the Knowledge Management Centre • HRIS officially launched by the Minister of Health • Portal being used by average of 15-19 users per day, with a total of over 1,430 visits in 2nd quarter of 2009 alone Planned link of district HRIS to MOH HRIS did not happen End of Project Evaluation of the Capacity Project | 79 Planned Activities Achievements by September 2009 Comments/ Shortfalls 1.2 Enhance MOH and District capacity for HRH policy and planning, reviews, and development • The HRIS used as a more reliable source of information for planning. This data used to revise the workforce projections; and to identify priorities for the HSSP III iHRIS Plan software being developed at IntraHealth Headquarters as the HRH Planning Model and tool not ready for adaptation and adoption • Staff from HRDD, Nursing Council, MOH, and HMDC supported to attend workshops and training courses • Worked with MOH to produce HRH Strategic Plan Supplement, HSSP II MTR report, and to undertake turnover study District HRH work plans and budget not developed due to budget constraints • Worked with MOH to review policy issues relating to the HRH component of the National Health Policy under formulation. • Support provided to the HWAB and HRWG Planned evaluation of HWAB not done because there were no clear Terms of Reference (TOR) • Capacity Project staff and Commissioner of Planning (MOH) attended HAF Review meeting in Amsterdam; project reports widely shared; worked with NUMAT to organize a District Leaders meeting in Northern Uganda; and organized the Second PEPFAR implementers’ meeting on HRH 1.3 Enhancing MOH capacity for long-term strategic health workforce planning, roll-out, and reviews • TA and facilitation provided to HRWG; introduced Health Action Framework (HAF) as a thinking tool for the Health workforce; HWAB formed and its mandate expanded beyond HRIS to include other HR issues; HWAB facilitated to meet Planned evaluation of HWAB not conducted • HRH District Action Workshop held during 2007/08 to translate policy and strategies into district operational plans • Two workshops on HRH organized in 2007 targeting district staff all over the country; attended by representatives from 78 out of 80 districts; focused on developing HRH plans; districts encouraged to implement HRH plans 1.4 Strengthen district capacity to translate HRH policy and strategic plan into action plans to address HRH priorities at the district level • Regional HRH action planning workshops conducted covering 78 districts • Districts supported to identify priority actions and formulate action plans Planned development of annual HRH plans for 10 districts not done 1.5 Strengthen networks and Strategic Alliances for HRH • Participated in networking meetings • Project briefs produced an disseminated to stakeholders • Good collaboration achieved with NUMAT in northern Uganda End of Project Evaluation of the Capacity Project | 80 Objective 2: To Strengthen Systems for Effective Performance-based Health Workforce Development Planned Activities Achievements by September 2009 Comments/ Shortfalls 2.1 Harmonizing pre-service and in-service training of the health workforce with integrated HIV/AIDS and reproductive health needs and priorities; strengthen systems for in￾service training and continuing professional development • HMDC supported to install e-granary—a resource for learning materials for trainers and health worker trainees 2.2 Developing and promoting approaches for effective performance improvement (mentoring, performance improvement, support supervision, action learning, etc). • A central level technical resource team trained in performance improvement • DHT members in nine districts sensitized in performance improvement • Performance assessment tools developed, assessments conducted in nine districts, and results disseminated • Supported Kabarole district to implement PI process. Activities included training the DHMT, undertaking performance assessment, developing a district intervention plan and health unit intervention plans, and providing infection control equipment to the district Only few health units developed an intervention plan • National-level trainers oriented to Learning for Performance Approach; • Learning for Performance training of facilitators and trainers conducted • District trainers in Kabarole supported to carry out mentoring training on infection control in 21 health facilities • Worked with MOH to monitor, evaluate, and document PI experiences and lesson and plan roll-out to other districts • Supported UPMB to mainstream PI approaches in support supervision activities 2.3 Developing strategies to motivate continuing professional development including standards for accreditation, certification, and licensure • Supported the MOH and the professional councils to develop IST/CPD standards and guidelines • Standards and guidelines for accreditation of IST/CPD launched • Updated professional ethics and codes of conduct for health workers launched Professional councils and associations not oriented in the use of IST/CPD accreditation guidelines due to the launch taking place very close to end of project 2.4 Strengthening the role of the health workforce professional councils and associations in in-service training and continuing professional development • Assessment carried out to determine nurses and midwives satisfaction with services from their association UNANM • A communication strategy for UNANM produced and implemented, including workshops and production of UNANM handbook and brochures • UNANM website developed Some of the planned activities not implemented due to lack of funds End of Project Evaluation of the Capacity Project | 81 Planned Activities Achievements by September 2009 Comments/ Shortfalls • Worked with professional councils, associations and MOH and other stakeholders to review the ethical codes of conduct. • Reviewed ethical codes of conduct launched • Copies of codes of conduct printed and distributed • Supported placement of MUST students to work in community contexts working with VHTs to provide PMTCT,HCT and Family Planning Services End of Project Evaluation of the Capacity Project | 82 Objective 3: To Identify and Promote Health Workforce Management Practices for Improved Performance and Retention Planned Activities Achievements by September 2009 Comments/ Shortfalls 3.1 Strengthening systems for health workforce performance management • Conducted mapping of HR processes; copies of report distributed • In collaboration with MOH, HMDC, AMREF, and Makerere University, organized and attended consultative meetings on health sector HR leadership and management development • Developed strategy and tools for performance management • Tested the concept of performance management using Results Oriented Management (ROM) in three districts, Oyam, Amolatar, and Kabarole; supported development of an implementation strategy for performance management in collaboration with MOH and MOPS 3.2 Developing strategies and systems for improved recruitment and deployment • Introduced a computer-aided short-listing program, which was used in Lira, Dokolo, Amolatar, Kabarole, Amuru districts, thus reducing the cost and time previously associated with manual short-listing • Supported districts to do centralized/joint advertising, thus reducing cost of advertising by more than half • Procedures of advertising of vacant posts modified to include radio and notice boards, not just newspapers • Supported DSCs to identify vacant posts, advertise, and recruit • Supported districts to induct newly recruited staff Amuru district not supported to induct staff • Nationwide staff audits conducted to inform district recruitment plans Recruitment plans not formulated 3.3 Developing tools and initiatives for improved job satisfaction and retention • Retention and turnover studies done; reports printed and disseminated • Comprehensive motivation and retention strategy finalized, printed, and disseminated • Payroll management training done for all districts 3.4 Enhancing systems for community participation in health services • Collaborated with MUST to implement the VHT strategy in Kabarole and Kasese districts using MOH guidelines • Supported community attachment of MUST students 3.5 Strengthening systems for workplace safety, protection, and care of the health workforce • Supported the MOH to develop the Workplace Safety Guidelines and policy; copies printed and disseminated; policy and guidelines disseminated; trained health workers in workplace safety and health in Kabarole district; formed Occupational and Safety Committees in Amolatar, Oyam, and Kabarole districts End of Project Evaluation of the Capacity Project | 83 Table A10 Capacity Project Activities not Completed as Scheduled Uncompleted Activities (as per project documents) Comment from COP 1.1 Planned off-site back-up not yet established by June 2009 Process was delayed due to unresolved debate about where to locate the back-up; External consultant supporting the process available in Uganda on and off for short periods of time Roll-out of HRIS happened in nine out of 12 planned districts Realized 12 districts were too ambitious; reduced target to nine Planned link of district HRIS to MOH HRIS did not happen Thought linking only nine districts would not have desired impact; decided to do the link when at 20 districts; have 19 so far 1.2 iHRIS Plan software being developed at IntraHealth Headquarters as the HRH Planning Model and tool not ready for adaptation and adoption Yes, software not received in time for adaptation District HRH work plans and budget not developed due to budget constraints District HRH work plans done—only that there was no follow-up to ensure that they were integrated into district plans and budgets 1.3 Planned evaluation of HWAB not done because there were no clear Terms of Reference (TOR) No TOR establishing the roles of the HWAB; these are now to be developed as the HWAB transitions into a program steering committee 2.2 Only few health units developed an intervention plan for PI Process of developing health unit intervention plans left to districts, expected to be supported by other actors e.g. NUMAT. Project supported these to lower levels only in the three core districts 2.3 Professional councils and associations not oriented in the use of IST/CPD accreditation guidelines due to the launch taking place very close to end of project Yes, launch of the guidelines happened very late; no time to do orientations 2.4 Some of the planned activities not implemented due to lack of funds CP had very little funding from field support to work with UNANM 3.2 Amuru district not supported to orient staff Amuru had internal problems including interdiction of the district Chief Administrative Officer (CAO); activity re￾allocated to Kamwenge district Recruitment plans not formulated Had planned to develop a national recruitment plan based on the staff audit report. Funding was to come from MOH partnership fund but was never realized End of Project Evaluation of the Capacity Project | 84 Appendix 11: Sample Output Data from HRIS Qualify Databases (Four health professional councils) Cadre Registered Male Female Total Nursing and Midwifery Council Registered Clinical Nursing Degree 31 121 152 Registered Health Tutor Midwifery Diploma 0 59 59 Registered Health Tutor Nurse Diploma 10 62 72 Registered Nurse Diploma 542 3803 4345 Registered Public Health Nurse Diploma 0 119 119 Registered Comprehensive Nurse Diploma 239 315 554 Registered Mental Health Nurse Diploma 227 262 489 Registered Pediatric Nurse Diploma 16 96 112 Registered Midwife Diploma 7 2884 2891 Enrolled Nurses Certificate 1235 5022 6257 Enrolled Comprehensive Nurse Certificate 530 1500 2030 Enrolled Mental Health Nurse Certificate 225 113 338 Enrolled Midwife Certificate 5 4293 4298 Total 3067 18649 21716 Allied Health Professionals Council Biomedical Sciences (Degree and above) 0 1 1 Anesthesia (pre-degree) 86 62 150 Theatre Techniques (Certificate) 41 35 77 Clinical Medicine and Community Health 1692 523 2228 Public Health Dental Assistants 121 58 179 Prosthetist and Orthodontist (Degree +) 1 0 1 Environmental Health Science (Degree +) 18 8 26 Environmental Health Science (Pre-degree) 790 278 1073 Medical Laboratory Technology (Degree +) 10 8 18 Medical Laboratory Technology (Pre-degree) 796 274 1071 Medical Radiology (Degree +) 14 3 17 Medical Radiology (Pre-degree) 65 10 75 Occupational therapy (Degree +) 0 1 1 Occupational therapy (Pre-degree) 33 17 50 Orthopedic medicine/technology (Pre-degree) 222 52 274 Dispensing/Pharmacy (Pre-degree) 144 62 207 Physiotherapy (Degree +) 3 9 12 Physiotherapy (Pre-degree) 37 21 59 Medical Entomology (Diploma) 79 15 92 Mental Health (Diploma) 35 11 46 Ophthalmic medicine (Diploma) 25 19 44 Cataract surgery (Diploma) 0 0 0 Ear Nose and Throat (ENT) (Diploma) 1 1 2 Public Health (Certificate) 0 1 1 Total 4213 1469 5704 End of Project Evaluation of the Capacity Project | 85 Cadre Registered Male Female Total Medical and Dental Professionals Medicine and Surgery (first degree) 2042 824 2866 Dental Surgery/Public Health (first degree) 160 51 211 Post-graduate diploma 48 29 77 Post-graduate degree (clinical) 75 15 90 Post graduate degree (Management/PH) 24 12 36 Member, Royal College 4 6 10 Fellow, Royal College 17 1 18 Doctorate/PhD 18 4 22 Total 2,388 942 3,330 Pharmacy Council Pharmacy – Degree 187 84 276 Pharmacy – Masters 23 11 34 Total 210 95 310 End of Project Evaluation of the Capacity Project | 86 Appendix 12: Performance Improvement Assessment Report The following example of an assessment performance report from Gulu District in northern Uganda shows the practice adopted the importance of the assessment tool in identification of the HRH issues and bringing them up for planning and decision making. Report of Performance Improvement Assessment in Gulu District Overview 1.1 INTRODUCTION 1.1.1 Geographical profile Gulu district is situated in Northern Uganda, located between longitudes 31° E-32° E, latitude 02° N - 04° N. It is bordered by Amuru district in the West, Pader district in the East, Kitgum district in the North-East, Lira and Apac districts in the South- East. The district total area is 344,908 square kilometers of which 96.9 sq km (0.8 percent) is covered by open water and swamps. It is 1.44 percent of the national land area. The district headquarters: Gulu Municipality is at a road distance of 332 kilometers from Kampala, the capital city. There is modern communication network. Administratively the district is divided into two counties and one municipality, 11 sub-counties, four municipal divisions, 69 parishes (of which 16 are municipal wards), and 233 villages. The counties make up the three health sub-districts (HSD) of Aswa, Gulu Municipality, and Omoro. The socio-economic condition of the people is very poor. Due to persistent insecurity, over 60 percent of the people continue to live in the internally displaced people’s camps and settlements. Due to the insecurity, the vast fertile land for cultivation remains unutilized except those very close to the camps. It is obvious from the above that very little socio-economic activities are possible as the people are predominantly cultivators. Livestock farming is likewise very limited since most of the livestock were looted. The people have been reduced to relying heavily on food relief from WFP, and non￾food items from relief agencies. End of Project Evaluation of the Capacity Project | 87 1.1.2 Demographic Features Table 2: Demographic profile, infrastructure, and manpower distribution HSD S/counties Population Infants Pregnant Mothers Health Unit Ownership Current Staffing Staffing Norm Gap Status ASWA Awach 12,631 543 632 Awach HCIV G 18 50 32 F Paibona HCII G G 0 9 9 C Bungatira 25,934 1,115 1,297 Pabwo HCIII G 0 19 19 C Punena HCII G G 0 9 9 C Rwotobilo HCII G G 0 9 9 C Coope E G 6 9 3 F Paicho 28,155 1,211 1,408 Kal Ali HCII G 5 9 4 F Te Ato HCII G 0 9 9 C Omel HCII G 0 9 9 C Cwero HCIII G 5 19 14 F Unyama HCII G 7 9 2 F PakweloHCII G 0 9 9 C Palaro 7,480 322 374 Labworomor HCIII G 3 19 16 F Oroko HCII G 0 9 9 C Patiko 9,450 406 472 Patiko HCIII G 6 19 13 F Pugwinyi HCII G 0 9 9 C Lugore E 2 9 7 F Total 83,650 3,597 4,183 GMC Bardege 41,489 1,784 2,075 Independent H P Bardege HCII G Lacor H NGO St Philip HCII Ngo End of Project Evaluation of the Capacity Project | 88 HSD S/counties Population Infants Pregnant Mothers Health Unit Ownership Current Staffing Staffing Norm Gap Status St. Luke HCII Ngo Laroo 24,011 1,035 1,201 Gulu H G Pece HCII G St Mauritz HCII Ngo Layibi 28,779 1,237 1,439 Techo HCII G Pece 40,896 1,759 2,045 Aywee HCII G Total 135,174 5,812 6,759 Bobi Bobi HCIII G 9 19 10 F Palenga E G 3 9 6 F Lelaobaro G 0 9 9 C Minakulu NGO Koro 20,536 883 1,027 Lapainat HCIII G 3 19 16 F Abili E G 3 9 6 F Lakwatomer HCII G 0 9 9 C St. Luke NGO 0 Lakwana 15,147 651 757 Opit HCIII NGO 6 19 13 F Lanenober HCIII G 0 19 19 C Parak HCII G 0 9 9 C Awoo HCII G 7 9 2 F Lalogi 20,723 891 1,036 Lalogi HCIV G 17 50 33 F Idobo HCII G 0 9 9 C Odek 27,443 1,180 1,372 Odek HCIII G 4 19 15 F Acet HCII G 5 9 4 F Awere NGO 7 9 2 F Dino G 3 9 6 F St. Peters NGO 0 9 9 C End of Project Evaluation of the Capacity Project | 89 HSD S/counties Population Infants Pregnant Mothers Health Unit Ownership Current Staffing Staffing Norm Gap Status Ongako 16,248 699 813 Ongako HCIII G 4 19 15 F 119,014 5,118 5,951 151,013 7,098 7,551 Source DDHS Status C = Closed due to insecurity and the staff deployed elsewhere F= Functional G = Government E = Emergency unit built in underserved camps End of Project Evaluation of the Capacity Project | 90 Infrastructure Both HCIV referral (Lalogi and Awach) are partially operational, although both theatres being incomplete and without equipment. Lalogi has no maternity ward; instead the theatre, which was badly built, is being used temporarily. Laboratory and staff accommodation is inadequate. All eight government HCIII are lacking either maternity or a general ward. Two of these have been abandoned due to unfavorable security. Generally the OPDs are very small, and staff accommodation inadequate. The HCIIIs are in dire need of rehabilitation. Most of them have to be upgraded in terms of adding wards, maternity units, and staff accommodation. There are 29 HCII, of which 15 are closed due to insecurity. Staff accommodation is inadequate in all the HCIIs with five without any staff housing at all. Most of the health units are in various states of disrepair, and the abandoned structures have been further damaged by disuse and termite damage. Human Resources During the year, about 75 percent of PHC staff allocated for the district was recruited, upgrading the staffing norm to about 60 percent according to the old structure. However, this picture is drastically reduced when the new structure is oprationalized. Attracting and retaining certain cadres of staff, especially doctors, laboratory staff, and midwives, has been difficult due to poorer service conditions in the district (insecurity, lack of incentives, etc). The health worker to patient ratio is unfavorable, as shown below. Staffing in Lower Level NGO Units is very miserable except in the HCIII. Table 3: Showing the ratio of health worker to population Category Gulu Uganda Doctor to population ratio 1:15,983 1:18,600 Nurse and midwife to population ratio 1:2,677 1: 2,870 End of Project Evaluation of the Capacity Project | 91 General Observations During the Assessment of PMTCT and Reproductive Health Services 1. A number of health facilities that are located near camps that were/are used for the internally displaced persons are realizing low antenatal attendances. Many people have responded to the government’s decision to send people back to their villages since there is security in the region now. This has consequently affected the delivery rate at the health facilities. Some of the affected units visited are: Labworomor HCIII, Patiko HC III and Odek HC III 2. In some of the health facilities, antenatal services are not held on daily basis as recommended by the MOH. 3. Some units had been used as training centers previous to the training of the DHT, which was done twice with Gulu. This opportunity of being assessed twice before made the health workers more knowledgeable and they performed very well e.g. Lalogi HCIV, Bobi HCIII. 4. The majority of units had no televisions at all. A few units had received them from NUMAT, and Pathfinder. The reasons for their not being functional were lack of power or fuel for running generators, or not being connected. 5. The recommended Policy Guidelines and Job Aides were not available in many units as they have not been supplied by the district. End of Project Evaluation of the Capacity Project | 92 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention Opit HCIII Opit Health Unit is an NGO facility under Lacor Hospital. It has good infrastructure. The compound was littered with polythene bags as a result of food distribution to HIV/AIDS clients, otherwise a clean environment Staffing 1 Clinical officer in charge 1 Registered nurse 1 Enrolled nurse 1 Enrolled midwife 1 Laboratory assistant 2 Assistant health educators 5 Nursing assistants 1 Data clerk 2 Support staff No guidelines on opportunistic infections, STI management and infection prevention in ANC and prescription room No schedules for health education (HE) talks nor plan for HE sessions However: Unit has both incinerator and medical waste pit in good use Has running water The patient/client load is high and HIV/AIDS clinic is well attended. This facility performed fairly well in the assessment The health worker assessed was an enrolled midwife. She performed well, especially in the areas of history-taking, physical and obstetric examination of the mother, and asking and recording of the necessary information on the mother’s integrated card The midwife however did not measure all the vital signs because there were no thermometers. There was only one blood pressure machine and one weighing scale in use at the outpatient’s department HB, VDRL/RPR, and urine tests for all mothers were not requested for at their first ANC visit Introduced herself and asked mothers to introduce themselves Spoke in low tone using the language understood by mothers However: She did not ask about past or current symptoms of STIs Did not educate on the use and provide mothers with ITNs as they were out of stock Did not counsel mothers regardless of their HIV status Did not give mothers information on routine HIV counseling and testing, lab tests on HB, syphilis, blood grouping, and urine testing Did not inquire nor administer tetanus toxoid (TT). She said TT is not given due to workload The unit had no decontamination containers and utility gloves. No posters were displayed on infection prevention in treatment room as they don’t have them Did not dry hands with a single use towel as towels are not available End of Project Evaluation of the Capacity Project | 93 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention Odek HCIII Name of Staff Omon Paul - SCO in charge Okello Labedo - CO Akullo Agnes Omara - EM Laker Milly Grace - EN Okite James - HA Ogal Ensio - NA Acan Evelyn - NA Adongo Teddy - N A Abalo Pamela - NA (maternity leave) Aciro Norah - NA (Staff for Binya HCII awaiting completion of their HC Akidi Alice - NA (as above) Odongo Yoel - HA (as above) Midwife was absent so the NAs were running the ANC and carrying out HIV testing. Unit has no lab assistant. Only tests done by the NA and trained village health workers are HIV This unit is located near a camp and has no accommodation for staff. One old house nearby houses six staff, three nursing assistants reside in two rooms adjacent to labor ward and one male nursing assistant resides in the YCC clinic. ANC services are offered on Tuesdays and Friday of every week. The person assessed was a nursing assistant who has limited knowledge in midwifery Health worker The Nursing Assistants running the clinic were limited in the knowledge to share with the mothers. There were many copies of the Birth Preparedness Plan on the table but they did not know what they were for No hand-washing facilities therefore the nurse did not wash her hands. Equipment is not sterilized as there is no paraffin. Use Jik when available Lab has no sink neither container for water Placentas are disposed off in a nearby pit latrine which is also still in use. Has no cover and exudes a terrible stench End of Project Evaluation of the Capacity Project | 94 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention HIV testing therefore is done only on those days so mothers who come on any other day do not get tested 50 ampoules of morphine injection and 25 vials of injection ketamine were found in the maternity store just on the slab. The NAs did not know their use or importance. CO reported that the drugs were a donation by a certain organization. Drugs were withdrawn and taken to DHO store for supply to the deserving health facility There is no placenta pit Bobi HCIII Unit has two COs, one EM, one EN, one lab assistant, one lab technician, five NAs, three support staff. All the qualified staff members were not in the unit. The NA was therefore running the unit Had no policy guidelines as they have not been provided by the district Test kits for HB had been out of stock for many months The person assessed was a nursing assistant who has limited knowledge in midwifery Did not introduce herself to the mother and never encouraged questions Important information was not given to the mother such as when to come back when ill, summary of findings etc Hand-washing facility not provided in the room. No pit for medical waste instead they use the old pit latrine, which is also full Laboromor HCIII Newly renovated by ICRC The WHO also supports the unit with various supplies UNICEF also provides some supplies such as Mama Kit The midwife is knowledgeable on required procedures. She checked for edema while patient is lying down. Said this is how Did not introduce her to the mothers and did not call them by name. Says that the mothers already knew her The health worker used the same towel for cleaning hands more than once. It permanently hangs by the washstand. End of Project Evaluation of the Capacity Project | 95 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention Unit has only two qualified staff, one CO, one MW, one HA, and three NAs they were trained Lalogi HCIV Supported by Medicines San Frontiers Have six midwives. Two are from government, while four are employed by MSF All drugs are available Midwife was knowledgeable although some procedures were being done to impress the assessors as they were not routine There were no decontamination containers The incinerators and waste bins were in place and in use Lapainat HCIII Government facility with a total number of 16 health workers and one support staff. Despite this big number of staff, only one NA and watchman were on duty Has good infrastructure including staff houses. Was constructed by World Vision No incinerator nor medical waste pit No recommended guidelines at all places. Say these had not been supplied No schedules for H/E talks. Did not know their importance RPR, HB, and Urine test kits were not available The person assessed was a nursing assistant who has limited knowledge in midwifery History taking, physical, and obstetric examination were not thoroughly performed. Other important procedures, like asking the mother to empty her bladder, were also omitted Provider tried to use the language spoken by the mother despite herself not knowing the language well However, she forgot to ask the mother whether she has any other general health problems Mothers were not counseled at all Summary of findings not given to mothers HE was given to mothers and their companions on personal hygiene. However, mothers were not told to come back in case of fever, cough, or per vagina bleeding Importance of taking iron, folic acid and multivitamin tablets not mentioned Did not educate on the use and provide mothers with ITNs as they were out of stock The unit had both sterile and non-sterile gloves and safety boxes for sharps. However, the NA did not wash hands before, between and after examining mothers (no soap) No running water, instead rain and bore hole water is used All rubbish is disposed off in a shallow pit. The three-bucket decontamination system is not in use due to lack of buckets and Jik Cwero HCIII The unit is understaffed. Qualified staff has left due to non-salary payment There were no mothers attending antenatal services so this area There were no mothers attending antenatal services so this area There were no mothers attending antenatal services so this area Hand-washing facilities not available No decontamination End of Project Evaluation of the Capacity Project | 96 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention Unit currently is ran by one CO and two NAs No drugs for opportunistic infections No TV ANC run on Tuesday and Friday. No revised PMTCT guidelines. No test kits for HIV, VDRL, and reagents for urine No integrated AN Cards was not assessed was not assessed was not assessed containers available No incinerator available. All rubbish is disposed in a shallow pit, which is only burnt when it gets full Ongako HCIII Government health facility with one CO, one EN, one EM, one lab assistant, and two volunteers Has OPD and maternity and patients/client attendance. HB test kits had been out of stock for many months. No guidelines, job aides in the unit The MW was on leave so the CO who is the OIC was assessed Did not ask the mother to empty her bladder Did not take vital signs as there were no weighing scales, BP machine Did not introduce herself nor did she ask the mothers to introduce themselves. Counseling was not done at all Did not give the mother a birth plan form and encourage her to share the plan with her husband Unit has no running water. Use borehole water After hand-washing, she dried hands with her handkerchief. This is because there were no hand towels There were no posters displayed on infection prevention Aywee HCIII Staff at the unit included one CO, one EM, one lab technician, and four NA support staff. Mothers’ ANC attendance is very low No revised PMTCT manual or STI management manual Audio/visual equipment present but not functional due to lack of power EM was assessed and attended to clients well. She took all measurements of vital signs but did not carry out a complete physical inspection of the mother. Did not palpate the neck for enlarged Did not ask mothers to introduce themselves Did not ask mothers to introduce themselves nor did she introduce herself to them She did not ask the past or current history of STI symptoms from Did not provide counseling on STI, or safer sex practices Did not provide information on safer breastfeeding option Did not counsel the mothers at all Did not tell the mothers Hand-washing facility not provided inside the room No safety boxes for sharps but use an improvised Jerry can End of Project Evaluation of the Capacity Project | 97 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention No posters on infection prevention Safe water for swallowing drugs was not available glands mothers Mothers were not asked to empty their bladders before examination to come back in case they felt unwell Awach HCIV Government unit supported by NUMAT, UNICEF, Baylor College of Medicine. Staff present were one CO, one EM, one RM, two NAs. Audio/visual equipment present but not working due to lack of power. No revised PMTCT model No STI management guidelines No HB or RPR for mothers for ANC Inadequate topics for HE ANC Clinic run by registered midwife Did not explain to the mother to the mother what her care will consist of Privacy was not offered to mothers as the door was kept open during examination and other persons were not restricted from entering the room Did not inform mothers about safer feeding options and signs and symptoms of labor onset Did not provide counseling on STIs Did not tell the mothers to come back in case they felt unwell No auto disposable syringes No incinerator. All rubbish in the Unit is disposed of in an old pit latrine No decontamination containers The placenta pit is not properly constructed Gulu Hospital Gulu is a government regional referral hospital with an MCH department that provides reproductive health and PMTCT services Has TV but department not connected to power No revised PMTCT policy guidelines No STI management guidelines No HB or RPR for mothers for ANC The RN/RM was trained in the provision of PMTCT services. She performed very well in most of the areas However, she did not ask about the past history or current symptoms of STIs Did not measure the vital signs except the BP and weight because there were no thermometers and Did not ask mothers to introduce themselves. Did not ask about the past history or current symptoms of STIs and any symptoms related to HIV Did not counsel clients at all Did not explain the importance of taking iron, folic acid, and multivitamin tablets Did not explain importance of a birth plan and mention the items in the birth plan to the mother Did not tell mothers to come back to health facility as soon as they experience fever, diarrhea, productive cough, rash, and PV Washed hands with water and soap but used her handkerchief to dry her hands because there were no hand towels End of Project Evaluation of the Capacity Project | 98 Local Management Clinical Competencies Provider-Client Interaction Information to Clients Infection Prevention height measure. Did not ask the mother to empty her bladder bleeding Patiko HCIII No PMTCT Policy Guidelines displayed/in use HB and urine testing not done in unit due to lack of reagents and no laboratory assistant The midwife was assessed. She was not consistent in carrying out physical examination Did not ask the clients if they had any questions Did not talk about the birth plan and its importance Mothers were not informed when to come back in case they are not feeling well No running water at the unit. The borehole was converted into motorized system but is not in use due to lack of fuel for running it Midwife did not wash hands before, between, and after examination of mothers Rubbish pit is full No posters on infection control/prevention End of Project Evaluation of the Capacity Project | 99 RECOMMENDATIONS 1. The District Health Team should ensure that all recruited staff members are at their stations of work by carrying out regular support supervision. 2. The district should employ qualified staff and ensure that all those employed are working. 3. The district should ensure that the salaries of the newly recruited staff are paid as soon as they resume duty. 4. The nursing assistants should be given more training skills in midwifery. 5. Decent accommodation for staff should be provided. 6. The district should employ qualified health personnel and ensure that there is adequate qualified staff offering services. 7. Support supervision to the health units should be done frequently. This will also help to curb absenteeism at the health facilities. 8. Supplies such as test kits should be provided. 9. The health districts and sub-districts should ensure availability of drugs and other essential supplies in all the health facilities. 10. New PMTCT policy guidelines should be sent to the district by the MOH. 11. The medical superintendent should allocate the weighing scale with weight measure, which is in the OPD Minor Theatre, to the MCH Department where it is most required. 12. The medical superintendent should buy a three meter piece of cloth for MCH to be cut into the size of face towels and used for hand-drying as single use towels. 13. The medical superintendent should collect posters on infection prevention and policy guidelines on PMTCT from the MOH and distribute to all departments in the hospital where they are required. 14. Revitalization of the Health Unit Management Committees should be completed. 15. The district assistant drug inspector should carry out support supervision to the health facilities regularly to verify drugs in use and coordinate with partners in the supply of required drugs. 16. The district should equip all the laboratories so that all tests can be done in the health units. Dressing rooms should also be equipped accordingly. 17. The district should ensure that all the health facilities have adequate hand-washing facilities. 18. Incinerators and placenta pits should be provided in all health facilities. 19. There is need to educate the health workers on the importance of the health education plans and schedules as they did not know what these were. Provision of required stationery is equally important as they have no papers and files for developing the schedules and plans. End of Project Evaluation of the Capacity Project | 100 ACTION PLAN Short Term No Activity Responsible Party 1. Organization of regular refresher courses for nursing assistants DHO’s office 2. Revised PMTCT policy guidelines should be availed and implemented in all the health units DHO’s office/HSD MOH 3. Provision of drugs to the health units DHO’s office 4. Immediate access to the payroll by newly recruited staff DHOS office, District Personnel Office 5. Support supervision to monitor staff performance DHO 6. Health units to have health education books/files DHO/NUMAT/CP 7. Revitalization of health unit committees DHO 8. Provision of hand-washing facilities and soap DHO/NUMAT/CP Long Term No Activity Responsible Party 1. Provision of accommodation to all health staff GLG, MOH 2. Employment of qualified health personnel GLG, MOH 3. Equipment of all laboratories with test kits, reagents, and equipment DHO’s office 4. Provision of incinerators, placenta pits, waste bins, decontamination containers in all the health units DHO’s office, 5. Development of the motivation policy for the health staff e.g. promotions and confirmations in service and transport DHO, GLG, Donors 6. Release of requested drugs and supplies to the health units to avoid stock outs, misuse, and expiry of drugs MOH, DHO End of Project Evaluation of the Capacity Project | 101 31 34 36 53 53 59 64 66 66 69 69 81 52 0 10 20 30 40 50 60 70 80 90 100 Percent Lapainat HCIII Labworomor HCIII Pabwo HCIII Gulu Hospital Awach HCIV Bobi HCIII Opit HCIII Cwero HCIII Aywee HCIII Ongako HCIII Odek HCIII Lalogi HCIV Gulu District Health Facility LOCAL MANAGEMENT The results show that Lalogi HCIV performed best in local management while Lapainat and Labworomor’s performance was low. Lalogi is supported fully by Medicines San Frontiers (MSF). MSF has recruited additional trained staff and provides medicines and supplies. It has improved on the existing buildings, some of which are housing the staff, and is constructing more buildings including an operating theatre. Lapainat and Labworomor have minimal support in all areas, and there is pervasive staff absenteeism, especially of qualified personnel who would be steering the health facility. End of Project Evaluation of the Capacity Project | 102 45 89 71 67 67 57 57 56 50 47 33 33 29 0 10 20 30 40 50 60 70 80 90 100 Percent Gulu District Ongako HCIII Lalogi HCIV Opit HCIII Lapainat HCIII Cwero HCIII Pabwo HCIII Bobi HCIII Gulu Hospital Labworomor HCIII Aywee HCIII Odek HCIII Awach HCIV Health Facility Infection Prevention End of Project Evaluation of the Capacity Project | 103 0 0 21 36 41 57 60 61 63 65 67 72 79 48 0 10 20 30 40 50 60 70 80 90 100 Percent Pabwo HCIII Cwero HCIII Lapainat HCIII Odek HCIII Patiko HCIII Gulu Hospital Aywee HCIII Lalogi HCIV Bobi HCIII Awach HCIV Labworomor HCIII Ongako HCIII Opit HCIII Gulu District Health Facility Clinical Technical Competence End of Project Evaluation of the Capacity Project | 104 0 36 50 56 58 62 65 69 72 75 79 82 58 0 10 20 30 40 50 60 70 80 90 100 Percent Pabwo HCIII Labworomor HCIII Cwero HCIII Lalogi HCIV Odek HCIII Lapainat HCIII Ongako HCIII Bobi HCIII Awach HCIV Opit HCIII Gulu Hospital Aywee HCIII Gulu District Health Facility Client-Provider Interaction End of Project Evaluation of the Capacity Project | 105 44 67 64 63 63 63 57 52 51 33 30 21 13 0 10 20 30 40 50 60 70 80 90 100 Percent Gulu District Bobi HCIII Cwero HCIII Labworomor HCIII Aywee HCIII Gulu Hospital Pabwo HCIII Lalogi HCIV Awach HCIV Opit HCIII Odek HCIII Ongako HCIII Lapainat HCIII Health Facility Provision of Information to Clients End of Project Evaluation of the Capacity Project | 106 PMTCT Performance Assessment Team for Gulu District 1. Mr. Lokach Okot G Ag DHO 2. Dr. Lawino Anna M/O Lalogi HCIV : Tel No 0772448801 3. Ms. Anena Grace Ag DHV 4. Mr. Yoweri Idiba Biostastician, DHO Office: Tel No 0775 3416 5. Mr. William Onyai Clinical Officer, Awach HCIV: Tel No. 0772939754 6. Mr. Opwonya Boniface Ag District Health Inspector 7. Mr. Ogwang Oscar District Cold Chain Technician 8. Mr. William Nyeko Senior Clinical Officer In charge HSD 9. Ms. Maginoh Olwit Ruth Senior Training Officer, MOH 10. Ms. Matte F Rita Program Officer, Capacity Project End of Project Evaluation of the Capacity Project | 107 Appendix 13(A): List of Consultants Available For Work on HRH # NAME AREA OF EXPERTISE CONTACT 1 Prof. Sam Luboga Health Worker Training 2 Prof. Anne Katahoire Social Research, HIV/AIDS 3 Mr. William W. Ogulei Human Resource Management 4 Dr. Andrew Mwanika Health Worker Training Tel: 0712-812294 5 Dr. Joseph H. Kyabaggu Health Systems Management/Development 6 Dr. Sam A. Okuonzi Health Policy and Planning 7 Dr. Elidad Mabumba Public Health Interventions Tel: 0772670319 8 Mr. W. Rwandembo Mugisha Curriculum Development/Review Tel: 0772-442833 9 Mr. Benjamin Udong Health Professional Councils’ Management 10 Dr. Sebastian Baine Health Care Financing sbaine@musph.ac.ug 11 Mr. Festus Kibuuka Capacity Building 12 Dr. Mbona Tumwesigye Health Statistics and Data Analysis 13 Dr. Elizeus Rutebemberwa Policy and Planning ellie@musph.ac.ug 14 Ms. Adrienne Karianah Marketing Research adrienne@transparencyresear ch.co.uk Source: Capacity Project End of Project Evaluation of the Capacity Project 108 Appendix 13 (B): Potential Private Consultants Available at the School of Public Health, University of Makerere This appendix is in an appended PDF document by the same name. It lists the staff, qualifications, and areas of expertise of academic staff at the School of Public Health University of Makerere, Kampala. Names Position Specialization Qualification E-mail Dr. J. Konde￾Lule Associate Professor Epidemiology and Research Methods MBChB, DPH, MSc jkonde@musph.ac.ug Dr. F. Wabwire￾Mangen Associate Professor Epidemiology & Biostatistics MBChB, MPH, PhD fwabwire@musph.ac.ug Dr. David Ndungutse Senior Lecturer Epidemiology and Research Methods MBChB, DPH, MSc dndungutse@musph.ac.ug Dr. David Guwatudde Senior Lecturer Biostatistics PhD dguwatudde@musph.ac.ug Dr. Nazarius Mbona Tumwesigye Lecturer Biostatistics B. Stat., MA (Demo), MSc, PhD tmbona@musph.ac.ug Dr. Fredrick Edward Makumbi Lecturer fmakumbi@musph.ac.ug Dr. Dan Bagenda Lecturer Biostatistics B. Stat., MSc, PhD dbagenda@musph.ac.ug Dr. Edith Nakku Joloba Lecturer Epidemiology MSc Epi, PhD Mr. Simon Kasasa Ass. Lecturer Biostatistics B. Stat., MSc skasasa@musph.ac.ug Mr. David Mukaga Odaka Ass. Lecturer Epidemiology B. Pharm, MPH dmukanga@afenet.net Dr. Joan Nankya Ass. Lecturer Epidemiology MBChB, MSc. Epi Mr. Francis Muwonge Teaching Assistant Demography B. Stat., M. Stat fmuwonge@musph.ac.ug Dr. Jude Olinga Teaching Assistant Health Informatics MBChB Dr. Nakacubo Gitta Sheba Research Fellow Epidemiology B. Dental Surgery, MPH sgitta@musph.ac.ug Dr. Roy William Mayega Research Fellow Epidemiology MBChB, MPH rmayega@musph.ac.ug Mrs. Tabusibwa Kirunda B. Eva Ass. Lecturer Epidemiology BSc. FST, MPH bkirunda@musph.ac.ug Dr. Juliet Sekandi Academic Coordinator MHSR Epidemiology MBChB, MPH jsekandi@musph.ac.ug Dr. Noah Kiwanuka Lecturer Epidemiology MBChB, MPH, PhD Noah.kiwanuka@case.edu Dr. George W. Senior Lecturer International MBChB, MSc, gpariyo@musph.ac.ug End of Project Evaluation of the Capacity Project 109 Pariyo Health/Health Systems Management PhD Dr. S.O. Baine Lecturer Health Care Financing MBChB, DTM&H, MPH, PhD sbaine@musph.ac.ug Dr. Olico-Okui Lecturer Health Management & PHC MBChB, DPH, MSc ookui@musph.ac.ug Dr. Suzanne Kiwanuka Lecturer Health Policy & Planning BDS, PhD skiwanuka@musph.ac.ug Dr. Freddie Ssengooba Ass. Lecturer Health Care Financing MBChB, MPH sengooba@musph.ac.ug Dr. Elizabeth Ekirapa Kiracho Ass. Lecturer Health Economics MBChB, MPH, MPH(Econ.) ekky@musph.ac.ug Dr. Peter Waiswa Ass. Lecturer Primary Health Care MBChB, MPH Mr. Chrispus Mayora Teaching Asst. Health Economics BA(Economics) Dr. Elizeus Rutebemberwa Research Fellow Health Planning & Management BA, BTh., MBChB, MPH ellie@musph.ac.ug Dr. Virgil Onama Research Fellow Health Systems Management MBChB, MSc. vonama@musph.ac.ug Mr. Edward Galowango Site Operations Coordinator, Iganga￾Mayuge DSS Demography BA, MA (Demography) egaliwango@musph.ac.ug Dr. Moses Arinaitwe Research Fellow Population & Reproductive Health MBChB, MSc. (Pop & RH) marinaitwe@musph.ac.ug Mr. Aloysius Mutebi Research Fellow Public Health/Statistics B. Stat. MPH amutebi@musph.ac.ug Dr. Christine Nalwadda Research Fellow Field Coordinator BDS, MPH enalwadda@musph.ac.ug Dr. Nicholas Ayebazibwe Research Fellow MBChB, MPH naybazibwe@afenet.net Ms. Enid Kemari Administrative Asst. Public Administration & Management BA(Arts), MA(PAM) ekemari@musph.ac.ug Mrs. Adikini Anne J. Oketch Administrative Asst./Secretary Administration BA(SS) PGD(HR) jadikini@musph.ac.ug Dr. David Serwadda Associate Professor Epid. & Comm. D’se Control MBChB, MMed, MPH dserwadda@musph.ac.ug Dr. William Bazeyo Senior Lecturer Occupational Health MBChB, MMed wbazeyo@musph.ac.ug Mr. M. Oryema￾Lalobo Senior Lecturer Entomology/Parasitolo gy BSc. MPH mlalobo@musph.ac.ug Dr. Fred Nuwaha Senior Lecturer Communicable Disease Control MBChB, MPH, PhD fnuwaha@musph.ac.ug Dr. Margaret Muganwa Lecturer Env. Health & Food Hygiene MBChB, MPH, DFH, MS mmuganwa@musph.ac.ug Dr. John Ssempebwa Lecturer Environmental Toxicology BSc., MSc. PhD jssempebwa@musph.ac.ug Dr. Juliet Babirye Ass. Lecturer Disease Control MBChB, MPH jnbabirye@MUph.ac.ug Dr. John Bosco Damulira Ass. Lecturer Public Health MBChB, MPH End of Project Evaluation of the Capacity Project 110 Dr. Esther Buregyeya Ass. Lecturer Occupational Health MBChB, M. Phil Health Science eburegyeya@MUph.ac.ug Mr. Abdallah Ali Halage Teaching Asst. Environmental Health BEHS Mr. Stephen Tusingwire Teaching Asst. Environmental Health BEHS Ms. Ruth Mubeezi Teaching Asst. Environmental Health BEHS, MSc. (Env. Sc.) rfmubeezi@yahoo.com Mr. Richard Mugambe Teaching Asst. Environmental Health BEHS, MSc. (Env. Sc.) Rose Namuyanyi Personal Secretary