Abt Associates Inc. n 4800 Montgomery Lane, Suite 600 Bethesda, Maryland 20814 n Tel: 301/913-0500 n Fax: 301/652-3916 In collaboration with: Development Associates, Inc. n Harvard School of Public Health n Howard University International Affairs Center n University Research Co., LLC Funded by: U.S. Agency for International Development Technical Report No. 57 Jordan Training Results Evaluation January 15, 2001 Prepared by: Shirl M. Smith, M.A. PHR Training Officer Howard University Lonna T. Milburn, Ph.D. PHR/Jordan Resident Advisor Abt Associates Inc. Jamal Abu-Saif, M.D. PHR Counterpart for Training Ministry of Health, Jordan Manal Shahrouri, B.A. Training Coordinator, PHR/Jordan Abt Associates Janet Edmond, M.P.H. Program Officer Development Associates Anthony Mensah, B.Sc. ANE Project Assistant University Research Corporation Joanne Jorissen, B.A. ANE Project Assistant Abt Associates Inc. Mission The Partnerships for Health Reform (PHR) Project seeks to improve people’s health in low- and middle-income countries by supporting health sector reforms that ensure equitable access to efficient, sustainable, quality health care services. In partnership with local stakeholders, PHR promotes an integrated approach to health reform and builds capacity in the following key areas: > better informed and more participatory policy processes in health sector reform; > more equitable and sustainable health financing systems; > improved incentives within health systems to encourage agents to use and deliver efficient and quality health services; and > enhanced organization and management of health care systems and institutions to support specific health sector reforms. PHR advances knowledge and methodologies to develop, implement, and monitor health reforms and their impact, and promotes the exchange of information on critical health reform issues. January 15, 2001 For additional copies of this report, contact the PHR Resource Center at PHR-InfoCenter@abtassoc.com or visit our website at www.phrproject.com. Contract No.: HRN-C-00-95-00024 Project No.: 936-5974.13 Submitted to: USAID/Amman and: Karen Cavanaugh, COTR Policy and Sector Reform Division Office of Health and Nutrition Center for Population, Health and Nutrition Bureau for Global Programs, Field Support and Research United States Agency for International Development The opinions stated in this document are solely those of the authors and do not necessarily reflect the views of USAID. Recommended Citation Smith, M.A., Shirl, Lonna T. Milburn, Ph.D., Jamal Abu-Saif, M.D., Manal Shahrouri, Janet Edmond, M.P.H., Anthony Mensah, and Joanne Jorissen. January, 2001. Jordan Training Results Evaluation. Technical Report No. 57. Bethesda, MD: Partnerships for Health Reform Project, Abt Associates Inc. Abstract As part of Partnerships for Health Reform Project (PHR) health sector reform activities in Jordan, extensive training and capacity-building programs were completed for Ministry of Health counterparts and staff, as well as various other key professional staff in the health sector. These training activities included customized courses, study tours to examine other national health systems, workshops, and on-the-job training. This report examines the results of these training activities, as well as the strengths, limitations, and potential future strategies for building staff capacity to initiate and manage health sector reform initiatives in Jordan, such as Decentralization, Hospital Autonomy, National Health Accounts, and Applied Research/Evidence-Based Decision-Making. Table of Contents iii Table of Contents Acronyms.....................................................................................................................................v Acknowledgments ......................................................................................................................vii Executive Summary.....................................................................................................................ix 1. Structure of PHR/Jordan Training Portfolio............................................................................1 2. Objectives and Methodology of the PHR Training Results Evaluation .....................................3 2.1 Objectives.....................................................................................................................3 2.2 Methodology ................................................................................................................3 3. Findings...............................................................................................................................5 3.1 View of Training Outcomes and Results Across the Project .............................................5 3.1.1 Counterpart Observations and Feedback..................................................................5 3.1.2 Summary of Feedback from the MOH Education and Training Directorate ................6 3.1.3 Summary of Feedback from PHR Technical Advisors..............................................7 3.2 Assessment of Training Activities Conducted Under the PHR Hospital Autonomy Component ...........................................................................................................................8 3.2.1 Benefits from PHR-Sponsored Training..................................................................8 3.2.2 Progress Towards Results.....................................................................................11 3.2.3 Observable Performance Improvements.................................................................12 3.2.3.1 Participant Observations ........................................................................12 3.2.3.2 Supervisor Observations ........................................................................12 3.2.3.3 Organizational-Level Improvements.......................................................13 3.2.4 Innovations and Trends........................................................................................13 3.2.5 Contributions to Policy Changes...........................................................................14 3.2.6 Systems Changes.................................................................................................14 3.2.7 Achievement of PHR Training Indicators for the Hospital Autonomy Component....14 3.3 Assessment of Training Activities Conducted Under the National Health Accounts Component .........................................................................................................................16 3.3.1 Benefits from PHR-Sponsored Training................................................................16 3.3.2 Progress Towards Results.....................................................................................17 3.3.3 Observable Performance Improvements.................................................................17 3.3.3.1 Participant Observations ........................................................................17 3.3.3.2 Supervisor Observations ........................................................................17 3.3.3.3 Organizational-Level Improvements.......................................................17 3.3.4 Policy and Systems Changes.................................................................................18 3.3.5 Progress Towards PHR Training Indicators for the NHA Component......................18 iv Table of Contents 3.4 Assessment of Training Activities Conducted Under the Applied Research Component...19 3.4.1 Benefits from PHR-Sponsored Training................................................................19 3.4.2 Observable Performance Improvements.................................................................21 3.4.2.1 Participant Observations ........................................................................21 3.4.2.2 Supervisor Observations ........................................................................21 3.4.2.3 Organizational-Level Improvements.......................................................22 3.4.3 Policy and Systems Changes.................................................................................22 3.4.4 Progress Towards PHR Training Indicators for the Applied Research Component....23 3.5 Foundation Building Training......................................................................................24 3.5.1 Benefits from PHR-sponsored Training.................................................................24 3.5.2 Performance Improvements..................................................................................25 3.5.2.1 Participant-level Improvements..............................................................25 3.5.2.2 Organizational-Level Improvements.......................................................25 3.5.2.3 Policy and Systems Changes ..................................................................25 4. Conclusions and Recommendations.....................................................................................27 4.1 Overall Assessment of PHR Training Component in Jordan...........................................27 4.2 Recommendations.......................................................................................................27 4.2.1 General Respondent Recommendations.................................................................27 4.2.2 Other Recommendations ......................................................................................28 4.2.3 PHR Recommendations.......................................................................................28 Annex A: Number of Participants Trained by Organization ..........................................................31 Annex B: Chronological History of PHR Training Activities, 1998–2000......................................41 Acronyms v Acronyms ANE Asia/Near East APHA American Public Health Association CAP Country Activity Plan EPI Expanded Programme on Immunisation JD Jordanian Dinar JUH Jordan University Hospital KSAs Knowledge, Skills, and Attitudes MOH Ministry of Health MPPMS Managing Procurement of Pharmaceuticals and Medical Supplies NHA National Health Accounts PHCI Primary Health Care Initiatives (USAID) PHR Partnerships for Health Reform Project (USAID) RMS Royal Medical Services SO3 Strategic Objective 3 (USAID) USAID United States Agency for International Development Acknowledgments vii Acknowledgments The authors would like to thank all of the respondents who agreed to be a part of this Training Results evaluation. Their candid feedback was instrumental to the success of this evaluation. In addition, the authors would like to give a special acknowledgement to Dr. Salwa Bitar Qtiet, for her guidance and support of this evaluation exercise, and to Partnerships for Health Reform Project (PHR)/Jordan Chief of Party Dr. Dwayne Banks, for his commitment and dedication in supporting all of the capacity-building and performance improvement activities undertaken by the project. A special note of appreciation is also extended to PHR Lead Counterpart and Head of the Ministry of Health (MOH) Planning Directorate, Dr. Taher Abu-Samen; PHR Program Manager, Dr. Hani Brosk; PHR Counterpart for Applied Research, Dr. Taissir Fardous; PHR Counterparts for Hospital Autonomy, Dr. Ayyob As-Sayideh, and Dr. Abdelrazzaq Shafei; and the Head of the MOH Education and Training Directorate, Dr. Osama El-Samawi. PHR would also like to acknowledge all of the tremendous support provided by Dr. Shugran, Director of Princess Raya Hospital; Dr. Sultan, Director of El Kerak Hospital; Dr. Zuheir Teif, Director of Al Bashir Hospital;and Dr. Salim Malkawi, Director of Al Ramtha Hospital. Executive Summary ix Executive Summary From 1998-2000, the Partnerships for Health Reform Project (PHR)/Jordan training program completed a series of skill-building and capacity-enhancing activities designed to strengthen Ministry of Health (MOH) counterparts and staff in various areas of health sector reform. The primary audience for these activities included staff from the MOH, Royal Medical Service, Jordan University Hospital, and other key partner organizations in Jordan’s health reform sector. A total of 355 Jordanian participants were trained as a result of the PHR/Jordan training program. Training activities included study tours to the United States and Tunisia, a series of customized health and management courses, workshops, and on-the-job informal training opportunities. Training topics ranged from applied health research to hospital autonomy to general management and accounting principals. In August 2000, as the training program was coming to a close, a diverse team of evaluators conducted an assessment of results from the one-year set of activities. The objectives of the evaluation were to: document short-term training results and outcomes; assess the “lessons learned” from program implementation of the training component; and develop a clear picture of future training needs and outcomes. Through in-depth interviews with a subset of training participants, the PHR evaluation team learned that the majority of participants cited improved performance due to skills gained as a positive outcome of the training program. In general, counterparts and key staff gained a deeper understanding of health reform issues, and in the case of hospital autonomy, counterparts implemented training course materials in daily job performance. In addition, the health worker motivation activity resulted in a high degree of visibility for the Education and Training Directorate within the MOH, as the department helped to shape a new policy for linking training and employee performance within the MOH system. Findings from this evaluation indicate that the PHR/Jordan training program completed the majority of its objectives outlined in the training plan. These training activities did not completely close the gap in training needs among MOH and professional staff. Interviews with key MOH staff revealed that staff need to have competency-based training programs in order to fully meet the expectations and requirements of health sector reform challenges in Jordan today. Based on the results of this training program evaluation, PHR has assembled a set of recommendations for future program activities, building upon the results in this report. Select recommendations for future training goals include: > Implement integrated training approaches from the start of the project and tie these programs to overall project objectives; > Focus training programs specifically toward improving job performance; > Continue the capacity-building process at hospitals already trained in hospital autonomy skills; and > Improve the process for follow-up and supervision between the MOH counterparts x Executive Summary and individuals receiving PHR-sponsored training. In addition, PHR recommends that the MOH Education and Training Directorate engage in a partnership with the Planning Directorate to assume responsibility for national health training activities at the central and local levels, involving primary, secondary, and tertiary levels of care. Together these departments could help to coordinate with donors, develop a sector-wide plan for skills development, and manage resources more effectively. 1. Structure of PHR/Jordan Training Portfolio 1 1. Structure of PHR/Jordan Training Portfolio The Partnerships for Health Reform Project (PHR)/Jordan training portfolio was structured to support the achievement of objectives and intermediate results as identified in its Country Activity Plan (CAP) for 1999-2001. All of the activities undertaken were in direct support of United States Agency for International Development (USAID)/Amman’s third Strategic Objective (SO3), “Improve access to quality reproductive health care services.” In 1999, PHR conducted a training needs assessment, the results of which were documented in the PHR publication, “PHR Training Plan1 ,” for the purpose of identifying the sets of skills that would strengthen the capacity of the Ministry of Health (MOH), Royal Medical Services (RMS), Jordan University Hospital (JUH), and other key partner organizations to lead broad-based reforms in the health sector. The resulting training plan defined the series of activities that PHR would conduct to support SO3, with emphasis on the roles and responsibilities held by PHR counterparts, hospital directors, and senior officials from the Palace, the MOH, the RMS, JUH, Princess Raya, El Kerak, Al Ramtha and Al Bashir Hospitals, Jordan University, and Jordan University for Science and Technology in achieving measurable performance improvements, as well as policy and systems changes on behalf of their respective institutions. From 1998-2000, PHR succeeded in training a total of a total of 355 participants from the public and academic sectors. (See Annex A for participant profiles and Annex B for a list of training activities for Jordanians.) Looking at a cross-section of participants who were surveyed for this evaluation, 49 percent of these participants held bachelors degrees; 2 percent had doctorate degrees; 20 percent had degrees in medicine; and 14 percent held masters level degrees at the time of the training. A detailed breakdown of these representative participant profiles appears in Table 1. This report will begin with a description of the approach used to evaluate the PHR/Jordan training program, followed by the presentation of general findings and findings specific to PHR’s major program components. The report closes with conclusions and recommendations that will hopefully contribute to the development of future efforts aimed at health systems strengthening. 1Smith, Shirl. (July) 1999. Jordan: PHR Training Plan. Bethesda, Maryland: Partnerships for Health Reform, Abt Associates Inc. 2 1. Structure of PHR/Jordan Training Portfolio Table 1: Representative Participant Profiles by Education and Current Position Number of Participants Level of Education Current Job Work Location Previous Experience Course M.D. Ph.D. Masters High Degree Bachelors Associate Degree High School Not Specified Doctor Director Project Manager Pharmacist Planning Officer Consultant/ Researcher/ Assistant Student Administrator/ Clerk Nurse Biostatistician Auditor/ Accountant Not Specified Al Bashir Hospital Al Ramtha Hospital Princess Raya Hospital Karak Hospital MOH Jordan University Hospital Royal Medical Services Audit Bureau PHR PHC Not Specified Research None Other Related Work Experience Related Coursework Not Specified Accounting and Data Interpretation 5 3 1 1 1 1 2 1 2 1 1 1 5 Applied Health Research 15 6 4 5 1 3 1 1 2 1 2 2 1 1 1 8 2 1 1 2 1 7 7 English Course 66 15 1 8 32 2 3 5 14 3 3 2 9 1 9 17 2 6 2 8 38 10 1 1 6 1 43 6 16 EPI Training 9 1 1 4 3 1 1 2 3 1 1 2 3 2 2 7 1 1 Financial Management 8 7 1 1 1 3 3 6 2 6 1 1 General Accounting 6 6 1 1 4 6 5 1 General Management 32 6 4 15 7 4 2 1 12 8 5 5 13 7 7 22 9 1 Medical Records 10 1 6 1 2 1 7 2 2 3 5 9 1 SPSS 4 1 1 1 1 1 1 1 1 2 1 1 4 TOTAL 155 31 3 22 1 76 2 13 7 21 9 6 5 3 20 2 33 28 1 17 10 10 21 16 58 24 6 4 3 4 1 8 2 10 8 1 25 19 Notes: Number of individuals who participated in two or more courses: 8 Real number of participants: 144 2. Objectives and Methodology of the PHR Training Results Evaluation 3 2. Objectives and Methodology of the PHR Training Results Evaluation 2.1 Objectives The objectives of the PHR Training Results evaluation were: 1. To capture and document short-term training results and outcomes from PHR-sponsored training investments undertaken as part of the 1999-2000 PHR/Jordan Training Plan. 2. To evaluate experiences with program implementation as a basis for strengthening the design of future capacity-building initiatives in Jordan. 3. To establish a clear picture of training outcomes accomplished by the PHR project and its main partner institutions, for the purpose of facilitating the design of training activities by USAID/Amman and its country counterparts under the new follow-on project for health sector reform and health systems strengthening in Jordan. 2.2 Methodology The structure of the PHR Training Results evaluation framework bears a direct correlation to the layout of the PHR/Jordan 1999-2000 Training Plan. This results evaluation is centered on the following research questions: 1. How have PHR trainees applied new knowledge, skills, and attitudes (KSAs) acquired from PHR-sponsored customized courses in Jordan, third country training (focusing on study tours and course enrollments in other countries), and US-based short-term training? 2. What specific performance improvements came about as an outcome of this application in the work place, including participant-level improvements, organizational performance improvements, and short-term policy or systems changes? 3. To what extent did PHR achieve the performance indicators stated in the 1999-2000 PHR/Jordan Training Plan, given the sets of training activities that were accomplished under the Hospital Autonomy, National Health Accounts (NHA), Applied Research, and Foundation Building Training components over a 12-month period? In order to answer these questions, data were gathered on the following three levels (see Table 2): > Input Level, referring to the actual training activity/event; > Process Level, which refers to the individual participant’s application of new KSAs and improved performance; and finally, > Impact level, which refers to the broad impact of training interventions on 4 2. Objectives and Methodology of the PHR Training Results Evaluation organizations, the health system, and clients. Table 2: Levels of Evaluation Input Level a) Assessment of Training Event b) Assessment of Training Provider c) Assessment of Management/Administrative Support d) Acquisition of KSAs Process Level a) Application of KSAs b) Performance Improvement Impact Level a) Organizational Change b) Health System Change c) Client/End-User Satisfaction In consultation with the PHR Counterpart for Training at the MOH, the PHR/Jordan team randomly selected three participants for each course evaluated under the Hospital Autonomy, NHA, Applied Research, and Foundation Building Training components. In the actual implementation of the evaluation exercise, PHR achieved a 79 percent response rate. A total of 41 participants and six supervisors were interviewed. Two survey forms were designed to collect the training results data. The first, entitled “Participant Interview Form,” was administered to the sample of trainees who attended the training courses assessed for this evaluation exercise. A second “Supervisor Evaluation Form” was developed for the purpose of recording supervisor observations about changes in employee performance, before and after the training received, in addition to providing a second tier of information about specific organizational changes that came about as a result of the training. The PHR evaluation team utilized the following methodology to administer the participant and supervisor surveys: 1. Standard written introductions were developed for each form to avoid skewed interpretation of survey; 2. PHR evaluators provided a standard oral introduction at the beginning of each encounter, including an orientation as to how the evaluation would be completed; and 3. Participants were encouraged to ask questions if something was unclear or not understood. 3. Findings 5 3. Findings 3.1 View of Training Outcomes and Results Across the Project 3.1.1 Counterpart Observations and Feedback Nearly all PHR counterparts stated that they had derived numerous tangible benefits from PHR￾funded training activities across the board. It was noted that the opportunities to work with PHR resident advisors and consultants helped the counterparts gain more confidence in the areas of report writing and preparing presentations. In addition, it was cited that on-the-job training, as a form of capacity building throughout the life of the project, was especially valuable as a learning vehicle. Most PHR counterparts felt that they now understood the meaning of reform, as well as the importance of partnerships with other sectors. It was felt that the timing of programs became an important issue, given that opportunities to apply new skills and knowledge were not immediately available upon their return to the workplace. In several instances, counterparts expressed concern about the participant selection process as a whole. The final selection of MOH participants is determined by the Minister of Health. As a result, a small number of PHR-funded participants did not apply their new skills to the context of PHR’s ongoing technical assistance to the MOH and other partner institutions. Because of this trend, it will be more difficult to quantify or measure the impact of these training investments over the long term. Overall, the counterparts observed that they now work very well as a team, and that PHR’s capacity-building initiatives have made a difference in the way they are performing in their current positions. In the future, the counterparts felt they would benefit by having training programs that are specifically geared to enhancing their roles as technical managers on a donor-funded project. In evaluating PHR’s effectiveness in addressing the performance gaps that were encountered in the MOH, PHR’s lead counterpart stated that PHR’s training investments were focused, but did not close the gap across the whole organization. PHR successfully forged an impact in some areas, primarily through its work with the counterparts and the staff of Al Ramtha, Al Bashir, El Kerak, and Princess Raya Hospitals. Changes in performance were readily observable in each of these hospital facilities. To improve the design and implementation of capacity-building programs in the future, all of the counterparts stressed the importance of competency-based training as a way to close performance gaps at the organizational and employee levels. Such an approach would transcend PHR’s original focus on foundation skills training, which was geared to provide a broader introduction to health reform concepts. As an extension of this sentiment, the counterparts requested involvement in project-initiated applied research studies from the outset, having direct responsibility for data interpretation and analysis. A recommendation for planning the training component of the successor project was to develop an integrated training plan collaboratively at the beginning of the project cycle, instead of mid-way into the project. The need to sustain the capacity built so far to support health systems reforms within the MOH system and among each of the Ministry’s partner institutions was also seen as a priority. Suggested support measures included: 6 3. Findings > Ensure that trained MOH staff continue in positions of leadership on current and future health reform initiatives; > Continue to build a critical mass of individuals who can provide leadership on health systems and policy reforms; > Better integrate training supported by PHR into PHR-initiated technical assistance activities and other donor-funded health reform projects; > Provide support to trained staff on other health sector reform-related projects and activities; and > Continue building more capacity by utilizing trained staff and counterparts to train new cadres. 3.1.2 Summary of Feedback from the MOH Education and Training Directorate The MOH Education and Training Directorate has been a strong supporter of the PHR training portfolio over the past two years. The Head of the Directorate contributed to the design of the PHR training plan in 1999, and has monitored the progress of trainees working at the central and hospital levels. From the standpoint of this directorate, PHR has been effective in addressing the performance gaps that were encountered in the MOH organizational structure. Training results have been attributed to high levels of interest among middle- and high-level managers to collaborate with the PHR project. Staff associated their participation in PHR training with increased management authority and autonomy in the workplace; essentially, the more staff are trained, the more they can feel confident in effecting management decisions. The quality of PHR’s training courses was viewed as satisfactory overall. With practice, trained staff will be in a position to implement what they learned. As a way of harnessing the capacity that has been established, the Education and Training Directorate would like to see how the MOH, PHR, and Primary Health Care Initiatives (PHCI)2 can combine resources. Under PHCI, for example, the MOH is helping to design a course for nurses, in addition to designing a database system for the purpose of tracking staff participation in training courses and tying their performance to accreditation. Ideally, the successor project to PHR would also be open to meeting the needs of the MOH beyond the current parameters of applied research, NHA, and hospital autonomy. The MOH Education and Training Directorate would like to see more emphasis placed on building capacity in health economics at the masters level; at this stage, the Ministry is lacking cadres with this specific background. Looking at PHR’s Hospital Autonomy component, the Head of the Education and Training Directorate observed a need for staff training to be accompanied by supporting systems changes which allow staff to have more authority to make decisions. Staff should be given opportunities to prove themselves as individuals who can take the right decisions; this will allow the MOH to have more confidence in their leadership and decision-making abilities. It was also observed that PHR’s Health Worker Motivation component has brought a new range of responsibility to the Education and Training Directorate as a department unto itself within the MOH organizational structure. As an outcome of taking part in the PHR workshops and policy forums on health worker motivation, the Education and Training Directorate has forged a higher 2 The Primary Health Care Initiatives project is USAID/Jordan’s flagship bilateral health project. 3. Findings 7 degree of visibility and presence in helping to shape a new policy for rationalizing the link between training and employee performance in the MOH system. 3.1.3 Summary of Feedback from PHR Technical Advisors PHR/Jordan Resident Advisor Dwayne Banks cited the immense progress that PHR counterparts had made over a two-year period. At the outset of training plan implementation, the counterparts progressed from a baseline of limited capacity and understanding of concepts related to health sector reform, to the point of providing technical direction and input into policy formulation and systems improvements at the hospital, governorate, and directorate levels. Citing the example of the PHR counterparts in charge of hospital autonomy and NHA, results have been most readily observable under conditions where the participants’ range of authority and responsibility for a specific reform could be increased incrementally over time. Dr. Banks noted that the counterparts for hospital autonomy are now implementing reforms independently, using a PHR-developed technical and regulatory framework for hospital decentralization. Dr. Lonna Milburn, PHR/Jordan Resident Advisor for Training and Applied Research, noted PHR’s success in implementing all components of the PHR/Jordan training plan within a one year period. Given the brevity of this timeframe, she felt that it would be difficult to observe intermediate results from PHR’s training investments at this stage. Looking back at the series of customized courses, workshops, and study tour activities implemented to date, Dr. Milburn outlined the extent of the participants’ progress in undertaking specific action-steps as a result of PHR training and technical assistance. (See Table 3.) Table 3: Impact of PHR Training Programs by Component Component Status of Action-Steps Taken Applied Research PHR started the applied research training portfolio in October 1999. As an outcome of concentrated research forums, in￾country workshops, and supervised applied and operational research studies, 100% of the participants trained took part in the development and presentation of policy recommendations for a) the creation of a national agenda for health policy research and b) the introduction of targeted performance improvements at Al Bashir and Al Ramtha Hospitals. Hospital Autonomy Upon completing customized skills training in computer applications, general management, and medical records, staff at El Kerak and Princess Raya pilot hospitals have successfully restructured systems for accounting, personnel management, and medical records. National Health Accounts The NHA team has completed their final report in collaboration with A.K. Nandakumar and Manjiri Bhalwakar. The team took steps to create a supportive policy environment for the institutionalization of NHA by tying it to the research-to-policy link that was explored during the PHR Applied Research Forum in July 2000. The following is a detailed look, for each of these components, at the benefits, progress towards results, observable performance improvements, and policy and systems changes that have resulted so far from PHR training activities. 8 3. Findings 3.2 Assessment of Training Activities Conducted Under the PHR Hospital Autonomy Component PHR CAP Result 2.4: Improved hospital performance and efficiency through delegation or decentralization of select responsibilities PHR Training Indicator: Practical concepts assessed and used in the Jordan context Courses Evaluated: Financial Management General Management Medical Records Managing Procurement of Pharmaceuticals and Medical Supplies (MPPMS) Hospital Autonomy Study Tour to Tunisia 3.2.1 Benefits from PHR-Sponsored Training Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace Financial Management Added skills in financial management concepts. How to design tables to simplify work procedures. The ability to prepare an independent budget. Designing tables to simplify accounting procedures in the hospital (especially related to expenditures); preparing the hospital budget; performing financial analyses. General Management Understanding of modern concepts of General Management; more organized; change in attitude. More knowledge and skills in general management of health systems. Use of concepts in health sector management; increased knowledge in principles of management. Management and planning. Communication skills; thinking and application methods. Increased confidence in making decisions. Tools for enhancing productivity. Using computers for data entry; applying new management skills on the job; short term planning. Reorganizing department; sharing activities and opinions; planning. Better use of staff; expanded use of teams in the hospital setting. 3. Findings 9 Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace Medical Records Increased knowledge of medical record concepts; learning about the experiences of other hospitals; computer skills. Understanding the relationship between the medical record systems in other organizations and our hospital. Understanding the problems encountered in other medical records centers; working to solve these problems along with other colleagues and managers. Learning about the standards of the medical research operating system, and how technology and quality of work contribute to productivity. Computer skills; planning; skills in problem solving; information sharing. Communication skills, used to persuade decision-makers to improve the medical records department by applying modern techniques. Sharing computer skills with colleagues. Planning and organizing responsibilities for staff. Problem-solving. Increasing efficiency in using the computer for data entry, filing, and statistical analysis. Participating in the decision-making process. Managing Procurement of Pharma￾ceuticals and Medical Supplies Seeing the experience of other countries (12 countries represented at course). Learning procedures for buying pharmaceuticals, preparing tenders and contracts, choosing suppliers, writing proposals, and writing specifications for drugs. Analyzing the performance suppliers as part of the selection criteria. Learning how to deal with insurance and shipment from country of origin; bulk purchasing and VEN (Vital, Essential, and Non-Vital Drugs) analysis. Learning about treatment protocols and essential drug lists, based on experiences of other countries. Seeing importance of establishing treatment modalities and protocols as a way to achieve cost savings. Made presentation at MOH to share these recommendations. Skills for drug purchasing at the hospital level after decentralization. (Drugs are currently procured from the MOH’s central medical stores.) Skills in tendering, purchasing, and contracting with suppliers. In South Africa, visited a medical store that supplies small hospitals. Training assistant to use new skills in order to be prepared for the future. Using negotiation method for direct purchases. Purchasing drugs at the lowest cost/best value. Leveraging between suppliers to get lowest price. Negotiating and obtaining quotations by phone. Tendering is in the future. 10 3. Findings Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace Hospital Autonomy Study Tour to Tunisia Seeing how the hospital board of trustees takes decisions through the director of the hospital. This approach helped the administration to deal more effectively without needing to conduct detailed studies. Seeing the Tunisia MOH’s referral system, capable of tracking the status and diagnosis of patients. The opportunity to observe the concept of delegation in practice. In Tunisia, staff rely less on the central Ministry because responsibility is formally delegated to the governorates. Observing staffing practices in the absence of a civil service commission. Hospital directors have the autonomy to develop and advertise job assignments, and to make hiring and firing decisions. Seeing the experience of hospitals before decentralization and how these hospitals and medical services improved after decentralization. The opportunity to exchange experiences with staff at the MOH headquarters and Raitha Hospital. Representatives of the Tunisia MOH departments of personnel, finance, and maintenance all encouraged us to go forward with decentralization. Seeing hospital staff manage their own budget. Hearing Tunisian staff compare the benefits of decentralization before and after (i.e., more efficiency, more time spent with physicians and technical staff). The importance of keeping patients informed about the different changes in the hospital, and encouraging consumer buy-in/participation. Upon returning to Jordan, study tour materials were shared and reviewed in conjunction with hospital staff by department (nursing, administration, personnel, etc.). Learned how to solve any problem in the hospital, and how to structure a staff organizational chart and use it as a management tool. Exploring ways of using this study tour to make changes in the administration of the hospital. It is not possible to implement everything in light of current MOH rules and regulations. We are looking at possible applications, such as criteria for selecting hospital directors. Plotting the hospital’s new organizational chart. Giving everyone in hospital exact and specific duties. Staff have ownership about the hospital’s new organizational chart. It clearly delineates the way in which staff perform specific functions in the hospital. The Nursing section is now more efficient. The hospital director and nursing staff meet once a month. Nurses are encouraged to work with confidence. Problems are much fewer now. Each [hospital] unit is now equipped to solve its own problems. The hospital autonomy working group has also prepared a booklet to inform hospital staff about their job responsibilities. Lessons learned from the study tour were shared with the hospital working groups; at the same time, staff have yet to fully apply this system because decentralization hasn't been given by the MOH. The MOH has given authority in some ways only, for example, the authority to spend up to 1000 JD without going to the MOH for permission, or the authority to decide how to hire and take control of the hospital. We did not have this authority before. 3. Findings 11 3.2.2 Progress Towards Results Under the Hospital Autonomy component, PHR’s customized training and study tour investments built a vision for how decentralization could operate and its potential benefits. However, the majority of the participants interviewed cited MOH rules and regulations as key constraints to their ability to apply new knowledge and skills in the workplace, combined with limited management authority to effect decisions that would impact the introduction of specific changes in job performance. In the case of the respondents who received skills training in procurement practices, for example, difficulties were cited in applying all of the skills acquired from the course. In this instance, the pace of higher level regulatory reforms in hospital drug purchasing practices had not yet coincided with skills development in tendering and international bidding procedures. At the time of this evaluation, hospital pharmacists were confined to purchasing drugs within the range of 200 Jordanian dinar (JD), whereas 500 to 1,000 JD are required to execute a tender for the purchase of drug supplies and medical equipment. As Jordan is about to become a member of the World Trade Organization in 2001, respondents based in the pilot hospital pharmacy departments were optimistic that further policy changes would support their ability to develop tenders at the hospital level. The Minister also recently approved a policy change that will allow each pilot hospital to spend up to 1,000 JD per month for drug purchases. (The previous budget ceiling used to be 200 JD per month.) In the area of medical records, the shortage of trained personnel, along with MOH rules and regulations, was cited as a barrier to skills application. Another barrier was a shortage of computers at each pilot hospital. Respondents felt that it would be possible to overcome these constraints by hiring more personnel and by increasing the number of computers in the medical records departments of El Kerak and Princess Raya Hospitals. Respondents based in the pilot hospital accounting departments felt that it would be important to continue to decentralize hospital management functions. This would give hospitals greater management authority to implement skills acquired from the customized accounting courses. Reforms in MOH rules and regulations surrounding financial management functions would help to create a more supportive policy environment in which to apply these new skills. Constraints cited in the application of skills from the general management course included: limited computer knowledge; timidity to converse in English; MOH regulations; and limited management authority. Solutions to these constraints were cited as within range of the respondents’ ability to impact specific changes in their respective work environments, namely, through increased delegation of authority, cooperation with senior managers at the hospital level to adopt new rules and regulations, and building a base of support for new ideas through continued information-sharing with colleagues. Respondents attending the hospital autonomy study tour in Tunisia observed that the rules and regulations in Jordan are highly centralized. Continued dialogue and policy communication with high-level decision-makers at the central ministry and parliamentary levels were viewed as tenable solutions to create a more supportive policy environment for hospital decentralization. The challenge of building staff confidence, ownership, and awareness of the concept of decentralization itself was also a constraint to applying lessons learned from the Tunisia study tour. To overcome this, management by participation was used as an approach to obtain staff buy-in around the concept. This approach involved having staff directly participate in the design of hospital-level policies surrounding personnel management, administration, purchasing, supply management, and financial management. 12 3. Findings From the standpoint of ensuring that training investments are tied more closely to the pace of policy and systems reforms occurring at the central and hospital levels, it was recommended that trainees in each of the pilot hospitals be encouraged to follow up on a more consistent basis with the MOH counterparts for hospital autonomy. This exchange of information and experience would serve to enhance planning next steps for future capacity-building interventions, in addition to creating a more supportive environment for trainees to apply new skills and knowledge. 3.2.3 Observable Performance Improvements 3.2.3.1 Participant Observations All respondents noted significant improvements in their own performance at work. Twenty percent of all respondents reported higher levels of confidence and productivity upon returning to the workplace. Other positive benefits included: having a more realistic job outlook; improved communication skills; and the ability to do more work in less time, without requiring the assistance of other departments in the hospital. As a result of acquiring new skills, 33 percent of all respondents interviewed under the Hospital Autonomy component have initiated performance improvements within the parameters of their current management authority. Examples included: developing requests for proposals for local vendors in line with a 200 JD per month budget; developing a plan to improve the medical records department; sharing new ideas and knowledge with hospital staff; designing protocols for medical records; and using computers to simplify hospital accounting and financial management procedures. The exposure to different country experiences also instilled a greater level of confidence on behalf of the middle-level managers who are responsible for guiding the introduction of systems and policy reforms within the central ministry and the two pilot hospitals. Differences in the way these officials approached their positions stemmed from a stronger understanding of decentralization as an organizational concept and the ability to implement it more than in the past. 3.2.3.2 Supervisor Observations The directors of El Kerak and Princess Raya pilot hospitals also noted changes in performance among hospital staff who participated in hospital autonomy-related training courses. Notable changes included: a change in attitude; a better understanding of individual job responsibilities; greater initiative; stronger skills in problem-solving without direct supervisory involvement; and increased staff motivation, involvement, and ownership in the day-to-day functions of the hospital. With its new computerized system, the medical records department is now more self-contained at Princess Raya Hospital. El Kerak Hospital’s new pharmacy inventory tracking system also provides an accurate picture on the number and types of drugs dispensed, thereby eliminating previous problems with oversupply and expiry. All drug purchases are now effectively planned in accordance with need. Both pilot hospital directors cited a higher comfort level in going before senior, high-ranking MOH officials and advocating for hospital reforms. Skills in advocacy and policy communication have fostered the ability to lobby with greater confidence on how the MOH can improve the hospital system within the confines of the new rules and regulations that are being developed. These skills have enabled the directors to change the way the MOH views the importance of hospital decentralization, resulting in greater efficiency, greater benefits for each hospital’s client base, and greater cost-savings in the long term. 3. Findings 13 3.2.3.3 Organizational-Level Improvements While it may be too soon to document medium-term training results across the board, 100 percent of all respondents observed short-term improvements in organizational performance at the two pilot hospitals and Al Bashir Hospital as an outcome of new skills and attitudes being applied in the workplace. Improvements included: > Use of new procedures for filing, work planning, and scheduling; > Discussions between the heads of the medical records departments and staff to improve coordination; > Team-based approaches to management and problem-solving; > Changes in employee attitudes towards work; > Improved relationships between the hospital directors and employees, resulting in improved performance and productivity; > Stronger collaboration between MOH decision-makers and hospital-level staff on systems changes needed for reform; > Increased authority for decision-making among middle-level managers; > Reduced patient waiting times for medical records; > Shortened time for repairing equipment and processing medications; > Greater patient/client satisfaction; and, > Initiative among trained staff to provide on-the-job training to co-workers. Al Bashir Hospital reported a tangible performance improvement in its medical records department. Staff participation in the customized medical records course, along with the placement of a new PHR computer in the medical records registries for the inpatient and outpatient departments, have cut down on records duplication. As a result of these measures, standards and protocols for quality control have improved. 3.2.4 Innovations and Trends A direct outcome from the study tour visit to Tunisia was the initiative on behalf of both pilot hospitals to develop new organizational charts. These charts were jointly developed through a process of mutual exchange between the hospital autonomy working groups of Princess Raya and El Kerak Hospitals, in collaboration with the MOH Planning Directorate. Under the old organizational chart, both hospitals had an Administrative Assistant and a Professional Assistant to supervise administrative and clinical functions. Under the new staffing configuration, functions have been further compartmentalized through the addition of Supply and Pharmacy Assistant positions. A new hospital board, intended to function like a steering committee, was also placed at the top of the organizational hierarchy to retain the highest administrative rank in the hospital, followed by the hospital director and main department heads. To enhance each hospital’s effectiveness in the areas of client satisfaction and community outreach, each hospital also added a Professional Medical Committee; the members of this committee include the main staff of the hospital and will ultimately be joined by two community representatives. Another important innovation resulting from the process of ongoing capacity building under the Hospital Autonomy component, is the leadership initiative undertaken by the MOH Planning and Education and Training Directorates to develop and implement customized training in administration, 14 3. Findings delegation, and financial management skills for pilot hospital staff. The PHR Counterparts for Hospital Autonomy, in collaboration with the Education and Training Directorate and senior department heads from the MOH Directorates of Administration and Financial Management, designed and led two month-long training programs to strengthen skills in delegation and budgeting. As a result of this competency-based training, both hospitals will be equipped to prepare their first autonomous budgets for presentation to the Minister of Health. Lastly, the directors of each pilot hospital have participated in high-level forums to share lessons learned from the decentralization process with hospital directors in other governorates. 3.2.5 Contributions to Policy Changes Approximately 61 percent of the respondents interviewed stated that they were able to make a contribution to policy changes at the hospital and/or central levels. Examples included: 1. Making suggestions for consideration by decision-makers; 2. Securing high-level support and permission to work on hospital autonomy priorities, which have led to changes that support hospital-level goals and objectives for reform; 3. Overcoming the MOH bureaucracy; 4. Developing policies for dealing with patients and staff; 5. Working with a new system of authority and delegation, plus a system for monitoring and evaluation and a new organizational chart for the hospitals; and 6. Creating a plan to raise employee awareness about new policies, procedures, and responsibilities related to hospital autonomy. The remaining 39 percent felt that they were not able to make a direct contribution due to the lack of authority associated with their current position. Respondents in this category felt that they could reasonably influence the process of policy change by way of offering suggestions or ideas to their superiors. Beyond this, each felt limited in terms of his or her own ability to make a significant impact on the development of a specific policy change. 3.2.6 Systems Changes While it is still early to determine if long-standing systems changes have occurred in each pilot hospital, it is evident that skills training has contributed to observable operational changes, including: 1. A decrease in the number of steps required to deliver services to patients, and more efficient service delivery, due to the fact that hospital staff have a stronger awareness of exact duties; 2. Increased community participation and ownership in the services of the hospital; and 3. New policies and procedures to deal with pharmaceutical procurement, personnel, and medical records. 3.2.7 Achievement of PHR Training Indicators for the Hospital Autonomy Component The above evaluation results demonstrate that the PHR training indicator for the Hospital Autonomy component, “Practical concepts assessed and used in the Jordan context,” has been achieved. Practical applications for hospital autonomy have resulted from a combination of training inputs, including customized courses, exposure to the experience of another country in the Asia/Near 3. Findings 15 East (ANE) region, and an off-the-shelf course on procurement. This training has directly impacted the pace of reform at Princess Raya and El Kerak Hospitals, in addition to strengthening the performance of senior- and middle-level managers who have a direct role in the implementation of management and regulatory reforms. 16 3. Findings 3.3 Assessment of Training Activities Conducted Under the National Health Accounts Component PHR CAP Result 1.1: Establish National Health Accounts as a policy tool on a sustainable basis PHR Training Indicators: 1. Capacity to analyze NHA data increased; 2. Steps for organization of NHA system completed at national level; 3. NHA institutionalized in the MOH SPSS Computer Skills Epi-Info Computer Skills Accounting and Interpretation of Data Courses Evaluated: NHA Regional Workshops 3.3.1 Benefits from PHR-Sponsored Training Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace SPSS Computer Skills Learning the software for statistical analysis. Data entry; processing and analyzing data in a statistical way. No opportunities reported. EPI-Info Computer Skills Learning and using the software application for tracking and analyzing epidemiological data and trends. Research skills; interpretation of data. Collecting data; tabulating data; report￾writing; disseminating research results. Accounting and Interpretation of Data Better understanding of NHA concepts; computer applications; increased productivity and motivation. Data analysis. Participated in writing NHA technical reports; analyzed NHA data to identify issues and policy recommendations. NHA Regional Workshops Learning what is required of public institutions regarding NHA data collection and reporting. Understanding the mechanics of NHA. Learning the experience of other countries with NHA.. Learning how to classify expenditures, including sources and uses, and provider expenditures; how to reallocate resources for primary health care. Understanding the importance of institutionalizing NHA, especially how to convince policy makers and how to look at results and explore policy options. How to prepare reports. How to be more active in discussions. Having a broader vision of financing systems across sectors, especially in looking at expenditures, savings, and reallocation. How to collect and validate data and ensure data transparency. Currently applying NHA in the RMS health and financial management information systems; computerized financial data system in line with NHA reporting requirements, and are performing a computerized study on how the system is working. Based on the results, will have all financial departments at the RMS collecting data on NHA by adding NHA data sets to current systems. Contributed to NHA final report by way of data collection, analysis, and interpretation of data. It is difficult to apply what was learned due to changes within the MOH; the time for implementing NHA has not been sufficient. 3. Findings 17 3.3.2 Progress Towards Results Respondents interviewed under the NHA component unanimously agreed that NHA was a valuable tool for improving resource allocation and evidence-based decision-making. At the same time, all respondents encountered constraints in applying new skills in the workplace. The most prevalent constraint cited was time management. Several respondents found it difficult to juggle the responsibilities for NHA data collection and analysis with their current job workloads, because of the time investment required. Limited supervisory support and incentives were also mentioned as obstacles, combined with the lack of a computer in one instance and the need for more counterparts with backgrounds in finance and accounting. The majority of the respondents interviewed held the view that greater cohesiveness among the institutional partners would be key to ensuring the future success of NHA, along with a shared commitment to move the policy process forward. A unification of the MOH, JUH, and RMS financial systems was suggested as a measure to institutionalize NHA, along with the creation of a national, multi-sectoral steering committee that would involve other ministries and health committees of the higher and lower parliament. The course inputs under the NHA component provided a mixed range of benefits for the respondents interviewed. Training in Epi-Info was viewed as helpful, but not directly relevant to one respondent’s role on the NHA team. Reasons for this were primarily attributed to a gap surrounding the customization of this course to NHA applications. The majority of the respondents gained more direct benefits from the customized skills training in SPSS and in accounting and interpretation of data, along with the process of deriving practical skills and concepts from the first NHA workshop in Jordan and the three ANE regional NHA workshops. 3.3.3 Observable Performance Improvements 3.3.3.1 Participant Observations Fifty percent of the respondents in this category felt that their job performance improved as a result of the training received. Improvements included: better communication skills; increased exposure to working with officials in the public, semi-public, and private sectors; skills in data analysis and interpretation and improved confidence. The remaining 50 percent stated that they did not see any improvements in their own individual performance at work, because the opportunity to fully apply these skills is contingent upon the start-up of related donor-funded activities. These activities include undertaking a cost study for primary health care centers under PHCI and implementation of a World Bank loan designed to improve the financial management systems of the RMS. In the face of these future initiatives, this group of respondents did not observe any changes in their job performance. 3.3.3.2 Supervisor Observations At the supervisory level, skill-building in NHA was seen to have a positive impact, as it established a critical mass of individuals who can apply NHA as a tool for policy decision-making. An interest was expressed in expanding this capacity through additional technical assistance on the ground. 3.3.3.3 Organizational-Level Improvements Given the early stages of NHA implementation, respondents cited a limited number of performance improvements at the organizational level. The Planning Department within the RMS, for example, is well positioned to support institutionalization of NHA now that each team member understands its benefits and applications. In addition, the RMS financial data system has been changed to be consistent with NHA datasets. 18 3. Findings 3.3.4 Policy and Systems Changes It is still too early to determine if training in NHA has contributed to a longer-term policy or systems change. All respondents felt that they were able to make a positive contribution to the process of policy and systems change in their respective organizations, including: re-configuring agency management information systems to collect NHA data; writing reports; leveraging the support of high-level decision-makers in multiple sectors; and using more proactive and participatory approaches to meeting and workshop planning. 3.3.5 Progress Towards PHR Training Indicators for the NHA Component In comparing the baseline data on the level of skills that the NHA team manifested at the beginning of the training process, it is evident that the first training indicator, “Capacity to analyze NHA data increased,” was fulfilled. There has been tangible progress towards the remaining two training indicators, but not fulfillment (“Steps for the organization of NHA system completed at national level; NHA institutionalized in the MOH”). 3. Findings 19 3.4 Assessment of Training Activities Conducted Under the Applied Research Component PHR CAP Result 1.1: Capacity of the MOH and other Jordanian institutions to take leadership in policy development and implementation of health sector reforms strengthened PHR Training Indicators: 1. Research agenda formulated; 2. Leadership capacity for research strengthened in the MOH Courses Evaluated: Applied Research Patient Flow Analysis PHR Scholars and Mentors Program American Public Health Association (APHA) Conference Linking Research to Policy/US Study Tour 3.4.1 Benefits from PHR-Sponsored Training Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace Applied Research New skills and research techniques tied to the Health Worker Motivation and 360 Degree studies; a hands-on instructional approach allowed actual implementation of lessons learned. Learning to undertake research studies in steps; time management and organizational skills. Developing skills in using the Data Show computer software. Developed three proposals, the first of which was funded by WHO. Applied Data Show techniques in training presentations made to MOH staff in four governorates. Patient Flow Analysis Communication skills; skills in data collection, coding, and analysis. Applying number coding to patient flow analysis data. The importance of timing in relation to patient flow analysis. Introduction of significant organizational changes at Al Bashir Hospital as a result of implementing data findings. PHR Scholars and Mentors Program Gaining experience in theory and practice of applied research. The opportunity to gain real research experience for the first time. Data collection. Qualitative analysis (by learning to do coding). Presentation skills; skills in policy communication to support interactions with high￾level decision-makers. Designed a study on health worker development activities, focusing on the perspectives of nurses at Al Bashir and Jordan University Hospitals. Communicated study research findings to the director of Al Bashir Hospital; study to be published by the Jordan Dental Association Journal. Designed a pilot study on job satisfaction for Al Bashir Hospital; formulated a research framework, survey instrument, and study sample. Presented findings at closing workshop for PHR scholars, involving representatives from the MOH, academic, and private sectors. 20 3. Findings Course Benefits Received from Training New Skills Acquired Skills Application in the Workplace APHA Conference Chance to meet different people and look at problems in different ways. At APHA, found that many of the problems were also prevalent in Jordan, i.e. smoking, cancers, chronic diseases, and health care delivery in under-served communities. How to deal with problems on all levels, i.e., within the private sector, government sector, not-for-profit organizations, and local communities. Working with local groups in Ramtha; now taking steps to establish committees in the hospital to improve the relationship between hospital personnel and the local community, so that the community has a more active role in the health process. Promoting the idea of health economy/healthy community. Linking Research to Policy/US Study Tour Opportunities for networking; practical orientation of the tour. Learning how information systems are used to support research; how research is designed and prioritized based on criteria. Gaining insights on how to fund not-for-profits. The opportunity for Jordanian decision-makers from the academic and government sectors to come together for the first time. Seeing the US manages its health system and uses research to tackle problems and come up with real data/results. Understanding the importance of monitoring and evaluation as a tool for coming up with evidence￾based recommendations. Learning about mechanisms for appropriating money for scientific research. Working with Prince Firas to help influence the creation of a Research/Public Health Unit in Amman. Communication skills; learning to communicate research results; how to perform applied research studies; how to fund research. How to utilize research findings. Procedures and methods for research dissemination. How to determine research priorities. Undertaking research (at the university level) and generating findings. Shared study tour results with high-level decision-makers in the academic and government sectors. Convinced department heads of the importance of evidence-based research, and integrated research-based methodologies into the design of curricula. Focusing on gaining high-level buy-in to make use of the research being undertaken. Sixty percent of the respondents interviewed under the Applied Research component stated that they had encountered constraints in applying what they learned in the workplace. The main constraint cited was the prevalence of limited supervisory and peer support to participate in studies or to implement actual research results. Other constraints included: the lack of effort to integrate those scholars and ministry staff trained in applied research methods into the day-to-day policy formulation and decision-making activities of the MOH; difficulties in obtaining baseline data; challenges with sampling due to the fact that respondents were not familiar with research concepts at the hospital 3. Findings 21 level, and thereby reluctant to participate in research studies; and limited support from high-level decision-makers to act on research findings. Strategies proposed for resolving these constraints included raising the awareness of the new Minister of Health; having PHR work more closely with internal MOH structures to create a base of support; more wide-spread policy communication and advocacy; and orienting the public to the importance of research. The remaining 40 percent of the respondents interviewed did not experience constraints in applying new skills and knowledge in their respective work environments. In these instances, the conditions for successful application were attributed to the strong political will that the Palace, the MOH, and collaborating academic institutions have been able to forge in support of both creating a national research center in Jordan and strengthening ways of utilizing research findings in a policy environment. Respondents involved in this high-level policy dialogue reported stronger short-term results in the application of new skills, given the ability to draw on the base of support that had been created prior to and during the research study tour visit to the US. The opportunity for officials to come together for the first time from the high government, the MOH, RMS, and university sectors was repeatedly acknowledged as a tangible and sustainable benefit from this study tour experience. 3.4.2 Observable Performance Improvements 3.4.2.1 Participant Observations Ninety percent of the respondents in this category felt that their job performance improved as a result of the training received. Improvements included: a stronger comfort level in writing proposals; better skills in decision-making and establishing research priorities; increased productivity; and higher levels of confidence and initiative in undertaking research studies. A success cited at Al Ramtha and Al Bashir Hospitals was the ability to use survey data as an effective tool for problem solving and improving staff motivation. The remaining respondents felt that it was still too early to observe specific improvements in their own individual performance at work. These respondents reported progress towards introduction of specific performance improvements in the workplace as a result of the training received, but changes were not apparent at this stage. 3.4.2.2 Supervisor Observations Supervisors at Al Bashir and Al Ramtha Hospitals observed significant changes in the way that trained staff approached their jobs before and after receiving PHR-funded training. The opportunities to participate in the Patient Flow Analysis, 360 Degree, and Health Worker Motivation studies caused staff to change their attitudes about their job functions. For example, nurses now are completing independent studies on nursing satisfaction, in addition to producing reports without direct supervision. As a result of the Patient Flow Analysis and Health Worker Motivation studies, Al Bashir Hospital identified bottlenecks in the outpatient, emergency, pediatrics, and ophthalmology departments and performed structural changes in how these departments were organized. The reorganization of the emergency department was considered an important success; this unit was divided into three parts as a way of better streamlining the delivery of services to patients. Patients are now examined in a pre-screening area. Depending on the severity of their cases, they are moved to a newly created triage section, sent home, or referred to the outpatient department of the hospital. These changes have brought about significant reductions in patient waiting times and eliminated recurrent problems with overcrowding. As a result of these changes, staff are more confident and “people are thanking them instead of complaining.” At Al Ramtha Hospital, trained staff display 22 3. Findings greater initiative and a willingness to solve problems independently. Staff are also participating in the creation of a new committee for motivation at the hospital. Before the Patient Flow Analysis and Health Worker Motivation studies were carried out at Al Bashir Hospital, staff were operating in the absence of set standards for performance. This environment was characterized by frequent mistakes in diagnosis and filing, and long patient waiting time. Supervisors embraced the attributes of applied research methodologies as a whole, citing the fact that these research studies helped to generate plausible solutions without requiring a lot of money. Within the PHR Scholars Program, mentors observed changes in the way that scholars approached the process of data analysis and interpretation over time. As a result of receiving competency-based training in applied research methodologies, the PHR Scholars have gained concrete practical experience in conducting research and presenting research findings. Valuable inputs to this capacity-building process included the skills training in SPSS to support data analysis. 3.4.2.3 Organizational-Level Improvements Improvements at Al Bashir and Al Ramtha Hospitals were cited most frequently during the participant and supervisor interviews. Apart from aforementioned changes in staff initiative, other changes included: greater respect and cooperation between employees as a whole; improving the paper trail as a way of ensuring that issues are followed up in a timely manner; and encouraging staff to either come early or report to work on time. An extension of this sentiment was that the use of research results also facilitated the decision-making process in each hospital facility. Organizational changes or performance improvements stemming from the research study tour were less apparent at the time of this evaluation. Respondents in this category felt that they were able to make headway in establishing a foundation for continued policy dialogue on the creation of a national agenda for health policy research. At the university level, one respondent emphasized the need to continue to focus their efforts towards advocacy and to gain the buy-in of senior faculty and department heads on the importance of applied research. Another important direction to be pursued was the need to direct greater attention towards the utilization of research data that is currently available in Jordan. 3.4.3 Policy and Systems Changes Policy and systems changes resulting from PHR training in applied research were most apparent at the hospital level, through the creation of a motivation committee to improve staff performance, as well as steps taken to prepare an educational guidebook for patients and health workers on the services of Al Ramtha Hospital. Respondents involved in supporting the development of a national research agenda felt that they were able to impact the course of policy dialogue, either directly or indirectly. Steps included making presentations to the Minister of Health on the importance of research as a tool for evidence-based decision-making, emphasizing the time- and money-saving features of this process. One respondent observed that it could take time to introduce such changes in a traditional culture. An institutional shift in the utilization of research at the MOH has been an expanded role on behalf of the Information Management Directorate to track all research being undertaken. 3. Findings 23 3.4.4 Progress Towards PHR Training Indicators for the Applied Research Component After one full year of training investments in applied research, the respondents interviewed have made progress towards the creation of a national research agenda. PHR fulfilled its target of strengthening leadership capacity for applied research within the MOH, as manifested through the training of key staff in the Planning, Food Safety, and Information Management Directorates. This capacity has also clearly moved into the hospital and university sectors, by way of concentrated skills development in applied research methodologies, as well as laying a foundation for the cross￾fertilization of this expertise within the hospital, university, and higher government sectors. 24 3. Findings 3.5 Foundation Building Training PHR CAP Result 1.1: Capacity of the MOH and other Jordanian institutions to take leadership in policy development and implementation of health sector reforms strengthened PHR Training Indicators: 1. Research agenda formulated; 2. Leadership capacity strengthened in the MOH English language training courses for counterparts and project-specific participants Computer skills training courses (Windows Applications) for counterparts and project￾specific participants Harvard University, School of Public Health Course on Managing Health Care in Developing Countries Boston University, School of Public Health Course on Financing Health Care in Developing Countries University of Connecticut, Institute for Public Service International Course on Training Design and Management World Bank Flagship Course on Health Sector Reform, Washington, DC Courses Provided: World Bank Flagship Course/Economic Development Institute, Lebanon 3.5.1 Benefits from PHR-sponsored Training Course Evaluated Benefits Received from Training New Skills Acquired Skills Application in the Workplace World Bank Flagship Course, Washington, DC Enhanced knowledge of health sector reform; learned about the concept of health sector reform in terms of its pluses and negatives, and possible application in hospital settings. How to improve project management skills; understanding of different country experiences. Applying to current work with PHCI; have better skills in facilitating training courses and designing case studies. Working in the trenches has brought about a change in vision. Foundation building training in health sector reform concepts and applications, computer skills, project management, and English language skills was a lynchpin to PHR’s strategy for strengthening leadership and management skills among senior- and middle-level managers. One respondent was interviewed under this category, for the purpose of determining the extent to which new skills, knowledge, and attitudes have has influenced the process of reform as undertaken by middle managers within PHR’s main partner organizations. The one respondent in this category cited an inability to fully apply new skills in the workplace, primarily due to a change in position shortly after the training occurred. In the respondent’s current capacity on PHCI, it is likely that future opportunities will arise, especially as the MOH and USAID look at ways of harnessing all the expertise and performance improvements that have evolved through MOH-, PHR-, and PHCI￾supported training investments. 3. Findings 25 3.5.2 Performance Improvements 3.5.2.1 Participant-level Improvements The respondent did not observe any immediate changes in performance, since the acquired skills have yet to be fully applied in the workplace. It is expected that opportunities for application will increase as PHR, PHCI, and the MOH Planning and Education and Training Directorates take steps to establish a base for coordination in the planning and monitoring of future training investments. The respondent observed short-term benefits from the training received, as manifested through a new range of responsibilities in her current position. These benefits included: the ability to conduct meetings with the heads of different general directorates and governorates; the ability to look at problems related to resource allocation and budget management at the governorate level; and working with governorates to make decisions and identify options. Better communication skills and increased confidence were noted as changes in the respondent’s performance as a result of the training. 3.5.2.2 Organizational-Level Improvements Observable organizational improvements were noted primarily in the respondent’s work in the MOH Education and Training Directorate. The respondent brought greater accuracy to the process of student selection at the university level, in addition to working with colleges and paramedical programs to develop long-range plans. These changes were approved by the Minister of Health. 3.5.2.3 Policy and Systems Changes Limited decision-making authority at the level of a middle manager was articulated as an obstacle in the respondent’s ability to effect policy or systems changes independently. The continued development and presentation of ideas to supervisors, combined with the ability to change strategies that directly contribute to policy or systems changes were identified as ways to influence the change process. Examples of the respondent’s ability to impact systems and policy reforms as part of her previous position in the MOH Education and Training Directorate included: revising curriculum and degree requirements for nursing and changing the requirements for training paramedicals in Jordan. 4. Conclusions and Recommendations 27 4. Conclusions and Recommendations 4.1 Overall Assessment of PHR Training Component in Jordan The findings from this evaluation indicate that the PHR Project achieved the majority of its objectives under the 1999-2000 PHR Training Plan for Jordan. The majority of the participants interviewed are applying new skills toward the achievement of reforms in the areas of hospital autonomy, applied research, NHA, and training. Nearly 100 percent of the time, any barriers to application were directly correlated to limited management authority among middle-level cadres, or the perception of having an external or indirect relationship to those who were well positioned to affect change at the systems and policy levels. Innovations were observed in cases where respondents worked within their range of authority and available resources to implement reforms, combined with direct support from their supervisors and senior decision-makers within their organizations. 4.2 Recommendations 4.2.1 General Respondent Recommendations There was a 100 percent response rate in the identification of recommendations to PHR and USAID/Amman, pointing to feedback on the quality of the training received and suggestions for improvement in the future. These respondent recommendations are listed as follows: 1. Conduct more training for private sector institutions, not only for the MOH, JUH, or RMS; 2. Expand the lead-time for planning training activities in the future; 3. Place more emphasis on competency-based training tied to enhancing job performance and the ability to undertake specific health system and policy reforms, now that a foundation has been put into place; 4. Ensure that customized courses are structured to include participants with similar backgrounds and disciplines; 5. Take steps to ensure that local training providers develop curricula which directly supports the achievement of training objectives and/or provides a conceptual framework that ties new skills with direct application in the workplace; 6. Continue with the capacity-building process at Princess Raya, El Kerak, Al Ramtha, and Al Bashir Hospitals, as a way of expanding the skill base that now exists in each institution; 7. Provide more training in qualitative data analysis and interpretation; 8. Provide additional support in applying skills in patient flow analysis, 360 degree analysis, and studies of health worker motivation in the surgery department of Al Bashir Hospital; 9. Take further steps to widen the critical mass of employees within the MOH who have skills to support hospital decentralization, especially in the areas of health care financing and management; 28 4. Conclusions and Recommendations 10. Provide more training to support the creation of a harmonized medical records system at El Kerak Hospital; 11. Specific emphasis on budget skills training for staff at El Kerak and Princess Raya Hospitals; 12. Offer more courses in English conversation, as opposed to skills training in grammar; 13. Extend the PHR project for two or three more years, after which time the results from PHR￾sponsored training will be more evident; 14. Require selected trainees to have a briefing with MOH staff before they leave for third country or US training venues; 15. Provide more opportunities to see examples of hospital autonomy in other countries, ideally in the ANE region; 16. Improve the linkage for follow-up and supervision between the MOH counterparts for hospital counterparts and participants who received PHR-sponsored training either locally or in third countries; and 17. Provide more skills training for counterparts, through courses that are specific to the topics of decentralization and project management. 4.2.2 Other Recommendations 1. Develop a “Medline” at Jordan University, containing abstracts on the body of research that has been developed. This will provide access to a greater number of resources in addition to formalizing the way research products are catalogued and accessed; 2. Continue the path of encouraging scientific research in the public and private sectors; 3. Assist with the completion of a study on the effectiveness of recruiting temporary staff in the event of patient flow increases occurring in different departments at Al Bashir Hospital; 4. Encourage donors, including USAID, to help in providing seed money for the creation of the national research unit in Jordan; and 5. Focus on institutionalizing NHA through the creation of an independent institution with clear rules and regulations, and systems for financial management and administration. 4.2.3 PHR Recommendations In line with the above feedback, PHR recommends the provision of ongoing training support, for the purpose of strengthening the creation of a critical mass of individuals who can lead the process of health sector reform in Jordan. Ideally, the recipients of training under the life of the current PHR project will be tapped more aggressively to provide direction and technical input into the design of future competency-based training. Now that a broad-based capacity has been established, it will be critical to continue to evaluate the performance gaps that exist within each partner organization and to ensure that training will be structured to meet those gaps. PHR also recommends that more emphasis and importance be placed on pre-defined conditions of training, so that selected trainees are better positioned to make tangible and meaningful contributions to the process of health sector reform in Jordan. Lastly, as PHR draws to a close, it would also be timely and appropriate to have the MOH Education and Training and Planning Directorates assume direct responsibility for working with the donor community to develop a multi-sectoral training plan to support continued skills development at the central, hospital, and primary health care center levels. Such a partnership will encourage a more 4. Conclusions and Recommendations 29 optimal utilization of resources, in addition to fostering clearly defined roles and systems for tracking trainee progress and gauging performance as an outcome of the training received. Annex A: Number of Participants Trained by Organization 31 Annex A: Number of Participants Trained by Organization 32 Annex A: Number of Participants Trained by Organization Number of Participants Trained from the Ministry of Health (Non-Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In￾country Out-country In￾country Out-country Type of Activity Directorates M F M F M F M F M F M F M F M F Directorate of Planning 1 1 5 1 Directorate of Education and Training 1 1 1 1 Directorate of Finance 1 1 2 1 1 1 Central Directorate 11 2 1 1 Directorate of Food Hygiene 1 1 Primary Health Care General Directorate 1 1 Internal Monitoring Directorate 1 1 2 Information Center 1 1 Research and Studies Directorate 2 3 1 General Directorate for Health Insurance 2 1 2 1 2 2 Directorate for Curative Services 1 1 1 General Directorate for Administrative Affairs 1 2 2 Directorate of Purchasing 1 1 2 Directorate of Supplies 1 1 Directorate of Services and Transportation 1 2 Directorate of Development 1 Legal Affairs Directorate 1 Directorate of Patient Affairs 1 Ma’an Health Directorate 2 Balqa Health Directorate 3 Tafieleh Health Directorate 1 Madba Health Directorate 2 Ajlon Health Directorate 2 Mafraq Health Directorate 2 Annex A: Number of Participants Trained by Organization 33 Number of Participants Trained from the Ministry of Health, Cont’d. (Non-Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In￾country Out-country In￾country Out-country Type of Activity Directorates M F M F M F M F M F M F M F M F Jerash Health Directorate 1 Zarqa health Directorate 2 Irbid Health Directorate 31 10 2 5 1 Al Karak Health Directorate 17 17 1 2 2 1 1 North Jordan Valley Health Directorate 1 Al-Rawaished Health Directorate 1 Al Bashir Hospital 15 10 1 2 3 1 2 1 Al Ramtha Hospital 18 2 2 1 1 Totals 89 45 7 0 0 0 19 1 59 11 4 0 6 1 4 1 Totals 134 7 0 20 70 4 7 5 Totals 141 20 74 12 Ministry of Health Trainees Total of the non-duplicated numbers of female trainees = 60 Total of the non-duplicated numbers of male trainees = 187 Total of the non-duplicated numbers of trainees = 247 34 Annex A: Number of Participants Trained by Organization Number of Participants Trained from Other Organizations (Non-Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In-country Out-country Type of Activity Organizations M F M F M F M F M F M F M F M F Royal Palace 1 Academics 3 2 2 4 2 8 8 7 Private Sector 2 1 4 Royal Medical Services 3 2 2 2 1 Jordan University Hospital 1 1 1 1 Audit Bureau 1 1 World Health Organization 2 2 PHCI 2 1 Jordan Hashemiate Fund 1 USAID 1 1 1 1 Diabetes Center 1 World Bank 1 2 3 Civil Service 1 1 Arab Bank 3 Social Security Corporation 2 Ministry Social Development 1 General Budgeting Department 1 Ministry of Panning 1 Ministry of Health, Morocco 1 1 Jordan Petroleum Refinery 2 National Health Insurance Company 2 The Arab Niser Insurance Company 1 Annex A: Number of Participants Trained by Organization 35 Number of Participants Trained from Other Organizations, Cont’d. (Non-Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In-country Out-country Type of Activity Organizations M F M F M F M F M F M F M F M F Ministry of Finance 1 Totals 9 2 0 0 0 0 5 4 28 13 3 0 33 12 0 0 Totals 11 0 0 9 41 3 45 0 Totals 11 9 44 44 Other Organization Trainees Total of the non-duplicated numbers of female trainees = 31 Total of the non-duplicated numbers of male trainees = 78 Total of the non-duplicated numbers of trainees = 108 Summary Total of the non-duplicated numbers of trainees from the Ministry of Health and the other organizations Total number of female trainees from Ministry of Health and the other organizations = 91 Total number of male trainees from the Ministry of Health and the other organizations = 265 Total number of trainees from the Ministry of Health and the other organizations = 355 36 Annex A: Number of Participants Trained by Organization Number of Participants Trained from the Ministry of Health (Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In￾country Out-country Type of Activity Directorates M F M F M F M F M F M F M F M F Directorate of Planning 46 5 8 55 4 4 18 Directorate for Education and Training 2 1 2 1 Directorate of Finance 4 3 1 2 6 1 3 Central Directorate 10 1 4 1 Directorate of Food Hygiene 2 1 1 1 Primary Health Care General Directorate 1 1 Internal Monitoring Directorate 2 1 2 1 Information Center 2 1 3 3 Research and Studies Directorate 2 1 3 2 General Directorate for Health Insurance 2 2 2 1 1 3 1 Directorate for Curative Services 2 1 4 General Directorate for Administrative Affairs 2 1 1 3 1 Directorate of Purchasing 1 1 4 2 Directorate of Supplies 2 1 1 1 2 Directorate of Services and Transportation 1 4 2 Legal Affairs Directorate Directorate of Patient Affairs 1 Directorate of Development 1 Ma’an Health Directorate 2 Balqa Health Directorate 4 Tafieleh Health Directorate 1 Madba Health Directorate 2 Ajlon Health Directorate 2 Annex A: Number of Participants Trained by Organization 37 Number of Participants Trained from the Ministry of Health, Cont’d. (Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In￾country Out-country Type of Activity Directorates M F M F M F M F M F M F M F M F Mafraq Health Directorate 1 Jerash Health Directorate 1 Irbid Health Directorate 74 17 2 2 49 11 1 1 1 Al Karak Health Directorate 57 36 2 1 57 18 1 2 1 Zarqa Health Directorate 3 North Jordan Valley Health Directorate 1 Al- Rawaished 1 Al Bashir Hospital 9 5 1 18 6 1 4 4 Al Ramtha Hospital 22 2 6 6 1 Totals 222 72 14 2 0 0 20 1 241 46 7 0 59 10 4 1 Totals 294 16 0 21 287 7 69 5 Totals 310 21 294 74 Totals of the duplicated numbers of trainees from the Ministry of Health Total number of female trainees = 132 Total number of male trainees = 567 Total number of female and male trainees from the Ministry of Health = 699 38 Annex A: Number of Participants Trained by Organization Number of Participants Trained from Other Organizations (Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In-country Out-country Type of Activity Organizations M F M F M F M F M F M F M F M F Royal Palace 1 1 Academics 11 6 3 4 5 5 22 12 Private Sector 2 4 1 Royal Medical Services 3 2 6 2 9 1 Jordan University Hospital 3 1 4 1 3 1 Audit Bureau 3 1 1 World Health Organization 1 3 PHCI 1 1 3 Jordan Hashemite Fund 1 USAID 3 7 2 1 Diabetes Center 1 1 World Bank 1 1 1 Civil Service 1 1 Arab Bank 1 Social Security Corporation 2 1 Ministry of Social Development 1 General Budgeting Department 1 1 Ministry of Planning 1 1 Annex A: Number of Participants Trained by Organization 39 Number of Participants Trained from Other Organizations, Cont’d. (Duplicated Numbers) Customized Courses Study Tours Workshops Conferences In-country Out-country In-country Out-country In-country Out-country In-country Out-country Type of Activity Organizations M F M F M F M F M F M F M F M F Ministry of Health, Morocco 1 1 Jordan Petroleum Refinery 2 National Health Insurance Company 2 The Arab Niser Insurance Company 1 Ministry of Finance 2 Totals 22 6 0 0 0 0 7 4 30 17 3 0 57 21 0 0 Totals 28 0 0 11 47 3 78 0 Totals 28 11 50 78 Totals of the duplicated numbers of trainees from the other organizations Total number of female trainees = 48 Total number of male trainees = 119 Total number of female and male trainees from the Ministry of Health = 167 Summary Totals of the duplicated numbers of trainees from the MOH and the other organizations Total number of female trainees from the Ministry of Health and the other organizations = 180 Total number of male trainees from the Ministry of Health and the other organizations = 686 Total number of trainees from the Ministry of Health and the other organizations = 866 Annex B: Chronological History of PHR Training Activities, 1998-2000 41 Annex B: Chronological History of PHR Training Activities, 1998–2000 Study Tours Tunisia Study Tour 16-22 January 1999 (NHA) Tunisia Study Tour 21-29 January 2000 (Hospital Autonomy) Health Policy Study Tour–Washington DC 20-29 February 2000 (Applied Research) Conferences World Bank Conference–Health Sector Reform–Washington DC 20 October-17 November 1999 (Applied Research) American Public Association Research Conference–Chicago 7-11 November 1999 (Applied Research) First Arab Conference on Health Care Management–Lebanon 3-13 April 2000 (Applied Research) National Health Accounts Dissemination Conference–Amman April 2000 (NHA) Applied Research Dissemination Conference–Amman 19-21 June 2000 (Applied Research ) National Policy Research Form Conference–Amman 31 July 2000 (Applied Research ) Workshops National Health Accounts Workshop–Amman 25-28 May 1998 (NHA) Hospital Autonomy Workshop–Amman 4 October 1998 (Hospital Autonomy) Insuring the Uninsured on Jordan -Amman 23-24 November 1998 (Universal Coverage) Hospital Costing Workshop-Cairo 28 January-5 February 1999 (Hospital Autonomy) NHA Regional Workshop–Lebanon June 1999 (NHA) World Health Organization Workshop–Syria October 1999 (Applied Research) 42 Annex B: Chronological History of PHR Training Activities, 1998-2000 360 Degree Assessment Workshop–Amman 24 October 1999 (Applied Research) Patient Flow Analysis Workshop–Amman; 22 November 1999 debriefing of APHA and the World Bank conferences (Applied Research) Hospital Autonomy Workshop–Amman April 1999 (Hospital Autonomy) Hospital Autonomy Workshop–Amman 27 July 1999 (Hospital Autonomy) Debriefing of South Africa Course (MPPMS)–Amman 16 September 1999 (Hospital Autonomy) Hospital Autonomy Workshop–Amman 12 October 1999 (Hospital Autonomy) Hospital Autonomy Workshop–Amman, and 17 February 2000 Tunisia Study Tour Debriefing (Hospital Autonomy) Research Study Tour Debriefing 11 April 2000 (Applied Research) National Health Accounts Regional–Amman May 2000 (NHA) Hospital Autonomy Workshop–Amman 8 May 2000 (Hospital Autonomy) Hospital Autonomy Workshop–Amman 18 July 2000 (Hospital Autonomy) Courses Management and Financing of Health Programs July- September,1998 in Developing Countries–Harvard (Capacity [Foundation] Building) Health Care and Policy–Boston September- December 1998 (Capacity [Foundation] Building) Hospital Managerial and Financial Management–Amman September 1998 (Capacity [Foundation] Building) English Language Skills–Amman July 98-April 2000 (Capacity [Foundation] Building) General Computer Skills–Amman June 98-May 2000 (Capacity [Foundation] Building) Procurement Management–Beirut June 1999 (Hospital Autonomy) Managing the Procurement of Pharmaceuticals–South Africa August-September 1999 (Hospital Autonomy) Computer Skills in SPSS January-February 2000 (NHA) Computer Skills in Epi-info February-April 2000 (NHA) Medical Records April 2000 (Hospital Autonomy) Applied Research Course December 1999-May 2000 Annex B: Chronological History of PHR Training Activities, 1998-2000 43 (Applied Research) General Management February-April 2000 (Hospital Autonomy) Financial Management February-June 2000 (Hospital Autonomy) Accounting and Interpretation of Data March-April 2000 (NHA) Training Design and Management–US July 2000 (Training Capacity Building)