Leadership Initiative for Public Health in East Africa Outcomes Assessment April 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by the assessment team contracted by JBS International (Mildred P. Howard, Peter L. French, Stephen E.D. Nsimba, Dawood H. Sultan) with the support of JBS International staff Oill Rhodes, Christine Allison). USAID FROM THE AMERICAN PEOPLE H Higher Ed.ue;ation for Development. LEADERSHIP INITIATIVE FOR PUBLIC HEALTH IN EAST AFRICA (LiPHEA): OUTCOMES ASSESSMENT Final Report Authors: Mildred P. Howard, Peter L. French, Stephen E.D. Nsimba, Dawood H. Sultan, Jill Rhodes Program. activity. or project number: USAID Development Partnership for University Cooperation and Development" - CFDA number 98.012 Sponsoring USAID office: EGAT/ED and Global Health Cooperative Agreement number: AEG-A-00-OS-00007-00 Contractor or grantee name: American Council on Education Date of publication: April 2012 Language of document: English The authors' view expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government Leadership Initiative for Public Health in East Africa Outcomes Assessment Acknowledgments This assessment exercise would not have been possible without the generous time and attention given to the team by the LlPHEA partners. We were able to arrange more than one teleconference in the United States with LlPHEA's U.S. Partner Institutions prior to departure to the field. Professor Gilbert Burnham and Research Associate Daniela Lewy at johns Hopkins Bloomberg School of Public Health, and Associate Professor Nancy Mock and Senior Program Manager Adam Papendieck at Tulane University, were welcoming and extremely helpful. They all provided invaluable insights and information in preparation for the assessment team's visits to LlPHEA partner countries. Once on the ground, Associate Professor William Bazeyo and LlPHEA Administrative Secretary Angela Nanyanzi at Makerere University School of Public Health in Uganda, and Professor japhet Killewo and LlPHEA Coordinator Dr. Simon Manuya at Muhimbili University of Health Sciences, School of Public Health and Social Sciences in Tanzania, helped to identify and introduce our team via e-mail to key informants in the three countries visited. Dr. Bazeyo graciously returned from leave and spent the better part of a morning giving our team an in depth briefing on LlPHEA, past, present and future. Staffer Angela Nanyanzi did an excellent job of facilitating our schedule in Uganda, and provided us with contact information for Kenya and Tanzania. We received the same warm welcome from Dr. japhet Killewo and his staff upon arrival in Tanzania. Dr. Manuya made sure that all key informants were contacted beforehand, and generally made our Tanzania itinerary run smoothly. Thanks go to Moi University for the outstanding reception and support provided to Dr. Peter French, one of our LlPHEA assessment team members, during his four days spent at the Moi University Campus in Eldoret. Professor Richard Mibey, Vice-Chancellor, Professor Samuel Gudu, Deputy Vice-Chancellor for Planning & Development, Dr. Paul Ayuo, Dean of the Medical School, Dr. Peter Gatongi, Dean of the School of Public Health, Dr. john Tabu Simiyu, Senior Lecturer at Moi University, and a host of Moi University faculty and staff ensured that the assessor received a thorough briefing of LlPHEA-related activities at MoL Special thanks to Dr. Tabu who handled all of the logistics for our team member's visit to Eldoret, and also linked us in Nairobi with the National Disaster Operations Center and members of the Center's multi-sectoral forum. Dr. Tabu ensured that the assessment team reached all of the key informants we needed to contact in Kenya. In wrapping up our field work and drafting of this assessment report, the assessment team leader also followed up with key USAID officers, Drs. Dale Gibb and Dennis Carroll. With them we were able to clarify certain issues relevant to the team's impressions from the field compared to USAID/HED's original expectations for LlPHEA. In sum, the assessment team wishes to extend sincere thanks to all who facilitated and provided thoughtful contributions to this assessment effort. JBS International Sincerely, The LlPHEA Assessment Team April 2012 p. i Leadership Initiative for Public Health in East Africa Outcomes Assessment Table of Contents Acronyms ...................................................................................................................................................................... iv Executive Summary ..................................................................................................................................................... v Introduction ................................................................................................................................................................. I Program Context ................................................................................................................................................... I Inception and Development of LlPHEA. ........................................................................................................... 3 Program Overview ................................................................................................................................................ 4 Call for a Program Assessment .......................................................................................................................... 4 Assessment Purpose & Methodology .................................................................................................................... 5 Scope and Design Considerations ..................................................................................................................... 5 Logical Framework for LlPHEA Assessment ................................................................................................... 6 Analyzing the Performance of LlPHEA Partners within a Development Context ................................. 7 Key Assessment Questions ............................................................................................................................. 9 A Systems Approach to Understanding the Institutional Capacities of LlPHEA Partner Institutions ........................................................................................................................................................... 9 Data Collection .................................................................................................................................................... I I Data Analysis & Limitations ............................................................................................................................... I I Report Organization ............................................................................................................................................ 12 Objective One. Create a Network that Links Public Health Schools, Ministries of Health, Public Health Practitioners, Regional Organizations, and Other Critical Stakeholders, to Facilitate Information and Resource Sharing ....................................................................................................................... 13 Networking and Formation of the HEALTH Alliance ................................................................................. 13 Information and Communications Technology ............................................................................................. 14 Objective Two. Create an Enabling Environment in Tanzania and Uganda for Public Health Activities by Building Leadership and Advocacy Skills among Public Health Faculty and Key Decision-makers. 19 LlPHEA Influence .................................................................................................................................................. 19 LlPHEAlHEALTH Alliance ................................................................................. , ........................................... 20 Findings ............................................................................................................................................................... 21 Lessons Learned & Recommendations ....................................................................................................... 23 Objective Three. Strengthen Teaching and Educational Programs that Integrate Leadership Training throughout the Curricula ....................................................................................................................................... 25 Relevance of T raining .......................................................................................................................................... 25 Undergraduate, Post-basic, and In-service Training ..................................................................................... 25 Development of Curriculum ......................................................................................................................... 26 HEALTH Alliance ................................................................................................................................................. 26 Public Sector Health Training ....................................................................................................................... 26 Findings ............................................................................................................................................................... 28 Lessons Learned & Recommendations ....................................................................................................... 29 JBS International p. ii Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective Four. Establish a Faculty Development Program that will Sponsor Promising Young Public Health Faculty from MUSPH and MUHAS through a Combination of Degree Programs and In-service Programs in Public Health Leadership ................................................................................................................. 30 Program for Faculty Development ................................................................................................................... 30 Faculty Training ..................................................................................................................................................... 30 Findings ............................................................................................................................................................... 30 Lessons Learned & Recommendations ....................................................................................................... 30 Objective Five. Improve the Teaching Infrastructure at MUSPH and MUHAS ........................................ 32 Teaching and Teaching Infrastructure ............................................................................................................. 32 Findings ............................................................................................................................................................... 32 Lessons Learned & Recommendations ....................................................................................................... 32 Additional Program Considerations ..................................................................................................................... 34 Partnership Management Structure and Roles .............................................................................................. 34 Program Documentation and Monitoring ...................................................................................................... 36 Material Support for LlPHEA Activities .......................................................................................................... 37 Challenges with Resources and Relationship Management ................................................................... 40 Findings ............................................................................................................................................................... 40 Lessons Learned & Recommendations ....................................................................................................... 41 Impact on U.S. Partner Institutions .................................................................................................................. 41 Assessing Operations in Four LlPHEAlHEALTH Alliance Partner Institutions ........................................ .42 Measures for Assessing Operational Practices and Strategic Purposes .................................................. 42 Uganda .................................................................................................................................................................... 42 The Context of LlPHEAlHealth Alliance Operations in Uganda ......................................................... 43 Makerere University ....................................................................................................................................... 43 Tanzania .................................................................................................................................................................. 49 The Context of LlPHEAlHealth Alliance Operations in Tanzania ....................................................... 50 MUHAS .............................................................................................................................................................. 50 Kenya ...................................................................................................................................................................... 55 The Context of LlPHEAlHEAL TH Alliance Operations in Kenya ....................................................... 55 Moi University .................................................................................................................................................. 55 University Of Nairobi ..................................................................................................................................... 59 Conclusion .................................................................................................................................................................. 63 Appendix A: List of LlPHEA Partners and Collaborators .......................................................................... 69 Appendix B: List of Individuals and Organizations Contacted .................................................................. 71 Appendix C: Assessment Interview Worksheet .......................................................................................... 74 JBS International p. iii Leadership Initiative for Public Health in East Africa Outcomes Assessment Acronyms AIDS AM PATH AMREF CABI DflD DRC EAJPH FHS GTZ (GIZ) HEALTH HED HEMP HIV ICT IT JBS JHU LlPHEA MOPHS MOSSP MPH MUCHS MUHAS MUSPH MUSPH/Kenya NDOC NGO NSF OCHA OHCEA OPM PMO SPH SPHSS UNDP UoN U.S. USAID WHO JBS International Acquired immune defiCiency syndrome Academic Model Providing Access to Healthcare African Medical Research Foundation CAB abstract database Department for International Development Democratic Republic of Congo East African Journal of Public Health Future Health Systems Deutsche Gesellschaft fur Internationale Zummenarbeit Higher Education Alliance for Leadership Through Health Higher Education for Development Health Emergency Management Program Human immunodeficiency virus Information and Communications Technology Information Technology JBS International, Inc. Johns Hopkins University Leadership Initiative for Public Health in East Africa Ministry of Public Health and Sanitation (Kenya) Ministry of State for Special Programs Master of Public Health Muhimbili University College of Health Sciences (now MUHAS) Muhimbili University of Health and Allied Sciences (formerly MUCHS) Makerere University School of Public Health Moi University School of Public Health National Disaster Operations Center (Kenya) Nongovernmental Organization National Science Foundation Office for the Coordination of Humanitarian Affairs (United Nations) One Health for Central and Eastern Africa Office of the Prime Minister Prime Minister's Office Schools of public health School of Public Health and Social Sciences (at MUHAS) United Nations Development Programme University of Nairobi United States United States Agency for International Development World Health Organization p. iv Leadership Initiative for Public Health in East Africa Outcomes Assessment Executive Summary In 20 I I, Higher Education for Development (HED) requested the Aguirre Division of JBS International, Inc. UBS/A) to assess the outcomes of the U.S. Agency for International Development (USAID/HED) Leadership Initiative for Public Health in East Africa (LiPHEA). Initiated in October 2005, LlPHEA was a five-year partnership among Muhimbili University of Health and Allied Sciences, School of Public Health and Social Sciences (MUHAS/SPHSS) in Tanzania, Makerere University School of Public Health (MUSPH) in Uganda, and Johns Hopkins University Bloomberg School of Public Health UHU), with additional support from Tulane University and George Washington University. The partnership arrangement ended in March 20 I I. The goal of the LlPHEA partnership was to strengthen public health leadership in East Africa by improving the capacity of local institutions to train mid- and senior-level health professionals in leadership skills. LlPHEA objectives, or expected outcomes, were to: • Create a network that would link public health schools, ministries of health, public health practitioners, regional organizations, and other critical stakeholders, to (at a minimum) facilitate information and resource sharing. • Create an enabling environment in Uganda and Tanzania for public health activities by building leadership and advocacy skills among public health faculty and key decision-makers. • Strengthen teaching and educational programs that integrate leadership skills development into the curricula of the Schools of Public Health and other university departments in Uganda and Tanzania. • Establish a faculty development program that would sponsor promising young publiC health faculty from the Schools of Public Health through a combination of degree programs and in￾service programs in public health leadership. • Improve teaching infrastructure in the Schools of Public Health in Uganda and Tanzania. Assessment Purpose and Methodology The purpose of this outcomes assessment was to determine the extent to which LlPHEA met its goals and targets: the human and institutional capacities that have been strengthened; the perceived impact of the program on its multiple stakeholders; and, the observable impact the program has had on national and regional public health developments. I This was done by examining the LlPHEA program's effectiveness and results measured against its reported outputs and its expected outcomes. The assessment team also proposed to conduct institutional assessments to gauge each institution's internal strengths and, to some extent, determine whether key stakeholders at each institution viewed LlPHEA as a critical factor contributing to the strength of the institution. Although this Assessment addresses impact-like issues, this study was not meant to be an impact evaluation as defined by USAID guidelines; rather, this Assessment meets the parameters of an outputs/outcomes-based performance assessment. As such, this Assessment relied primarily on qualitative data and provides insights into what happened during the life of the program, but is limited in being able to definitively address evaluation questions. The assessment team used a mixed-method assessment design to support formulation of findings and conclusions. Information on program inputs and outputs was obtained from written reports and data summaries. A Key Informants Opinion design was used to elicit the perceptions and perspectives of partners and stakeholders regarding their experiences with the program, and their opinions about program performance, benefits and impact. Key informant information was primarily obtained through structured interviews. An Expert Opinion design was used to triangulate the impressions and observations of the assessment team. Inquiry and analysis was facilitated by use of a conceptual framework and the cumulative expertise of the team. This framework is a construct developed by the assessment team to depict, based on its best judgment, a selected number of critical factors or elements within the internal (institutional) environment and external (public health development) I USAID/HED Request for Proposal (RFP) for the LlPHEA Higher Education Partnership. dated February 7. 20 II. JBS International p.v Leadership Initiative for Public Health in East Africa Outcomes Assessment environment that it believes are important to review within the context of an end-of-project, outcomes assessment of the LlPHEA program. Data Collection During a five-week period, the assessment team conducted research, developed instruments and the assessment framework, held teleconferences with U.S. Partner Institutions, visited both African Partner Institution countries (Uganda and Tanzania), one HEALTH Alliance member country (Kenya) with visits to two of Kenya's participating Schools of Public Health, and held a formal conference call interview with the representative of another HEALTH Alliance member country (Rwanda) based at the National Rwanda School of Public Health. Organizers of the Assessment made a decision during planning for the Assessment that the team would not travel to Rwanda, Democratic Republic of Congo, and Ethiopia due to time and logistical constraints. The assessment team conducted key informant interviews (n=82) and focus group discussions (n=5) with individuals and organizational representatives drawn from a range of LlPHEA stakeholders, including its funders. Discussions with the program administrator, who also funded this assessment, were limited to requests for clarification and additional information on the programmatic changes and program description of Years 3-5 of LlPHEA and total funding for the activity and its distribution among the partners. Contacts were made with four of the six LlPHEAlHEALTH Alliance countries, representing approximately 80 percent of the program.2 However, it should be noted that compared to the thousands of individuals who have participated and benefitted from LlPHEAlHEAL TH Alliance during its five-year period, the Assessment sample size is not large enough to generalize the findings to the whole of the benefiCiary popUlation. Data Analysis & Limitations The team collected qualitative data in their interviews and focus groups, and notes from each session were shared with all team members to ensure that they incorporated all of the points each team member had in his or her notes. The data analysis was then based on their field notes, with different team members being responsible for writing different sections of the report. Each team member reviewed all the relevant interview and focus group notes multiple times to ensure thorough analysis of each theme within the report. There were several challenges the team faced in conducting this assessment. First and foremost, few resources were available and the number of days for each team member was defined by the contract. Therefore, only the team leader was allocated time to interview the U.S. Partner Institutions, and the rest of the team never had the opportunity to ask questions of their key stakeholders. In addition, only three days were permitted for each of the team members (with two more days for the team leader) to complete data analysis and report writing after four weeks of field work. The lack of resources also precluded involving all seven African Partner Institutions in the assessment. In discussing how limited resources could yield the most information, the assessment team and HED concluded that the team should visit the two primary partners, MUSPH and MUHAS/SPHSS, and the institutions in Kenya, due to logistical simplicity and the possibility of capturing two additional institutions rather than one (as would have been the case in the other countries). The team also spoke with the LlPHEA Coordinator in Rwanda, who was able to contribute his experiences to the overall analysis. Further, the sample of beneficiaries interviewed was small, and the results of those interviews cannot be generalized to the rest of the beneficiary population. 2 This estimate was provided by a JHU spokesperson. JBS International p. vi Leadership Initiative for Public Health in East Africa Outcomes Assessment Findings, Conclusions, and Recommendations Objective One. Create a Network that Links Public Health Schools, Ministries of Health, Public Health Practitioners, Regional Organizations, and Other Critical Stakeholders, to Facilitate Information and Resource Sharing LlPHEA was highly successful in establishing a functional legacy organization to continue the LlPHEA vision. A major achievement of LlPHEA was strengthening the LlPHEAlHEAL TH Alliance capacity in management of information, communications, and instructional technology systems. There have been major inputs of equipment and systems development with impressive results. Resident ICT technical resource people (and champions) remain limited, but an information culture has been "seeded," and demand for all types of information and instructional technology is high. Assistance in building information systems and instructional technology was a good investment under LlPHEA, and would be a worthwhile investment for future institution strengthening programs for aSSistance-ready schools of public health in East Africa. The LlPHEAlHEAL TH website at MUSPH is a useful archive on historical LlPHEAlHEAL TH Alliance reports and program documentation, but it serves very little national or regional public health development purpose at this time. The website represents a first generation development, and is in need of substantial, on-going technical assistance and development to reach its potential as originally envisioned. Furthermore, LlPHEA leaves a legacy institution in place for the future. In keeping with its stated objective to "create a network," LlPHEA successfully created and institutionalized a regional academic network to address leadership gaps and other challenges in public health program management. Shortly after inception of LlPHEA in January 2006, the LlPHEA Partner Institutions had convened a first ever meeting of deans and directors of selected Schools of Public Health in East Africa by October 2006 to discuss possible long-term collaboration on activities and opportunities that might be of mutual interest and benefit This initiative eventually led to the formal establishment of a regional alliance of seven public health institutions in six East African countries (Kenya, Rwanda, the Democratic Republic of the Congo, and Ethiopia in addition to Uganda and Tanzania) known as the "Health Education Alliance for Leadership through Health (HEALTH)." The HEALTH Alliance, as it has generally come to be known, is a chartered organization that was registered in Uganda in 2008. The Alliance can receive and disburse funds, effectively transforming the LlPHEA grant-based partnership involving only two African institutions into a permanent membership organization providing a regional public health platform from which to operate. Objectives of the HEALTH Alliance, similar to those of its parent LlPHEA program, are to generally promote leadership competencies in public health; to strengthen the public health educational capacities of partner institutions through academic collaboration; to improve institutional infrastructure for teaching, particularly in improved use of information and communications technology; and to serve as a technical/teaching resource in addressing public health development problems in the region. During its past three years of operation, the HEALTH Alliance has demonstrated potential for long-term sustainability, albeit somewhat fragile. In the near-term, the HEALTH Alliance has increasingly gained local, regional, and international recognition as a committed and coordinated resource for research, leadership development, information technology and training capacity in a range of public health areas, including emerging and pandemic diseases, emergency and disaster management, and preventive health care. Although the HEALTH Alliance is correctly described as an outgrowth of early deliberations of the Deans and Directors Meetings and is generally thought of as a strictly regional phenomenon engendered by the future vision of the Deans and Directors, the U.S. partners also played a role in these early developments by encouraging and fostering LlPHEA entrepreneurship. In the best spirit of regional cooperation, the periodic meetings of Deans and Directors, in effect, functioned as the JBS International p. vii Leadership Initiative for Public Health in East Africa Outcomes Assessment HEALTH Alliance's management committee. As such, this may have substantially substituted for and reduced the level of strategic and technical guidance that may have been otherwise forthcoming from the U.S. partners. Objective Two. Create an Enabling Environment in Tanzania and Uganda for Public Health Activities by Building Leadership and Advocacy Skills among Public Health Faculty and Key Decision-makers LlPHEA is generally recognized as having contributed to increased awareness and demand for effective leadership in Schools of Public Health and among public health stakeholders in the region. LlPHEA has enabled thousands of students, faculty and professionals to be exposed to various aspects of leadership development within the context of public health program management. The regional reach of LlPHEAlHEAL TH Alliance activities and the program's ability to raise awareness and increase knowledge and skills of public health students and professionals in six countries, the region, and globally, are notable achievements. The outputs and numbers of people whose personal and professional lives have been touched by LlPHEAlHEAL TH Alliance are well￾documented and borne out by numerous anecdotal accounts of people that were interviewed during this Assessment. There are indications that LlPHEAlHEAL TH Alliance Health Emergency Management Program (HEMP) had a verifiable programmatic spread effect in Uganda and Kenya. In Uganda, training of senior government personnel was instrumental in formation of the government's disaster management office. At Moi University in Kenya, HEMP training had migrated to several other programs within the university (e.g., certificate, diploma, and e-Iearning courses), and the program was in high demand among public health professionals, government, donor partners, and nongovernment public health stakeholders. Lessons Learned In structuring its leadership development training program, LlPHEA did not appear to have adequately consulted with government and nongovernment implementers (as potential users of personnel trained in the program). As a result, LlPHEA's generic post-basic in-service leadership development training program did not create effective and sustainable demand, and needed to be revamped to address needs/interests that were more relevant to government and development partners. In development and delivery of HEMP training (during LlPHEA, Years 3-5), there was considerable variation in the nature and extent of consultations initiated by participating LlPHEAlHEAL TH Alliance schools with government and development partners, resulting in varying degrees of success (quality, use and sustainability) of the country HEMPs. As relates to both leadership and HEMP training, there did not appear to be effective mechanisms in place to identify needs and provide on-going assistance to participating schools, as necessary, in forging sound working relationships with government and development partners. Although the U.S. Partners OHU and Tulane) had opportunities during LlPHEA instructional technology training to make some inroads in terms of strategidtechnical approaches to design/development of public health development training programs, there may have been some reticence to do so. The regional nature of the partnership as well as USAID grant-funding and grant management guidelines that caution against substantial involvement of grantors may have been presented constraints. Recommendations In partnership programs that embrace a regional approach, USAID and HED may wish to consider ways in which host country partner institutions can more readily benefit from strategic and technical gUidance that might be needed and forthcoming from U.S. Partner Institutions. Partnering roles and responsibilities related to provision of such guidance may need to be made more explicit in grant JBS International p. viii Leadership Initiative for Public Health in East Africa Outcomes Assessment agreements, if possible. Alternatively, such understandings may need to be established in the working relationships of the partners. Objective Three. Strengthen Teaching and Educational Programs that Integrate Leadership Training throughout the Curricula Government partners in two of the three LlPHEAlHEAL TH Alliance countries visited felt that all training had been useful, and had at least served the general purposes of increasing public health leadership and professionalism in their countries and the region. In Uganda, MUSPH had not only been responsive to the government's needs/interest in increasing the numbers of trained health emergency and disaster management personnel, but had been instrumental in the development of the emergency and disaster response program. Government implementers such as the NDOC in Kenya appeared to be very satisfied with the relevance of the Moi University disaster management training. In contrast, the Office of the Prime Minister in Tanzania felt that MUHAS/SPHSS was not attuned to government operations or governance issues, and thus did not have the technical competencies to cover government policies, structures and administrative guidelines in its disaster management training curriculum. Aside from the advantages of user-informed competency-based training design, two other principles emerged from the LlPHEA experience. First, competency-based regional training programs need to consider regional disparities and have the capacity to customize country programs to the needs/interests and constraints of the local setting. Informants observed that disparities in administrative structures in LlPHEAlHEAL TH Alliance countries made it difficult to replicate training. In addition and to some extent, language differences impeded the free exchange of trained individuals and HEALTH Alliance faculty with the Rwanda and DRC because of English/French language differences. Secondly, regardless of how well in-service training programs have been constructed to meet the practical, competency-based needs of trainees, training results/outcomes will be diminished if the most appropriate trainees are not selected. "Appropriate" refers to those with the most capacity and motivation to learn, and most likely to be re-deployed to work at sites where training can be applied. For example, in one district in Uganda it was observed that selection of district representatives for LlPHEAlHEAL TH trainings was left totally to the discretion of a local District Chief Administrative Officer without the benefit of any standardized criteria or guidelines for selection. This might have increased the possibility of biased selection, or selection of trainees who do not have genuine interest or positive motivation to participate in the training program. Lessons Learned & Recommendations When identifying training priorities, LlPHEA tended to confer with public health professionals and students and less specifically with governmental and nongovernmental implementers. This may have resulted, at least in part, in LlPHEA making a decision not to offer health care financing courses, thereby missing what may have been an opportunity to advocate for and assume leadership in development of Africa-specific health care financing expertise. Post-basic and in-service training provided by LlPHEAlHealth Alliance was viewed as useful, practical and relevant in two of the three countries visited, and LlPHEAlHEALTH Alliance planners used technically sound, standard training program design/development methods including needs assessments. LlPHEAlHEAL TH Alliance training program planners in the third country might have benefited from strategic guidance and technical assistance in strengthening its coordination with implementers (in this case, government). Training designs could have been further strengthened by involving those stakeholders with the best understanding of worker competencies such as potential employers or supervisors. Recommendation: In the interest of optimizing post-basic and in￾service training investments, USAID and HED should ensure that grants include a mandate, technical assistance and resources for training program developers to optimally engage government and JBS International p. ix Leadership Initiative for Public Health in East Africa Outcomes Assessment nongovernment implementers and effectively incorporate their needs/interests into design and development of competency-based training. Criteria for systematic selection of in-service trainees appeared to be weak or absent in the civil service systems from which trainees were drawn. Regardless of how well in-service training programs have been constructed to meet the practical, competency-based needs of trainees, training results/outcomes may be diminished if the most appropriate trainees are not selected. Recommendation: In the interest of optimizing post-basic and in-service training investments, USAID and HED should ensure that grants include a mandate, technical assistance and resources for training program developers to enter into effective dialogue with government and nongovernment implementers to standardize structures and procedures for selection of training program participants. Regional competency-based training programs need to consider regional disparities and have the capacity to customize country programs to needs/interests and constraints of the local setting (such as differences in governance, governmental administration and structures, and language). Recommendation: In support of regional training programs, USAID and HED should ensure that grants include technical assistance and resources to assist implementers in effectively customizing regional approaches and curricula to meet local country requirements. Objective Four. Establish a Faculty Development Program that will Sponsor Promising Young Public Health Faculty from MUSPH and MUHAS through a Combination of Degree Programs and In-service Programs in Public Health Leadership Although many productive teaching, teaching infrastructure improvements, and staff development activities were undertaken, no formal faculty development plans complete with long and near term objectives and action steps were described or otherwise detected by the assessment team. In this respect, LlPHEA did not meet a key program objective of establishing a formal staff development program. The fact that a formal staff development plan was not developed over a five-year period is a program management oversight that should have been picked up through the program monitoring and evaluation process. Lessons Learned & Recommendations Each of the primary partner universities should have instituted staff development plans over the life of the actiVity. LlPHEA might have strengthened the content, validity, and relevance of its training, and provided learning opportunities for its undergraduate and graduate students and research opportunities for professors and graduate students through more strategic linkages between training and research opportunities. However, this approach does not appear to have been pursued with any vigor by LlPHEA. Not only does this signal the possibility of lost opportunity, but also speaks to an absence of technical guidance from U.S. Partner Institutions, which all have extensive experience with resource leveraging in resource-constrained academic environments. Objective Five. Improve the Teaching Infrastructure at MUSPH and MUHAS There was limited attention to improving teaching generally. The activities undertaken were focused on content and presume good adult learning practices are widely used, which mayor may not be a good assumption. While teaching infrastructure is very important, teaching tools cannot replace good pedagogy. LlPHEA may not have met fully the intent of the key program objective of improving teaching infrastructure. Lessons Learned & Recommendations Many productive teaching infrastructure improvements and staff development activities were undertaken. However, the absence of a staff development plan may have contributed to what appears generally to be a low level of staff and faculty training and development on technical teaching and training methods. An interrelated plan that linked improvements in teaching infrastructure, JBS International p.x Leadership Initiative for Public Health in East Africa Outcomes Assessment instructional technology and staff development together would likely have improved outputs and outcomes for all three areas. Additional Program Considerations Partnership Management Structure and Roles LlPHEA began with a relatively well-defined management structure that appears to have evolved in subtle ways over the five-year period as new dimensions of the program were introduced and new disciplines added such as medicine, and veterinary science. These changes to the originally envisioned structure, roles and technical support had an impact on the responsibilities of the U.S. consortium and appear to be agreed upon by the partners in line with the field-driven, regional principles of LlPHEA. The organizational structure of LlPHEA called for an Advisory Committee consisting of well-respected policy leaders and principal customers (i.e., program graduates) who would assist a Management Committee in ensuring the training process remained needs driven. Although convening the Deans and Directors group was not done for the specific purpose of staffing the Management Committee,3 it appears that the Deans and Directors annual meetings, in effect, became the LlPHEA Management Committee, at least for purposes of strategic and technical direction. Budgetary and financial management responsibilities originally envisioned for the Management Committee were moved to Makerere because the university had systems to adequately receive and disperse funds. The considerable global strategic and technical advisory capacity of the U.S. partners appears to have been "on-call," but not necessarily called upon to the fullest extent it might have been. The reported delays in permanently filling the JHU support manager position in Baltimore apparently limited the opportunities for the U.S. partners to contribute, as opposed to react, to the strategic and technical direction of the program. Thus, close operational liaison between the U.S. consortium and the African Partner Institutions may have been less than optimal during the first two years of the program. Program Documentation and Monitoring The Assessment Terms of Reference did not immediately highlight monitoring and evaluation (M&E) issues. Rather, interest in M&E emerged when preliminary findings suggested that program outcomes could have been improved with more precise program monitoring and identification of areas that could have benefitted from additional strategic and technical guidance. An understanding of the program's monitoring was drawn from LlPHEA's available documentation, literature and reports, and interviews. Though referenced, no actual work plans could be found or reviewed. However, extensive implementation level work planning appears to have been carried out, particularly in Years I and 2, but also in later years. This impression is reinforced by the fact that throughout its five years of operation, LlPHEA was documented to be a highly productive program at the output level. A set of semi-annual reports and a small volume of historical documentation on the HEALTH Alliance website attest to a program that vigorously pursued its planned activities. The semi-annual reports have a standard format that captures both qualitative and quantitative data. While the reports were informative and readable, a comparison of data gained from the field visits with how events were reported led to the following observations: • The reporting format provided extensive opportunities to describe achievements, but did not adequately cover problems and constraints at a sufficient level of detail. • Related to this, the reporting format called for a rolling projection on what activities were planned in the coming six months, but did not require the reporter to recount what activities had been planned, but did not occur. 3 The Deans and Directors group representing selected East African schools of public health was first convened in October 2006 for the purpose of collaboration in areas of mutual institutional interests which eventually lead to establishment of the LlPHEA legacy HEALTH Alliance which was chartered in 2008. JBS International p. xi Leadership Initiative for Public Health in East Africa Outcomes Assessment • Because of a lack of detail about problems, constraints, and issues that may warrant management follow-up, the reports took on the character of newsletters rather than management tools. As such, these reports may not have provided as robust an understanding of the program as they might have. • It was not clear who was involved in the writing of these reports, but it appeared that reports were prepared by JHU in Baltimore with some input from the African partners. If this is the case, it would have been more productive management-wise for the partners to use the occasion of preparation of reports for a face-to-face joint strategic, technical, and operational review. Notably, however, despite the depth of conversations with staff in three countries there was very little detail provided regarding planning and project evaluation for mid-course assessment beyond the major shift from academic to competency-based learning which shifted leadership training to disaster management response training. Funding The U.S. and African Partner Institutions, and to a lesser extent, LlPHEAlHEAL TH Alliance partners, received core funding of USD 5.87 million for the purposes of financing specified program activities such as university and in-service training programs in leadership development and health emergency management, SPH faculty and staff training in leadership, instructional technology, and a massive infusion of information and communications equipment, etc. It appeared that funds were generally used for the purposes intended. In addition to USAID's initial contributions to LlPHEA, some of the partners reported to the assessment team that they were successful in leveraging other funds. Three major international development agencies have recognized the utility of working with the HEALTH Alliance and networks formerly established through LlPHEA. The HEALTH Alliance was able to offer the practical advantages of having a legal and financial management structure that allows it to handle large grants (which newer grantees may not have). The HEALTH Alliance has been asked to join in three major new global initiatives: USAID- funded RESPOND, DflD-funded Future Health Systems (FHS), and a National Science Foundation (NSF)-funded natural disaster research project. Of note, MUSPH/Uganda was particularly successful in leveraging its leadership role and association with the HEALTH Alliance regional platform, and its reputation for excellence in delivery of HEMP and other LlPHEA training programs. MUSPH has been able to garner the following additional support: a) commitment from the Ugandan Minister of Finance to construct a state-of-the-art building at the MUSPH campus; b) funding for an International Conference on Reproductive Health and Family Planning; and c) a large grant from the Gates Foundation to establish a Center of Excellence in Reproductive Health and Family Planning. Challenges with Resources and Relationship Management4 As reported from Kenya, some members of the LlPHEAlHealth Alliance feel they did not benefit equitably from LlPHEA funding, including the Rockefeller Foundation contribution that was available during the 2008-20 I 0 period. Recommendation: It may be instructive for HED to review and clarify how equitably LlPHEA funds for HEMP training and other institution building activities (such as ICT) were shared during the 2008-20 I 0 period, and to review the patterns of accountability in funds disbursement across all seven LlPHEAlHEAL TH Alliance partners. Impact on U.S. Partner Although the issue of reciprocal benefit of partnerships to U.S. Partner Institutions is of interest to policy makers who promote international partnerships, neither U.S. nor African Partners appeared to strategically focus on this concern. Recommendation: To encourage U.S. Partner Institutions 4 A finanCial review of LlPHEA was outside the scope of this Assessment. However, there were some (mostly unsoliCited) impressions related to funding levels and distribution of financial resources that emerged during the course of the Assessment and warrant comment. JBS International p. xii Leadership Initiative for Public Health in East Africa Outcomes Assessment and their counterparts to think and act more strategically about the benefits that might accrue to their institutions as a result of partnership arrangements, USAID and HED might want to consider establishment of a standardized profile for project design that would ensure these needs have been adequately assessed. Finally, the basic standard reporting format used for semi-annual reports to higher levels did not encourage partners to report challenges in such a way that would have informed higher level managers and assist the monitoring and management follow-up process. Recommendation: HED may wish to review partnership reporting formats to ensure that information being provided includes references to operations, communications and assessment or mid-course corrections to assist project monitoring and trigger management interventions and course amendments when indicated. Conclusion Overall, LlPHEA was found to be a highly productive and consequential program. It was successful to a large degree in its activities engaging other universities, governmental organizations, and NGOs. There were, however, two areas that could have been strengthened, even in light of the strong institutional bases that three of the four universities included in the study had, and these are related to improving faculty development and teaching and teaching infrastructure. The African Partner Institutions and the HEALTH Alliance partners might have benefitted from increased strategic and technical guidance from the U.S. Partner Institutions. In addition, closer program monitoring at the HED level might have caught and corrected certain strategic and management weaknesses in the program earlier-on. Beyond these broad observations, this Assessment yielded detailed findings, lessons learned, and recommendations which are described under each objective. JBS International p. xiii Leadership Initiative for Public Health in East Africa Outcomes Assessment Introduction In 2011, Higher Education for Development (HED) requested the Aguirre Division of jBS International, Inc. UBStA) to assess the outcomes of the U.S. Agency for International Development (USAIDtHED) Leadership Initiative for Public Health in East Africa (LiPHEA). Initiated in October 2005, LlPHEA was a five-year partnership among Muhimbili University of Health and Allied Sciences, School of Public Health and Social Sciences (MUHAStSPHSS) in Tanzania, Makerere University School of Public Health (MUSPH) in Uganda, and johns Hopkins University Bloomberg School of Public Health UHU), with additional support from Tulane University and George Washington University. The partnership arrangement ended in March 2011. Program Context Public health conditions in sub-Saharan Africa are among the most acute in the world. Since the mid￾I 990s many of the public health gains made over the previous decade in this region have stagnated and in some cases been reversed. The emergence of HIVtAIDS as the leading cause of mortality, the spread of drug resistant microbes, the rising incidence of undernourishment, and, in many parts of the continent the decline in the quality and availability of public health Higher Education for Development works in close partnership with the United States Agency· for International Development and.. the .nation's· six presidential higher educatiori . associations to support the involvement of higher . education in development issues worldwide. HED primarily carries out .. its mission by funding innovativepartilerships that partner US. colleges .or universities with institutions of higher learning in developing nations. . Since 1992. HED; forl11erly known as the Association. Liaison Office for·. University Cooperation in Development (ALO),has been working with U.S. universities. colleges. and community colleges to build worldwide partnerships that address international development issues. HED has established 350 past . and present higher education partnerships in more than 66 developing countries. services are graphic indicators of the public health challenges facing the region and a measure of the limitations of current health systems capacities to effectively respond to health challenges. Among the most telling trends of the past decade are: • Increasing infant and child mortality in many countries in the region; • Growing erosions in the availability and quality of essential services, such as immunizations; • Dramatic reductions in life expectancy in high HIV prevalence countries in East Africa; • Upsurge in tuberculosis-related deaths; • Emergence and spread of drug resistant malaria across the region, leading to increased incidence of severe malaria and mortality; • Increased levels of malnutrition, which are now an underlying cause of an estimated 50 percent of all child mortality; • Stagnation in the numbers of skilled birth attendants; and • In some countries, dramatic increases in maternal mortality. At the same time, the steady growth of chronic diseases, injuries, consequences of environmental stresses, and the global emergence and spread of infectious diseases such as Type A Influenzas, are posing new challenges to the health of the people of Africa and threaten to overwhelm its under￾funded and chronically fragile health systems. While occupied by just 13 percent of the world's population, the region bears 24 percent of the global burden of disease.' The continent's immense disease burden and frail health systems are impacted by the broader context of poverty, underdevelopment, conflict, and weak or ill-managed government institutions that shape each country's ability to deliver effective health care. As a result of these and other factors, the area is I Cooke. J. (2009) Public Health in Africa: A Report of the CSIS Global Health Policy Center. Center for Strategic and International Studies. Washington. D.C. jBS International p. I Leadership Initiative for Public Health in East Africa Outcomes Assessment prone to both natural and man-made disasters. The International Federation of Red Cross and Red Crescent Societies reports that in 2008, Africa confronted 159 disasters, more than double the number experienced a decade ago.2 Experts3 predict a "continental health disaster" in the making as a result of climate change, especially global warming, fearing that the emergence, resurgence, and spread of infectious diseases, especially those that are mosquito-transmitted from livestock to humans, could accelerate as a result of global warming. Disasters have devastating effects on a personal, national, and regional level, and in many cases "can wipe out decades of development in a matter of hours."4 Essential to improving health conditions in East Africa is the need to focus on strengthening health systems to Global Health Workforce 30 numb~b of health workers per 1000 10 pop o 24.8 provide more comprehensive care, strengthened networks of procurement and outreach, and improved prediction and planning capabilities, making systems more responsive to local needs. Central to effective health systems and a major and largely neglected barrier to the effective use of these resources is the African health workforce, particularly at the middle and senior management level, who are responsible for envisioning, planning and overseeing the delivery of solutions to these health problems. A further concern is the lack of effective communication and collaboration among ministries of health and ministries responsible for planning, finance and human resource management, whose decisions on budgets and workforce often undermine efforts for effective public health action. African countries are carrying out extensive reforms of their health sectors to meet the increasingly complex needs of their population. A companion reform of their Figure 1- Regional Differences in the Global Health Workforce training institutions to provide the human resources with the appropriate skills for effective health action is necessary. Increasingly, emerging African leaders will see health as key to poverty alleviation and development in their countries. Schools of Public Health are, or can be, key actors in efforts to strengthen the continent's ability to deal more effectively with its own health problems and development potential. Relevant to meeting the Millennium Development Goals and to achieving better health equity among all populations, the focus of USAID support to global health has been to 2 International Federation of Red Cross and Red Crescent Societies (2009) World Disasters Report 3 Climatologists, veterinarians, and medical researchers at a meeting in Nairobi, Kenya discussed the need to work together in Africa to combat the health effects of climate change. VOA News (September 19, 2008): 'Experts Warn Climate Change Could Cause Health Disaster in Africa.' http://www.voanews.com/english/news/a-13-2oo8-09-19-voaSO-66688527.htmIAccessed March 4. 20 II. 4 IRIN Humanitarian News and AnalysiS Qune 2005) In-Depth: Disaster reduction and the human cost of disaster JBS International p. 2 Leadership Initiative for Public Health in East Africa Outcomes Assessment strengthen host-country health systems by developing leadership capacities of public health professionals. By almost any health indicator, low numbers of health workers is associated with poor health status. In Europe, there are 18.9 health workers per 1000 persons and in the Americas, 24.8; whereas, in Africa, there are only 2.35 (Figure I). Furthermore, the capacity to train additional public health specialists is particularly weak. Many who fill public health positions have not had the training necessary to effectively implement the sweeping health systems changes that are so needed. Higher education initiatives, including those that would increase the public health workforce, rarely have been a priority for donors. Moreover, the dynamics driving many of the continent's problems are sub-regional in nature, yet donor efforts target the country level. Africa needs strong regional coalitions to promote health initiatives that span national borders; therefore, building human capacity across borders should be part of any coordinated regional strategy. To fully benefit from the opportunities provided by greater financial resources, health officials cannot take a business-as-usual approach to managing health sector development in the region. The new financial health initiatives demand strong leadership and foresight. To convert financial resources into real health gains, the region needs public health leaders who are able to manage financial and human resources strategically, collaborate across sectors, envision new ways of operating, and effectively set priorities and allocate resources. Inception and Development of L1PHEA In 2005, USAID committed resources toward the development of an East African Public Health Leadership Initiative that would address the region's critical needs for public health leadership. Considerable thought and deliberation went into conceptualizing and structuring of LlPHEA. In the final analYSis, it was the strategic vision of USAID which resonated with HED and attracted the U.S. and African institutions that eventually formed the LlPHEA partnership. HED issued a Request for Concept Papers followed by a Request for Applications from U.S. consortia of higher education institutions. Applications received final review in September 2005. The Johns Hopkins University/Tulane University/George Washington University partners were selected by the review team, which included the African partners who figured prominently in the selection process. Formal LlPHEA activities began in January 2006.6 Although LlPHEA formally brought together two prominent institutions of higher education in East Africa, the partnership between the Makerere University School of Public Health (MUSPH) in Uganda and the Muhimbili University of Health and Allied Science (MUHAS/SPHSSF in Tanzania actually predated LlPHEA. More accurately, that partnership was born out of a common interest in doing something proactive about the troubled state of public health human resource capacity in Uganda and Tanzania and across Africa, in general. Based on their prior connections and mutual interest, a joint MUSPH and MUHAS/SPHSS research team conducted a meta-analysis in 2004 on the state of public health education and practice in sub-Saharan Africa. That study led policy makers to conclude that undergraduate and graduate curricula in sub-Saharan Africa lacked components that systematically addressed "leadership" issues. Moreover, a direct link to health development was made by identifying lack of structured instruction on leadership at the university level as a key factor in the failure of public health practice. 5 These figures are from the 2006 World Health Report. http://www.who.int/whr/2006/whr06_en.pdf. "Ibid. 7 Muhimbili University of Health and Allied Sciences (MUHAS) is the successor to the Muhimbili University College of Health Sciences. JBS International p. 3 Leadership Initiative for Public Health in East Africa Outcomes Assessment By mid-200s, USAID was at the advanced stages of identifying and selecting LlPHEA partners. Both MUSPH and MUHAS/SPHSS attended a consultative interview with the selection team in Geneva in 2005, and USAID ultimately selected MUSPH and MUHAS/SPHSS as the LlPHEA African Partner Institutions. As was related to the LlPHEA assessment team, the alliance of MUSPH and MUHAS/SPHSS brought a fresh philosophical perspective to the LlPHEA partnership. They felt that LlPHEA presented an important opportunity for true regional cooperation. While U.S. Partner Institutions were brought into LlPHEA (in keeping with HED's traditional partnership model), the spirit of regional cooperation was also maintained in the sense that the African Partner Institutions played a critical role in identifying and selecting the U.S. partners (not the reverse, which is most often the case). Viewed strategically, the African Partner Institutions wanted their U.S. counterpart(s) to be able to contribute relevant institution strengthening capacities to the partnership. Johns Hopkins University and Tulane University had long histories of involvement in public health higher education in East Africa, and strong programs in instructional technology and leadership development. It was thought that the strength of these programs could elevate LlPHEA's public profile as a leader of leadership development in the region. LlPHEA's strong commitment to regional cooperation would be evident throughout the lifetime of LlPHEA as the African Partner Institutions sought to build ever-increasing linkages and connectivity with other schools of public health in the region. Bringing together deans from selected East African schools of public health as the core of the LlPHEA network eventually lead to formation of the "Higher Education Alliance for Leadership Through Health" (HEALTH Alliance), a LlPHEA legacy institution which was chartered as a regional institution (separate from the LlPHEA program) in Uganda in 2008. During the period 2005-20 I 0, the LlPHEA program and the HEALTH Alliance operated jointly. This joint operation is generally referred to throughout this Report as the "LlPHEAlHEAL TH Alliance." Program Overview The goal of the LlPHEA partnership was to strengthen public health leadership in East Africa by improving the capacity of local institutions to train mid- and senior-level health professionals in leadership skills. LlPHEA objectives (Le., expected outcomes) were to: • Create a network that would link public health schools, ministries of health, public health practitioners, regional organizations, and other critical stakeholders, to (at a minimum) facilitate information and resource sharing. • Create an enabling environment in Uganda and Tanzania for public health activities by bUilding leadership and advocacy skills among public health faculty and key decision-makers. • Strengthen teaching and educational programs that integrate leadership skills development into the curricula of the Schools of Public Health and other university departments in Uganda and Tanzania. • Establish a faculty development program that would sponsor promising young public health faculty from the Schools of Public Health through a combination of degree programs and in￾service programs in public health leadership. • Improve teaching infrastructure in the Schools of Public Health in Uganda and Tanzania. Call for a Program Assessment While the achievements of the five-year long LlPHEA partnership and its legacy HEALTH Alliance are generally known, the completion of the LlPHEA grant-supported program presented an important opportunity for a more in-depth examination of the LlPHEA model with a view to better understanding its significance and the factors that may have contributed to, or impeded, its achievements. HED thus requested an assessment of the program's effectiveness and results measured against program objectives (expected outcomes). JBS International p.4 Leadership Initiative for Public Health in East Africa Outcomes Assessment Assessment Purpose & Methodology The purpose of this assessment was to determine the extent to which LlPHEA met its goals and targets, specifically: • Creating a network that would link public health schools, ministries of health, public health practitioners, regional organizations, and other critical stakeholders, to (at a minimum) facilitate information and resource sharing. • Creating an enabling environment in Uganda and Tanzania for public health activities by bUilding leadership and advocacy skills among public health faculty and key decision-makers. • Strengthening teaching and educational programs that integrate leadership skills development into the curricula of the Schools of Public Health and other university departments in Uganda and Tanzania. • Establishing a faculty development program that would sponsor promising young public health faculty from the Schools of Public Health through a combination of degree programs and in-service programs in public health leadership. • Improving teaching infrastructure in the Schools of Public Health in Uganda and Tanzania. The assessment team also proposed to conduct institutional assessments to gauge each institution's internal strengths and, to some extent, determine whether key stakeholders at each institution viewed LlPHEA as a critical factor contributing to the strength of the institution. Scope and Design Considerations According to USAID guidelines, an impact evaluation assesses the changes that can be attributed to a particular intervention, such as a project, program or policy. This involves counterfactual analysis, that is, a comparison between what actually happened and what would have happened in the absence of the intervention. Although this Assessment addresses impact-like issues, this study was not meant to be an impact evaluation as defined by USAID guidelines; rather, this Assessment meets the parameters of an outputs/outcomes-based performance assessment, which was indicated explicitly in the contract. As such, this Assessment relied primarily on qualitative data and provides insights into what happened during the life of the program, but is limited in being able to definitively address evaluation questions (for example, the extent to which institutional changes that may have occurred during the life of the program can be attributed to the intervention). As depicted in the adjacent diagram, the assessment team used a mixed-method assessment design to support formulation of findings and conclusions. Information on program inputs and outputs was obtained from written reports and data summaries. A Key Informants Opinion design was used to elicit the perceptions and perspectives of partners and stakeholders regarding their experiences with the program, and their opinions about program performance, benefits and impact. Key informant information was primarily obtained through structured interviews.S An Expert Opinion design was used to triangulate the impressions and observations of assessment team. Inquiry and analysis was facilitated by JUDGEMENlSANDIMPRESSlONS COMIINED BY ASSESSMENTTEAMTO FnRMlllATF RNmNr;.c;;ANO mwnllSlflltli;; use of a conceptual framework and the cumulative expertise of the team. In the absence of a pre-determined Logical Framework or Results Framework with defined expected outcomes and targets, the assessment team had to construct an alternative means of analyzing 8 With the exception of six questionnaires that were alternatively distributed to the three HEALTH Alliance countries (Rwanda. DRC. and Ethiopia) that were not visited. JBS International p.5 Leadership Initiative for Public Health in East Africa Outcomes Assessment program outcomes, and assessing the factors that may have led to or hindered program success. Capturing the multiple perspectives and sources of assessment information required a combination of assessment approaches. The team triangulated data and information to form judgments about the program's outcomes and long-term sustainability. Logical Framework for L1PHEA Assessment According to Bickman, et al (1993),9 every program assessment, whether explicitly or implicitly, is based on a conceptual framework or model that specifies how the program is purported to work. Consistent with this notion, the L1PHEA assessment team found that analysis and assessment of a strategically complex human, institutional capacity building, and public health development program such as the L1PHEAlHEAL TH Alliance required adaptation of a well-defined assessment framework. The team sought to construct a framework that would assist in understanding the principles, processes and mechanisms by which the program might be expected to achieve its intended goals, and communicate these understandings to its readership. By its nature, there are two different contexts or environments in which the L1PHEAlHEAL TH Alliance has operated over the past five years: • The institutional context: L1PHEA's network of universities and Schools of Public Health each operate within the internal environments of their higher education teaching/training institutions. The L1PHEAlHEAL TH Alliance Schools of Public Health are engaged, within their university systems, in developing their own institutional management capacities to more effectively provide capacity building teaching/training services to the public health community (within and external to the university). • The development context: In contrast, the specific teaching/training programs developed and offered by L1PHEA must be relevant and responsive to a broader external public health development environment. L1PHEAlHEAL TH Alliance teaching/training programs must cater to the human resource knowledge, skills and competencies required by government and non-government organizations that implement community, national and regional public health-related programs and projects (utilizing personnel that have been trained, in large part, by the public health schools). The L1PHEA Outcomes Assessment needed to consider both the developmental and institutional context in which the program has operated. These concepts, incorporating both the external (developmental) and internal (institutional) context of the L1PHEAlHEALTH Alliance program are shown below in Table I - Logical Framework for the L1PHEA Outcomes Assessment. The Assessment Framework is a construct developed by the assessment team to depict, based on its best judgment, a selected number of critical factors or elements within the internal (institutional) environment and external (public health development) environment that it believes are important to review within the context of an end-of-project, outcomes assessment of the L1PHEA program. Elements shown in the schematic are based on standard logical (input/output/outcomes) framework concepts. 10 The Logical Framework shows the desired relationship of program elements to each other, and serves to define the desired gUiding principles, characteristics and qualities that should be evident in an effective higher education public health development partnership program. The Framework also is informed by the criteria for selection of institutional partners established in a pre￾grant award L1PHEA conference, and specifically informs Key Questions formulated to address the needs and interests of this Assessment's readership. Elements shown in the internal institutional environment (the lower part of the schematic) refer to the managerial process capacities that, according to a systems approach to assessing capacity in 9 Bickman, L, Hedrick, T, and Rog., D.j., "Applied Research Design: A Practical Guide", Sage Publications, 1993, p. 5. 10 Based on USAID Logical Framework method, as discussed in McCawley, P.F., "The Logic Method for Program Planning and Evaluation", University of Idaho Extension, (N.D.), and Mclaughlin, jA and jordan, GB., "Logic Models: a tool for telling your program's performance story", Evaluation and Planning (journal) 22, 65-72, 1999. JBS International p. 6 Leadership Initiative for Public Health in East Africa Outcomes Assessment successful teaching/training institutions, should exist (or be developed). The institutional change hypothesis is predictive (not attributable) and suggests that an institution that possesses, or is able to build, strong institutional managerial capacities through the program will be best positioned to perform well in meeting the strategic public health objectives (development context) of the program. Analyzing the Performance of LlPHEA Partners within a Development Context The USAID Center for Development Information and Evaluation provides guidance on the need and purpose of institutional capacity building in programs such as the LlPHEA partnership. This gUidance postulates that in the course of planning, implementing, and measuring their programs, USAID managers often find that a partner's or customer's lack of capacity is an obstacle to achieving program results. Increasing the capacity of partner and customer organizations helps them carry out their mandate effectively and function more efficiently, thereby making those organizations more competitive and sustainable. Strong organizations, it has been determined, are more able to accomplish their mission and provide for their own needs in the long run. According to this guidance, institution strengthening programs are intended to improve an organization's ability to provide quality and effective services, while also increasing the organization's viability. This means supporting an organization, or organizations such as the LlPHEA Partner Institutions, to be programmatically sustainable (Le., able to provide useful services), as well as organizationally sustainable (Le., with strong leadership and having necessary management systems and procedures), while ensuring that it can independently attract and maintain support, and efficiently utilize resources (e.g., human, financial, and material). The support, such as is represented by the LlPHEA program, must help the partner institutions to, "understand the external environment (e.g., political, economic, and social) in which the institution operates, to effectively coordinate with development partners, and to develop productive relationships within that environment that are sufficiently stable and predictable." II These institutional qualities serve to define the characteristics expected by USAID from its investments in institutional strengthening. 12 Although an assessment of the management systems of the LlPHEA Partner Institutions to receive and efficiently utilize grant resources (e.g., human, financial, and material) was not the focus of this Assessment, the assessment team did consider the institution's capacities to function strategically and be relevant and responsive to the development markets it pursues. The institutional capacities specifically related to the ability of LlPHEA Partner Institutions to understand and be responsive to the external environment, and coordinate with development partners were analyzed and appear as elements (as interpreted by the assessment team) in the external environment in the Logical Framework for the LlPHEA Assessment (see Table I, below). These elements describe essential project management functions that, if working well, will present the optimal environment for partnering and transfer of capacity between partner institutions. If these basic management functions are weak in the partner institutions that are the target of an institution￾strengthening intervention, the assisting partners should supplement and/or facilitate development of these abilities along with whatever other technical assistance is being provided. 13 This Assessment will refer to these elements in determining the extent to which the LlPHEA program has improved the performance of the African Partner Institutions to manage programs within a development context. II New TransCentury Foundation, 1996, p.1 of vol.3, as quoted in "Measuring Institutional Capacity, Recent Practices in Monitoring and Evaluation," TIPS IS, USAID, 2000, PNACG612[1]. 12 Also see the footnote 5 and 6, above, and USAID criteria for selection of institutional partners which was established in a pre-grant award LlPHEA conference (See Figure 3, below). 13 Howard, M., Omaswa, F., Okuonzi, S., and Muhangi, D., "CapaCity Building and Leadership Enhancement in Uganda." The Mitchell Group, 2010. JBS International p. 7 ., ~ C o U ., c CU E Q. o - CU ~ C ., ~ C o U fa c o 0.p ::s ., :p \I) C to abilities of SPHs to address strategic and technical priorities of public health implementers and development partners -Untested demand and market for SPHs as a resource to provide public health dpvplonmpnt services Need to strengthen institutional capacities: -Learning } -Complexity -Flexibility -Productivity Connectivity* -Durability Table I - Logical Framework for LIPHEA Outcomes Assessment planning -Assessments. design/development of teaching. training. research and public health program development services -Technical assistance and guidance from program partners -Funding/grant support for SPHs to demonstrate their abilities to carry out public health research. systems development. and teaching/training activities -Instructional technology -Staff training and sabbaticals -Strategic and technical assistance and guidance from program partners -Equipment and funds for activities -Funds for coordination. ICT equipment and systems to develop communications networks a) Funding/grant support b)Technical assistance and creation of a regional platform (HEALTH Alliance) from which to market services -Expansion/integration of development course offerings into SPHs and other university undergraduate programs -Conduct of basic. post-basic and in-service public health training programs -Conduct of public health research and program development services -Teaching methods and infrastructure improved -Appropriate systems and human resources to carry out program objectives put in place -Increased program outputs during grant period -Durable institutional and regional information and communications capacity a)Systems and human resources developed b) demonstrated ability to market services public health and leadership development courses -Increased opportunities to advocate for and promote a public health leadership development agenda -Establishment of an in-service public health training track record -Establishment of a public health research and program development track record -Permanent mechanisms for on-going identification of new systems. methods. and technologies are in place -Organizational growth and develop is accompanied by best use of systems and human resources -Productivity levels are sustained at end of grant -Institutional and regionallC capacity will continue to grow and develop a)Sustainable systems and human resources are in place connectivity of SPHs to the public health community JBS International p.8 Leadership Initiative for Public Health in East Africa Outcomes Assessment Key Assessment Questions It was the intent of this outcomes assessment to examine the LIPHEA program's effectiveness and results measured against its reported outputs and its expected outcomes. The assessment measured the extent to which the program met its goals: the human and institutional capacities that have been strengthened; the perceived impact of the program on its multiple stakeholders; and, the observable impact the program has had on national and regional public health developments. IS SpecificalIy, the assessment team was requested to determine the extent to which LIPHEA was successful in meeting its five objectives: • Creating a network that would link public health schools, ministries of health, public health practitioners, regional organizations, and other critical stakeholders, to (at a minimum) facilitate information and resource sharing. • Creating an enabling environment in Uganda and Tanzania for public health activities by building leadership and advocacy skills among public health faculty and key decision-makers. • Strengthening teaching and educational programs that integrate leadership skilIs development into the curricula of the Schools of Public Health and other university departments in Uganda and Tanzania. • Establishing a faculty development program that would sponsor promising young public health faculty from the Schools of Public Health through a combination of degree programs and in￾service programs in public health leadership. • Improving teaching infrastructure in the Schools of Public Health in Uganda and Tanzania. A Systems Approach to Understanding the Institutional Capacities of L1PHEA Partner Institutions In addition to addressing key questions regarding LIPHEA's outcomes, the assessment team assessed the institutional capacity of each of the partner institutions visited during the field work. To engage in analysis of concepts and strategies concerned with human capacity and institution building, the team applied systems theory with a special emphasis on teachingllearning systems. As shown in Table I - Logical Framework for the LIPHEA Outcomes Assessment, organizations can be thought of as having six organic dimensions: 19 Iearning, complexity, flexibility, productivity, connectivity, and durability. With these dimensions, criteria (or indicators) can be established to measure the extent of institutional capacity that appears to exist at the time of review (that is, a "snap shot").2o AlI of the six criteria which are used as a basis for assessing institutional capacity can be measured quantitatively using a rating scale. Of these criteria, learning is viewed as the most important measure because it involves the capacity to take in new information and combine it with previously learned information to produce new insights. Like indiViduals, institutions engage in learning. However, institutional learning is a more complex phenomenon that reqUires a structural and cultural congruence that is not so easily developed in settings where historic cultures are attempting to work within the framework of imported structures. Nonetheless, alI institutions need to demonstrate the ability to learn in order to grow and develop. This is most easily done by training administrative staff in management techniques and, most importantly, in the elements of decision-making and systems analysis. Such training involves both analysis of empirical data and sophisticated decisional analysis to strategicalIy increase the complexi~ and growth on the 1BUSAID/HED Request for Proposal (RFP) for the LlPHEA Higher Education Partnership. dated February 7. 20 II. 19 A sixth dimension. connectivity. was added to this basic systems concept to reflect. in the case of the LlPHEA program design. a call in the stated goals of the program to create networks and to improve communications and information sharing on subjects of common interest. 20 Although some attempt was made to assess how the program has contributed the professionalism of individuals. the scope of the Evaluation precluded an in-depth study of individuals. per se. and instead concentrated on institutional strengthening. JBS International p.9 Leadership Initiative for Public Health in East Africa Outcomes Assessment organization. With both new learning and utilization of decisional analysis to increase organizational complexity, institutions are positioned to apply tools of management in a flexible manner in responding to challenges from various external and internal constituencies. Increasing the ability to interact with external and internal constituencies, to share information, communications and connectivity, fosters learning, complexity and growth of the organization. Effective and skilled management is critical to the preservation of balance or equilibrium that is conducive to maximum productivity of an academic institution. The more an institution is sensitive to the importance of using administrative learning built upon sound theoretical premises to fleXibly respond to institutional needs, the strength of the institution will improve and its sustainability and durability will be more Iikely.21 Institutional growth and development requires effective leadership and governance. Leadership describes the ability of an individual to scan the environment, to create an attractive vision and strategies, and to inspire and align actors and interests for action to achieve an agreed goal. Good leadership requires both personal and institutional attributes. At a personal level, it requires a person who not only has the knowledge, skills, and experience, but can make bold and firm decisions and originate successful initiatives. To do this, the leader requires a good and supportive institutional arrangement within an enabling political, cultural, and social environment. The required institutional attributes include a functional legal framework, effective and efficient administrative procedures, and available resources. Governance, on the other hand, refers to the wayan organization is run. Generally speaking, good governance requires a legal framework; transparent processes; clear (and system appropriate) assignments of roles, responsibilities, and powers; a system for disciplining those who abuse their roles, responsibilities, or powers; and a system of redress or recourse for those within the organization who believe their rights have been abrogated. To measure leadership and governance in this Assessment, the team used proxy measures based on the responses of the key informants and an analysis of the structural and cultural changes reported by key informants. A systems approach to assessing institutional performance (including issues of leadership and governance) is intended to highlight institutional and management processes and performance that may benefit from institutional strengthening and change. However, the approach is not intended to substitute for an audit-type examination of an institution's financial and program management performance, (which was seen by the assessment team as beyond the scope of this Assessment). These systemic organizational/institutional performance concepts were applied in case studies of four of the participating institutions: MUSPH, MUHAS/SPHSS, Moi University, and University of Nairobi. These case studies apply the systems approach to institutional assessment to provide a snap shot of each school's institutional management processes and capacities. Use of the systems approach as an assessment tool demonstrates that: • The approach and tool have value as a qualitative/quantitative assessment tool that can yield useful information about the status of an institution; • The approach and tool can be used to make comparisons between two or more institutions; and • This approach and tool have potentially wide application in assessing higher education development partnership programs. 21 Regarding the measure of the complexity variable, the teaching of individuals in terms of theory or methods of analysis assists in constructing webs of understanding resulting in fuller appreciation of the intricacy of a specific subject. The application of knowledge and theory can be assessed in instructional methodologies as they explore an individual's capacity to merge empirical and normative theory in the application of data to produce informed conclusions. This strengthening of intellectual vitality then provides flexibility in merging previously learned data and theory with recently acquired information to form new syntheses of insight that contribute to understanding complex phenomena. The application of new learning to newly learned paradigms encourages the syntheSis of theory. JBS International p. 10 Leadership Initiative for Public Health in East Africa Outcomes Assessment Data Collection The assessment team conducted key informant interviews (n=82) and focus group discussions (n=5) with individuals and organizational representatives drawn from a range of LlPHEA stakeholders, including its funders. Discussions with the program administrator, who also funded this assessment, were limited to requests for clarification and additional information on the programmatic changes and program description of Years 3-5 of LlPHEA and total funding for the activity and its distribution among the partners. Contacts were made with four of the six LlPHEAlHEAL TH Alliance countries, representing approximately 80 percent of the programP However, it should be noted that compared to the thousands of individuals who have participated and benefitted from LlPHEAlHEAL TH Alliance during its five-year period, the Assessment sample size is not large enough to statistically generalize the findings to the rest of the beneficiary population. The sample does, however, offer a general pattern of opinions and perspectives that, when triangulated, offer evaluative "impressions." During a five-week period, the assessment team conducted research, developed instruments and the assessment framework, held teleconferences with U.S. Partner Institutions, visited both African Partner Institution countries (Uganda and Tanzania), one HEALTH Alliance member country (Kenya) with visits to two of Kenya's participating Schools of Public Health, and held a formal conference call interview with the representative of another HEALTH Alliance member country (Rwanda) based at the National Rwanda School of Public Health. Organizers of the Assessment made a decision during planning for the Assessment that the team would not travel to Rwanda, Democratic Republic of Congo, and Ethiopia due to time and logistical constraints. The categories of LlPHEA stakeholders included in the sample are shown in Appendix A - List of LlPHEA Partners and Collaborators. Identification of individuals and organizations to be included in the sample was substantially assisted (and influenced) by designated assessment facilitators in each country, and was limited to some degree by the availability of the interviewees. Appendix B - List of Individuals and Organizations Contacted, provides the names of individuals included in the Key Informant Opinion Survey. Each interview event involved a pre-determined appointment (often at the interviewee's place of business), and was usually attended by the maximum number of assessment team members available - in most instances, all four team members. The assessment team employed a structured conversational interview style that was guided by a standardized Assessment Interview Worksheet (designed for internal use only). Following the interview, each assessor was responsible for transcribing his/her interview notes onto the standardized Worksheet. Worksheets or hand-written notes were shared among team members, and an archived set of these raw survey data has been saved in an archived e￾file.23 A completed, macro-enabled Assessment Interview Worksheet is included as Appendix C￾Sample Key Informant Interview Survey Instrument. Data Analysis & Limitations The team collected qualitative data in their interviews and focus groups, and notes from each session were shared with all team members to ensure that they incorporated all of the points each team member had in his or her notes. The data analysis was then based on their field notes, with different team members being responsible for writing different sections of the report. Each team member reviewed all the relevant interview and focus group notes multiple times to ensure thorough analysis of each theme within the report. 22 This estimate was provided by a JHU spokesperson. 23 The archived e-file is not part of this Evaluation Report. J BS International p. II Leadership Initiative for Public Health in East Africa Outcomes Assessment There were several challenges the team faced in conducting this assessment. First and foremost, few resources were available and the contract explicitly stated the number of days for each team member. Therefore, only the team leader was allocated time to interview the U.S. Partner Institutions, and the rest of the team never had the opportunity to ask questions of their key stakeholders. In addition, only three days were permitted for each of the team members (with two more days for the team leader) to complete data analysis and report writing, which was not sufficient for the amount of data collected over four weeks of field work. Given those time constraints, the team was only able to use very limited qualitative data analysis techniques. Each of the interviews was reviewed three times (in a few cases, more) to ensure that the information from each was incorporated into the relevant sections of the report, and the team triangulated data from multiple sources to provide the broadest range of views. However, no qualitative data analysis tools, such as MAXQDA or ATLAS.ti could be used to analyze the textual data more rigorously given the timeframe. The lack of resources also precluded involving all seven African Partner Institutions in the assessment. In discussing how limited resources could yield the most information, the assessment team and HED concluded that the team should visit the two primary partners, MUSPH and MUHASISPHSS, and the institutions in Kenya, due to logistical simplicity and the possibility of capturing two additional institutions rather than one (as would have been the case in the other countries). The team also spoke with several key stakeholders in Rwanda, who were able to contribute their experiences to the overall analysis. Further, as noted above, the sample of beneficiaries interviewed was small, and the results of those interviews cannot be generalized to the rest of the beneficiary population. Report Organization The report is organized by discussions of each of LlPHEA's five objectives followed by an assessment of each of the four institutions the team visited and the team's findings and recommendations. JBS International p. 12 Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective One. Create a Network that Links Public Health Schools, Ministries of Health, Public Health Practitioners, Regional Organizations, and Other Critical Stakeholders, to Facilitate Information and Resource Sharing Networking and Formation of the HEALTH Alliance A key objective of LlPHEA was to "create a network that would link public health schools, ministries of health, public health practitioners, regional organizations, and other critical stakeholders, to (at a minimum) facilitate information and resource sharing." LlPHEA principals were to determine how this mandate would be implemented. For instance, convening of regular regional deans meetings was not a specified activity in the original program descriptions. Rather, this activity was an interpretation of the partnership's mandate to "build networks." These meetings eventually led to the formation of the HEALTH Alliance. In the earlier years of LlPHEA, the extensive documentation shows that schools of public health in the Alliance network were very interested in being involved with LlPHEA-supported institutional strengthening activities. Partners considered the possibility of the field-based regional cooperation strategy, in the form of student, faculty, instructional, information and communications exchanges, as a way to strengthen implementation of LlPHEA. This implementation strategy would involve expanding the network of regional collaborators to create a regional platform for institutional strengthening. The partners developed detailed institutional profiles in order to share information about each institution's strengths. LlPHEA did not appear to provide any significant regional technical assistance or funding to improve institutional profiles where weaknesses may have existed. One exception may be the on-going consultative support to the UniverSity of Nairobi in its (now successful) efforts to transition from a departmental status to that of a full-fledged school of public health, but it is not clear whether LlPHEA was instrumental in that process or merely helpful. During Years 1-2 ofthe program, LlPHEA partners laid much ofthe groundwork for enriching the institutional, national, regional, and global professional information sharing environments for public health issues. Networking systems and activities were being undertaken, including student and faculty exchanges, cross-campus collaboration in the form of External Examiner exchanges, and instructional, information and communications exchanges. The partners created profeSSional public health information and communications networks by offering an extensive array of workshops, conferences, colloquia, publication of a professional journal, and creation of a virtual knowledge center. While these networking developments were underway, the creation of the LlPHEA legacy HEALTH Alliance was progressing. Although the HEALTH Alliance is correctly described as an outgrowth of early deliberations of the Deans and Directors Meetings and is generally thought of as a "grass roots" movement engendered by the deans and directors, the U.S. partners also played a role in these early developments by encouraging LlPHEA entrepreneurship. For example, at the very first Deans and Directors Meeting in October 2006, the theme of the agenda was to "develop a shared vision of leadership roles [that could be played by schools of public health] in addressing health problems in East Africa." The keynote presentation by Eamon Kelly from Tulane University was tellingly entitled, "Resource Development: Fund Raising." Among other concepts, Dr. Kelly's message substantially dealt with the need to develop systems and human resources to improve competitiveness in schools of public health. JBS International p.13 Leadership Initiative for Public Health in East Africa Outcomes Assessment According to interviewees, in 2008, the HEALTH Alliance was granted legal status and became registered in Uganda with authority to receive and disperse funds in the East Africa region. Formation of the HEALTH Alliance, complete with strategic purpose, organizational structure, operational modalities and regional scope, made it an attractive institution from which to launch a variety of public health developments in East Africa. During the period 2008-20 I I, the Alliance functioned under the mantle of L1PHEA and was generally referred to as "LIPHEAlHEAL TH Alliance." As described in the L1PHEA Final Report, the establishment of the HEALTH Alliance as a legalized network of schools of public health in Uganda, Tanzania, Kenya, Rwanda, DRC, and Ethiopia is perceived by some as the L1PHEA partnership's greatest success. For the first time in the history of Eastern Africa, schools of public health formally collaborated across borders to improve public health for the region. Deans, faculty, ministers, and district level officers from the several countries have contributed to the development of programs and opportunities in the areas of research, policy support, public health education and training in area of mutual interest to the HEALTH Alliance, as well as to government and donor public health development partners. In spite of a few identifiable strategic and technical weaknesses in the program, formation of the HEALTH Alliance has been described as "an unexpected triumph" and "a success story." In practical terms, the Deans and Directors group played a stewardship role in the formation of the HEALTH Alliance as a legal entity, and also provided on-going strategic and technical oversight to the L1PHEA Directors. This latter role may not have been obvious to many L1PHEA observers, leading to questions about the achievements of the Dean's Conferences. The HEALTH Alliance, however, was noted for being internally, rather than externally, driven. A review of HEALTH Alliance documents seems to confirm this perception. The following are some examples of policy, strategy, and technical deliberations that were normally processed by the annual Deans and Directors Meetings: • The 4th Deans and Directors Meeting in 2008 reviewed the Alliance legal status documents with the Makerere Legal Officer. • The 6th Deans and Directors Meeting in 2009 reviewed the IT status in all Alliance SPHs and IT technical assistance priorities and activities proposed by Tulane. • The 7th Deans and Directors Meeting in 2009 discussed the HEALTH Alliance strategic plan. Information and Communications Technology All partners saw upgrading Information and Communications Technology (ICT) at Muhimbili and Makerere Schools of Public Health as critical to meeting this objective. To assist with meeting this objective, JHU primarily focused on provision of hardware, and Tulane developed information and communications systems and provided training to faculty and staff. Although the two African Partner Institutions were the main focus of ICT strengthening, limited IT assistance was also provided to each school in the L1PHEAlHEALTH Alliance, and a regional "across￾systems IT network" was established among the seven schools. Further, a HEALTH Alliance website was developed at Tulane and was permanently transferred to Makerere University where it is now hosted. The provision of systems development and training in Information Technology (IT) was a major outcome of L1PHEA. There is ample evidence at Makerere University that the training was effective in providing skills to the two original L1PHEA Partner Institutions. When the program was expanded via the Health Alliance, the new members of the regional consortium were invited to send personnel to Makerere for IT training. Building ICT Capacity JBS International p. 14 Leadership Initiative for Public Health in East Africa Outcomes Assessment LlPHEA support provided a notable boost to ICT capacity to LlPHEA partners and the HEALTH Alliance. Earlier problems with bandwidth were identified in assessments (in 2008) of ICT status at MUSPH and MUHAS/SPHSS and resolved with use of fiber optics. This was coupled with training of ICT staff from the various HEALTH schools on Instructional Design at Tulane. These developments allowed for the successful migration of a LlPHEAlHEALTH server from Tulane University to MUSPH and for linking computers across HEALTH's schools. The ICT system at Makerere University at large is described by Deogratious Sebuwufu, the MUSPH ICT coordinator, as being "chaotic." Thus, the autonomous ICT system which was built through LlPHEA at MUSPH is considered a "model" at Makerere for the following reasons: • It has a stable electrical power input (unlike the University's system which suffers frequent power outages). • It is sustainable. The system's open-source nature allows for continuous free upgrades. • The system is open for access (albeit in a limited manner) to the local community (business and residential) which immediately borders the school of public health. • It provides outreach ICT training to students from other Makerere ICT systems. LlPHEA funding and training enabled the creation of multiple redundancies at MUHAS/SPHSS which will ensure sufficient backing-up of content and provide guarantee against loss of functionality if the main system suffers fatal failure. Except for Makerere University where the SPH IT system is a stand-alone system, ICT systems at other HEALTH Alliance schools are part of overall university computational infrastructures and subject to all of the weaknesses of those systems. However, the HEALTH Alliance systems managed at Makerere are reported to work seamlessly and to be in sync. ICT administrators reportedly often work cooperatively to resolve localized or across-HEALTH ICT problems. Two potential limitations of across-HEALTH ICT system functionality exist. These include problems with bandwidth at MUHAS/SPHSS (a situation that is likely to exist in other schools), and the lack of personnel or infrastructural capabilities to provide sufficient support for robust e-Iearning platforms such as BlackBoard or Moodie. Additionally, the LlPHEAlHEAL TH Alliance server does not allow public access to available permanent instructional materials. ICT activities continue to be under-marketed. The LlPHEAlHEAL TH Alliance computational infrastructure does not show the complexity necessary for evolution, as a resource for national health care capacity building, as an information/data clearinghouse, or as a resource for distal health care system [node] capacity building. As it stands, the LlPHEAlHEAL TH website at MUSPH is a useful archive on historical LlPHEAlHEAL TH Alliance reports and program documentation, but it serves very little national or regional public health development purpose. For instance, it is not open-sourced and not connected directly to national or regional government authorities in charge of national disaster management. The disaster management instructional materials it contains are not readily accessible by the "public" directly affected by disaster. Hence, its use as a public health policy tool is extremely limited and rather questionable. The website represents a first generation development, and is in need of substantial, on-going technical assistance and development to reach its potential as originally envisioned. Establishing a Local Web-hosting Environment In 2009, Tulane received supplemental funding from HED to migrate the LlPHEA server from Tulane in the U.S. to MUSPH in Uganda. The overall objective of this special one-year partnership between Tulane and MUSPH was to transfer the ownership, management, and maintenance of the LlPHEA website, also referred to as the "Knowledge Center." The transfer of the website empowered MUSPH to lead the effort in activating and developing an e-Network among the seven member schools of the HEALTH Alliance. The primary achievements and outcomes under this short-term collaboration included: JBS International p. 15 Leadership Initiative for Public Health in East Africa Outcomes Assessment • Identification, purchase, and delivery of equipment to establish a flexible regional web-hosting environment at Makerere University; • ICT needs assessments of the seven HEALTH Alliance institutions; • Presentation of outcomes at the 6th Deans and Directors meeting in Dar es Salaam, Tanzania; • Successful installation and configuration of equipment and migration of all existing LlPHEA web applications to the MUSPH development server (both the LlPHEA Knowledge Center and new HEALTH Alliance site can now be downloaded ten times faster by local users); and • Training of 15 ICT representatives from the HEALTH Alliance schools during two workshops and one on-line certificate course. According to the Tulane partners, the establishment of a high-quality regional web-hosting environment at a LlPHEA member institution was a critical step in building the capacity of the LlPHEA network to host and further develop shared web applications and resources. These web applications were intended to enhance connectivity among LlPHEA members, thereby facilitating collaboration and communication between institutions. leT Assistance The two African Partner Institutions worked closely with JHU and Tulane to map out a program of ICT assistance. The first step was an assessment of basic ICT needs, and provision of ICT equipment: CPUs and a Medical Illustration Unit at Muhimbili and CPUs at Makerere. Specific activities completed under LlPHEA with Tulane's assistance include: • Specified, purchased, and delivered hosting equipment for website. • Specified and purchased equipment for classroom-in-a-bag. • Completed an assessment of six of seven LlPHEA institutions.24 • Presented preliminary findings from IT assessment.25 • Strengthened MUSPH infrastructure and human resources: o Configured a functioning hosting environment at MUSPH; bandwidth managed, production web server, virtualization server, capable of hosting application for HEALTH partners. o Produced a help desk application http://helpdesk.musph.ac.ug. o Produced strategy documents to formalize IT processes at MUSPH: http://helpdesk.musph.ac.ug/blog. • Migrated all LlPHEA Web applications successfully to the MUSPH development server located in Kampala.26 • Cached LlPHEA Knowledge Center and the new HEALTH Alliance Site for lOx faster page load speeds for local users. • Created low bandwidth versions of the sites to be accessible via the Finch caching service and prominently added a "Mobile or Low Bandwidth" link to the front pagesP • Brought together nine total ICT representatives from six of the seven LlPHEAlHEAL TH Alliance institutions to participate in the LlPHEAlHealth Alliance E-Learning and Networking Technologies Workshop: Building a Regional Community of Practice.28 • Identified and assembled a regional "Public Health IT" Community of Practice. • Identified priority domains for future collaborative learning and development of shared knowledge assets.29 24 Draft assessment report can be found online at: http://ljphea.org/libraryllibrary-documents/surveys/web-tech-and-capacity-assessmentlview. 25 Meeting materials and presentation can be found online at: http://liphea.orglliphea-in-action/conferences/fifth-deans-and-directors-meeting￾kampala-uganda. 26 These can be found at: http://liphea.org. http://courses.liphea.org. and http://media.liphea.org. 27 High band: http://halliance.org ; Low band: http://finch.ploogy.netlfinch/halliance.org I. 28 See http://halliance.org/groups/LlPHEA%20web%20and%20e-learning%20workshop. JBS International p. 16 Leadership Initiative for Public Health in East Africa Outcomes Assessment • Held web conference meetings among HEALTH Alliance IT personnel by the end of 2009. • Developed a development version of the HEALTH Alliance website hosted at MUSPH: http://halliance.org. • Trained six participants in the use of open source authoring tools for online collaboration, web authoring and the creation of e-Iearning modules, and a series of example course modules provided to course participants.3o • Strengthened capacity of the web publishing team to process, feed, and support existing web applications. • Identified and trained tech services and help desk entity in HEALTH Alliance site admin at MUSPH. • Strengthened capacity to independently maintain and develop new web hosting environment through training in production of documentation) I Two outputs of ICT assistance need special mention. First, the classroom-in-a-bag equipment and the associated training workshop were key steps in establishing and formalizing a regional web publishing team and process. The classroom-in-a-bag was designed to make the capture of courses, presentations and web-based conferences more straightforward and to facilitate the dissemination of these knowledge assets over the web via the LlPHEA Knowledge Center, the LlPHEA Course Commons and other shared LlPHEA web resources. Second, the establishment of a high quality regional web hosting environment at a LlPHEA member institution is a critical step in building the capacity of the LlPHEA network to host and further develop shared web applications and resources. These web applications will help maintain connectivity among LlPHEA alliance members, by facilitating collaboration and communication among institutions. Also during 2009, an ICT assessment was undertaken to provide a thorough technical analysis of ICT achievements up to that point, and to identify areas in need of further development within the LlPHEAlHEAL TH Alliance information and communications network systems. The identified issues were used to guide equipment specification and training priorities for capacity building moving forward. Findings of the 2009 assessment were reviewed by HEALTH Alliance members at the 5th Deans Conference. Currently, there are no readily available resources that have been identified by MUSPH, MUHASISPHSS, or the HEALTH Alliance for continued support to ICT systems development. The MUHASISPHSS ICT Director expressed gratitude for LlPHEA support, and stated that his department has developed a comprehensive strategic plan describing near and long term vision and assistance requirements. Findings & Lessons Learned LlPHEA was highly successful in establishing a functional legacy organization to continue the LlPHEA vision. A major achievement of LlPHEA was strengthening the LlPHEAlHEALTH Alliance capacity in management of information, communications, and instructional technology systems. There have been major inputs of equipment, systems development with impressive results. Resident ICT technical resource people (and champions) remain limited, but an information culture has been "seeded," and demand for all types of information and instructional technology is high. Assistance in building information systems and instructional technology was a good investment under LlPHEA, and would be a worthwhile investment for future institution strengthening programs for aSSistance-ready schools of public health in East Africa. 29 See http://halliance.or'l/groups/LiPHEA%20web%20and%20e-learning%20 workshop/synthesis/Meeting%20Synthesis%20HEAL TH%20Alliance%20Priorit;y%20 ICT%20Domains.html. 30 http://halliance.org/groupslteci. 31 http://helpdesk.halliance.orgldocumentation/web-application-administration/document.2009-0920.6187644783. JBS International p. 17 Leadership Initiative for Public Health in East Africa Outcomes Assessment The LlPHEAlHEAL TH website at MUSPH is a useful archive on historical LlPHEAlHEAL TH Alliance reports and program documentation, but it serves very little national or regional public health development purpose at this time. The website represents a first generation development, and is in need of substantial, on-going technical assistance and development to reach its potential as originally envisioned. Furthermore, LlPHEA leaves a legacy institution in place for the future. In keeping with its stated objective to "create a network," LlPHEA successfully created and institutionalized a regional academic network to address leadership gaps and other challenges in public health program management. Shortly after inception of LlPHEA in January 2006, the LlPHEA Partner Institutions had convened a first ever meeting by October 2006 of deans and directors of selected Schools of Public Health in East Africa to discuss possible long-term collaboration on activities and opportunities that might be of mutual interest and benefit. This initiative eventually led to the formal establishment of a regional alliance of seven public health institutions in six East African countries (Kenya, Rwanda, the Democratic Republic of the Congo, and Ethiopia in addition to Uganda and Tanzania) known as the "Health Education Alliance for Leadership through Health (HEALTH)." The HEALTH Alliance, as it has generally come to be known, is a chartered organization that was registered in Uganda in 2008. The Alliance can receive and disburse funds, effectively transforming the LlPHEA grant-based partnership involving only two African institutions into a permanent membership organization providing a regional public health platform from which to operate. Objectives of the HEALTH Alliance, similar to those of its parent LlPHEA program, are to generally promote leadership competencies in public health; to strengthen the public health educational capacities of partner institutions through academic collaboration; to improve institutional infrastructure for teaching, particularly in improved use of information and communications technology; and to serve as a technicallteaching resource in addressing pUblic health development problems in the region. During its past three years of operation, the HEALTH Alliance has demonstrated potential for long-term sustainability, albeit somewhat fragile. In the near-term, the HEALTH Alliance has increasingly gained local, regional, and international recognition as a committed and coordinated resource for research, leadership development, information technology and training capacity in a range of public health areas, including emerging and pandemic diseases, emergency and disaster management, and preventive health care. Although the HEALTH Alliance is correctly described as an outgrowth of early deliberations of the Deans and Directors Meetings and is generally thought of as a strictly regional phenomenon engendered by the future vision of the Deans and Directors, the U.S. partners also played a role in these early developments by encouraging and fostering LlPHEA entrepreneurship. In the best spirit of regional cooperation, the periodic meetings of Deans and Directors, in effect, functioned as the HEALTH Alliance's management committee. As such, this may have substantially substituted for and reduced the level of strategic and technical guidance that may have been otherwise forthcoming from the U.S. partners. JBS International p. 18 Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective Two. Create an Enabling Environment in Tanzania and Uganda for Public Health Activities by Building Leadership and Advocacy Skills among Public Health Faculty and Key Decision-makers LlPHEA Influence In the first formative two years of the LlPHEA program, the two African Institutional Partners proactively built working relationships with government and other development partners by . championing the importance of leadership development in general, and public health leadership in particular. During that period, LlPHEA presented leadership development concepts in small workshops and large conference settings to over 770 parliamentarians, public health professionals, and academicians. While primarily for the purpose of raising awareness and demand for leadership development training, these many forums were also opportunities for LlPHEA to better understand its subject matter and the needs and interests of its audiences. However, the LlPHEA partners did not develop, to the assessment team's knowledge, any systematic procedures that solicited guidance from government or development partner participants as potential users/employers of trained public health leaders and managers. LlPHEA offered other leadership development training courses and workshops (undergraduate and in￾service) during these formative years of the program. While needs assessment activities to determine the focus and content of training took place, those activities may have been targeted to potential public health leaders/managers themselves, not to government or development partners as potential users/employers of trained public health professionals. The determination of which leadership development courses to offer appeared to have been based primarily on the best technical judgment of course developers. Many of the leadership development courses were offered only once and not repeated, possibly suggesting low demand and/or limited known relevance to potential employers or sponsors. In one instance, what appeared to be a critical course on publiC health financing was never fully developed. Such a course might have been of high demand to government and development partners, but it was not clear if course developers had conferred with government or development partners regarding the need/interest in this course offering. The apparent lack of direct consultations with government and development partners on the focus and alignment of LlPHEA's leadership development training program may serve to explain why the program appeared to flounder, to some extent, toward the end of the first two years of the program. Demand for undergraduate public health education, including leadership development subject matter, was, and continues to be, high; however, on-going demand, and financial support, for generic post-basic and in￾service public health leadership development training was limited. The USAID-funded Health Emergency Management Program (HEMP) thus presented a timely opportunity for LlPHEA to make a strategic shift away from generic post-basic leadership training to more practically-oriented public health training, with leadership skills development built into the curricula. The realities summarized above serve to illustrate a vital need existed for much more strategic and technical gUidance to assist LlPHEA in these formative stages to strengthen the developmental approaches of the program. Three factors may have been at play: • USAID grant-funding and grant management guidelines caution against direct and substantial . involvement by the grantor in the strategic/technical decisions of the grantee so long as overall performance is satisfactory and objectives appear to be achievable. In the case of LlPHEA, JBS International p.19 Leadership Initiative for Public Health in East Africa Outcomes Assessment although jHU was a technical partner, it also functioned as an intermediary grantor. Even if USAID, HED, or jHU had recognized some critical strategic/technical weaknesses in the way the program was approaching a task, the grantors were potentially constrained by the substantial involvement rule. • As technical collaborators, the U.S. Partner Institutions clearly had opportunities to share certain strategic principles with LlPHEA planners, however, for whatever reasons, it does not appear that these more strategic and technical approaches to ensure the relevance of public health development training were tackled effectively. • And finally, the notion of regional cooperation may have also hindered direct intervention by the U.S. partners in strategic approaches of LlPHEA. LlPHEAlHEAL TH Alliance The LlPHEAlHEALTH Alliance HEMP training program in Kenya, Tanzania, Uganda, and Rwanda during Years 3-5 presented very different scenarios regarding the degree of working collaboration between the LlPHEAlHEAL TH Alliance network and governmental and nongovernmental implementers. The design and development of the HEMP training program in close coordination with governments in HEALTH Alliance countries presented an opportunity for the LlPHEAlHEALTH Alliance to apply its core emphasis on leadership development to a technical area of public health service delivery. Many interviewees confirmed that the theme of leadership development was never abandoned or lost. Rather, leadership skills were redefined within the context of the more technically-oriented emergency and disaster management training content. For example, disaster management trainees were required to develop work plans which required them to learn the principles of organizing others and/or working under the leadership of other coordinators/leaders. In 2009, USAID personnel were instrumental in encouraging the LlPHEAlHEALTH Alliance to transition the focus of HEMP to epi-zoonotic emergencies in public health disaster management training. To meet the capacity requirements of the new program focus on epi-zoonotic diseases, district training teams were expanded from six to nine members as each alliance country added three epi-zoonotic experts in veterinary medicine and other fields. As a result, the University of Nairobi opted to form new alliances with a direction for disaster management training that it believes to be more in tune with opportunities and markets; other Alliance members prefer not to stray too far away from LlPHEAlHEALTH's broad and historical core emphasis on leadership in public health. Overall, a total of 1,005 multi-sectoral personnel from 161 districts in regions that are at high risk for disasters were trained in leadership in health emergency and disaster response.32 In addition to providing LlPHEAlHEAL TH Alliance with a platform to continue its core leadership development theme, HEMP also allowed the HEALTH Alliance to make direct programmatic connections between research, policy, and practice. For example, the HEALTH Alliance had an opportunity to carry out research on the disaster-related effects of drought on malnutrition in children. While HEALTH Alliance involvement in this disaster-related research activity demonstrated LlPHEA's capacity to make linkages with these types of programs, it also highlighted the fact that LlPHEA performance was very limited in this area. LlPHEA might have strengthened the content, validity, and relevance of its training, and provided learning opportunities for its undergraduate and graduate students through more strategic linkages between training and research opportunities. However, the approach does not appear to have been pursued with any vigor by LlPHEA. Not only does this signal the possibility of lost opportunity, but also speaks to an absence of technical guidance from those who have extensive experience with these approaches. 32 Source: LlPHEA Closeout Report FINAL, May 20 II. jBS International p.20 Leadership Initiative for Public Health in East Africa Outcomes Assessment Disseminating Knowledge In another area of influence, a course for journalists was developed to improve their technical understanding of public health, and improve the ability of these journalists to accurately portray public health issues to readers and media consumers. Makerere University has a page in a local newspaper, as well as full-time employees who are dedicated to transmitting scientific evidence to the media community. More than 24 articles and radio shows have highlighted the work of LlPHEA in Tanzania, and even more have been published in Uganda. In addition, another LlPHEA program outcome was the establishment and publication the East African Journal of Public Health (EAJPH). Thirty issues with 182 articles from 26 countries worldwide have been published and supported by LlPHEA since October 2005 and can be accessed through African Index Medicus, African Journals Online, Bioline International, PubMed and CABI Global Health Abstract Database. Countries that have contributed include Argentina, Australia, Bangladesh, Central African Republic, China, Democratic Republic of Congo, Ethiopia, the Gambia, Germany, Ghana, India, Iran, Kenya, Malawi, Morocco, the Netherlands, Nigeria, Norway, Qatar, South Africa, Swaziland, Tanzania, Uganda, United Kingdom, the United States, and Zambia. It should be noted as a caveat to the enormous success of the EAJPH, concern was expressed that the EAJPH appears to be dominated by MUSPH and MUHASISPHSS faculty. The two universities are over￾represented in Board membership (practically more than 75 percent of the Editorial Board members are either MUSPH or MUHASISPHSS faculty members). Although academic value neutrality is expected, this over-representation seems to have affected editorial decisions to favor publication of submissions by faculty and graduate students from the two institutions. A review of the content of the latest issue of the journal confirms this perception. Findings LlPHEA is generally recognized as having contributed to increased awareness and demand for effective leadership in Schools of Public Health and among public health stakeholders in the region. LlPHEA has enabled thousands of students, faculty and professionals to be exposed to various aspects of leadership development within the context of public health program management. The regional reach of LlPHEAlHEAL TH Alliance activities and the program's ability to raise awareness and increase knowledge and skills of public health students and professionals in six countries, the region, and globally, are notable achievements. The outputs and numbers of people whose personal and professional lives have been touched by LlPHEAlHEALTH Alliance are well-documented and borne out by numerous anecdotal accounts of people that were interviewed during this Assessment. There are indications that LlPHEAlHEALTH Alliance HEMP had a verifiable programmatic spread effect in Uganda and Kenya. In Uganda, training of senior government personnel was instrumental in formation of the government's disaster management office. At Moi University in Kenya, HEMP training had migrated to several other programs within the university (e.g., certificate, diploma, and e-Iearning courses), and the program was in high demand among publiC health professionals, government, donor partners, and nongovernment public health stakeholders. The differences in the approaches and results found in each country are discussed below. Makerere University MUSPH/Uganda clearly demonstrated its capacity to engage government interest at the most senior levels and be responsive to government's needs and interests. During LlPHEA Years 2-3, MUSPH/Uganda was an effective catalyst for bringing disaster management to the attention of JBS International p.21 Leadership Initiative for Public Health in East Africa Outcomes Assessment government policy makers. Uganda's Department of Disaster Management, which is in charge of risk reduction, preparedness and response, is headed by a senior officer in the Office of the Prime Minister (OPM) who was trained as a trainer, in the LlPHEAlHealth Alliance techniques for disaster response management and has been substantially influenced by the LlPHEA leadership and disaster management training programs. His training is credited for broadening his knowledge on policy issues, public health issues, disaster management and newly emerging diseases. He sees himself as a teacher/trainer and catalyst for policy development and advocacy for disaster management, and is now a spokesman and advocate for leadership development, disaster management policy and program development with his own leadership and among peers. MUSPH now has a solid working relationship with the Office of the Prime Minister. The School participates in the national Disaster Management Forum which allows the School to stay informed about the training priorities of government and other stakeholders. The School is also seen by the OPM as a technical resource. For instance, when OPM recently undertook its own internal assessment of staff training needs, it approached the School for suggestions about how to design the study. Muhimbili University Although MUHAS/SPHSS has delivered a productive disaster management training program in Tanzania with linkages at the technical level in the Ministry of Health, MUHAS/SPHSS' dialogue and working relationship with government at the level of the Prime Minister's Office (PMO) appeared troubled. Overall, the PMO spokesman emphasized that the LlPHEA training had been relevant and helpful; it had opened minds to the principles of holistic, multi-sectoral coordination. However, MUHAS/SPHSS and the disaster management training program missed appropriate linkages and on-going dialogue with the PMO. Once the training program was approved by government, MUHAS/SPHSS proceeded to carry out its activities without adequate and continuing liaison with the PMO. The PMO had concerns about resulting gaps in the training content as well as dissatisfaction with some aspects of fund administration. In particular, the PMO was concerned that the MUHAS/SPHSS training did not adequately cover the government's policy and operational framework, and did not appropriately engage the government as a teaching resource. This dissonance between MUHAS/SPHSS and the Prime Minister's Office was mirrored in the impressions of one agency in the donor community. A representative of that agency felt that MUHAS/SPHSS had not developed a sufficiently rich dialogue with the PMO or marketed its productivity in a way that would ensure its continued involvement in the next generation of donor-supported disaster management training. As one officer on the PMO's side said, "There appeared to be a fundamental difference of opinion on conceptual ownership [of the training], who is in charge, and who invites whom to participate." This officer went on to say tha~, in donor-supported development activities, the PMO's office must be seen as more than just an endorser of a program. The Office must be involved at initial stages of an activity when the grantee first begins dialogue with a potential donor. These perceptions and issues point to differences in Tanzania on how the government and the higher education institution view their roles and each other, with the university viewing its role as an academic authority on the subject training, while the PMO's office viewing MUHAS/SPHSS as a training contractor who should have been obligated to work with and through its office. Moi University In Kenya, the MUSPH/Kenya forged successful working relationships with the Ministry of Provincial Administration and Internal Security, the National Disaster Operations Center (NDOC), and the Ministry of State for Special Programmes related to in-service training of government and nongovernment workers in health emergency and disaster management. Government informants reported that HEMP offered a high quality in-service training program that is of strategic relevance to the government's development objectives. As part of the training, each district team was to produce a JBS International p.22 Leadership Initiative for Public Health in East Africa Outcomes Assessment disaster management plan. Districts are accountable to NDOC for submission of plans to the Center, and disaster management coordinators refer to plans when a disaster occurs. The overall picture was quite positive, and MUSPH/Kenya should be able to build upon existing working relationships with government for continued or similar future public health development training and/or technical services (e.g., public health research). MUSPH/Kenya likely will be able to sustain its strategic relevance and the marketability of its public health education and other technical services into the future. Government officials interviewed were in general agreement about the need for additional disaster management training (e.g., preparedness, risk reduction and expanded coverage at district and sub-district levels), and other public health training needs may also emerge. MUSPH/Kenya is well￾positioned to become a key training and technical resource for these developments. University of Nairobi In stark contrast, there appears, at best, to be "distance" in the working relationship between the University of Nairobi (UoN), School of Public Health and central policy makers in the Ministry of Public Health and Sanitation (MOPHS). This impression comes from senior officials in MOPHS who felt that in the case of health emergency and disaster management, the School was not moving in a direction that was relevant to Ministry's' needs. The Ministry did not specifically discredit the importance of training in disaster management; however, in MOPHS's view, the most important contribution the health ministries can make to health emergency and disaster management is to improve Kenya's overall national capacity for forecasting, early warning, and risk reduction in health-related disasters. The Ministry apparently does not believe the UoN School of Public Health is "on board" with this objective, and instead works with the University Department of Public Health and the U.S. Centers for Disease Control to build applied epidemiological capacity through its Field Epidemiology and Laboratory Training Program. Still, MOPHS is engaged with the rest of the Kenyan government on disaster management, and has close ties to the NDOC and has a full-time health officer seconded to the Center. However, when it came to identifying a training resource for disaster management training, the MOPHS referred UNDP to Moi University in preference to the University of Nairobi. For its part, the UoN School of Public Health did not participate in the LlPHEAlHEALTH Alliance HEMP training at all, and claimed not to be interested due to staff shortages and competing priorities.33 National University of Rwanda School of Public Health By the end of LlPHEA in March 20 I I, the training program had not yet been initiated because the government required the School to enter into a memorandum of understanding (MOU) to gain the necessary authorities to conduct the training. In this case, there were not only communications constraints, but bureaucratic constraints in partnering with government. Lessons Learned & Recommendations In structuring its leadership development training program, LlPHEA did not appear to have adequately consulted with government and nongovernment implementers (as potential users of personnel trained in the program). As a result, LlPHEA's generic post-basic in-service leadership development training program did not create effective and sustainable demand, and needed to be revamped to address needs/interests that were more relevant to government and development partners. In development and delivery of HEMP training (during LlPHEA, Years 3-5), there was considerable variation in the nature and extent of consultations initiated by participating LlPHEAlHEALTH Alliance schools with government and development partners, resulting in varying degrees of success (quality, use 33 An underlying issue related to UniverSity of Nairobi ultimately not participating in the HEMP training program may have concerned delays and difficulties with LlPHEA funds flow during that period (see p 55). JBS International p.23 Leadership Initiative for Public Health in East Africa Outcomes Assessment and sustainability) of the country HEMPs. Recommendation: Lead partner institutions may consider developing guidelines for partners' consultations with government and other partners to ensure consistency of approach and implementation of effective practices across the program. As relates to both leadership and HEMP training, there did not appear to be effective mechanisms in place to identify needs and provide on-going assistance to participating schools, as necessary, in forging sound working relationships with government and development partners. Although the U.S. Partners OHU and Tulane) had opportunities during LlPHEA instructional technology training to make some inroads in terms of strategic/technical approaches to design/development of public health development training programs, there may have been some reticence to do so. The regional nature of the partnership as well as USAID grant-funding and grant management guidelines that caution against substantial involvement of grantors may have been presented constraints. Recommendation: In partnership programs that embrace a regional approach, USAID and HED may wish to consider ways in which host country partner institutions can more readily benefit from strategic and technical guidance that might be needed and forthcoming from U.S. Partner Institutions. Partnering roles and responsibilities related to provision of such guidance may need to be made more explicit in grant agreements, if possible. Alternatively, such understandings may need to be established in the working relationships of the partners. JBS International p.24 Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective Three. Strengthen Teaching and Educational Programs that Integrate Leadership Training throughout the Curricula Relevance of Training A major challenge of leadership development is to impart not only knowledge and skills, but most importantly the motivation that will help the newly trained and re-deployed leader to use the newly￾acquired skills to improve the performance of an organization. The key objective of leadership training should be to ensure the curriculum includes practical knowledge and skills that will aid the leader in working effectively in a challenging public setting. The accomplish this objective, a first stage in the development of the LlPHEA program was a meta￾analysis was conducted by MUSPH and MUHAS/SPHSS research teams in 2003-2004, to assess the state of public health education and practice in sub-Saharan Africa. The meta-analysis identified lack of formal instruction on leadership at the university level and failures of practical leadership in public health practice as the major gaps in public health education. The meta-analysis was based primarily on a synthesis of results from various studies and inputs from schools of public health experts, and included some dialogue with government and nongovernmental implementers although such dialogue may not have been systematic or strategic. Once the LlPHEA project was inaugurated three other major situation analyses were completed: • A Study of Leadership and Management in Uganda and Tanzania (October 2006) focused more specifically on training/teaching needs; • Disaster Preparedness and Response in HEALTH Alliance Countries Ouly 2008); and • A study of epi-zoonotic diseases in HEALTH Alliance countries Oune 2009). All three studies received input from a range of public health profeSSionals and stakeholders. As with the formative meta-analysis discussed above, the degree of systematic or strategic involvement of government and nongovernment implementers as informants in these various studies was not explicit or obvious. Undergraduate, Post-basic, and In-service Training As described in the LlPHEA Final Report, leadership development was incorporated into the teaching and educational programs at both primary African Partner Institutions. This was achieved by restructuring curricula to provide a greater emphasis on leadership skills for public health graduates, and to provide short courses for public health practitioners to increase their leadership capacities. At MUSPH, this led to leadership being included as a module in the Environmental Health undergraduate curriculum and in the Master of Public Health (MPH) program. In Tanzania at MUHAS/SPHSS, leadership is a compulsory two-term undergraduate course which 5,095 undergraduate students have completed and continues without the need for additional funds. In total, over 5,330 students, faculty and practitioners in Tanzania and Uganda have been trained in leadership. LlPHEA has also trained a JBS International p. 25 Leadership Initiative for Public Health in East Africa Outcomes Assessment total of 264 in-service public health practitioners from Uganda and Tanzania in leadership and management. Participants for these trainings came from districts throughout both countries and included members of the Ministry of Health, NGO workers, physicians, nurses and community health workers. Moreover, large-scale events were hosted at both institutions to engage stakeholders and policy makers. For example, over 566 Ugandans and Tanzanians attended a leadership lecture by Tulane President Emeritus, Eamon Kelly, and over 212 participants in Uganda attended a lecture on leadership by Israeli Ambassador Jacob Keiser. Development of Curriculum During Years 1-3, the formative stage of the program, MUSPH, MUHAS/SPHSS, and LlPHEA stakeholders reached an agreement on a training curriculum that focused on strengthening leadership capacity among public health practitioners and institutions as training hubs for leadership courses. Careful consideration was given to (I) course offerings, (2) course content, and (3) course themes. Overall course design was geared towards public health challenges and emphasized and required a teamwork approach to learning. Courses on (I) Leadership in Public Health in East Africa, (2) The Media and Communication, (3) Leadership in Quality of Health Care, and (4) Strategic Leadership in Health were developed and offered. According to the LlPHEA Director at MUHAS/SPHSS, one of the major achievements of LlPHEA was obtaining high-level institutional approval for the leadership curriculum and integration of curriculum modules (e.g., Leadership in Public Health in East Africa and Leadership in Quality of Health Care) into university undergraduate and graduate degree programs. At MUSPH/Uganda, a leadership module was added to the undergraduate Environment Health curriculum and to the MPH degree program. However, it was not clear if training modules were adjusted to variations in the educational levels and levels of comprehension of undergraduate students. In a number of cases, districts in Tanzania did report completing post-training assessments at the post-basic training level, no statistical or substantive evidence of training curriculum-related pre-tests and post-tests of participants was readily available. This lack of information made it impossible to determine levels of comprehension and variation in training achievement among trainees in terms of their educational levels. One course on health care financing, "Investing in Health," was considered but not developed. This course was rated as "not a priority" by public health practitioners and potential trainees during a needs assessment, and was consequently dropped from the leadership training curriculum. Given the relevance of health care financing to public health leadership and the extremely high priority generally placed by government and development partners on health care financing competencies, the low rating of the course is difficult to understand. One possibility for this low rating is the sample of informants used in the needs assessment design and process. The sample may have been biased toward input of professionals and potential trainees, rather than potential users of personnel (Le., government and nongovernmental public health implementers) where demand would likely have been quite high. HEALTH Alliance Public Sector Health Training JBS International p.26 Leadership Initiative for Public Health in East Africa Outcomes Assessment Leadership training for multi-sector district teams as part of the extended funding for LlPHEA was started in 2007 and introduced new roles for the Schools of Medicine and Veterinary Medicine at both MUHAS and Makerere University. Needs assessment at the outset indicated a critical need among district-based personnel for a shift from leadership training to disaster response management and leadership training. To establish detail for such training, a baseline analysis was conducted by Dr. Christopher Orach of MUSPH to identify knowledge gaps at regional and district administrative levels. The analysis led to a focus on upper-level and middle-level district officers and the design and implementation of two-week training programs using a carefully crafted instructional manual entitled Public Health Emergency Planning and Management: Training Manual for Disasters in Eastern Africa. The manual provided instruction on (I) disaster management theory and terminology, (2) relevant disaster management policy, and the (3) development of draft district disaster response plan. In line with the LlPHEAlHEALTH's paradigmatic focus on competency training, half of the training on disaster management was of a practical nature. In 2008, this manual and curriculum materials on managing health emergencies at district level were developed among faculty at HEALTH colleges and consolidated into HEMP, which became an integral part of the overall LlPHEAlHEAL TH program. Of note, the HEMP curriculum adopted an approach which included all emergencies (all hazards). Over time, continuous assessment of training needs led to the development of a Risk Reduction curriculum which included hands-on simulations (e.g., First Aid, fire extinguisher use, management of epidemic outbreaks). Courses developed in the LlPHEAlHEALTH program have been used by organizations outside of the Health Alliance including other East African universities, the European Union program on HIV, and Moi University partners such as African Medical Research Foundation. Demand for disaster management training extended beyond the traditional public health community and was requested by the Ugandan military, Ugandan Anglican Church (Bishops) and the Ugandan Catholic Church (clergy), all of which contracted with MUSPH and received the training. Thus, the HEMP curriculum was popular and considered to be useful to a variety of public health organizations and stakeholders far beyond the intended target audiences. LlPHEAlAlliance Training Development Process The LIPH EAlHEAL TH Alliance training development process was reviewed for evidence of planners acknowledging the practical value and relevance of the training programs they were developing. The team found that during the planning phases of the training programs, those involved paid considerable attention to process detail. Needs assessments and evaluation served as the basis of the leadership curriculum design in Uganda and Tanzania, and the institutionalization and integration of leadership training in the broader university curriculum was a strategic mechanism for long-term public health leadership capacity building. This led to the formal inclusion of leadership-focused educational modules in undergraduate (at both MUSPH and MUHAS/SPHSS) and graduate (at MUHAS/SPHSS) learning curricula. The inclusion of these modules is a logical response to expressed or inferred needs/interests of the stakeholders who will eventually hire, supervise, or otherwise work with those persons trained through the program. Involvement of stakeholders in informing design/development of training programs was even more apparent at the post-graduate and in-service training level. While impact evaluation of LlPHEA leadership and/or HEMP training was beyond the scope of this assessment, compelling anecdotal evidence suggests that LlPHEAlHEAL TH Alliance training programs had practical value and have been appreciated as such by participants. A lecturer at MUHAS/SPHSS said that she had benefitted greatly from the instructional technology training she received which she now applies in the classroom. Another senior government informant recognized that the training he received had made a difference in how he approached his management tasks, and thought that future leadership training would have even greater demand and impact if more emphasis were placed on "managing resources, and mitigating regulations and JBS International p.27 Leadership Initiative for Public Health in East Africa Outcomes Assessment administrative procedures; more skills in managing constraints through change management." In yet another instance, the senior government officials who participated in the training-of-trainers program for HEMP training went on to form and manage the government's national health emergency and disaster response program. In Uganda, in spite of good and productive working relationships between MUSPH and OPM, the training program appeared to have a technical glitch that had not been adequately addressed by either party. All district teams were expected to draft disaster management plans once they returned to their work stations. These plans were supposed to be widely discussed and vetted locally, and forwarded to MUSPH. According to discussions with Chief Administrative Officers and district teams in the two districts visited, other district officers were not widely involved in discussing the plans, and it could not be determined if plans were actually completed or forwarded to MUSPH. Furthermore, the district teams as well as OPM felt that MUSPH should have followed up its training by monitoring progress on plan preparation. It may have been more effective for the responsibility for training follow up and monitoring for district plan preparations to be assumed by OPM, not MUSPH, since MUSPH served as a technical resource for training on behalf of OPM, but OPM oversees disaster management. OPM has the authority to demand accountability from district administrators, such as completion and submission of plans, while MUSPH does not. This was an instance where the considerable technical background and experience of JHU and Tulane might have come into play. Findings Government partners in two of the three L1PHEAlHEAL TH Alliance countries visited felt that all training had been useful, and had at least served the general purposes of increasing public health leadership and professionalism in their countries and the region. In Uganda, MUSPH had not only been responsive to government's needs/interest in increasing the numbers of trained health emergency and disaster management personnel, but had been instrumental in the development of the emergency and disaster response program. Government implementers such as the NDOC in Kenya appeared to be very satisfied with the relevance of the Moi University disaster management training. In contrast, the Office of the Prime Minister in Tanzania felt that MUHAS/SPHSS was not attuned to government operations or governance issues, and thus did not have the technical competencies to cover government policies, structures and administrative guidelines in its disaster management training curriculum. Aside from the advantages of user-informed competency-based training design, two other principles emerged from the L1PHEA experience. First, competency-based regional training programs need to consider regional disparities and have the capacity to customize country programs to needs/interests and constraints of the local setting. Informants observed that disparities in administrative structures in L1PHEAlHEALTH Alliance countries made it difficult to replicate training. In addition and to some extent, language differences impeded the free exchange of trained individuals and HEALTH Alliance faculty with the Rwanda and DRC because of English/French language differences. Secondly, regardless of how well in-service training programs have been constructed to meet the practical, competency-based needs of trainees, training results/outcomes will be diminished if the most appropriate trainees are not selected. "Appropriate" refers to those with the most capacity and motivation to learn, and most likely to be re-deployed to work sites where training can be applied. For example, in one district in Uganda it was observed that selection of district representatives for L1PHEAlHEAL TH trainings was left totally to the discretion of the local District Chief Administrative Officer without the benefit of any standardized criteria or gUidelines for selection. This might have increased the possibility of biased selection, or selection of trainees who do not have genuine interest or positive motivation to participate in the training program. JBS International p.28 Leadership Initiative for Public Health in East Africa Outcomes Assessment Lessons Learned & Recommendations When identifying training priorities, L1PHEA tended to confer with public health professionals and students and less specifically with governmental and nongovernmental implementers. This may have resulted, at least in part, in L1PHEA making a decision not to offer health care financing courses, thereby missing what may have been an opportunity to advocate for and assume leadership in development of Africa-specific health care financing expertise. Post-basic and in-service training provided by L1PHEAlHealth Alliance was viewed as useful, practical and relevant in two of the three countries visited, and L1PHEAlHEAL TH Alliance planners used technically sound, standard training program design/development methods including needs assessments. L1PHEAlHEAL TH Alliance training program planners in the third country might have benefited from strategic guidance and technical assistance in strengthening its coordination with implementers (in this case, government). Training designs could have been further strengthened by involving those stakeholders with the best understanding of worker competencies such as potential employers or supervisors. Recommendation: In the interest of optimizing post-basic and in-service training investments, USAID and HED should ensure that grants include a mandate, technical assistance and resources for training program developers to optimally engage government and nongovernment implementers and effectively incorporate their needs/interests into design and development of competency-based training. Criteria for systematic selection of in-service trainees appeared to be weak or absent in the civil service systems from which trainees were drawn. Regardless of how well in-service training programs have been constructed to meet the practical, competency-based needs of trainees, training results/outcomes may be diminished if the most appropriate trainees are not selected. Recommendation: In the interest of optimizing post-basic and in-service training investments, USAID and HED should ensure that grants include a mandate, technical assistance and resources for training program developers to enter into effective dialogue with government and nongovernment implementers to standardize structures and procedures for selection of training program participants. Regional competency-based training programs need to consider regional disparities and have the capacity to customize country programs to needslinterests and constraints of the local setting (such as differences in governance, governmental administration and structures, and language). Recommendation: In support of regional training programs, USAID and HED should ensure that grants include technical assistance and resources to assist implementers in effectively customizing regional approaches and curricula to meet local country requirements. JBS International p. 29 Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective Four. Establish a Faculty Development Program that will Sponsor Promising Young Public Health Faculty from MUSPH and MUHAS through a Combination of Degree Programs and In-service Programs in Public Health Leadership Program for Faculty Development The existence of a formal faculty development program was not observed, although there was evidence of some strategies being undertaken. Dr. William Bazeyo, the LlPHEA Host Country Partnership Director, described MUSPH as being concerned with mentoring and encouraging career advancement of selected faculty members and with engaging faculty as team members in the growth and development of the school and its programs. Mentoring is viewed as an important tool for retaining skilled talent and building a future generation of leaders within MUSPH. Between 2006 and 2008, two faculty members were supported for sabbaticals at Johns Hopkins and Tulane. They concentrated on instructional technology and use of electronic tools in instructional design. The assessors attended separate presentations by each of these individuals, and were able to observe their acquired competencies in e-Iearning methods which each readily attributed to the training that had been received through the LlPHEA program. However, two sponsored sabbaticals in five years appears to be inappropriately low, given the overall resources available to LlPHEA. Faculty Training All HEALTH Alliance schools offered short-term training to faculty in order to improve technical skills in teaching disaster response and management to their students and communities. An online Enhancing Teaching and Learning course was created to help faculty to develop engaging and effective curriculum using the latest in teaching methods. The online course, developed by Johns Hopkins University and Tulane University, has now been rolled out to all seven schools of public health in the HEALTH Alliance through a training-of-trainers program. The curriculum for Enhancing Teaching and Learning provides a step by step pathway to develop effective and engaging courses. Modules include: course design; course content; course delivery; course assessment; and course evaluation. In addition, a learning-to-Iearn program developed by Tulane has helped many faculty in all of the HEALTH Alliance schools to make better use of electronic and technical materials in their lectures. Findings Although many productive teaching, teaching infrastructure improvements, and staff development activities were undertaken, no formal faculty development plans complete with long and near term objectives and action steps were described or otherwise detected by the assessment team. In this respect, LlPHEA did not meet a key program objective of establishing a formal staff development program. The fact that a formal staff development plan was not developed over a five-year period is a program management oversight that should have been picked up through the program monitoring and evaluation process. Lessons Learned & Recommendations Each of the primary partner universities should have instituted staff development plans over the life of the activity. LlPHEA might have strengthened the content, validity, and relevance of its training, and provided learning opportunities for its undergraduate and graduate students and research opportunities for professors and graduate students through more strategic linkages between training and research JBS International p.30 Leadership Initiative for Public Health in East Africa Outcomes Assessment opportunities. However, this approach does not appear to have been pursued with any vigor by LlPHEA. Not only does this signal the possibility of lost opportunity, but also speaks to an absence of technical guidance from U.S. Partner Institutions, which all have extensive experience with resource leveraging in resource-constrained academic environments. Recommendation: Lead partner institutions must ensure that the institutional development goals of the partnership are not lost in the push to deliver services to external partners. Early on in the partnership planning, institutional development goals should be linked with the external activities to the extent possible and should be given equal priority. JBS International p.31 Leadership Initiative for Public Health in East Africa Outcomes Assessment Objective Five. Improve the Teaching Infrastructure at MUSPH and MUHAS Teaching and Teaching Infrastructure Effective teaching is key to the success of students and trainees in acquiring knowledge and skills. As described in the LlPHEA Final report, activities were implemented to improve the technical skills of faculty to become more effective facilitators and educators in teaching leadership skills to students and course participants. There were also significant efforts to ensure that teaching of disaster management courses was effective. However, there appears to have been little focus on pedagogy or adult learning methodologies at any point. The timeline for the assessment precluded the team observing any significant sample of classes or training sessions in progress, so the team's impressions are based on what was reported to them by the stakeholders they interviewed at each university. The team was much more readily able to assess the changes in teaching infrastructure. The ability of SPHs to meet the great demand for public health training in the East Africa region was seriously hampered by very limited teaching facilities. Therefore, the U.S. and African Partner Institutions considered improvements in teaching infrastructure as critical in the LlPHEA program. To expand teaching infrastructure, computer labs were established at MUSPH and MUHASISPHSS, and more technology was made available to faculty for effective teaching. For example, after completing training in Instructional Design at Tulane University in 2007, Dr. Roy Mayega participated in introducing e-Iearning to the entire MUSPH campus. The assessment team observed the computer learning labs in use. Desks, tables, and chairs were purchased and resulted in faculty and students being able to learn in a more comfortable and conducive environment. In addition, the establishment of a LlPHEA and HEALTH Alliance website (http://halliance.org/) led to an electronic learning platform where schools of public health from Uganda, Tanzania, Kenya, Rwanda, DRC and Ethiopia can all share materials, courses, lectures, and events. The site now has the potential for establishing online courses throughout the region. However, toward the end of LlPHEA, the ICT systems, in general, had aged, and the potential for sustainability of this and other innovations had waned. A new infusion of resources and technical assistance would be required to build upon gains from this investment. In general, LlPHEA outputs, particularly ICT equipment, were in evidence in appropriate places and resources appeared to have been distributed where they were reported to have been sent. Funds received from the Rockefeller Foundation appear to have been distributed to HEMP programs in Rwanda, Democratic Republic of Congo, and Ethiopia as intended. Findings Although many productive teaching infrastructure improvements were undertaken, there was limited attention to improving teaching generally. The activities undertaken were focused on content and presume good adult learning practices are widely used, which mayor may not be a good assumption. While teaching infrastructure is very important, teaching tools cannot replace good pedagogy. LlPHEA may not have met fully the intent of the key program objective of improving teaching infrastructure. Lessons Learned & Recommendations Many productive teaching infrastructure improvements and staff development activities were undertaken. However, the absence of a staff development plan may have contributed to what appears generally to be a low level of staff and faculty training and development on technical teaching and training JBS International p.32 Leadership Initiative for Public Health in East Africa Outcomes Assessment methods. An interrelated plan that linked improvements in teaching infrastructure, instructional technology and staff development together would likely have improved outputs and outcomes for all three areas. Recommendation: Lead partner institutions must ensure that the institutional development goals of the partnership are not lost in the push to deliver services to external partners. Early on in the partnership planning, institutional development goals should be linked with the external activities to the extent possible and should be given equal priority. JBS International p. 33 Leadership Initiative for Public Health in East Africa Outcomes Assessment Additional Program Considerations Partnership Management Structure and Roles L1PHEA began with a relatively well-defined management structure that appears to have evolved in subtle ways over the five-year period as new dimensions of the program were introduced and new disciplines added such as medicine, and veterinary science. These changes to the originally envisioned structure, roles and technical support had an impact on the responsibilities of the U.S. consortium and appear to be agreed upon by the partners in line with the field-driven, regional principles of L1PHEA. As stated in its grant application, Johns Hopkins, as the primary grantee, had envisioned that its main role would be to support the priorities and activities selected by Muhimbili and Makerere that related to the program's goals. JHU did not foresee an invasive hands-on partnership relationship, reporting that it would "make technical and management support available as required by the program and support, and mentor those faculty attending Johns Hopkins and Tulane for further training." As described by the African Partner Institutions, both JHU and Tulane worked in close collaboration with their counterparts to jointly plan and agree on work plans and schedules. However, it was clear that priorities and technical direction were intended to be set by the African partners. Although this field-driven approach is consistent with the philosophy of a regional partnership, designers and implementers of such partnership arrangements might note that a hierarchy of advisory levels is even more critical in such complex organizations. The need for advisory levels perspective was built into the original management plan, but was modified in reality, and may have led to problems with the strategic direction of the HEALTH Alliance. A case in point is the apparent confusion, and some degree of frustration, that emerged over an unsuccessful bid by HEALTH Alliance for an USAID epi-zoonotic technical assistance project. A lack of perspective and strategic/tactical direction may have contributed to the unsuccessful bid, and might have been predicted from the subtle changes in L1PHEAlHEALTH Alliance's organogram where strategic consultation with U.S. partners, perhaps including HED and USAID, was less than optimal. The organizational structure of L1PHEA, as shown below, called for an Advisory Committee consisting of well-respected policy leaders and principal customers (Le., program graduates) who would assist a Management Committee in ensuring the training process remained needs driven. The Advisory Committee would be headed by President Emeritus Eamon Kelly. The Deans of JHU and Tulane Schools of Public Health would also be members, as would public health leaders who have risen to executive positions in government. The JHU Project Director, Gilbert Burnham, and the Tulane Deputy Project Director, Nancy Mock, would each devote 25-30 percent of their time to this program. Additional time would be set aside for faculty consultants who would include David Peters, Bill Bertrand, Bill Brieger and Sambe Duale. The Project Director would have overall managerial responsibility for the project and achievement of its goals and objectives. He would be the primary contact for HED. The Deputy Director would have responsibility for specifically overseeing the technical components, and arranging faculty support. A support manager in Baltimore would spend up to 50 percent of his or her time managing funds moved to the Africa partners, following up on reports, tracking project costs, and supporting faculty from the two schools studying in the U.S., either in doctoral programs or on sabbaticals. At the beginning of each year the work plan would be established and the budget agreed by all parties. At this point the support needed from the U.S. partners would be identified and scheduled.34 34 Leadership initiative for Public Health in East Africa. Grant Application and Proposal outline 26 August 2005. page 17. JBS International p.34 Leadership Initiative for Public Health in East Africa Outcomes Assessment Figure 2. LlPHEA Organogram 2005-2007 MUHAS!SPHSS U.S. Consortium Johns Hopkins and Tulane T ethnical Working Committees ] MUSPH/Uganda The partners anticipated that the rapid transition to an East Africa orientation built on shared resources (e.g., curriculum, degree programs, and human resources), comparative advantages (Le., niches) of the partners, a platform for sharing lessons learned, and joint strategic activities, was desirable. Year I strategic planning activities were to include key institutions from the region. From the partnership, a broader network of international and regional expertise would be mobilized based on initial planning. For example, expertise from the World Bank or the London School of Economics might be brought in to work on health sector financing and macro-economic planning. Since JHU faculty had taught their programs to policy makers through the World Bank Institute, and had hosted the World Bank Institute flagship training program overseas, notably in Afghanistan, the university might have served as a key resource in establishing similar programs within the LlPHEA program. JHU at that time was also a collaborator with the World Bank and borrowing countries on the analysis of health sector reforms and monitoring and evaluation of health sector assistance; such expertise and networks could be brought to bear for LlPHEA. Similarly, plans envisioned involvement of the business programs in Johns Hopkins or Tulane as needed. The stated intent was to approach input to strategic and technical direction of the program from a field-driven perspective.35 Compared to the foregoing vision, it appears that the AdviSOry Committee did not actually function as planned. The proposed Committee Chairman, Professor Eamon Kelly, keynoted two major LlPHEA conferences, and he and other high profile international public health professionals attended some of the Deans and Directors Meetings that began in October 2006. However, it could not be determined if the Advisory Committee was ever fully formed or if it ever purposefully functioned as an entity separate from the Management Committee. If such a convergence of adviSOry and management committee functions did occur, the program may have been denied the sustained energy, stature and high level public health development leadership that could have been brought by the high profile Advisory Committee. Functions of the Management Committee, as planned, also seem to have shifted in subtle ways. The Management Committee, as originally envisioned, would have technical and budgetary oversight and 3S Leadership initiative for Public Health in East Africa. JHU Application and Proposal outline 26 August 2005, page 8. JBS International p. 35 Leadership Initiative for Public Health in East Africa Outcomes Assessment approval authority over LlPHEA funds disbursed by JHU to the two African Partner Institutions. The Management Committee would have three persons from each of the two schools, and would be expanded later to include representatives from other schools in the region. The JHU Application and Proposal, 2005, provides some idea of how the strategic and technical planning partnership was to work. The proposal describes how the partners would form a core partnership involving Makerere, Muhimbili, Johns Hopkins University, and Tulane University through a strategic planning process. An early activity would include mapping strengths, deficits, assets and needs, thus making a determination about how the schools can reinforce each other and how the U.S. partners can facilitate this process. The strategic planning process and continual input of stakeholders via the Advisory Committee would help to ensure that training, programming and applied research would serve the needs and goals of key stakeholders such as governmental and nongovernmental implementers. Although convening the Deans and Directors group was not done for the specific purpose of staffing the Management Committee,36 it appears that the Deans and Directors annual meetings, in effect, became the LlPHEA Management Committee, at least for purposes of strategic and technical direction. Budgetary and financial management responsibilities originally envisioned for the Management Committee, were moved to Makerere because the university had systems to adequately receive and disperse funds. The considerable global strategic and technical advisory capacity of the U.S. partners appears to have been "on-call," but not necessarily called upon to the fullest extent it might have been. The reported delays in permanently filling the JHU support manager position in Baltimore apparently limited the opportunities for the U.S. partners to contribute, as opposed to react, to the strategic and technical direction of the program. Thus, close operational liaison between the U.S. consortium and the African Partner Institutions may have been less than optimal during the first two years of the program. This impression is reinforced by the fact that toward the end of Year 2, the program was reporting "delays" in its semi-annual report. Program Documentation and Monitoring The LlPHEA Partnership had considerable latitude in implementing its vision and mandate, consistent with the parameters of USAID/HED grant funding and grant management, and also in encouraging the best principles of regional cooperation. To a large extent, the program was allowed to evolve rather than be structured to operate within an explicit program framework with specifically defined objectives, activities, and targets. For instance, convening of regular regional deans meetings was not a specified activity in the original program design. Rather, this activity was an interpretation of the partnership's mandate to "build networks." Aside from broad expected outcomes, program planning and development progressed according to the best thinking of the partners. Although the program implemented many of the institution-building activities that are standard approaches for institutions of higher education (e.g., academic publications in technical/professional journals), the program directors also pursued program development opportunities, took what they believed were the best logical courses of action and made course corrections as they saw fit. The Assessment Terms of Reference did not immediately highlight monitoring and evaluation (M&E) issues. Rather, interest in M&E emerged when preliminary findings suggested that program outcomes could have been improved with more precise program monitoring and identification of areas that could have benefitted from additional strategic and technical guidance. An understanding of the program's monitoring was drawn from LlPHEA's available documentation, literature and reports, and interviews. Though referenced, no actual work plans could be found or reviewed. However, extensive implementation level work planning appears to have been carried out, particularly in Years I and 2, but 36 The Deans and Directors group representing selected East African schools of public health was first convened in October 2006 for the purpose of collaboration in areas of mutual institutional interests which eventually lead to establishment of the LlPHEA legacy HEALTH Alliance which was chartered in 2008. JBS International p. 36 Leadership Initiative for Public Health in East Africa Outcomes Assessment also in later years. This impression is reinforced by the fact that throughout its five years of operation, LlPHEA was documented to be a highly productive program at the output level. A set of semi-annual reports and a small volume of historical documentation on the HEALTH Alliance website attest to a program that Vigorously pursued its planned activities. The semi-annual reports have a standard format that captures both qualitative and quantitative data. While the reports were informative and readable, a comparison of data gained from the field visits with how events were reported led to the following observations: • The reporting format provided extensive opportunities to describe achievements, but did not adequately cover problems and constraints at a sufficient level of detail. • Related to this, the reporting format called for a rolling projection on what activities were planned in the coming six months, but did not require the reporter to recount what activities had been planned, but did not occur. • Because of a lack of detail about problems, constraints, and issues that may warrant management follow-up, the reports took on the character of newsletters rather than management tools. As such, these reports may not have provided as robust an understanding of the program as they might have. • It was not clear who was involved in the writing of these reports, but it appeared that reports were prepared by JHU in Baltimore with some input from the African partners. If this is the case, it would have been more productive management-wise for the partners to use the occasion of preparation of reports for a face-to-face joint strategic, technical, and operational review. Notably, however, despite the depth of conversations with staff in three countries there was very little detail provided regarding planning and project evaluation for mid-course assessment beyond the major shift from academic to competency based learning which shifting for leadership training to disaster management response training. Financial management data was very limited, with the only financial data available being of a general nature and drawn from the public domain. Since the Assessment did not extend to perusal of official financial records, observations were confined to those that could be made on the basis of the cursory financial information available. However, questions did arise about how LlPHEA funds were shared during HEMP training and other institutional building activities, and embedding a financial review in any planned end-of-project financial audit could have assisted in providing greater understanding of these concerns. Material Support for LlPHEA Activities Available LlPHEA funding and financial management information was initially limited to data in the pUblic domain. Table 2 below, summarizes core USAID/HED funding for FY 2006 - 2010. Source: HED, 20 I I JBS International p. 37 Leadership Initiative for Public Health in East Africa Outcomes Assessment In addition to USAID's initial contributions to L1PHEA, some of the partners reported to the assessment team that they were successful in leveraging other funds (as shown in Table 3 below). Three major international development agencies have recognized the utility of working with the HEALTH Alliance and networks formerly established through L1PHEA. The HEALTH Alliance was able to offer the practical advantages of having a legal and financial management structure that allows it to handle large grants (which newer grantees may not have). The HEALTH Alliance was been asked to join in three major new global initiatives: USAID- funded RESPOND, DfID-funded Future Health Systems (FHS), and a National Science Foundation (NSF)-funded natural disaster research project. The USAID RESPOND project is now working through the HEALTH Alliance as its African Hub for epi￾zoonotic disaster response trainings. This five-year USD 185 million cooperative agreement aims to strengthen the human capacity of countries to identify and respond to outbreaks of newly emergent diseases in a timely and sustainable manner. The project is led by DAI and includes the University of Minnesota, Tufts University, Training Resources Group, Inc., and Ecology and Environment, Inc. It is worth noting that the HEALTH Alliance bid on the RESPOND activity on its own, but was unsuccessful in that bid. The DfID-funded Future Health Systems consortium works through the HEALTH Alliance as a hub to improve health systems research throughout the region. The goal of FHS is to generate knowledge that shapes health systems to benefit the world's poor. It is a consortium of research institutions in eight countries including Afghanistan, Bangladesh, Ethiopia, India, Rwanda, Uganda, UK, and United States. FHS received a new six-year award in 20 II and selected the HEALTH Alliance to be its African research hub. In other leveraged developments, the Rockefeller Foundation provided USD 350,000 through USAID to augment institutional support and resources for three HEALTH Alliance schools (Rwanda, DRC, and Ethiopia), to undertake HEMP training. It should be noted however, that the Foundation's funds may have been intended for general institution capacity building in the recipient schools which could possibly include funding for a specific training program. The Rockefeller funds appear to have been issued to each of the designated schools, but without much assistance in implementing the training programs. The exact status of HEMP training in DRC and Ethiopia, or any impact their participation in the HEMP training may have had on their institutions, was not determined. In contrast, HEMP training in Uganda, Tanzania and Kenya were all highly productive and trained hundreds of district level responders. In addition, the HEALTH Alliance facilitated an unprecedented level of regional cooperation among public health practitioners. As an example, NSF-funded researchers heard about the HEMP training and asked to partner with the HEALTH Alliance and HEMP in the Bududa and Butaleja landslide and flood affected regions of Uganda. Kenyans trained by HEMP traveled to Uganda to assist Uganda's response and recovery efforts. Similarly, during the Kenyan post-election JBS International p. 38 Leadership Initiative for Public Health in East Africa Outcomes Assessment riots, HEMP-trained Ugandans went to help Kenyans. In addition, NSF researchers worked alongside faculty and students from MUSPH/Uganda to assess the impact of the natural disasters on local communities, and evaluate the response efforts. The NSF grant enabled the participation of MUSPH/Uganda faculty and students in creating an innovative mapping tool to determine regions at-risk for disasters. Of note, MUSPH/Uganda was particularly successful in leveraging its leadership role and association with the HEALTH Alliance regional platform, and its reputation for excellence in delivery of HEMP and other LlPHEA training programs. MUSPH has been able to garner the following additional support: a) commitment from the Ugandan Minister of Finance to construct a state-of-the-art building at the MUSPH campus, b) funding for an International Conference on Reproductive Health and Family Planning, and c) a large grant from the Gates Foundation to establish a Center of Excellence in Reproductive Health and Family Planning. UNDP/OCHA Foundation CDC-PEPFAR USAID-RESPOND Future Health Systems JBS International Research collaboration Funding Conduct disaster response research in HEMP-trained regions in Uganda Support HEALTH Deans and Directors Meetings #8 and #9 s to Systems Research Capacity based on ten trainings at 30,000/trai 30,000 9/30/2011; 1,250,000 expected between 2011-2016 p. 39 Leadership Initiative for Public Health in East Africa Outcomes Assessment Source: LlPHEA Final Partnership Report, May 20 II *Not reported as official or proposed cost share Challenges with Resources and Relationship Management37 Kenyan informants characterized the LlPHEA funding as limited support compared to the generally known program resources that have been made available by USAID/HED. This perception may have been fostered by a complex funding and funds management process. As the assessment team understands it, funds were channeled from HED via the Johns Hopkins University through Makerere University to Acclaim, Inc., a financial management firm in Kampala that was contracted by Makerere University for that purpose. Funds could be drawn down only with authorization by Makerere University and, under the original LlPHEA grant, could only be transferred to the (then) Muhimbili University College of Health Sciences (for the School of Public Health in Tanzania), or the Institute of Public Health at Makerere (for the School of Public Health in Uganda). This method of funds management was used to make disbursements more efficient and eliminate delays caused by complex fiscal procedures in either of the universities. The creation of the Health Alliance complicated this funding stream. The five additional Health Alliance schools were not authorized to receive LlPHEA funds under a very precisely defined charter which established the LlPHEA grant as described by officials at the Makerere Institute of Public of Health. When the original LlPHEA grant (2005-2007) expired, it was re-funded with different criteria related to epi-zoonotic disease emergencies and made it possible for other schools, including public health and veterinary medicine, to receive LlPHEA funds. As a new partner in the Health Alliance, Moi University required support funding to conduct planned HEMP training. Having developed the curriculum, and the funds from LlPHEA not having been ready, Moi University School of Public Health secured funding from UNOCHAIUNDP for its initial two trainings in November/December 2008 that were channeled through the Research Projects Office (RSPO) fund managers; the RSPO is an office of USAID's Academic Model Providing Access to Healthcare (AMPATH) project in Eld 0 ret, Kenya and Moi University School of Medicine. Moi University also secured additional funding and was able to host a total of 10 trainings by the time of the LlPHEA Assessment. The Dean of the School of Medicine confirmed that the Medical School served only as a processor of funds destined for HEMP in the School of Public Health. The Department of Community Health/School of Public Health at the University of Nairobi received no comparable funding from the LlPHEAlHealth Alliance grant. While acknowledging that LlPHEA and the Health Alliance's interests were important in encouraging the founding of the School of Public Health at the University of Nairobi, officials there stated that there has been "little funds to support trickling down on our side" and" ... no equal sharing of [administrative support] costs." With the expiration of the LlPHEAlHealth Alliance grant in March of 20 II, and uncertainty as to how the HEALTH Alliance will find additional funding to preserve its vigor, both of the Kenyan universities are fully aware they will have to find other funding sources if follow-on training in various forms of disaster management and leadership are to continue (e.g., new initiatives in disaster risk management). Findings The U.S. and African Partner Institutions, and to a lesser extent, LlPHEAlHEALTH Alliance partners, received core funding of USD 5.87 million for the purposes of financing specified program activities such as conduct of university and in-service training programs in leadership development and health 37 A financial review of LlPHEA was outside the scope of this Assessment. However, there were some (mostly unsolicited) impressions related to funding levels and distribution of financial resources that emerged during the course of the Assessment and warrant comment. JBS International p.40 Leadership Initiative for Public Health in East Africa Outcomes Assessment emergency management, training of SPH faculty and staff in leadership, instructional technology, and a massive infusion of information and communications equipment, etc. It appeared that funds were generally used for the purposes intended. Lessons Learned & Recommendations As reported from Kenya, some members of the LlPHEAlHealth Alliance feel they did not benefit equitably from LlPHEA funding, including the Rockefeller Foundation contribution that was available during the 2008-20 10 period. Recommendation: It may be instructive for HED to review and clarify how equitably LlPHEA funds for HEMP training and other institution building activities (such as leT) were shared during the 2008-20 I 0 period, and to review the patterns of accountability in funds disbursement across all seven LlPHEAlHEAL TH Alliance partners. Impact on U.S. Partner Institutions The U.S. Partner Institutions have benefited generally from participation in the LlPHEA partnership. Their institutional experience has increased substantially, and their institutional profiles have been enhanced. The close working association and access to a regional platform of Schools of Public Health in East Africa have increased their leverage in being able to attract additional programs, projects, and funding for themselves and their African counterparts. This association and access, as well as expansion of public health research and development programs, offers an excellent training laboratory for their faculty and staff. One partner reported that the opportunities for administrative personnel, such as finance officers, as well as for faculty and students, to gain field-based experience improved the international health competencies of the school. Although the question of benefits that might accrue to the U.S. Partner Institutions was posed frequently, the manner of response suggested that this is not an issue that is thought about frequently or deeply. The absence of strategic focus on this specific issue in the program design or descriptions is noteworthy, especially when it is of importance to policy makers who promote non-regional partnerships. It is possible that with the concept of benefits to U.S. Partner Institutions built in and articulated in the program design, U.S. Partner Institutions and their counterparts would be more inclined to give this area greater strategic attention and planning. Lessons Learned & Recommendations Although the issue of reciprocal benefit of partnerships to U.S. partner institutions is of interest to policy makers who promote non-regional partnerships, neither U.S. nor African Partners appeared to strategically focus on this concern. Recommendation: To encourage U.S. Partner Institutions and their counterparts to think and act more strategically about the benefits that might accrue to their institutions as a result of partnership arrangements, USAID and HED might want to consider establishment of a standardized profile for project design that would ensure these needs have been adequately assessed. Finally, the basic standard reporting format used for semi-annual reports to higher levels did not encourage partners to report challenges in such a way that would have informed higher level managers and assist the monitoring and management follow-up process. Recommendation: HED may wish to review partnership reporting formats to ensure that information being provided includes references to operations, communications and assessment or mid-course corrections to assist project monitoring and trigger management interventions and course amendments when indicated. JBS International p.41 Leadership Initiative for Public Health in East Africa Outcomes Assessment Assessing Operations in Four LIPHEAlHEAL TH Alliance Partner Institutions The assessment of four structurally and functionally comparable East African universities in the L1PHENHealth Alliance provides a unique opportunity to consider how political culture and diverse national experience shape efforts to create regional cooperation in Africa. Moreover, by analyzing these four institutions using a common set of comparative measures, a framework is established for deriving a richer set of conclusions and lessons learned in this unique initiative in regional collaboration with cognate support from partner institutions in the U.S. In the case of the four universities considered in this report, all were developed on the same academic model, having begun as affiliated colleges to the University of London or as a more recent institution that drew on the British model for organizational structure. Makerere University was established in 1955 and is the oldest of the four institutions. The University of Nairobi began as the Royal College of Nairobi in 1957. The Muhimbili College of Medicine in the University of Dar es Salaam (est. 1961) became the Muhimbili University of Health and Allied Sciences (MUHAS) in 1995. Finally, in 1985, Moi University was created as Kenya's second university. The issue of political culture can be contrasted across all three countries. Uganda had faced decades of violence and though the country has achieved a measure of stability in recent years, the high visibility of the military in the public sphere and the continuing fight in the north against the Lord's Resistance Army tends to perpetuate a "high security" environment. Kenya has a political culture of political order which has only once been disrupted during the past half century. Political conflict has occurred, most notably in late 2007, and a "moderate security" environment makes people careful, but the culture as a whole is intent on moving beyond an aberration in civilian rule. Finally, Tanzania has what could be characterized as a "low security" environment where there is trust in government practices if not in all of its politicians. This third reality has contributed significantly to the work that has been done nationally in implementing the training programs at the district level for responding to disasters with effective leadership techniques. Measures for Assessing Operational Practices and Strategic Purposes Each of the profiles considers the data gathered in two ways. Practice-based evidence is examined to identify the quality of the practices employed in five different categories: planning, coordination and communication, implementation, assessment during the project, and overall evaluation. These criteria consider how objectives or tactical choices are addressed. The degree to which there is success in these five areas serves to inform conclusions on the strategic goals being pursued by the project. Those goals are human capacity building and institutional strengthening. Both forms of analysis can be made subject to performance scales with suitable metrics of arriving at conclusions on the outcomes and comparing results from other projects of similar nature. In these four profiles, no metric evaluation has been applied to the tactical practices sections given the short term of the field review and the limitations on certain kinds of data. However, a metric approach is applied to each of the four profiles which creates comparative conclusions on institutional strengthening in the four universities as a result of the L1PHENHealth Alliance project. Uganda In October 2012, Uganda will celebrate a half century of political independence. The first half of this period was characterized by significant political turmoil and some of the most horrific violence experienced in post-independent Africa. It was not until 1986 and the seizure of power by Yoweri Museveni's National Resistance Army that a period of extended calm was created for Uganda's people. JBS International p.42 Leadership Initiative for Public Health in East Africa Outcomes Assessment In the succeeding quarter of a century, the economy has enjoyed very substantial growth and generated resources so that the social and economic infrastructure can begin to meet the extensive needs of the population in all the areas of human activity. The growth in the capital city of Kampala now spreads across all of the seven hills which have been the centerpiece of the "Pearl of Africa." President Museveni's re-election to a five-year term in 20 II gives some promise of a continuing record of stability and calm in the country despite violence in the north related to fighting the Lord's Resistance Army. However, the long period of violence that was most horrifically characterized by the Amin era continues to affect Uganda's people and the political culture of the state. The continuing military presence in the capital and across the land does not dispel this sense of worry, as it suggests the maintenance of a "high security" environment. It also works to preserve the sense of uncertainty among the Ugandan people and encourages a culture of order where few would take risks on policies or program of development. While this condition has an evident impact on the work of the Emergency Management Training, it does not seem to intrude on the conversation within Makerere University. However, it is a variable that affects outcomes in terms of the productivity realized by the Health Alliance training programs. The Context of L1PHEAlHealth Alliance Operations in Uganda The earliest discussions of LlPHEA took place at a I 990s conference in South Africa where Makerere's Dean of the School of Public Health, William Bazeyo, and former Dean of the School of Public Health, Professor David Serwadda, were present, along with Dennis Carroll. Later, Makerere was represented in Washington, D.C. for the selection of the U.S. Partner Institutions: JHU, Tulane University, and George Washington University. JHU's Gilbert Bingham was named as one of the co-Principal Investigators along with the Deans of MUSPH and MUHAS/SPHSS. Historic linkages between JHU and Makerere ultimately determined that Makerere would take the lead in close cooperation with JHU for major portions of the project operation. Makerere University Makerere University, located in Uganda's capital city of Kampala, is the central site of the LlPHEAlHealth Alliance project in East Africa. Among the seven participating schools of public health in six countries, Makerere has taken the lead in the development of curriculum materials for training and served as the link between the U.S. partners and the teaching and training of university students and civil service personnel. The Makerere campus shows great awareness of the project's history, the methods employed to implement the project's goals, and appreciation of what the future holds after the expiry of the LlPHEAlHealth Alliance funding at the end of March 20 II. This comprehensive knowledge of the planning process of the LlPHEA project contributes to a sense of the potential for regional cooperation on public health issues. Moreover, there is a great awareness of the viable inter-agency linkages that can be encouraged within the international community. Operationally, the Makerere experience has been characterized by significant cross-disciplinary training and research within the university community, perhaps more so than the other universities. It has seen significant success with its on-campus activities such as training, journal development, collaboration with U.S. partners and development of IT capacity and curriculum materials. At Makerere, LlPHEA is viewed as having established a foundation and a legacy for other agencies to build on even if the activities in the original LlPHEA program are not sustained. Planning Project Proposal The original project proposal included extensive activities that could be accomplished by LlPHEA, though it lacked needed specificity in terms of specific objectives to be achieved. This lack of specificity compromised some of the project's outcomes. For example, funding mandates were not tied to specific JBS International p.43 Leadership Initiative for Public Health in East Africa Outcomes Assessment projects and significant resources were placed in a category of general expense with regard to program components to be carried out by Makerere and MUHAS. The tasks assigned to the U.S. partners were much more precise and expenditure patterns were well-controlled. A more precise proposal overall could have benefitted the project; however, the project, with its regional orientation and anticipation of draWing in other schools of public health, was such a departure from historic grant development by USAID that credit must be given for taking some considerable risks in supporting the project. Operations Planning Operations planning was effective in implementing a number of activities. These activities included the creation of multi-disciplinary collaboration across the Makerere University campus, the conduct of training, the flow of personnel to the U.S. for training and curriculum development work, the creation of a potentially useful website, and publication of the East African Journal of Public Health. Financial Management The financial management of the project was negotiated in a very detailed manner under which funds could only be spent by two partners. This was a sound approach for the first two years of the project, but was not as useful when the funding was extended for an additional three years. Other than funding a very modest level of training in Kampala and supporting the Deans Conferences, very little support was provided to the three new countries joining the regional grouping as members of the Health Alliance. In retrospect, there would have been value in writing an entirely new document for financial management of the extended project. A second feature of financial management that created some issues for operations was the structural/functional realities of the partner institutions. Universities based on the British model historically have "Directorates of Finance" and "Directorates of Academics." As a general rule, these two bodies interact only minimally. Dependence on "incremental budgeting" as standard practice further widens the gap between academic needs/program operations and resource support for those activities. Makerere and MUHAS agree that this is a reality and a problem for utilizing grant or programmatic support in a timely manner. To accommodate this reality, an independent financial services agency was engaged by Makerere UniverSity to speed the process of funds flowing from JHU to Makerere and MUHAS. In a complex approach, Makerere would request funds from JHU. JHU then would approve a transfer to the financial services agency which could then disburse funds to Makerere. If the arrival of funding was inordinately delayed, MUSPH had provisional authority to make loans from its own resources to departments within the University or to MUHAS. This must be seen as irregular from an accounting standpoint. Use of a financial services agency is a reasonable practice given the constraints of Anglo-African university models; however, it should be characterized by greater oversight. Coordination and Communication Internal to the University Communication within the University was good. For example, through LlPHEA, the Department of Environmental Health was able to attract interest from the departments of engineering, construction technology, hydrology, sewage and water treatment, veterinary medicine and from the Ministries of Agriculture, Natural Resources and Community Work. A module was developed for all of these departments on water sanitation, excreta disposal, vector and environmental control and food hygiene. LlPHEA also enabled the department to start the Bachelor's degree in Environmental Health Sciences. Communication in the Region At the beginning of the Health Alliance stage of operations, the Deans in the seven schools were not close. These relationships have improved over time. As a result, universities are now more effectively JBS International p.44 Leadership Initiative for Public Health in East Africa Outcomes Assessment exchanging external examiners for evaluations of theses. The Deans Conference is the single most important vehicle developed for communication within the region because it was maintained on a regular basis and acquired a routine characteristic that ensured discussion would occur. The substantive content of those meetings deserves further study. Operations Development and Use of Training Materials for L1PHEAlHealth Alliance Dr. Mayega received the leadership training and qualified as a trainer. He also received training at Tulane in instructional design and was made responsible for the various modules for training at Makerere, including the web-based tools for students that permitted lectures to be uploaded to the website, thus giving a more developed and paperless approach to the training materials. Dr. Mayega has also completed courses in virtual leadership, public health distance learning and e-learning. He is currently pursuing another Ph.D. in instructional technology under the faculty development portion of the project resources. Dr. Juliet Babirye develops short course training materials in leadership, and described the components of leadership courses as including team building, communication strategy, strategic leadership practice, leading change and ethical practice. These courses are attracting participants from across the university and from the Ministry of Health, including MPH students and undergraduates in environmental health. A media training course is presented for journalists to assist with more accurate reporting and to build mutual trust in communities. The course's goals are to create an efficient format to share information with journalists and encourage journalists to share information with public health practitioners before the material is released for publication. Loss of a key staff person has put this course on hold at the current time. Information Technology One of the great strengths of the L1PHEA project was the IT operation developed by Director Deogratius Sebuwufu who holds a diploma in engineering, a degree in statistics, and an MBA. This experience made him ideally suited for development of a successful business model, and provided a capacity to negotiate effectively with vendors and the capability of solving all of the technical problems that arose. Moreover, he was provided the resources to build an autonomous system with 24/7 connectivity that can be a model for any African university. The industrial size generator and stand-by power lines ensure the IT operation can continue uploading files, make information accessible on the website and ensure students have access to all of the e-line training materials. Externally, the IT operation has routine and uninterrupted connectivity to Tulane University. The staff and students are reported to appreciate the availability and speed of the system. The seven L1PHEAlHealth Alliance IT directors in the six countries are also connected and interact frequently with each other as need arises. In addition to the system created, the Director trains IT students (both undergraduates and IT-interns) from other disciplines on a rotational basis of three to six month practicums. While establishing these IT operations, the Director has received training under the L1PHEA grant for e-learning and social science so that he can interact more effectively with U.S. counterparts and other African colleagues. Health Alliance/Health Emergency Management Planning (HEMP) During the development of the second phase of project operations with its proposed focus on disaster management training, Makerere's key person, Dr. Christopher Orach, advised the university on the organization of country training at the district and zonal levels so as to achieve maximum operational viability. His background is in developing infrastructure and human capacity throughout Africa. He encouraged the exchange of facilitators across the region and making Makerere the coordinating center for all L1PHEAlHealth Alliance training. In the process of developing training materials, it became apparent that the different countries had varying gaps in health care. A team of four academicians and four non-academicians was selected to make recommendations on how the training should be JBS International p.45 Leadership Initiative for Public Health in East Africa Outcomes Assessment organized. These recommendations led to the development of a one-week course for District Chief Administrative Officers and Disaster Officers in six countries. The course was held in Kampala and set the stage for the eventual district level course consisting of two to three days of training with "practicals" on developing district disaster response plans, including a budget. Dr. Orach noted that the original LlPHEA materials were a foundation for the disaster management practical training. Makerere University and the Government of Uganda The relationship between Makerere University and the Government of Uganda must be seen at two levels: the formal level of government bureaucracy and the operational level of work occurring in the administrative districts. Vincent Woboyo of OPM, which coordinates disaster prepared management and all of LlPHEA initiatives, spoke positively about the training for disaster response management. Mr. Woboyo, who has been trained in the methods of disaster management, works in concert with the Ministry of Health, the Ministry of Agriculture, the Red Cross, and the Veterinary School. While OPM is responsible for Commissioners for Disaster Preparedness and Refugees and for Relief Services, and a Disaster Forum comprising representatives from government agencies, UN agencies, NGOs, and Makerere School of Public Health exists, the Government of Uganda has yet to promulgate a draft policy on disaster preparedness and, consequently, there is no specific funding for disaster management. Districts are funded by the Ministry of Local Government, and funds for disaster management are not provided. Therefore, even though the best will in the world exists in Mr. Woboyo's office and in various ministries, the Ugandan Government is not financially committed to supporting the HEMP training. The consequence of this limitation on disaster management training is revealed in surveying two districts close to Kampala: Mukono and Jinja districts. Both districts had presumably created the required response plans follOWing their training, but neither district could produce a copy of the plan. Secondly, the plans were supposed to be reviewed and approved with input from the School of Public Health. In both cases there had been no follow-up. This reality speaks to a larger issue. The Ministry of Local Government only provides funding to meet 60 percent of the operating and personnel costs of a district. The Chief Administrative Officer (CAO) is expected to raise the balance of the funds locally. In Jinja, two new districts have been created from the district and departed with the revenues that might have been raised in those two places. Additionally, a town has been given separate status and has taken more of the resource base. The CAO noted that his "chief responsibility is to find the money to pay the staff and put petrol in the vehicles." He went on to explain that there were no additional funds. Throughout the country, planning is incomplete and no funding exists for implementing plans if a crisis arises. Despite Mr. Woboyo's efforts in OPM, these problems exist and aid continues to come in times of crisis from disaster relief agencies. A third critical factor in the training program is the constant rotation of personnel. While a full complement of people may be trained in a district, within a year's time more than half the team may have moved to other locations. In the case of Jinja district, some of the members of the disaster response team will be moved to the two newly created districts. Makerere University and the Donor Community Makerere University's relationship with the donor community is complex. Representatives from the donor community are aware of the role LlPHEA has played in drawing attention to leadership needs in the public health sector, but are critical of the LlPHEA model for training at the local levels. Consequently, some are borrowing from LlPHEA while some are using other models. The School of Public Health sees its work as being "copied" and there is some bitterness that LlPHEA was not the recipient of continued funding and folded into RESPOND. While still being enormously proud of the successes they have created and the foundations they have laid for new partnerships to emerge, both JBS International p.46 Leadership Initiative for Public Health in East Africa Outcomes Assessment the current and former deans of the School of Public Health are aware that they have not been included in future initiatives. Assessment and Evaluation LlPHEA engaged in a significant mid-course correction: the change in focus from an academic program primarily in the university community to a competency-based program for training personnel at the district level. An official from the Ministry of Health reported that without LlPHEA, the transition to a focus on zoonotic diseases would have been very difficult. Through LlPHEA, epidemic diseases were incorporated into the Forum of Response Preparedness in the Office of the Prime Minister. This was followed by training that brought together representatives from the University, the Ministry of Health, and the Office of the Prime Minister. This training produced seven compact discs with training stories and commentaries from participants. Further, the Ministry of Health has been able to mobilize all veterinarians with epidemiological training in the Ministry and connect them to veterinarians in Veterinary School. These types of actions create a base for implementing the training at the district level with government support. For the Health Alliance, the development of a whole new curriculum in "competency education" for training hundreds of government and agency personnel across the three east African countries was required. The reconfiguration of the program at the mid-point was impressive; however, this reconfiguration would have been strengthened if the method of allocating funds to the participating partner states had also been amended. Assessing Effective Practices and Capacity Building at Makerere University Measuring Capacity Building Makerere University's partnership in LlPHEA and the Health Alliance lends itself to a quantitative rating as described in the Assessment Methodology section of this report. Makerere University's institutional profile can be subjected to a systems approach rating scale that considers five elements that can be assessed to define effective systemic performance. These elements are: learning, complexity, flexibility, productivity, and durability. A sixth performance element was requested by HED to be included in the analysis: connectivity. This condition emerges from integration theory and is appropriate here as a means of considering the level of functional integration that has occurred within the Health Alliance. The six elements used for assessing system performance are measured on a scale of 1-5 in each category with a I being low and a 5 being high. A total possible score is 30, and a score above 24 would be considered a desirable outcome. Using this measure for assessment of Makerere University's participation in the LlPHEAlHealth Alliance project, the following conclusions are made: • Learning: The efforts made by Makerere and its School of Public Health demonstrate a verifiable capacity to engage in new "learning" as it designed and secured funding for the LlPHEA project. The level of thought that combined to create the regional initiative was exceptional in scope. This was followed by a significant level of "tactical" planning in the operations of the project in terms of cooperation with its partner MUHAS to create curriculum and training materials, inaugurate the training at the graduate level at Makerere and build a set of linkages within the university. The "learning capacity" was somewhat compromised when that forward thinking capacity was not carried into the second phase of the project. The Health Alliance was treated as an add-on and not properly funded in order to build an effective regional network. Retaining all of the funding within the two original partners was unhelpful. In this respect, the learning capacity was notably compromised. Score: 4 • Complexity: When the LlPHEA project commenced, Makerere University was already the most mature of all the institutions of higher learning in East Africa. LlPHEA served to build on the existing level of complexity by creating a much denser set of linkages between the sciences and the arts. In Phase Two, linkages were deepened between various dimensions of the sciences such as the health sciences and veterinary sciences. Inter-disciplinary collaboration became a JBS International p.47 Leadership Initiative for Public Health in East Africa Outcomes Assessment standard condition of the work of the project. An additional layer of networks was added in the establishment of a model IT capacity with the aid and assistance most notably of Tulane University. This structural change supported a whole new set of functional capacities in terms of e-Iearning and curriculum development. The extension of communication across the whole geographic region, an exercise in functional integration, was a further effort in the creation of a complex entity. While the regional integration, or connectivity, exercise was not fully realized during the duration of the project, the foundation was created on which to build future initiatives in regional health improvement. Score: 5 • Flexibility: The dominant condition which demonstrated institutional strengthening was the willingness of Makerere to re-vamp the project midway and move from an academic focus to a more competency-based program of training people to respond to particular types of conditions, notably, disaster response management. The willingness to incorporate the biological sciences and the whole set of concerns associated with zoonotic diseases is not typical of academic communities. In this instance it worked very well; perhaps in part due to the mixed nature of curricula in a School of Public Health. However, by not making the changed focus in the second phase of the program a fully collaborative undertaking, the flexibility capacity of the project was impaired. The restrictions on the use of funds and concentrated control in the hands of Makerere and MUHAS had a negative impact on building on the strong foundation laid down in the initial stage of the project. The universities are continuing to be constrained from a developmental standpoint by the structural limitations that separate finance from program operations. Given that set of structural constraints, programmatic funding will continue to be an issue in attempting to engage in institutional strengthening. Score: 4 • Productivity: The LlPHEAlHealth Alliance project was exceptionally productive in terms of measurable outputs as defined by the number of people trained in the university and in the districts of Uganda. This district level training may be suspect because detailed district level disaster response plans are not fully in place, staff rotations compromise the maintenance of viable teams in various districts, and funding support from the Ugandan government is absent. However, the productivity must be considered in terms of permanence of facilities. The IT structure at Makerere is permanent. The curriculum has a permanence that can be amended to meet changing conditions. The skills in e-Iearning are a permanent reservoir of talent at the university. There is also a notable "spread effect" or "spillover." It is unknown how the linkages made will mature to lead to new developments in the area of health programs across the region. The sharing of External Examiners on an expanded basis is just one small unintended consequence of the colleagueship derived from the Deans Conferences. The website is a permanent vehicle that can be used far into the future as internet connectivity in the region is expanded. The permanent foundational conditions that remain at Makerere after the expiry are a measure of how the institution has been strengthened by the project. Score: 5 • Connectivity: The literature on the creation of international regional communities suggests that one of the most effective ways to create such linkages is to begin with functional forms of integration. In moving to an expanded version of the LlPHEA project with the establishment of the Health Alliance, an exercise in functional integration for support of public health needs was started. However, given the economic resources available, this condition remained in a fledgling stage. Only minimal amounts of the new LlPHEA funding flowed to the four new members of the group, primarily for a limited amount of training conducted in Kampala and for support of the Deans Conferences. Linkages between Makerere and the Ugandan government were strong in the Office of the Prime Minister, but not sufficiently broad either horizontally across various ministries or vertically across various levels within ministries as to influence policy to gain JBS International p.48 Leadership Initiative for Public Health in East Africa Outcomes Assessment support (by contrast see the work done in Kenya to build "contact points" at all levels). This lack of sufficient connectivity compromised capacity to secure funding support for the Health Alliance portion of the program. Finally, the LlPHEA project seemed to have generated a number of new programs or encouraged programs that were increasingly in competition with LlPHEA, including the African Field Epidemiology Network and RESPOND. Score: 4 • Durability: One of the key conditions of interest to the donor community in funding any new project is the likelihood that the initiative can be sustained beyond the lifetime of the funding cycle. Institutional strength is built on the creation of conditions of learning, complexity, flexibility and productivity that will produce a long-term level of strength for the system under review. The expiry of the funding for LlPHEA has revealed that a sound durable structure was not created for the long term. Things were not done which would have helped. For example, of the more than two dozen editions of the East African Journal of Public Health supported by LlPHEA, at least one edition per year should have been devoted to the achievements of LlPHEA and the Health Alliance. Successes were not being written up on a routine basis to create a powerful tool when seeking continuing funding. It is also apparent that in the creation of One Health for Central and Eastern Africa (OHCEA), individuals were moving away from LlPHEA to pursue their own related initiatives. The issue of institutional funding is and will remain an impediment to building long term durable initiatives in specific areas of university operations. It is only likely to be improved with a shift to a budgeting model that links program operations to patterns of revenue and expenditure flow. Finally, the champions of LlPHEA on the campuses of both the major partners also tend to "age-out" and new champions must be encouraged as deans retire. The structural strengths noted in productivity above will remain, but Makerere probably did not do all it could to preserve the vitality of the LlPHEA model beyond the life time of the funding cycle. Score: 3 A final score of 25 out of 30 establishes that the LlPHENHealth Alliance project at Makerere University's School of Public Health was very successful. It demonstrates how imaginative thinking transformed into an operational idea can attract the interest of major donors. It is in examining the implementation, or tactical/operational elements, of the program that limitations are exposed and potential opportunities seen to slip away. Those losses eventually have an impact on the overall strategic goals of human capacity building and institutional strengthening. This assessment of LlPHENHealth Alliance at Makerere was of a too limited duration to examine in detail the amount of human capacity building that occurred, although anecdotal commentaries suggest such capacity building was substantial. However, sufficient information was compiled to make informed judgments on the level of institutional strengthening that occurred at Makerere University. Very significant success was verifiable. Even greater success was possible. New iterations of donor supported initiatives in public health will be the beneficiaries of the achievements of LlPHEA and the Health Alliance at Makerere by the many committed academicians, senior administrators and collaborators from government and international non-governmental agency sector. Tanzania In 20 II, the Republic of Tanzania celebrated a half century of independence since the mainland portion of the country, Tanganyika, was created as an independent entity in 1961. FollOWing the grant of independence to the isle of Zanzibar in late 1963, the two countries merged in 1964 to create the Republic of Tanzania. In the succeeding five decades, the nation has had a remarkable history of political stability and civil order that is in part a tribute to ethnic homogeneity, as all tribal groups are of Bantu origin, and the legacy of effective leadership in the early decades under first president Julius Nyerere. Moreover, the creation of a philosophical plan for development, Ujamaa socialism, resulted in the country attracting very significant support from the international donor community to offset the paucity JBS International p.49 Leadership Initiative for Public Health in East Africa Outcomes Assessment of natural resources. Ujamaa socialism contributed to an organizing sense of process for political development that has persisted across the country. Tanzania today lacks anxiety over historic images of internal national security conditions or the underlying realities of ethnic tribal animosities that can compromise developmental progress. Simply, Tanzania is not like Uganda or Kenya. This combination of a reasonably peaceful civic culture, ethnic homogeneity, and an orderly and positive perception of the role of government is critical to arriving at conclusions as to how MUHAS was successful in realizing the range of its successes in implementing the LlPHEA program and its successor effort in disaster management. The Context of LlPHEAlHealth Alliance Operations in Tanzania Muhimbili University of Health and Allied Sciences and its School of Public Health and Social Science constitute one of the original two partners in the LlPHEA project. First established as a college within the University of Dar es Salaam in 1991 and as a separate institution in 2007, MUHAS has five schools that include medicine, dentistry, pharmacy, nursing and SPHSS. The School of Public Health and Social Science, in which the UPHEA project was hosted, is an amalgam of the former Institutes of Public Health, Development Studies and Primary Care/Continuing Education. SPHSS has six areas of focus where fully articulated departments are in the process of being created: behavioral science; community health; developmental studies; epidemiology and bio-statistics; parasitology; and environmental and occupational health. All of the School's facilities are located in close proximity and the newness of the institution as a separate entity has created a dynamism that results in a very strong case for concluding that MUHAS was the most productive of the two original LlPHEA partners and has a better shared academic and operational sense of mission than might be found at Makerere University.38 Planning for the creation of UPHEA began in 2003-2004 with the core idea was that provision of public health services was not a matter of technical competence but of leadership. Conversations involving Professor Ishabari of MUHAS and Dean Bazeyo at Makerere occurred in South Africa and in Geneva, meetings at which representatives of USAID indicated that if the two universities could create a collaborative design, USAID would be interested in supporting that concept. MUHAS/SPHSS and MUSPH agreed to combine talent and resources to counteract disease by no longer working alone. Very early in these discussions it was agreed there would just be the two African partners who would then collaborate with Washington to identify the U.S. partners. When the proposal was written in 2005, there were as many as eight candidate U.S. institutions wishing to be involved. Johns Hopkins University emerged as a lead choice and expressed the preference for two other institutions to be involved. Those selected were Tulane University and George Washington University. MUHAS The assessment of UPHEAlHealth Alliance implementation activities at MUHAS applies the five categories for conSidering tactical objectives of project implementation. The fulfillment of implementation tasks from planning to evaluation is the foundation for assessing the success of more strategic interests that characterize the task of institutional strengthening and capacity building. Planning The planning process for the implementation of the project was conducted in two phases and involved a number of complex issues. Phase One planning included consideration of the training of MUHAS faculty at Tulane and JHU, the development of the curriculum offered at MUHAS, and the development of training in IT that would internally link the MUHAS campus and provide better contact with Makerere. There were also complex financial arrangements to be worked out including how to address differing 38 The one compromising factor in completing an assessment of MUHAS was the timing of the inquiry during the "long vacation" in June when there are few students present and many faculty are away. This compromised opportunities to observe first-hand the training that is occurring on campus at both the under-graduate and graduate levels. JBS International p. 50 Leadership Initiative for Public Health in East Africa Outcomes Assessment policies between JHU and Tulane and MUHAS and Makerere on amounts to be designated as indirect costs. MUHAS had a very low rate of eight percent that the University was initially insistent on maintaining. To facilitate the flow of funds to MUHAS and Makerere, a funds management agency headquartered in Kampala was used. The University's Directorate of Finance insisted on stringent procedures with the funding, which MUHAS/SPHSS fully accepted. Later, the specific condition that LlPHEA funds could only be expended on behalf of the two original partners proved to be a limiting factor. In terms of authorizing expenditures, JHU was designated to have the lead role by responding to requests flowing from East Africa and approving disbursement of resources. Makerere had the flexibility to loan funds internally and to MUHAS while awaiting funds to be channeled via JHU and the facilitating funds management agency. Also incorporated into this planning phase were ideas regarding support for the East African Journal of Public Health and the development of a comprehensive website that would support the growth of LlPHEA. Finally, it was agreed that the Deans of the respective schools of public health would meet routinely on a collaborative basis to ensure the project was meeting its projected objectives. All of these activities completed Phase One of the planning cycles for LlPHEA. Phase Two planning at MUHAS came with the expiry of the original LlPHEA grant. This second round was linked to an expanded perception of how leadership in public health could be usefully applied, and the chosen focus of leadership training, initially to address zoonotic diseases, was expanded to the management of various disaster conditions. Recognizing that wilderness areas were diminished in size due to population pressures and that most new diseases are emerging from those wilderness spaces, LlPHEA funding was linked to disaster planning including floods, landslides, drought, and starvation. According to Dean Killewo, it was the desire to focus on an actual problem that led to HEMP. The proposal for Phase Two was written by Deans Killewo and Bazeyo and presented to USAID which was not interested in renewing the grant to only involve the two original schools and wanted additional partners to be sought. With the new leadership focus, the delivery of training was planned and included the notion of expanding the leadership concept to other countries and schools of public health in eastern Africa. This planning concept resulted in Kenya, Ethiopia, and Rwanda being incorporated into the grouping that was ultimately formalized as the HEALTH Alliance. Coordination and Communication One of the strengths of the LlPHEA program at MUHAS was the level of coordination and communication within the university. MUHAS, and particularly SPHSS, is a small community that interacts easily and effectively both formally and informally despite having limited facilities. This interaction created a broad awareness of both the LlPHEA initiative and its successor program of HEMP training for the faculty, staff and students at MUHAS. With the partners, a high level of communication and coordination took place with Makerere in the planning stages and thereafter. The degree coordination and communication with JHU is less apparent, but the University reported that Professor Bingham was heavily involved in the planning processes. It was noted that a direct connection in terms of submitting reports, particularly those concerning expenditures, between MUHAS and JHU was important, as reports that went via Makerere tended to get side tracked and delayed for substantial periods of time. Operations The university system is stable for both undergraduates and graduate students and university personnel have long-term training available to them. MPH students receive leadership training in response to J BS International p.51 Leadership Initiative for Public Health in East Africa Outcomes Assessment disasters. Short-term training such as disaster management is for currently employed civil servants. In￾service training is incorporated at the university level as certificate programs in the graduate curriculum. In-service certificates are also offered at the regional level. The decision-making at MUHAS begins with information shared with the deans, who then effectively communicate with lower-level participants. The Director of the Institute for Traditional Medicine noted that all Deans and Directors received computers from L1PHEA. Also, three servicers were provided: one for the system and the staff, one for e-Iearning, and one for the deans. In terms of faculty exchange, MUHAS faculty went for short courses of up to three months in the U.S. Those who participated reported that the summer program on emergencies in large populations was excellent, provided a lot of new material, and enriched understanding and improvement for all of the post-graduate work. HEMP Curriculum The HEMP curriculum provided students with skills in hazard assessment, more effective communication in regional areas, leadership at the district level, and developing district level plans. In implementing the HEMP program, the Head of the Office of Emergency Preparation and Response participated in both the planning and the training that took place in Kampala. Training at the district levels was recognized as a priority because of the nature of disaster response. Thirty-five districts, from a total of 176, sent teams of six to receive training. Districts, in order to receive funding for disaster management, must include disaster management as a line item in their annual budget. In critical cases, the Office of the Prime Minister can order front line ministries to contribute to disaster response. Under the second part of the grant, the training provided by the CDC in 2009 to country participants provided even more expertise at the country level. Regarding zoonotic diseases, the Ministry of Livestock and the Veterinary School collaborated with the Ministry of Health to bring together sectoral representatives in various parts of the country. The work of the districts is balanced with a commitment from the central government, which produces a training manual. The Office of Emergency Preparedness and Response sees its role as the focal point for response and consideration has been given to pre-positioning of supplies around the country to reduce the response time on disasters. In the Kikaba District some 40 miles west of Dar es Salaam, the leader of the district team confirmed the HEMP program was well presented and that district personnel learned how to recognize and effectively respond to a disaster. The district team comprised seven individuals with various specialties. However, while the District Commissioner chairs the disaster response committee and generally takes a hands-on approach, not all of the districts are as efficient in setting up their plans and then adhering to them. MUHAS Interface with Government and the Donor Community MUHAS's relations with the Office of the Prime Minister and other donor agencies are not optimal. MUHAS and the Office of the Prime Minister do not have a close working relationship. The Office of the Prime Minister noted that MUHAS failed to maintain proper liaison on the district level training programs. The National Disaster Management policy is not always understood at the lower levels from the provincial level downwards, and having government officials come to the training at the ward level, district level and at the provincial level would be helpful. UNICEF noted that MUHAS has not been proactive enough in the presentation of plans for continued funding from the government or external agencies. While a draft plan had been created, it was not brought forward and there was inadequate consideration of how to get it sourced. JBS International p.52 Leadership Initiative for Public Health in East Africa Outcomes Assessment MUHAS and USAID It was very apparent in the extended key interviews with Dean Killewo and Professor Leshabari that there was disappointment at the end of the project with the decisions by USAID to pursue a new initiative as incorporated into USAID RESPOND. The key informants readily acknowledge that they did not adequately perceive the consequences of being less than an aggressive in the pursuit of sustained funding. They also acknowledged that leadership was weakened by the emergence of One Health and RESPOND. Consequently, the Health Alliance is also weakened without a specific source of continued funding. Assessment and Evaluation The clearest evidence of assessment of the LlPHEA program was evident in the shift from leadership training to disaster management leadership training under the HEMP program. In the process of implementing the training and encountering real life situations, the partners also recognized the need for a core curriculum with specialized regional additions that meet local conditions. Assessing Effective Practices and Capacity Building at MUHAS Measuring Capacity Building The MUHAS institutional profile can be subjected to a systems approach rating scale that considers six elements the assessment team would like to see working well in an effective public health development partnership. These elements are: learning, complexity, fleXibility, productivity, connectivity, and durability. Performance is measured on a scale of 1-5 in each category with a I being low and a 5 being high. A total possible score would be 30, and a score above 24 would be desirable. Using this approach for measuring institutional strengthening and capacity building, MUHAS/SPHSS can be judged in the following manner: • Learning: One of the strongest dimensions of the LlPHEAlHealth Alliance project is the long history of thinking that went in to creating the project. MUHAS recognized the need for leadership training and then continued that learning process as it reassessed results at a mid￾point and turned to a functional approach that gave a greater emphasis to building competence in displaying the qualities of leadership in the specific area of disaster management. This learning process was multi-disciplinary and multi-institutional in nature and was then extended into the area of international functional integration theory with the move to establish the Health Alliance in all of eastern Africa. Learning was also evident in curriculum design and the micro-levels of training at the district level. Limitations on learning were noted in failure to appreciate fully the obstacles to technology use and the constraints on institutional, national and inter-national connectivity. A special tribute must be paid to the leadership at MUHAS. Score: 5 • Complexity: MUHAS was able to build a complex operation structure within a new institution. The university and five of its schools combined with the School of Public Health and its emergent departments to create an extensive learning program where all students received instruction and leadership training was delivered from across disciplines. MUHAS then undertook to build a network of linkages to provide HEMP training in 35 selected districts to hundreds of personnel who were mostly at a pre-university level of education. An information system was also created that worked well in some areas, but not as well when confronted with the challenges of technology transfer, due in large part to the limitations on internet connectivity. The rapid development of technology led to equipment aging out quickly and placed long term impact on viability without very significant inputs of additional resources. The external liaison with the Tanzanian government also added to the complexity, with the government able to focus on domestic priorities and not be distracted by security issues. Common understandings of administrative procedures inherited from the time of independence were present, though the limitations in the network construction were a conflict between the JBS International p. 53 Leadership Initiative for Public Health in East Africa Outcomes Assessment world of academia and the world of bureaucracy. Additional effort needed to be expended to make this linkage of the university and the public sector an even more complex and interwoven enterprise. Score: 4 • Flexibility: At the end of the second year of the project, MUHAS fully redeployed its efforts in the midst of an on-going effort. It was increasingly clear to the leadership that the teaching in leadership for all students at MUHAS while valuable was not likely to have a near term impact on national needs. Correspondingly, there was awareness of real national needs in the various dimensions of disaster management. With useful input from USAID, the focus was switched to the HEMP training which has proven verifiably useful in field situations such as the flooding disaster in Kilosa district. Along the way MUHAS has learned that one type of training does not fit all situations when preparing personnel at the district level. It has also realized that people trained must become the trainers of trainers as they prepare other staff to assume responsibility in a civil service where there is significant rotation of staff within the provinces and districts. The one Significant area where a greater degree of flexibility is sought is in the relationship with the government so that the country can benefit from the full capacity of underutilized intellectual manpower. Score: 4 • Productivity: The combination of planning, coordination, operational control and continuous assessment are the tactical objectives that contribute successful strategic outcomes. It is apparent that the LlPHEAlHealth Alliance project at MUHAS has produced a broad array of outcomes consistent with the objectives spelled out in the original proposal. With over 6000 university students exposed to leadership training, hundreds of district level personnel, multiple Dean's conferences and the spread effect from more than two dozen issues of the East African Journal of Public Health are just some of the indicators of productivity for the project. Score: 5 • Connectivity: The establishment of the Deans Conference and the commitment to meet on a regular basis is to be applauded. However, it is apparent that a full level of connectivity across national lines awaits further development in the future. Internally, MUHAS was much better on building a national network and liaison framework to support work at the district level. Improved connectivity with government, notably the Office of the Prime Minister, and the Office of Disaster Management and Response, needs to be more fully encouraged. Score: 3 • Durability: The conclusion of the LlPHEAlHealth Alliance project served to reveal that the limited amount of attention paid to the longer term of project operations is most significant in compromising the whole condition of institutional strengthening. A standard condition of most grant programs is the obligation to demonstrate sustainability beyond the lifetime of the grant support. This did not happen at MUHAS. The university was successful in redesigning its operations after two years to adopt a more practical focus. However, it did not display continuing efforts to assess what would happen after the additional three years of support. The result is a very good program, but unless it becomes the beneficiary of additional financial support in the near term, it will tend to decline in terms of operational effectiveness. Faculty will pursue other options for research and professional development. This limitation on durability also reveals another limitation on the entire LlPHEAlHealth Alliance project. It could be considered too complex. The regional dimension of planning was worthwhile, and bringing in the African partners to select the U.S. counterparts was also a useful innovation. The financing of the project was more problematic. The use of an intermediary financial management agent was helpful, but there did not seem to be enough controls. Score: 3. JBS International p.54 Leadership Initiative for Public Health in East Africa Outcomes Assessment A final score of 24 out of 30 establishes that the L1PHEAlHealth Alliance project at MUHAS was very successful. It also indicates that with some greater attentiveness devoted to the tactical/objective tasks of project implementation, the more strategic dimensions of institutional strengthening could have been materially addressed and a higher score earned. Kenya The Republic of Kenya is the only country in the expanded Health Alliance that was part of the field evaluation of the L1PHEAlHealth Alliance project. It is also the only country in the Alliance represented by two Universities: the University of Nairobi and Moi University. After the L1PHEA program received continuing funding in 2008, both institutions were invited to join the emerging regional coalition. Since independence in 1963, Kenya's population has increased seven fold to nearly 40 million, more than 60 percent of whom are under the age of 15. Its resource base, long centered in agriculture and tourism, is challenged by changing climatic conditions, advancing desertification, and rapidly increasing costs of food production. Its development efforts have at times been undermined by malfeasant business practices. While the economy displays a healthy annual growth rate, Kenya still suffers from double digit inflation and an inability to absorb young people into the job market at a satisfactory rate. These conditions, irrespective of performance issues by various public and private personnel, compromise the ability of the nation to adequately meet demands for health, education, social services and infrastructure to support this rapidly changing profile of a young nation. Among the measures most widely used to measure civil order are press freedom, electoral regularity, and treatment of opposition views. According to these measures, Kenya is not perfect, but has an enviable record among the nations of the African continent. Moreover, it has built a rapidly expanding middle class which is generally considered vital to longer term civil order. It also provides a stability that permits attention to be directed to national crises that include 6.3 percent prevalence for HIV/AIDS in the entire population and, more disturbingly, 13.4 percent prevalence in Nyanza province. Clearly, there are "disasters" in zoonotic diseases, natural calamities and social conditions requiring both leadership and management. It is within such realities as presented here that the L1PHEA project and its successor, the HEALTH Alliance, have been conducted in Kenya. The Context of L1PHEAlHEAL TH Alliance Operations in Kenya Kenyan institutions were not original participants in the L1PHEA initiative; however, at the invitation of Muhimbili UniverSity, the two Kenyan institutions joined discussions related to the establishment of what would become the HEALTH Alliance. In 2008, when the original L1PHEA program evolved to include "leadership to prepare for and more effectively manage disaster conditions," the University of Nairobi and Moi University began their participation in the regional alignment. Both Kenyan universities became members of the Dean's Conference. These representatives included Dr. Peter Gatongi, Dean of the Moi University School of Public Health and Prof. M. Mwanthi, founding Dean of the School of Public Health at the University of Nairobi as it converted from a Department of Community Health in the College of Health Sciences to a separate school in 20 I O. Professor Mwanthi was also a founding member of the Dean's Conference. Moi University Established in 1999, the Moi University School of Public Health currently has a faculty of 27 full-time members and 33 part-time faculty members, providing an institutional strength, organizational coherence, and a notable record of performance to support the L1PHEAlHealth Alliance initiative. It enrolls 206 undergraduates at the Eldoret campus, and 334 post-graduate students at the Eldoret JBS International p. 55 Leadership Initiative for Public Health in East Africa Outcomes Assessment campus and two satellite campuses in Nairobi. Further, it also enrolls post-graduate students in Eldoret in the departments of health management, epidemiology and nutrition, and environmental health. The University's School of Public Health was invited to be a member of the LlPHEAlHealth Alliance consortium in 2008, an affiliation that linked it with six other schools of public health in the eastern Africa region. When Moi University joined this grouping, the discussions about a move from leadership training to a more functional focus on leadership training for response to disaster conditions, including epi-zoonotic outbreaks that are increasingly common in the eastern African region, were in advanced stages. The organizational means for implementing such a program was formalized in the HEMP. Because the work of HEMP extended beyond the field of public health, the Health Alliance also brought veterinary schools and programs in wildlife management into the collaboration. The result was a much broader inter-disciplinary approach to responding to medical as well as natural disasters such as flooding, landslides and famine relief. Planning While Moi University had a very limited role in planning for the HEMP initiative by its two partners, Makerere and Muhimbili Universities, it is clear that once invited into the HEALTH Alliance, the School of Public Health at Moi was heavily engaged in planning within the departments of the SPH, government departments and the donor community. The Moi University HEMP team has shown dedication to the complete range of tasks for providing leadership and disaster management training, follow-up contact and communication with district committees, and evaluation of the provisional disaster management plans which are required by the Government of Kenya for all districts that have received training. The Moi University HEMP team receives these provisional plans after they have been reviewed at the District Council level, places these plans in a standardized format, and submits them to the NDOC of the Kenyan Government which has oversight on all matters related to disaster preparedness and response in the country. Communication and Coordination The University is notable for intense standards of communication and coordination that are both internal and external. This involves constant interaction within the College of Health Sciences in Eldoret and attentiveness to contact with the Government of Kenya and the donor community. Moreover, the School of Public Health faculty and staff are fully aware that they must constantly increase their range of "focal points" or contacts within ministries and donor agencies. The HEMP team was placed in operational conditions when seeking to mitigate the violence arising from the 2007 elections during which Eldoret in the Rift Valley was considered the epicenter of serious disruption of public order. This included working with AMPATH to arrange for a HEMP team member, a Kikuyu tribesman, to be flown to a safer place until the disturbances subsided. The HEMP team was also called into service as part of contingency planning team during the referendum on the new Kenyan constitution in August 20 I I. HEMP operations extend to the satellite campuses, one of which is at the African Medical Research Foundation's (AMREF) international headquarters located in Nairobi. The AM REF Director of Capacity Building and Human Resources, Dr. Alice Lakati, teaches in the HEMP training program, and the HEMP team or other Moi University part-time faculty members participate in the programs for the certificate in leadership in management, the diploma in community health, and MPH programs offered by AM REF. For the MPH program, Moi University provides the classroom curriculum and AM REF provides the practicums required to complete the program. AMREF and Moi also collaborate on short courses in disaster management and an e-Iearning course for nurses at Moi University Teaching and Referral Hospital. The e-Iearning course substantially shortens the time required for the program. This JBS International p.56 Leadership Initiative for Public Health in East Africa Outcomes Assessment collaboration is another example of the practical manner in which Moi University spreads its commitment to national service in Kenya. Moi University's astute strategy is due to its innate institutional capacities and is not in any sense attributable to the LlPHEA program, as such. Operations Regarding project operations, the School of Public Health has an enviable record of training and hands￾on experience in "disaster management" conditions, e.g. responsiveness to the 2007 political violence and service during the constitutional referendum period of 20 I O. The work conducted under HEMP at Moi shows the need for flexibility, and trainings must be tailored to local conditions. As of July, 20 II, the Moi University HEMP team had completed 10 trainings programs involving 59 of Kenya's 284 districts and 315 individuals in all 8 provinces. It is worth noting that there has been a continuous change in the number of Districts, from 47 at the time of independence in 1963 to the current 284. The current plan under the new 20 I 0 constitution is to return to the original 47 counties that existed at independence. However, this will have limited impact because the sub-district administrative units will still exist and the development needs will still be there. In order to have some impact, the government and development partners such as Moi University will have to continue to seek the resources to cover the training needs of as many administrative units as possible. Even with these challenges, Moi University has conducted more trainings involving more individuals than either Makerere University or Muhimbili University and more individuals than have been trained in Rwanda, the Democratic Republic of Congo and Ethiopia combined. After three years, Moi University had established itself as the leading institution in the Health Alliance in terms of supporting the efforts of the Government of Kenya to have appropriate leadership and disaster management training all across the nation. However, this success has been achieved with minimal support from LlPHEA or the Health Alliance beyond funding for attendance at the Dean's Conferences and the week of training for six people in Kampala at Makerere. LlPHEA gets full credit for establishing the regional programs. Moi gets credit for having the institutional capacity to fully exploit opportunities such as HEMP to build its own relationships and reputation. Moi University's Interface with the Government and the Donor Community One of the critical factors in building and maintaining an initiative like the HEMP program at the Moi University School of Social Work is to have constant interface with government agencies and with the donor community. This interface is particularly important in African universities because these institutions typically do not have efficient budgeting systems as they are still organized along procedural lines developed in colonial times and do not have a close connection between required operating costs and revenue provided from government coffers. Ministries tend to operate on an incremental budgeting basis, for example, adding three percent more to the previous year's budget, even if the institution has been mandated to increase enrollments by 15 percent. This places each institution at a disadvantage. A second feature that contributes to the need for constant liaison is the absence of well-developed offices of institutional advancement or fund raising in cultures not used to institutional philanthropy. Moreover, these institutions are often not well-versed in proposal writing or grant management. These universities are gradually realizing the need for an office of grants management or, preferably, a Pro Vice-Chancellor for Research and Development. The Moi University School of Public Health understands these needs while it is forced to work within an institutional budget system and develop grant opportunities. There is realization that some ministries are difficult to work with, an example being some offices in Kenya's Ministry of Health. However, the Office of the President (e.g., the Ministry of Provincial Administration and Internal Security and the Ministry of J BS International p.57 Leadership Initiative for Public Health in East Africa Outcomes Assessment State for Special Programmes) is viewed as having a much more open view of the role of external donors and university interests in helping to solve problems. The Government of Kenya has realized the great need for additional training in disaster preparedness and management. Moi University School of Public Health can potentially gain support from the newly established Ministry of State for Special Programs (MOSSP), as funding for these programs flows through the Ministry. This link between the University and the government was key to securing funding for HEMP training that was not available via LlPHEA or the Alliance. In general, Moi's School of Public Health leadership and initiative have led to the success in connecting to MOSSP, NDOC, the Ministry of Health and the Ministry of Provincial Administration and National Security. This strategy of maintaining interface with both ministries and the donor community was affirmed in discussions with the Disaster Risk Reduction Unit of UNDP. This Unit would consider additional funding for Moi that would be channeled through the government, pending review of the Moi University HEMP program., Continuous contact with donors can assist programs like HEMP to identify possible funding sources. Assessment and Evaluation This constant assessment effort featuring mid-course corrections shows the level of attention devoted making sure the best work is provided. It is also reflected in the follow-up and assessment activities conducted following trainings in individual districts. Finally, the evidence produced on a summary CD shows how the HEMP team evaluates its performance on a routine basis. A review of one of the routine meetings of the HEMP team chaired by Dr. Simiyu exposed the range of interests of the team. Comments ranged from drought conditions in Garissa to the deepening drug problems, especially for youth in Malindi. The team discussed how the format for the "provisional district emergency management plans" could be more precise in the "concretization" of information, more emphasis on fire as a disaster threat, consideration of gender balance in responding to HEMP emergencies, and personal assessment of the value of skills learned. There was discussion of a broader range of disaster conditions referencing drought and famine, HIV/AIDS, transportation accidents, drug abuse, deforestation, terrorism and fire as well as epidemics and natural disasters. The team looked forward to developing preparedness and risk reduction manuals as a way of enriching the curriculum. Measuring Institutional Capacity Building at Moi University Moi University's institutional profile can be subjected to a systems approach rating scale that considers six elements working together in an effective public health development partnership. These elements are: learning, complexity, flexibility, productivity, connectivity, and durability. Performance is measured on a scale of 1-5 in each category with a I being low and a 5 being high, with a total possible score of 30; a score above 24 would be desirable. Using this measure for assessment of the Moi University School of Public Health as a member of the Health Alliance, the following conclusions are made: • Learning: The HEMP team and the Moi School of Public Health in general have demonstrated notable ability to first mount an excellent program of training, expand it via the use of its satellite campuses, amend it to meet the special needs of the various regions of the country and seek to improve the tactical elements of operations, i.e., the most effective practices in operations. Score: 5 • Complexity: The Moi School of Public Health has built an increasingly dense network of action points as it collaborates with AMPATH, engages in joint programming with AMREF, extends the range of its contacts with various Government of Kenya Ministries, seeks new sources of support from the donor community, extends the nature of its resource base on the Moi JBS International p.58 Leadership Initiative for Public Health in East Africa Outcomes Assessment University Eldoret campus and seeks to develop new programs including the master's degree in risk management and early consideration of a doctoral program in risk management. Score: 4 • Flexibility: The Moi School of Public Health has demonstrated very significant capacity for flexibility as it collaborates with AMREF on HEMP training and participation in certificate, diploma and master's degree programs offered by AMREF to provide alternative forms of leadership training in the fields of public health. It has also demonstrated a willingness to amend training techniques and content to recognize the realistic needs of different regions of the country (e.g., from response to preparedness and risk reduction). Thirdly, it has significantly expanded the range of conditions occurring in Kenya that should be considered disasters needing emergency management mitigation work. Score: 5 • Productivity: Despite not being an original member of the Health Alliance and having received limited financial support from the sources of support connected to the Alliance, the Moi University School of Public Health HEMP team has trained more district level personnel than either Makerere or Muhimbili universities and more personnel than Ethiopia, Rwanda, and DRe HEMP programs combined. Moreover, the Moi University School of Public Health has a better record of follow-up, submission of district emergency management plans to the Kenyan government and overall evaluation of district plans created at the district level following training. Score: 5 • Connectivity: A key factor in the Moi Univeristy School of Public Health's performance in all of the above areas is the school's environment of reaching out and connecting with customers, communities, public health professionals and networks. This is clearly demonstrated in the school's being a highly productive and contributing member of the HEALTH Alliance, and is also reflected in the school being able to build productive working relationships with government and nongovernment pUblic health actors throughout Kenya. Score: 5 • Durability: At a time when there is concern regarding the fiscal future of the Health Alliance, the Moi University School of Public Health is positioned well due to early and continuous administrative work in maintaining contacts with the Kenyan government, expansion of "focal points" or contacts within the government, pursuit of potential funding sources within the donor community such as UNDP, encouragement of strong one-campus leadership in HEMP programming and general senior administrative support of the University. It is fully considering new programs to be offered by the School of Public Health and in so doing is constantly thinking through the importance of "extending its product line" in order to invite new support. These latter actions are too often neglected by academic institutions and the School of Public Health is to be commended for its foreSight. Score: 5 The achievement of a score of 29 out of 30 by the Moi University School of Public Health marks it as a model member of the Health Alliance HEMP program. It has done so many things correctly that it should be examined more closely by other members of the Alliance so that some of its practices can be emulated elsewhere. University Of Nairobi The University of Nairobi has a demonstrated interest in the LlPHEAlHealth Alliance effort. However, it has not sufficiently developed its new School of Public Health to provide the kind of training in leadership and disaster management to match the commitments of Moi University. Historically, the University of Nairobi had only a department of public health and it was not until August of 20 I 0 that a School of Public Health was formally created. The new School of Public Health at the University of JBS International p.59 Leadership Initiative for Public Health in East Africa Outcomes Assessment Nairobi has not developed the organizational strength and spread of competencies that require a large faculty to articulate specific departments within its programs. This has no bearing on LlPHEA. Financial exigencies dictate that the SPH has not matured into a strong institution after only one year in existence. In June 20 I I, the faculty numbered I I and there were only a few part-time faculty members. Moreover, the more than 400 undergraduates currently being taught placed a significant burden on the teaching staff and compromised capacity to undertake new initiatives. The University of Nairobi is also positioned in a metropolitan setting where there are significant alternative entrepreneurial options that compete for faculty interest, notably the availability of consultancy opportunities in a place where donor agencies are represented fully. Despite having only a department of public health, the University of Nairobi was invited to join the Health Alliance even before an invitation was extended to Moi University. Prof. M. Mwanthi has been a member of the Health Alliance since its inception. The LlPHEA program was helpful in encouraging the creation of the School of Public Health and provided financial support in conducting an organizational workshop, funding attendance at deans meetings, and provision of training for four academic staff and one IT staff person at workshops at Makerere University. The University of Nairobi School of Public Health is a marginal academic program at this stage and not consistent with the expectations of an entity providing graduate education at the master's degree level. The University of Nairobi's School of Public Health is not a strong member in the Health Alliance as its attention is directed to other venues for support and development. Further, it remains currently an institutionally weak program. Dean Ongore of the University of Nairobi School of Public Health joined a new venture, OHCEA, where he serves as a founding director and he has worked to bring in 10 members from the Ministries of Agriculture, Health, Environment, Water Resources and Public Health. This new organization is supported by Tufts University and the University of Minnesota. Dean Ongore expressed that LlPHEA gave birth to the Health Alliance and the Alliance gave birth to OHCEA. He explicitly stated that the Alliance and OHCEA are different entities and that there is a problem of sustainability for the Alliance as its funding has come to an end. While OHCEA and the HEALTH Alliance are separate entities, the Dean may be too harsh and pessimistic about the future of the HEALTH Alliance since by the end of the core LlPHEA program in March 20 II, the Alliance had several long-term grants already in place and operating. Planning The University of Nairobi is beginning to write a strategic plan, but has not done serious planning relating to its affiliation with the Health Alliance. It does not have strong belief in the future of the Health Alliance. The fact that its Dean has moved to focus on another organization (OHCEA) to the disparagement of the Health Alliance and its predecessor LlPHEA is a telling condition. This compromises implementation of new programs beyond just a lack of funding. Communication and Coordination The University of Nairobi School of Public Health lacks staff, support staff, funding, organizational coherence, effective communication internally, evidence of sustained support from senior administration, collaborative partnerships with government and notable interest from the donor community. Dean Ongore believed that LlPHEA and the Health Alliance fully supported the HEMP program at Moi University, while a lack of similar support for Nairobi encouraged disinterest among the School of Public Health's faculty who "saw nothing was new to them." Currently there are no plans for maintaining the Health Alliance as all meetings were mostly funded by LlPHEA and on rare occasions by the Alliance. He is of the opinion that the Alliance has set aside no funding for attending further meetings or training organized either by LlPHEA or the Alliance. JBS International p.60 Leadership Initiative for Public Health in East Africa Outcomes Assessment Operations The University of Nairobi School of Public Health did not create a HEMP team and has not trained any district level personnel under the Health Alliance initiative. It is likely that the school declined to participate in the HEMP program for several reasons: I) LlPHEA funds were limited and could only support one school in Kenya, and the Ministry of Public Health and Sanitation had already expressed preference for working with Moi University as a more capable institution to provide the training; 2) The school may not have had faculty resources available to conduct the training since its faculty resources are stretched; and 3) There is a preference for faculty to seek lucrative external consultancies, and personal advantages of participation in the HEMP program would have been limited. The founding dean of the School, Prof. Mwanthi, acknowledges the value of the LlPHEA leadership training in which he was a participant and for which he was appointed a "regional trainer in leadership." He has used this experience to develop a leadership exercise in training to be done with Prof. Wangombe in July 20 II, at Moi University and involving Schools of Public Health in Kenya, Tanzania, and Uganda and funded by the Government of Tanzania. Information Technology Regarding IT competencies at the University of Nairobi, one individual participated in two trainings that were conducted by Makerere University, supported by LlPHEA and led by faculty from Tulane University. These trainings were noted as not useful since the e-Iearning platform at the University of Nairobi is the Carolina Platform (Tufts University Sciences Knowledgebase) from Boston University and is incompatible with the Tulane materials. Unless Tulane was in a position to finance and facilitate conversion of the platform, the University of Nairobi probably should not have been included in the training. Additionally, other materials, while relevant, could not be easily used because undergraduate students have limited computer time in the IT labs at the University. Moreover, students are not full-time and attend in the evenings. Overall, it is not an optimum form of instruction for the University of Nairobi School of Public Health. The IT programming is compromised by the fact that main IT staff consists of one individual who has been employed for six years, but is not a permanent staff member and has no support staff. Representatives from Tulane appeared to believe that the University employed full-time IT staff. This misunderstanding about whether or not UoN had the basic management structures in place, such as permanent IT staff, to benefit from Tulane assistance speaks to the larger question of whether or not UoN should have been invited to join the HEALTH Alliance at such a formative stage of its development. While the IT staff member has implemented a Medical Education Program Initiative for the medical students, he cannot accommodate a diversity of disciplines with one common e-Iearning tool, e.g., the department of surgery compared to the department of obstetrics and gynecology. He is constantly looking for new tools for faculty. The resource-constrained environment in which he works may have led him to believe that the U.S. partners have benefitted more than the University of Nairobi, when the U.S. partners, particularly Tulane, seem to have no such issues. These perceptions combined with those of his superiors point to a troubled relationship with LlPHEA and the HEALTH Alliance; however, these troubles likely extend beyond mere breakdowns in communication. It is pOSSible that the University of Nairobi was not at a sufficient threshold to join and be a productive member of the Alliance. It did not, at the time of its joining the LlPHEAlHEAL TH Alliance, possess any of the criteria originally envisioned by architects of LlPHEA as desirable in an institutional partner. The school's institutional weaknesses were many; LlPHEA resources available JBS International p.61 Leadership Initiative for Public Health in East Africa Outcomes Assessment through the U.S. partners were restrictive; and HEALTH Alliance, a nascent regional organization may not have been in a position to help much. Assessment and Evaluation On-going evaluation is not present and overall assessment of operations on a routine basis does not seem to be present. Measuring Institutional Capacity Building at the University of Nairobi The criteria for measuring institutional capacity cited above are applied here. The scores and rationale are as follows: • Learning: The School of Public Health has not displayed evidence of effective learning capacity. It laments it poor state but does not learn even after years of experience as a department of community health. It has not illustrated evidence of planning capacity that is fundamental to institutional learning. It is taking some tentative steps towards operational planning. It has not defined a mission except to abandon the Alliance and promote OHCEA. In these conditions it is floundering. Score: 2 • Complexity: The School of Public Health is continuing to act like a department. It is not building the internal, associational networks in its own community or in the Alliance and is failing to establish networks with Government. It remains focused on the burdens of undergraduate education. It cannot engage its faculty in building the program. For these reasons, the Score: 2 • Flexibility: The School of Public Health remains mired in old thinking. It is apparent that it did not conduct a standard readiness study that should precede the decision on the creation of a new school of study. With few resources, both human and material, and required instructional obligations, the School has very limited options or reallocation of its limited fungible assets. Score: 2 • Connectivity: The School is immobilized by its internal institutional challenges and appears to be too distracted to reach out to better inform itself through information and experience sharing with counterpart institutions or the customers, and professional communities it is intended to serve. Although it is not completely isolated, its networking efforts all seem to center on institutional needs. Score 2 • Productivity: The School has some vague ideas of future directions, but it is limited to undergraduate training and educating self-paying MPH students in a night school program while having few assets. Until it secures notable commitments from the senior University of Nairobi administration, e.g. Vice-Chancellor, Deputy Vice-Chancellor and University Council, that the school is a priority, it will remain not qualified to offer graduate education. Score: 3 • Durability: Until the School of Public Health receives substantial additional resources either from Government or massive external aid, its future will be precarious. It will be unable to attract good faculty. It will be unable to conduct meaningful research. It will struggle to provide true university level learning. Score: 2 The score of 13 out of a possible 30 is a failing rate for institutional capacity building. The school opened without proper planning or a readiness study. It is unsupported and does not seem to have the strong commitment of its Dean to support the HEALTH Alliance. JBS International p.62 Leadership Initiative for Public Health in East Africa Outcomes Assessment Conclusion Overall, LlPHEA was found to be a highly productive and consequential program. It was successful to a large degree in its activities engaging other universities, governmental organizations, and NGOs. There were, however, two areas that could have been strengthened, even in light of the strong institutional bases that three of the four universities included in the study had, and these are related to improving faculty development and teaching and teaching infrastructure. The African Partner Institutions and the HEALTH Alliance partners might have benefitted from increased strategic and technical guidance from the U.S. Partner Institutions. In addition, closer program monitoring at the HED level might have caught and corrected certain strategic and management weaknesses in the program earlier-on. Beyond these broad observations, this Assessment yielded detailed findings, lessons learned, and recommendations which are described under each objective. These are recapitulated from the text and presented below, organized by programmatic objective. • LlPHEA successfully created and institutionalized a regional academic network to address leadership gaps and other challenges in public health program • A major achievement of LlPHEA was strengthening the LlPHEAlHEAL TH Alliance capacity in management of information, communications, and instructional • The HEALTH Alliance, a regional alliance of seven public health institutions in six East African countries, is a chartered organization that was registered in Uganda in 2008, which can receive and disburse funds, effectively transforming the LlPHEA grant-based partnership involving only two African institutions into a permanent membership organization providing a regional public health platform from which • potential for long-term sustainability, albeit somewhat fragile. In the near-term, the HEALTH Alliance has increasingly gained local, regional, and international recognition as a committed and coordinated resource for research, leadership development, information technology and training capacity in a range of public health areas, includ and diseases, JBS International p. 63 Leadership Initiative for Public Health in East Africa Outcomes Assessment emergency and disaster management, and preventive health care. • Although the HEALTH Alliance is correctly described as an outgrowth of early deliberations of the Deans and Directors Meetings and is generally thought of as a strictly regional phenomenon engendered by the future vision of the Deans and Directors, the U.S. partners also played a role in these early developments by encouraging and fostering LlPHEA entrepreneurship. In the best spirit of regional cooperation, the periodic meetings of Deans and Directors, in effect, functioned as the HEALTH Alliance's • LlPHEA is generally recognized as having contributed to increased awareness and demand for effective leadership in Schools of Public Health and among public health stakeholders in the region. LlPHEA has enabled thousands of students, faculty and professionals to be exposed to various aspects of leadership development within the context of public health program • The regional reach of LlPHEAlHEAL TH Alliance activities and the program's ability to raise awareness and increase knowledge and skills of public health students and professionals in six countries, the region, and globally, are notable achievements. The outputs and numbers of people whose personal and professional lives have been touched by LlPHEAlHEALTH Alliance are well-documented and borne out by numerous anecdotal accounts of people that were interviewed during this Assessment. • There are indications that LlPHEAlHEALTH Alliance HEMP had a verifiable programmatic spread effect in Uganda and Kenya. In Uganda, training of senior government personnel was instrumental in formation of the government's disaster management office. At Moi University in HEMP trai had migrated to JBS International p.64 • • • Leadership Initiative for Public Health in East Africa Outcomes Assessment several other programs within the university (e.g., certificate, diploma, and e￾learning courses), and the program was in high demand among public health professionals, government, donor partners, and nongovernment public health stakeholders. In structuring its leadership development training program, LlPHEA did not appear to have adequately consulted with government and nongovernment implementers (as potential users of personnel trained in the program). As a result, LlPHEA's generic post-basic in-service leadership development training program did not create effective and sustainable demand. In development and delivery of HEMP training (during LlPHEA, Years 3-5), there was considerable variation in the nature and extent of consultations initiated by participating LlPHEAlHEALTH Alliance schools with government and development partners, resulting in varying degrees of success (quality, use and sustainability) of the cou HEMPs. As relates to both leadership and HEMP training, there did not appear to be effective mechanisms in place to identify needs and provide on-going assistance to participating schools, as necessary, in forging sound working relationships with government and development partners. Although the U.S. Partners OHU and Tulane) had opportunities during LlPHEA instructional technology training to make some inroads in terms of strategic/technical approaches to design/development of public health development training programs, there may have been some reticence to do so. The regional nature of the partnership as well as USAID grant-funding and grant management guidelines that caution against substantial involvement of grantors may have been nr.,.,on .. ""rI Government partners in two of the three LlPHEAlHEAL TH Alliance countries visited felt that all training had been useful, and had at least served the of In partnership programs that embrace a approach, USAID and HED may wish to consider ways in which host country partner institutions can more readily benefit from strategic and technical guidance that might be needed and forthcoming from U.S. Partner Institutions. Partnering roles and responsibilities related to provision of such guidance may need to be made more explicit in grant agreements, if possible. Alternatively, such understandings may need to be established in the working relationships of the partners. JBS International p.65 Leadership Initiative for Public Health in East Africa Outcomes Assessment increasing public health leadership and program developers to optimally engage professionalism in their countries and the government and nongovernment implementers region. L1PHENHEALTH Alliance training and effectively incorporate their needs/interests program planners in the third country into design and development of competency￾might have benefited from strategic based training. guidance and technical assistance in strengthening its coordination with implementers (in this case, government). Training designs could have been further strengthened by more specifically involving those stakeholders with best understanding of worker competencies such as potential employers or supervisors. • Regardless of how well in-service training Competency-based regional training programs programs have been constructed to meet need to consider regional disparities and have the the practical, competency-based needs of capacity to customize country programs to trainees, training results/outcomes will be needslinterests and constraints of the local diminished if the most appropriate trainees setting. are not selected. • When identifying training priorities, L1PHEA tended to confer with public health professionals and students and less specifically with governmental and nongovernmental implementers. This may have resulted, at least in part, in L1PHEA making a decision not to offer health care financing courses, thereby missing what may have been an opportunity to advocate for and assume leadership in development of Africa-specific health care financing expertise. • Post-basic and in-service training provided In the interest of optimizing post-basic and in￾by L1PHENHealth Alliance was viewed as service training investments, USAID and HED useful, practical and relevant in two of the should ensure that grants include a mandate, three countries visited, and technical assistance and resources for training L1PHENHEALTH Alliance planners used program developers to optimally engage technically sound, standard training government and nongovernment implementers program design/development methods and effectively incorporate their needs/interests including needs assessments. into design and development of competency￾L1PHENHEALTH Alliance training program based training. planners in the third country might have benefited from strategic guidance and technical assistance in strengthening its coordination with implementers (in this case, government). Training designs could have been further strengthened by involving those stakeholders with the best understanding of worker competencies such as potential employers or supervisors. • Criteria for systematic selection of in- In the interest of optimizing post-basic and in￾JBS International p. 66 Leadership Initiative for Public Health in East Africa Outcomes Assessment service trainees appeared to be weak or absent in the civil service systems from which trainees were drawn. Regardless of how well in-service training programs have been constructed to meet the practical, competency-based needs of trainees, training results/outcomes may be diminished if the most appropriate trainees are not selected. • Regional competency-based training programs need to consider regional disparities and have the capacity to customize country programs to needs/interests and constraints of the local setting (such as differences in governance, governmental administration and 1~I!~s~tructures, an~djiim~~== • No formal faculty development plans complete with long and near term objectives and action steps were described or otherwise detected by the assessment team. This is a program management oversight that should have been picked up through the program monitoring and evaluation n .. r,ro.oc: • LlPHEA might have strengthened the content, validity, and relevance of its training, and provided learning opportunities for its undergraduate and graduate students and research opportunities for professors and graduate students through more strategic linkages between training and research nnnn,-ri nities. • Although many productive teaching infrastructure improvements were undertaken, there was limited attention to improving teaching generally. The activities undertaken were focused on content and presume good adult learning practices are • used. The absence of a staff development plan may have contributed to what appears generally to be a low level of staff and faculty training and development on technical teaching and train methods. JBS International service training investments, should ensure that grants include a mandate, technical assistance and resources for training program developers to enter into effective dialogue with government and nongovernment implementers to standardize structures and procedures for selection of training program participants. In support region training programs, and HED should ensure that grants include technical assistance and resources to assist implementers in effectively customizing regional approaches and curricula to meet local country requirements. Each of the primary partner universities should institute staff development plans, and ongoing monitoring should ensure progress on this objective. partners opportunities to strengthen their courses and training programs, and U.S. Partner Institutions, which all have extensive experience with resource leveraging in resource-constrained academic environments, should provide technical guidance to assist in this process. improvements in teaching infrastructure, instructional technology and staff development together would likely have improved outputs and outcomes for all three areas. p. 67 Leadership Initiative for Public Health in East Africa Outcomes Assessment • Some members of the LlPHENHealth Alliance feel they did not benefit equitably from LlPHEA funding, including the Rockefeller Foundation contribution that was available during the 2008-20 I 0 period. • Although the issue of reciprocal benefit of partnerships to U.S. partner institutions is of interest to policy makers who promote non-regional partnerships, neither U.S. nor African Partners appeared to strategically focus on this concern. • The basic standard reporting format used for semi-annual reports to higher levels did not encourage partners to report challenges in such a way that would have informed higher level managers and assist the monitoring and ",.",,,,,,,,,,,,,r, .. follow-up process. JBS International may be instructive for HED to review and clarify how equitably LlPHEA funds for HEMP training and other institution building activities (such as leT) were shared during the 2008-20 10 period, and to review the patterns of accountability in funds disbursement across all seven LlPHENHEAL TH Alliance "'., .... "' ...... ~ H may wish to review partnership reporting formats to ensure that information being provided includes references to operations, communications and assessment or mid-course corrections to assist project monitoring and trigger management interventions and course amendments when indicated. p.68 Leadership Initiative for Public Health in East Africa Outcomes Assessment Appendix A: List of LlPHEA Partners and Collaborators CATEGORY Funding agencies Program Administrator U.S. Partner Institutions Host Country Partner lristitutions International technical collaborators JBS International PARTICIPATING ORGANIZATIONS • Primary/core LlPHEA funding: U.S. Agency for International Development • Supplementary/complementary funding: · The Rockefeller Foundation · The United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA) · The United Nations Children's Fund (UNICEF) · The World Bank Higher Education for Development (HED) • Johns Hopkins University (Partnership Lead) · Bloomberg School of Public Health • Tulane University · The Payson Center for Int'l Development · School of Public Health and Tropical Medicine • George Washington University School of Public Policy and Public Administration • Makerere University • Muhimbili University of Health Sciences • Centers for Disease Control (CDC) • African Field Epidemiology Network (AFENET) • Management Sciences for Health, Center for Leadership and Management (MSH/CLM) • IBM Research Group, East Africa • African Medical Research Foundation (AM REF) • Lehman College of New York (NSF grant) • University of Minnesota and Tufts University (USAID/RESPOND grant) • Johns Hopkins Bloomberg School of Public Health and the Future Health Systems Consortium (DFID grant) LOCATION Washington DC Washington DC Baltimore MD New Orleans LA New Orleans LA Washington DC Uganda Tanzania Atlanta GA Kampala, Uganda Cambridge MA Nairobi, Kenya Nairobi, Kenya New York NY New York NY Baltimore MD p. 69 Leadership Initiative for Public Health in East Africa Outcomes Assessment HEALTH Alliance Institutions Higher Education Alliance for Leadership Through Health (HEALTH) Host Government Collaborators Other Collaborating Host Country Institutions (Limited Sample Focus Group to be identified by Partner Institutions) Individual beneficiaries (Limited Sample Focus Group(s) to be drawn from partner institutions, the HEALTH Alliance, government collaborators, other host country collaborating institutions) JBS International • • • • • • • • • • • • • • • • • • • • • • • Makerere University School of Public Health Muhimbili University of Health and Allied Sciences Moi University School of Public Health University of Nairobi, College of Community Health Jimma University College of Public Health and Medical Sciences Universite de Kinshasa Ecole de Sante Publique National University of Rwanda School of Public Health Office of the Prime Minister (for disaster management) Ministry of Health (national and district level) Ministries of Agriculture and Veterinary Services The AIDS Support Organization (TASO) MildMay International Straight Talk Foundation Joint Medical Stores Kitovu Hospital, Uganda Martyrs University Uganda Protestant Medical Board African Evangelistic Enterprise Kampala Diocese of Church of Uganda Baylor College of Medicine Foundation Mulago Mbarara JOint AIDS Program World Vision International Kabwohe Clinical Research Centre To be identified by suggestion of assessment coordinators, and as available Uganda Tanzania Kenya Kenya Ethiopia DR of Congo Rwanda Uganda Tanzania Uganda Tanzania Uganda Tanzania Uganda p. 70 Leadership Initiative for Public Health in East Africa Outcomes Assessment Appendix B: List of Individuals and Organizations Contacted United States Dr. Dale Gibb, USAID/Global Health Bureau Dr. Dennis Carroll, USAID/Global Health Bureau Professor Gilbert Burnham, Johns Hopkins Bloomberg School of Public Health Daniela Lewy, Research Associate, Johns Hopkins Bloomberg School of Public Health Associate Professor Nancy Mock, Tulane University Adam Papendieck, Senior Program Manager, Tulane University Uganda Professor David Serwadda, Chairman, HEALTH Alliance Associate Professor William Bazeyo, Principal Investigator, LlPHEA, and Dean, Makerere University School of Public Health Dr. John Ssempebwa, Head, Dept. of Disease Control & Environmental Health, Makerere University School of Public Health Dr. Christopher Garimoi Orach, Technical Advisor, Health Emergency Management Project (HEMP) Dr. Roy Mayega, Team Leader, Health Emergency Management Project (HEMP) Dr. Terence Odoch, Lecturer/Researcher, Country Team Member, Health Emergency Management Project (HEMP), and Dept. of Veterinary Public and Preventive Medicine, Makerere University Dr. Juliette Babirye, Strategic Leadership Short Courses Coordinator, Makerere University School of Public Health Deogratious Sebuwufu, IT Manager, Makerere University School of Public Health Dr. Elizabeth Ekirapa-Kiracho, Project Manager, Future Health Systems Project, Makerere University School of Public Health Dr. Immaculate Nabukenya, Country Team Member, Health Emergency Management Project (HEMP), and Veterinary Epidemiologist, Ministry of Health Vincent Woboya, Senior Assistance Secretary, Office of the Prime Minister Francis Odap, Deputy Chief Administrative Officer (DCAO), Mukono District James Ntege, Probation Child Welfare Officer, Mukono District Ben Otim Ogwette, Chief Administrative Officer (CAO), Jinja District Martin Kiplangat, Deputy Chief Administrative Officer (DCAO), Jinja District Mugweni Badira, Principal Associate Secretariat, Jinja District Male William Kayiwa, Branch Manager, Uganda Red Cross Society, Jinja Dr. Peter Oyogo, Agricultural Development Officer, Jinja District Gideon Kawekwa, Statistician, Jinja District Dr. Stephen Kiwemba, District Production Marketing Officer, Jinja District Dr. Geoffrey Kabagombe Rugambo, Program Manager, One Health Central & Eastern Africa, (OHCEA) Andrew Omale, Officer in Charge, Red Cross Disaster Response Lendell Foan, Regional Manager, Emerging Pandemic Threats Program (USAID/RESPOND) Dr. Monica Musenero Masanza, Senior Program Officer, African Field Epidemiology Network (AFENET) Tanzania Dr. Simon Mamuya, LlPHEA Coordinator, MUHAS School of Public Health and Social Sciences JBS International p.71 Leadership Initiative for Public Health in East Africa Outcomes Assessment Idda Mosha, Lecturer, MUHAS School of Public Health and Social Sciences Paulina Nahato, Lecturer, MUHAS School of Public Health and Social Sciences Asanali Msangi, Information Technology, MUHAS School of Public Health and Social Sciences Kishashu Yahaya, MUHAS School of Public Health and Social Sciences Dr. Gideon Kwesigabo, Dean, MUHAS School of Public Health and Social Sciences Prof. Mainen Moshi, Director of Information Technology, MUHAS Felix Sukum, Information Technology Specialist, MUHAS Dr. joseph Chuwa, Disaster Management Coordination, Ministry of Health Prof. japhet Killewo, L1PHEA Principal Investigator, MUHAS, School of Public Health and Social Sciences judith Bihondwa, Emergency and Disaster Management Liaison Officer, UNICEF Dr. Kaniki Issessanda, District Medical Officer, Kibaha Sayuni Ngai District Agricultural Officer, Kibaha Lumumba Amiri, Health Officer, Kibaha Prof. Medzedeck Leshabari, MUHAS School of Public Health and Social Sciences Willy Mathew, Regional Coordinator, Red Cross Harrisson Chinyuka, Department of Disaster Management, Prime Minister's Office Kenya Prof. Richard Mibey, Vice-Chancellor, Moi University Prof. Samuel Gudu, Deputy Vice-Chancellor for Planning & Development, Moi University Dr. Paul Ayuo, Dean, Moi University Medical School Dr. Peter Gatongi, Dean, Moi University School of Public Health Dr. Mabel Nangami, Head, Department of Health Policy and Management, Moi University School of Public Health Frederick Etyangi, Senior Secretary, Moi University School of Public Health Dr. john Tabu Simiyu, Senior Lecturer, Moi University School of Public Health The Moi University Health Emergency Management Team (HEMP): Dr. Peter Koskei IvyWanyama Allan Kamanda Frederick Majiwa Nancy Nygoha Christina Otieno Bernice Biomndo, Director of Public Relations, Academic Model Providing Access to Healthcare (AMPATH) Dr. Alice Kakati, Training Coordinator for MPH and Diploma in Community Health, African Medical and Research Foundation (AM REF) and part-time faculty at Moi University Satellite Campus in Nairobi Mwihaki Kimura Muragani, Associate Director, The Rockefeller Foundation in Nairobi Oscar Ebalu, Programme Officer, Disaster Reduction Unit, UNDP, Nairobi Dr. james Teprey, Coordinator, Emergency Humanitarian Action Sector, World Health Organization Dr. Edward K. Kiema, National Disaster Operations Center Philip Mulkuska, Ministry of Public Health and Sanitation Dr. Richard Ayah, Lecturer, Nairobi University School of Health Dr. Dismas Ongore, Dean, University of Nairobi School of Public Health Dr. Mutuku A. Mwanthi, former Dean of University of Nairobi School of Public Health Dr. S. K. Sharif, Director of Public Health and Sanitation, Ministry of Public Health and Sanitation Dr. Amira, Director of Disaster Management, Ministry of Public Health and Sanitation Fred Ndungu, Director, Training and Reforms, Ministry of Provincial Administration and Internal Security Col. (RET.) Vincent Lee Anami, Director, National Disaster Operations Center, Office of the President JBS International p. 72 Leadership Initiative for Public Health in East Africa Outcomes Assessment Col. (RET.) Joseph Kingori, Deputy Director, National Disaster Operations Centre, Office of the President Rwanda Dr. Etienne Rugigana, Team Leader, Health Emergency Management Project (HEMP), National University of Rwanda School of Public Health JBS International p. 73 Leadership Initiative for Public Health in East Africa Outcomes Assessment Appendix C: Assessment Interview Worksheet Name of Organization E-mail contact Respondent category: Host Country Partner Institution Respondent(s) Names/Titles Length of involvement with LlPHEA Date RESPONSE TO KEY QUESTIONS AS APPLICABLE: • From your perspective, what were the strengths and weaknesses of teaching and teaching infrastructure at the SPH prior to LlPHEA? What are they now? Assessor's comments/ rating INTERNAL USE ONLY • To what extent has leadership training been incorporated into curricula, and how was this accomplished? • To what extent has a formal faculty development program been established at your SPH? How have young faculty been mentored? Have you personally benefited from these faculty development and/or mentoring activities? If so, how? • How successful has the HEALTH Alliance been? What factors contributed to its success? What approaches could have been improved? • Describe the SPH and LlPHEA working relationships with government partners. Have relationships improved during the lifetime of the LlPHEA program? If so, how? • Describe the SPH and LlPHEA working relationships with public health practitioners and professional associations. Have relationships improved during the lifetime of the LlPHEA program? Explain. • How do you believe your U.S. partner institutions have benefited from the LlPHEA partnership? • What are your impressions of the extent to which LlPHEA has improved the knowledge of public health and emergency management among faculty, decision-makers, district health officers and first responders? Give examples. • Do you believe LlPHEA participants are able to apply what they learned through the program to their daily work? Give examples. • Has the LlPHEA had an impact on university structures and processes that extends beyond the SPH? If so, how? • Projecting 5 years into the future, what do you believe will be seen as the main legacy of the LlPHEA? What lessons have been learned? J BS International p. 74