1 APRIL 2014 This publication was produced at the request of the United States Agency for International Development (USAID) Tanzania (TZ) Mission. It was prepared by: Lily Asrat, Dr.P.H., Temitayo Ifafore, M.P.H., Jeffrey Blander, Sc.D., Angela P. Mwaikambo, M.A., and Regine Jean-Francois, M.P.H. Leveraging the power of Public Private Partnerships to improve Human Resources for Health in the Lake Zone of Tanzania: an End of Project Evaluation Report ii Cover photo courtesy of Paul Joynson-Hicks (2008) iii LEVERAGING THE POWER OF PUBLIC PRIVATE PARTNERSHIPS TO IMPROVE HUMAN RESOURCES FOR HEALTH IN THE LAKE ZONE OF TANZANIA: AN END OF PROJECT EVALUATION REPORT April 2014 Project Number: No. 621-A-00-11-00002-00 Development Experience Clearinghouse catalog number: PB-AAA-331 Project Title: "Strengthening the Tanzanian Health System by Increasing Number and Quality of Health Workers and Improving the Health Delivery Mechanisms in the Lake Zone of Tanzania” DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government iv This document is available online and can be located through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: Angela Mwaikambo Monitoring and Evaluation Specialist Program Office USAID/Tanzania Tel: +255-22-2668490 ext. 4611 Fax: +255-22-2294421 amwaikambo@usaid.gov v ACKNOWLEDGEMENTS We would first and foremost like to thank the United States Agency for International Development (USAID) Tanzania Mission, and in particular the Mission Director Sharon Cromer, the Program Office Director Craig Hart, the Health Team Lead Alisa Cameron, and the Health System Strengthening Team Lead Susna De. We thank them for inviting us to conduct this important evaluation and for the guidance and collaboration they offered us in its planning and execution. We would like to pay a special note of thanks to Gene Peuse, the Agreement Officer Representative (AOR) for the Touch Foundation partnership. His guidance, collaboration, advice, and support throughout the planning, implementation, analysis, and report writing has been unwavering. We would also like to thank the entire Touch team, from the respective offices including: Dar es Salaam, Mwanza, and New York. The team gave generously of their time, energy, and willingness to answer our endless stream of questions and shared the data and information they have collected over many years. We thank them for recounting their successes as well as their honest and candid descriptions of the challenges they faced along the way. They worked tirelessly to contact numerous stakeholders on our behalf to ensure that we had access to variety of information, resources, and perspectives necessary to synthesize findings on the project’s implementation, successes, and areas for improvement. Their commitment and dedication to the implementation of this project in order to improve the quality and supply of human resources in the Lake Zone is irrefutable. The evaluation team would also like to thank the Government of the Republic of Tanzania (GoT) for its dedication and support of Human Resource for Health (HRH) interventions and the need to evaluate and take stock of investments already made in order to inform future directions. The willingness of government representatives to speak to us at both the national and regional levels was highly appreciated. Finally, we would like to pay a special note of thanks to the beneficiary institutions, their leadership, their staff, their students, and their alumnae who provided candid and useful thoughts, perspectives, insights, and recommendations. We appreciate the time that they took away from their busy schedules to: give us tours of facilities, meet with us one-on-one, and provide us with all of the information that we requested. This evaluation could not have been carried out without their generosity of spirit and genuine desire to provide information that could benefit future programming and lead towards the improvement of the health service delivery systems in Tanzania. To all of you, we say asanteni sana! Thank you! vi EVALUATION TEAM Abridged biographies of the evaluation team members are below. Full biographies are in Annex 1. Lily Asrat, Dr.P.H. (Evaluation Team Lead) Senior Evaluation Advisor United States Agency for International Development (USAID)/Washington Dr. Asrat serves as the Senior Evaluation Advisor in the Strategic Planning, Evaluation, and Reporting (SPER) Division of the Office of HIV/AIDS (OHA) at USAID. Temitayo Ifafore, M.P.H. Health Workforce Technical Advisor United States Agency for International Development/Washington Ms. Ifafore serves as the Health Workforce Technical Advisor in the Service Delivery Improvement (SDI) Division of the Office of Population and Reproductive Health (PRH) at USAID. Jeffrey Blander, Sc.D. Deputy Director Private Sector Engagement Office of the Global AIDS Coordinator U.S. Department of State Dr. Blander serves as the Deputy Director of the Office of Private Sector Engagement (PSE) in the Office of the Global AIDS Coordinator (OGAC). Angela P. Mwaikambo, M.A. Monitoring and Evaluation Specialist United States Agency for International Development/Tanzania Ms. Mwaikambo serves as the Monitoring and Evaluation Specialist for at USAID/Tanzania with the Strategic Planning and Program Support team at USAID/Tanzania. Regine Jean-Francois, M.P.H. Analyst United States Agency for International Development/Washington Ms. Jean-Francois serves as an Analyst in the Strategic Planning, Evaluation, and Reporting (SPER) Division of the Office of HIV/AIDS at USAID. vii TABLE OF CONTENTS EVALUATION TEAM…………………………………………………………………………………………………………....vi ACRONYMS ...........................................................................................................................................................ix EXECUTIVE SUMMARY.................................................................................................................................................. xi INTRODUCTION................................................................................................................................................................. 1 Evaluation Rationale.......................................................................................................................................................... 1 Evaluation Questions......................................................................................................................................................... 1 Project Background............................................................................................................................................................ 1 Project objectives................................................................................................................................................................ 2 Evaluation design, methods limitations .................................................................................................................... 2 EVALUATION PURPOSE, USE, & AUDIENCE ............................................................................................... 3 Purpose.................................................................................................................................................................................... 3 Use ............................................................................................................................................................................................. 3 Audience.................................................................................................................................................................................. 3 EVALUATION QUESTIONS ............................................................................................................................... 4 Effectiveness ......................................................................................................................................................................... 4 Reproducibility .................................................................................................................................................................... 4 Sustainability ........................................................................................................................................................................ 4 PROJECT BACKGROUND................................................................................................................................................ 7 HRH in Tanzania.................................................................................................................................................................. 7 Project history ...................................................................................................................................................................... 8 Project objectives................................................................................................................................................................ 9 EVALUATION METHODS & LIMITATIONS............................................................................................... 13 Evaluation type and methods ...................................................................................................................................... 13 Desk review......................................................................................................................................................................... 14 Key informant interviews.............................................................................................................................................. 14 Project data.......................................................................................................................................................................... 15 Sampling ............................................................................................................................................................................... 15 Informed consent.............................................................................................................................................................. 15 Data collection and analysis ......................................................................................................................................... 15 Limitations........................................................................................................................................................................... 16 FINDINGS, CONCLUSIONS & RECOMMENDATIONS.............................................................................. 17 Findings: Effectiveness ................................................................................................................................................... 17 Findings: Reproducibility .............................................................................................................................................. 28 Findings: Sustainability .................................................................................................................................................. 33 CONCLUSIONS................................................................................................................................................... 41 Effectiveness ....................................................................................................................................................................... 41 Reproducibility .................................................................................................................................................................. 41 Sustainability ...................................................................................................................................................................... 43 RECOMMENDATIONS..................................................................................................................................... 43 Effectiveness ....................................................................................................................................................................... 43 Reproducibility................................................................................................................................................................... 45 Sustainability....................................................................................................................................................................... 46 viii ANNEXES............................................................................................................................................................. 49 Annex I: Evaluation Team Full Biographies ........................................................................................................... 49 Annex II: Evaluation Scope of Work.......................................................................................................................... 53 Annex III: Data Collection Instruments.................................................................................................................... 62 Annex IV: Documents Reviewed................................................................................................................................. 75 Annex V: Key Informants ............................................................................................................................................... 77 TABLES Table 1: Summary Table of Challenges and Recommendations………………………………………………..xix Table 2: Health Worker Shortage by Cadre…………………………………………………………………………….....8 Table 3: Key Findings, Conclusions, And Recommendations (Effectiveness)……………………………..27 Table 4: Key Findings, Conclusions, And Recommendations ……………………………………………………32 Table 5: Key Findings, Conclusins, and Recommendations (Sustainability)……………………………….39 FIGURES Figure 1: Touch Foundation Logical Framework………………………………………………………………………12 Figure 2: Treat and Train Conceptual Framework……………………………………………………………………13 Figure 3: Touch vs. non-Touch contribution to CUHAS to income....…………………………………………..37 Figure 4: Touch contribution as a percentage of CUHAS total income………………………………………..37 Figure 5: Total cost of Treat & Train Network extension at Sengerema hospital………………………..38 ix ACRONYMS AMO AMREF AOR APHFTA BMC CSSC Co-Ag COP CPD CSR CUHAS DrPH FBO GoT HCM HRD HRH HRMIS HSS ISPE ICT IT KCMC LMG MD M&E MEPI MMED MO MOH MOF MOHSW MOU MPH MUHAS OGAC PEPFAR PMORALG PMP POC POPSM PPP PSE PRH QI Assistant Medical Officer (formerly) African Medical and Research Foundation Agreement Officers Representative Association of Private Health Facilities of Tanzania Bugando Medical Center Christian Social Services Commission Cooperative Agreement Country Operational Plan Continuing Professional Development Corporate Social Responsibility Catholic University of Health & Allied Sciences Doctorate of Public Health Faith Based Organization Government of the United Republic of Tanzania Healthcare Management Human Resource Development Human Resources for Health Human Resources Management Information Systems Health Systems Strengthening Information for Strategic Planning and Evaluation Information and Communication Technologies Information Technology Kilimanjaro Christian Medical College Leadership, Management and Governance Medical Doctor Monitoring and Evaluation Medical Education Partnership Initiative Masters in Medicine Medical Officer Ministry of Health Ministry of Finance Ministry of Health and Social Welfare Memorandum of Understanding Masters of Public Health Muhimbili University of Health and Allied Sciences Office of the Global AIDS Coordinator President’s Emergency Plan for AIDS Relief Prime Minister’s Office of Regional and Local Government Performance Monitoring Plan Point of Contact President's Office, Public Service Management Public Private Partnership Private Sector Engagement Population and Reproductive Health Quality Improvement x RAS RUMC SDDH STRH SDI Regional Administrative Secretaries Rush University Medical Center Sengerema Designated District Hospital Sekou Toure Regional Hospital Service Delivery Improvement SOW SPER TZ T&T TWG USAID USAID/TZ USG WHO ZRC Scope of Work Strategic Planning Evaluation and Reporting Tanzania Treat and Train Technical Working Group U.S. Agency for International Development U.S. Agency for International Development/Tanzania United States Government World Health Organization Zonal Resource Centres xi EXECUTIVE SUMMARY Evaluation Purpose The United States Agency for International Development (USAID) Mission in Tanzania (TZ) requested this end-of-project performance evaluation to refine the Mission’s Human Resources for Health (HRH) and Public Private Partnership (PPP) development strategy. The Mission hopes to understand results achieved by the project and to use the evaluation findings, conclusions, and recommendations to inform management and financial decisions moving forward. Evaluation Questions The key evaluation questions are centered on three domains: effectiveness, reproducibility (ability to reproduce and bring the intervention to scale), and sustainability. The questions are: 1) How effective is the intervention model developed by the Touch?; 2) To what extent is the model reproducible beyond the Lake Zone to other regions of the country and what are the potential opportunities and obstacles to bringing the model to scale?; and 3) What is needed to ensure that the intervention model developed by the Touch Foundation (henceforth referred to as Touch) and its’ institutional partners is sustained beyond the life of the project? Project Background The USG-Touch partnership began with an initial investment of $1,000,000 in 2005/2006, followed by another partnership from 2007 – 2010 where USG contributed $3,190,000, followed by a third partnership between 2010 – 2014 (extended by one year) with a USG contribution of $8,500,000. The funding source has primarily been the President's Emergency Plan for AIDS Relief (PEPFAR). Since 2007, the Touch-USAID partnership has focused on increasing the quantity and quality of the healthcare workforce in Tanzania. Initially, Touch focused on strengthening Bugando Medical Centre (BMC) and its affiliated medical university, the Catholic University of Health & Allied Sciences (CUHAS). Primary support was through direct financial operational support, infrastructure upgrades, and capacity building in finance, management, faculty development, and information technology. In 2011, Touch established the Treat & Train (T&T) program, extending the training of health workers to regional and district hospitals through clinical rotations. The T&T program is intended to: reduce congestion at BMC; improve the quality of medical education at CUHAS/BMC; and strengthen the health system in the Lake Zone. xii Project objectives Touch’s work focused on five main objectives: 1) Increase the number of health students trained and improve the quality of education at CUHAS and BMC; 2) Establish CUHAS and BMC as self-sustaining institutions; 3) Strengthen the health system in the Lake Zone; 4) Enhance healthcare management across Tanzania and, in particular, in the Lake Zone; and 5) Create Touch Tanzania as the institutional platform for future local program implementation and scale-up. Evaluation design, methods, and limitations This end-of-project performance evaluation took place in January 2014 and used a non-experimental design. Qualitative methods were used focusing on descriptive and normative questions to assess: 1) what the project achieved; 2) how it was valued and perceived by stakeholders; and 3) to what extent the model is able to be reproduced and sustained. Data from multiple sources were triangulated to identify results. The evaluation team used three primary data collection methods: 1) desk review; 2) key informant interviews of key stakeholders; and 3) review of project data collected throughout the course of implementation. Interviews were conducted through site visits or on the phone as appropriate. The evaluation team visited Mwanza, where CUHAS and BMC are co-located, and two T&T satellite sites, Sekou Toure Regional Hospital (STRH) and Sengerema Designated District Hospital (SDDH). The team also met with regional and national government representatives as well as associations and local private sector sponsors. Potential methodological limitations of the evaluation include respondent, selection, and recall bias. As an internal evaluation, consisting of staff from the funding agency, some inherent bias is possible. To mitigate this, staff directly involved in the design, implementation, or oversight of the project were not involved in the evaluation. The risk for response bias also exists as beneficiary institutions and individuals may have tailored responses to moderate potential negative impact on future funding. This was mitigated through thorough informed consent, probing, and triangulation. Recall bias may also have been present as key informants were asked to recall past events. To the extent possible, key informants were asked for their observations and feelings, and not specific details that may be difficult to remember. There is always potential for selection bias in performance evaluations that rely primarily on key informants. To address this, the team included multiple stakeholders, at different levels, and from institutions including national and local government, private sector, and national associations in addition to the beneficiary institutions and their staff. xiii Summary of findings and conclusions CUHAS/BMC, with Touch support has had many accomplishments in the past 10 years. The institutions have graduated 278 physicians, 70 specialists, and approximately 1,700 other health professionals. Touch has sponsored CUHAS faculty to attend various capacity building activities in the US, which has benefited both individual faculty development and institutional improvements in curriculum and teaching methods. Significant efforts were made to improve the quality of medical training through the implementation of T&T, which extends medical student training from tertiary hospitals to district ones. As a result of the T&T, there is improved capacity to provide higher quality of services in Sengerema, a rural district, through the provision of specialist care, as well as improvements in infrastructure and equipment availability. AMO clinical rotations are replicable to other parts of the region and country and there is potential to reproduce T&T at government, Faith Based, and private sector facilities alike.The secondment of specialists in the flagship and peripheral institutions has the potential to: transfer knowledge and skills to not only students but also facility staff; bring specialist care closer to home for patients; and reduce the burden at the referral facility. Touch has established partnerships with local institutions (CUHAS, BMC, SDDH, STRH) as well as some private sector companies (African Barrick Gold) as well as strong partnerships with international institutions such as: Cornell University, Baylor University, McKinsey & Company, and U.S. Peace Corps, amongst others. While the project has made many accomplishments, there are some challenges which should be considered. There are some perceptions amongst beneficiary institutions of limited collaborative planning and decision-making on the project’s planned activities and budgets, which have led to some feelings of resentment and exclusion by some stakeholders. While there are some examples of institutional capacity strengthening of CUHAS, the institution’s primary revenue source is derived from increasing student enrollment through student tuition fees, which has the potential for decreasing the quality of education. Faculty and staff have benefited from numerous capacity building efforts, but one missing component is the capacity of faculty, as well as the institution, to mobilize resources through fundraising, partnerships, and research grants. The T&T model has many attributes, including the availability of specialist care to populations surrounding the satellite T&T sites. However, limited data is available to show evidence of improved health outcomes as a result. In addition, local capacity to: 1) reproduce and scale up the T&T model, 2) finance the expenses of running the program, 3) mobilize resource to fund it, and 4) manage it locally are unknown. These capacities must be assessed and strengthened to ensure sustainability. xiv After 10 years of program implementation in the Lake Zone, Touch Foundation has reached an important cross-road regarding its HRH interventions in Tanzania and must assess, strategize, and plan the way forward. The expansion of T&T to the village level would be valuable to reach the most rural populations; however, many questions remain about feasibility and utility. The T&T model thus far has not sufficiently received local institutional support and engagement from the community, local government, councils, private businesses and other stakeholders who would stand to benefit from the outcomes of the program. The health care management fellows who have completed the fellowship program appear to have benefited personally from the experience, but little evidence is available to demonstrate quality improvement outcomes. The health care management fellowship program appears to be the least integrated of Touch activities and least likely to be replicable in its current form. Summary of recommendations Communication and joint planning between Touch and beneficiary institutional management and staff should be strengthened to create a joint sense of ownership, inclusion, and mutual accountability. Capacity building of both individuals and institutional systems should be targeted to prioritize individual professional development, as well as institutional strengthening in areas such as resource mobilization (fundraising, research grants, and partnerships). Clinical rotation at STRH should be improved through infrastructure and equipment investments as well as better communication regarding scheduling and accountability of clinical instructors. Touch should consider placing two more specialists in the departments of OB/GNY and surgery who are dedicated to teaching students. A preference should be given to local specialists to ensure sustainability in lieu of the expatriate model, which is expensive and not likely to be reproducible, scalable, or sustainable. To reproduce or expand the T&T model, a decision should be made to either: I) continue working with additional regional, district, and designated district hospitals, and consider scale up to both public and private health facilities in the region; 2) reproduce the model to other regions, but remain at the district hospital level, replicating the model that has already been tested; OR 3) reproduce the model at the health center level (village level) with modifications to align with this context. To expand to the village level, Touch will have to carefully consider its own operational capacity to support the program in remote areas, the reproducibility of the model at lower levels, and the feasibility of upgrading health centers. Special attention should be paid to the utility of bringing T&T to the the lower level as Health Centers may not have the volume of patients needed for clinical teaching. xv In order to sustain the T&T model, the transfer of the financial and programmatic management of the model to local stakeholders will require significant technical and financial support and efforts should be made to move in this direction. Alternatives to the expatriate model should be explored by recruiting local and regional specialists, creating incentives for their recruitment and retention, and ensuring that the specialists have explicit roles in teaching students, facility staff, and treating patients. Analysis and documentation of quality improvements in health care management resulting directly from fellows’ participation in the program is needed to demonstrate results of the fellowship program. Touch should work more closely with other training institutes like Kilimanjaro Christian Medical College (KCMC), Muhimbili University of Health and Allied Sciences, and Mzumbe University on healthcare management integration into the medical curriculum. This should be coupled with advocacy for the professionalization of healthcare management field in Tanzania. Strengthening local institutions that can conduct the health care management training moving forward is more feasible, acceptable, scalable, and sustainable than the model that Touch has used to date. Finally, a step-by-step documentation of the Touch model, including costs and lessons learned, should be developed and disseminated widely. This will help inform and prepare in-country stakeholders to assume ownership and be able to reproduce and expand best-practices which are locally owned and able to be sustained. xvi TABLE 1: SUMMARY TABLE OF CHALLENGES AND RECOMMENDATIONS CHALLENGES RECOMMENDATIONS Perceived limited collaborative planning and decision-making on the project’s planned activities and budgets have led to some feelings of resentment and exclusion by some stakeholders. Communication and joint planning between Touch and beneficiary institutional management and staff should be improved to create a joint sense of ownership, inclusion, and mutual accountability. CUHAS’s primary revenue source is from increasing student enrollment through student tuition fees. This increases the teacher/student ratios, has the potential for decreasing the quality of education, and places much of the resource generation burden on one revenue source. Efforts should be made to ensure that the rapid increase in enrollment of CUHAS students, does not compromise the quality of education provided to students. Continued effort is needed for balanced teacher/student ratios and maintained quality of medical training. While faculty and staff have benefited from numerous capacity building efforts, one significant and missing component is the capacity of faculty to develop research proposals, succeed in acquiring research grants, implement research, and publish and present findings in national and international journals and conferences. CUHAS faculty and staff should continue to benefit from professional development opportunities to improve their capacity to utilize effective teaching methods and approaches. They should also receive targeted training and one-on-one mentorship in grant-writing, research design and scientific writing/publication. The clinical rotation for medical students at STRH provides a less conducive environment for medical student learning. Students may be getting high quality training in two departments and less so in the remaining two, due to the lack of dedicated clinical teaching staff. Clinical rotation at STRH should be improved through infrastructure and equipment investments and better communication regarding scheduling and accountability of clinical instructors. Touch should consider placing two more specialists in the departments of OB/GNY and surgery who are dedicated to teaching students. A preference should be given to local specialists to ensure sustainability in lieu of the expatriate model. Graduate tracking has been achieved through Touch’s own human resources, efforts, and initiative, and has not been institutionalized or owned by CUHAS/BMC. Graduate tracking systems and procedures should be refined, transferred to local institutions, and institutionalized to facilitate tracking of graduates, to assess placement and retention, and to establish and realize the full potential of alumnae networks. There is a lack of clarity as to what data the scribes are collecting and how it is or will be used. The purpose, function, and roles of the scribes should be clarified and communicated to beneficiary institutions. Feedback should be disseminated to the institutional management and their staff with a focus on information that can benefit students, faculty/staff, and even patient care. Though improved access to care for rural populations is evident through the availability of specialist care to populations surrounding the satellite T&T sites, limited data is available to show evidence of improved health outcomes as a result. Strengthened data collection and analysis of the health outcomes of the availability of specialist care in rural settings can produce evidence to support this claim. Local capacity to reproduce and scale up the T&T model, to finance the expenses of running the program, to mobilize resource to fund it, and the capacity to manage it are still in infancy. The transfer of the financial and programmatic management of the T&T model to local stakeholders will require significant technical and financial support and efforts should be made to move in this direction. The value of the expansion of the T&T model for AMOs is widely considered a good idea, but careful attention should be paid to how and where the model is rolled out . Expansion of the T&T for AMOs should be considered at other medical colleges. Some suggestions identified by key informants include: Tanga, Mbeya and Kilimanjaro Christian Medical College (KCMC), Ifakara in xvii Morogoro region, Songea (bordering Mozambique), and other schools affiliated with AMOs. After 10 years of program implementation in the Lake Zone, Touch Foundation has reached an important cross-road regarding its HRH interventions in Tanzania and must assess, strategize, and plan the way forward. Touch should make a decision regarding program expansion from several options including: I) continue working with additional regional, district, and designated district hospitals, and consider scale up to both public and private health facilities in the region; 2) reproduce the model to other regions, but remain at the district hospital level, replicating the model that has already been tested; OR 3) reproduce the model at the health center level (village level) with modifications to align with this context. The expansion of T&T to the village level would be valuable to reach the most rural populations; however, many questions remain about feasibility and utility. To expand to the village level, Touch will have to carefully consider its own operational capacity to support the program in remote areas, the reproducibility of the model at lower levels, and the feasibility of upgrading health centers. Health centers may also not have the volume of patients needed for clinical teaching. The T&T model thus far has not sufficiently received local institutional support and engagement from the community, local government, councils, private businesses and the like who all benefit from the outcomes of the program in a variety of ways. Touch should solicit in-kind contributions of land, housing, human resources, and local support from the community, local government, councils, and private businesses. The 6 health care management fellows who have completed the fellowship program appear to have benefited personally from the experience, but little evidence is available to demonstrate quality improvement outcomes in their respective departments. Analysis and documentation of the quality improvements in health care management resulting directly from fellows’ participation in the program is needed to demonstrate results of the fellowship program. The health care management fellowship program appears to be the least integrated of Touch activities and least likely to be replicable in its current form. The model, which has thus far produced 6 fellows, may not be the most efficient way to improve health care management training. Touch should work more closely with other training institutes like KCMC, Muhimbili, and Mzumbe on healthcare management integration into medical curriculum. This should be coupled with professionalization of healthcare management. Strengthening local institutions that can conduct the health care management training is more feasible, acceptable, scalable, and sustainable. Seconding expatriate specialists at T&T sites is neither scalable nor sustainable. Efforts should be made to find alternatives to the expatriate model by using local and regional specialists, creating incentives for their recruitment and retention, and ensuring that the specialists have explicit roles in teaching students, facility staff, and treating patients. Touch reports used for knowledge dissemination are: of high quality, informative, and widely accessible on the Internet. However, these documents do not describe the Touch model in adequate detail for the purposes of replication. A step-by-step documentation of the Touch model, including costs and lessons learned, should be developed and disseminated widely and will help inform and prepare in-country partners to assume ownership and be able to reproduce and expand best-practices which are also locally sustainable. CUHAS/BMC has benefited from many of the partnerships that Touch has brought to the table, but this has not necessarily strengthened the institutions’ abilities to network and build partnerships independently. Touch should also work to facilitate CUHAS/BMC’s ability to mobilize resources, build its global network, and foster its own partnerships, which can be sustained outside of Touch. A focus on strengthening CUHAS’s ability to generate resources is needed to improve sustainability of the institution. xviii Touch has had limited collaboration with national private associations such as Association of Private Health Facilities of Tanzania (APHFTA) and Christian Social Services Commission (CSSC), to discuss many areas of interest including HRH retention policies. Touch has also had limited collaboration with other NEPI/MEPI (KCMC) initiative institutions in Tanzania. Touch should strengthen existing partners for the T&T program to maximize the impact of their investments to date by continuing to strategically support BMC and CUHAS while moving towards transition to local ownership. The foundation should continue to develop partnerships (both local and global) that provide shared value as well as technical specialist and financial support (grants, research). Touch has had limited interaction with the Ministry of Health and Social Welfare (MOHSW) PPP-Technical Working Group (TWG) and could benefit from participation to streamline communications with national, regional, and local authorities around key financing and Human Resource Development (HRD) polices. Touch should make an effort to work more closely with government to develop strategies that can help strengthen polices for absorptive capacity and retention of health workers within the region. Touch should help to strengthen targeted recruitment and retention policies at the national and local (regional and district) levels. 1 INTRODUCTION Evaluation Rationale The United States Agency for International Development (USAID) Mission in Tanzania (TZ) requested this end-of-project performance evaluation in order to learn about the effectiveness of the project in order to utilize the recommendations to refine the Mission’s Human Resources for Health (HRH) and Public Private Partnership (PPP) development strategy. The evaluation sought to assess the impact of the United States Government (USG) investments in HRH through PPP in Tanzania. The Mission hopes to understand which components of the project were successful and produced desired results, and utilize the findings and lessons-learned to inform management and financial decision-making moving forward. Evaluation Questions The key evaluation questions are divided into three domains including: effectiveness, reproducibility (ability to reproduce and bring the intervention to scale), and sustainability. The questions include: 1) How effective has the intervention model developed by the Touch been; 2) To what extent is the model scalable beyond the Lake Zone to other regions of the country and what are the potential opportunities and obstacles to bringing the model to scale; 3) What is needed to ensure that the intervention model developed by the Touch and its’ institutional partners is sustained beyond the life of the project? Project Background The USG-Touch Foundation partnership began with an initial investment of $1,000,0001 in 2005/2006 from the PEPFAR Office of the Global AIDS Coordinator (OGAC), followed by another partnership from 2007 – 2010 where USG contributed $3,190,000 (Touch Foundation contributed $6,626,010), followed by a third partnership between 2010 – 2014 (extended by one year) with a USG contribution of $8,500,000 (Touch Foundation is to contribute $8,500,000). Touch not only brought financial inputs into the partnership, but also management, business, and medical technical expertise as added value to the project. The funding source for the Touch Foundation’s work in the country has primarily been the President's Emergency Plan for AIDS Relief (PEPFAR). Since 2007, the Touch-USAID partnership has focused on increasing the quantity and quality of the healthcare workforce in Tanzania. Initially, Touch focused on strengthening the Bugando Medical Centre (BMC), the specialty referral hospital for the Lake Zone, and its affiliated medical university, the Catholic University of Health & Allied Sciences (CUHAS) through direct financial operational support, infrastructure upgrades, and capacity building in areas of finance, management, faculty development, and 1 All $ amounts in the document are in United States dollars 2 information technology. In 2011, Touch established the Treat & Train (T&T) program that extends the training of health workers to regional and district hospitals. The T&T program is intended to improve the quality of medical education at CUHAS/BMC and also strengthen the health system in the Lake Zone. Project objectives Touch’s work focused on five main objectives: 1) increase the number of health students trained and improve the quality of education at CUHAS and BMC; 2) establish CUHAS and BMC as self-sustaining institutions; 3) strengthen the health system in the Lake Zone; 4) enhance healthcare management across Tanzania and, in particular, in the Lake Zone; and 5) create Touch/Tanzania as the institutional platform for future local program implementation and scale-up. Evaluation design, methods limitations This end-of-project performance evaluation used a non-experimental design using qualitative methods focusing on descriptive and normative questions to assess: 1) whether expected results were achieved; 2) how it was implemented; and 3) how it was valued and perceived by stakeholders. Data from multiple sources were triangulated to identify results. The evaluation team used four primary data collection methods: 1) desk review; 2) key informant interviews of key stakeholders through site visits; 3) observations of institutions to observe infrastructure and equipment purchases and upgrades, and 4) analysis of project data collected throughout the course of implementation. The evaluation team members visited Mwanza, where CUHAS and BMC are co￾located, and two T&T satellite sites, Sekou Toure Regional Hospital (STRH) and Sengerema Designated District Hospital (SDDH). During visits to all four sites, the evaluation team conducted interviews with management, staff, and beneficiaries where possible, and also observed infrastructure and equipment upgrades The team also met with regional and national government representatives as well as associations and local private sector sponsors. Potential methodological limitations of the evaluation include respondent, selection, and recall bias. As an internal evaluation, consisting of staff from the funding agency, some inherent bias is possible. To mitigate this, staff directly involved in the design, implementation, or oversight of the project were not involved in the evaluation. The risk for response bias also exists as beneficiary institutions and individuals may have tailored responses to moderate potential negative impact on future funding. This was mitigated through thorough informed consent, probing, and triangulation. Recall bias may also have been present as key informants were asked to recall past events. To the extent possible, key informants were asked for their observations and feelings, and not specific details that may be difficult to remember. There is always potential for selection bias in performance evaluations that rely primarily on key informants. To address this, the team included multiple stakeholders, at different levels, and from institutions including national and local government, private sector, and national associations in addition to the beneficiary institutions and their staff. 3 EVALUATION PURPOSE, USE, & AUDIENCE Purpose There are two overall purposes for conducting this evaluation: learning and accountability. As an end of project evaluation, the Mission is interested to learn from the findings of the evaluation, which will contribute to USAID Tanzania’s (USAID/TZ) development strategy as it relates to both HRH and PPP. The Mission hopes to understand the context and results of the intervention that was supported, understand which components of the intervention were successful and produced desired results, and learn from the lessons-learned from the project’s implementation to make management and financial decisions moving forward. The second overall purpose of conducting this evaluation is accountability. The Mission must consider the value for money gained for its investments in this area, and account for the public expenditure to key stakeholders including Congress and the U.S. taxpayers. While this evaluation focuses on the most recent cooperative agreement (Co-Ag) between Touch Foundation (henceforth referred to as Touch) and USAID Tanzania (2010-2013), it is worth noting that this is the third in a series of partnerships between USG and Touch which will be further explained in the project description. As such, the USG has made considerable investments for a series of projects in HRH in the Lake Zone and has determined a need to identify the impact of these investments in making a difference. Therefore, some of the findings included in this report refer to overall results of this collaboration, and not only the most recent Co-Ag. Use The primary uses of the evaluation findings are to inform funding decisions in Country Operational Plan (COP) 2014 and subsequent fiscal years. The Health Systems Strengthening (HSS) team will determine whether to continue to fund this partnership and if so by how much. The Mission will also use the findings, if they determine to continue to support the partnership, to engage in dialogue with the implementing partner to inform the direction and focus of future collaboration. The Mission will identify the strengths, weaknesses, successes, shortcomings, and the current and changing context of the human resource needs in the country to make programmatic and funding decisions moving forward. Audience The primary audiences of the evaluation findings are: USAID/Tanzania, other PEPFAR implementing agencies in Tanzania, OGAC, the Government of the United Republic of Tanzania (GoT), and Touch. In addition, the recipients of the technical assistance, names the two flagship institutions, BMC and CUHAS, as well as the tertiary institutions can also learn from the findings and can shape their own strategy for technical assistance needs and support. Other Missions currently using PPP to implement HRH interventions may be interested to review the findings to inform their own partnerships and decisions. 4 EVALUATION QUESTIONS Effectiveness 1. How effective has the intervention model developed by the Touch been? 1a. What has been the impact of the model on the quality of medical education provided to CUHAS and BMC students? 1b. How has Treat and Train program enhanced access to care for the more rural population? Reproducibility 2. To what extent is the model scalable beyond the Lake Zone to other regions of the country and what are the potential opportunities and obstacles to bringing the model to scale? 2a. After seven years, what more is needed in the Touch’s approach to continue to advance this work and/or bring it to scale? 2b. Which components have been the most difficult to make progress on and are not possible or advisable to reproduce or scale-up? 2c. Which components of the model have the greatest potential for replication? Sustainability 3. What is needed to ensure that the intervention model developed by the Touch and its’ institutional partners is sustained beyond the life of the project? 3a. How successfully has Touch managed to reduce CUHAS and BMC dependency on Touch operational support, utilize the more strategic/capital projects support, and move toward self-sustaining, private institutions? 3b. Are there things that can allow for the project to have more sustainable impact? 3c. How and how long would it take to make the model self-sustaining without external assistance in the form of grants from the donor community? Human Resources for Health Human resources for health are important for effective health services delivery. According to WHO, “health workers are important for advancing health and the quality of health care and achieving the health-related Millennium Development Goals,” (1). For USAID, “human resources for health are critical for a functioning healthy system,” (2). Tanzania’s MOHSW has a five-year (2008-2013) strategic plan for HRH and envisions that strengthening human resources for health will address extreme health worker shortages, improve disease-specific responses to malaria, HIV and AIDS, and enact the policy, regulatory and finance structures required to retain the country’s human resources (3). Outside of the health sector goals, the Ministry of Health and Social Welfare (MOHSW) also predicts that strengthening HRH will contribute to Tanzania’s development goal to move from a least developing to a middle-income country by 2025 (4). 5 Investing in HRH represents a potentially broad scope of work to improve health workforce functioning and provision of HIV and AIDS services. Improving HRH outcomes includes a wide array of activities and results including: the overall numbers of health workers produced, their geographic distribution, the skills mix and population needs per cadre, and their performance including the quality and productivity of health workers. Working in this area involves addressing several inter-related dimensions, including Human Resource Management Information Systems (HRMIS), policies, leadership and finance, educational systems, and partnerships. The Office of the Global AIDS Coordinator (OGAC) HRH Technical Working Group (TWG) describes six priority objectives: 1. Support national HRH planning and management, including development of human resource information systems; 2. Strengthen pre-service education institutions to improve the quality and output of graduates; 3. Ensure the standardization, quality, and coordination of in-service training through, for example, continuing professional development programs; 4. Advance innovative and cost effective models of service delivery and skill mix, including task-shifting/sharing, introduction of new cadres, integrating community health workers into the continuum of response, developing multi-disciplinary teams, and supporting implementation science; 5. Investigate and apply recruitment/retention strategies, especially in rural and underserved areas; 6. Advance health worker regulation and policy, including capacity building of regulatory bodies and professional associations. It is important that PEPFAR funded HRH interventions strengthen and foster sustainability. Addressing HRH challenges is necessary for a sustained HIV response in Tanzania. Historically, the PEPFAR strategy in Tanzania was to support the increased production of health care personnel. In recent years, the approach has shifted away from an emergency response towards supporting HRH planning and management, looking at innovative and cost effective models of service delivery and skill mix, applying recruitment/retention strategies and advancing health worker regulation and policy. The MOHSW of Tanzania encourages PPPs as one of several approaches to support the planning and management of HRH. Of the six priority objectives stated above, the evaluation focused on Sustainability, Country Ownership and Reproducibility1 . These criteria provided a framework for evaluating the Touch-USAID partnership in Tanzania through a HRH lens. PUBLIC PRIVATE PARTNERSHIPS Private sector engagement (PSE) and public-private partnerships play a critical role in strengthening and extending the principle of shared responsibility in the PEPFAR Blueprint to achieve an AIDS-free generation. Key Private Sector Engagement PEPFAR Blueprint strategies include: 1) Maintain and expand current partnerships, as well 6 develop new partnerships that enhance country ownership and shared responsibility; 2) Create collaborations around private health sector delivery of services to expand coverage and quality of care; 3) Support reporting and evaluation of private sector engagement to assess impact and share lessons learned; and 4) Actively seek and apply the core competencies of the private sector in strengthening the global HIV/AIDS response at every level, including local, regional, and global (5). In the United States Leadership against HIV/AIDS, Tuberculosis, and Malaria Act of 2003 (P.L. 108-25), Congress authorized PEPFAR to promote PPPs as a priority element of the U.S. strategy to combat the HIV/AIDS epidemic. Congress's commitment to PPPs and reporting on PEPFAR's engagement with the private sector was further strengthened by the PEPFAR Stewardship and Oversight Act of 2013 (P.L. 113-56). Since 2006, PEPFAR has made significant strides in brokering PPPs and establishing relationships with key private sector entities. The ultimate goal of each PPP is to allow more people to benefit due to additional resources—whether monetary or technical—brought to the partnership by the private sector organization. Doing so can increase efficiency, increase effectiveness, and harness the comparative advantages of all partners. It can also be a tool to build capacity of local country partners. PEPFAR defines Public Private Partnerships as collaborative endeavors that combine resources from the public sector with resources from the private sector to accomplish HIV/AIDS prevention, care and treatment goals. By leveraging private sector resources (financial and expertise), PPPs enable the U.S. government and private sector entities to enhance their efforts. PPPs are characterized by jointly defined objectives, program design and implementation, and the sharing of resources, risks, and results. The following are critical core elements of PPPs: 1. Coherence with country strategy and PEPFAR goals in prevention, care and treatment and orphans and vulnerable children: PPPs must help advance programs and reach PEPFAR targets; 2. Added value: PPPs reach more beneficiaries with additional resources; 3. Quality and sustainability: PPPs should include transition strategies that will allow for the integration and mainstreaming of program activities within the existing host country infrastructure (e.g., health care systems); 4. Effective monitoring and evaluation: Monitoring and evaluation of PPPs is required to document results, enable cost-effectiveness analysis, and ensure accountability; and 5. Resources Leveraged: PPPs by definition must include resource inputs from PEPFAR and from private sector partner(s), and meet the requirement of a 1:1 leverage. In the event the private sector partner contributes resources in-kind, country teams should monetize the contribution (6). 7 It is important to assess the impact of PPPs on core PEPFAR goals as well as on the dimensions of innovation, sustainability, and scalability. The following are recommendations for reporting on the quality of the partnership: 1. Impact: Description of impact related to care, treatment, prevention, and health systems country and global PEPFAR goals. The PPP should generate measurable outputs that strive to compare favorably with current PEPFAR programmatic methods; 2. Innovation: How is the PPP program, product or service perceived by the local community as being new or novel to the local setting of implementation? 3. Sustainability: Is there potential for development of a social business or non-profit model for financial sustainability within a period of five to ten years? Does the PPP have the ability to cover full or partial operating expenses with either operating revenues or shared streams of income from a diverse number of committed partners beyond the initial PEPFAR investment? 4. Scalability: Does the PPP have potential to grow by an order of magnitude beyond the initial proposal (i.e. 5x number of clients served, providers trained, facilities accredited, or geographies served) within five to ten years? Is there an experienced dedicated professional team to help grow the PPP within country or regionally; and 5. Financial contributions: Resource inputs from PEPFAR and private sector partner(s) that leverage a 1:1 match. In the event the private sector partner contributes resources in-kind, country teams should monetize the contribution (7). PPPs are complex and can require significant time to manage. Successful PPPs are driven by identifying needs and gaps in the field, adhere to country ownership principles, and have a local champion to guide them through implementation. PROJECT BACKGROUND HRH in Tanzania In order to understand the problem that this project was designed to address, it is important to provide the context through a brief description of HRH challenges in Tanzania. Tanzania is a large country with a population that is spread out, presenting numerous challenges to health care delivery. This requires the distribution of health workers throughout the country and at various levels of health facilities, from dispensary at the lowest to tertiary hospitals at the highest levels. The health sector is understaffed with a total staffing in the public health sector at 35% of the actual need, according to defined national staffing norms. The available number of professional health workers in the public sector is 35,202 leaving a deficit is 90,722 (8). Shortages in the private sector, especially in Faith Based Organization (FBO) institutions are also immense, though not quantified. There is an enormous shortage of human resources for health across all cadres: clinicians, nurses, pharmaceutical technicians, laboratory technicians, radiographers, physiotherapists, health officers and health administration cadres. The shortage is more 8 severe in rural districts. The high attrition rate is a threat and is compounded by the HIV/AIDS epidemic. Table 2 gives a summary of total HRH deficits by cadre. From the GoT sources, there is a total of 45.6% deficit for the country, with the most severe shortages amongst specialist doctors. Table 2: Health Worker Shortage by Cadre (9) Cadre Required Available Deficit Percent Medical Doctors 910 489 421 46.3% Specialist Doctors 268 94 174 64.9% Trained Nurses 9,761 6,382 3,379 34.6% Enrolled Nurses 17,053 7,796 9,257 54.3% Pharmacist/Technician 645 330 315 48.8% Chemist 274 126 148 54.0% Assistant Medical Officer 2,238 1,417 821 36.7% Laboratory Technician 1,036 568 468 45.2% Health Officer 2,660 1,177 1,483 55.8% Radiographer 222 120 102 45.9% MCH Aide 702 1,038 -336 47.9% Clinical Officer 492 347 145 29.5% Assistant Clinical Officer 1,787 826 961 53.8% Total 38,048 20,710 17,338 45.6% The capacity of health training institutions is limited. Consequently there has traditionally been a lower output of trained health personnel to meet the national demand. Training institutions suffer several obstacles including: funding, management, understaffing, and inadequate infrastructure and equipment. Health care staff in the field requires reliable and accessible Continuing Professional Development (CPD) to meet training needs, but capacity building of staff is often fragmented, linked to vertical programs, and not always targeting the right cadres. There is little follow-up to ensure that health workers indeed use acquired skills. The impact of such capacity building is limited. The Zonal Resource Centres (ZRCs), which are the regional government institutions responsible for on-going in-service training within their zones, lack the capacity to effectively support the training needs within their regions. Project history The HRH situation in Tanzania requires a multi-pronged effort to improve human resource planning at the national level, improve human resource allocation throughout the country, strengthen approaches to recruitment and retention to ensure equitable access to health care for all, and strengthening of both in-service and pre-service training institutions. One project cannot tackle all of these vast and complicated HRH challenges. However, the Touch project sought to address some of these areas and will be further described below. The partnership between the USG and Touch began in 2005/6 when the Office of the Global AIDS Coordinator entered into a Memorandum of Understanding (MOU) with the President’s Office of the Government of Tanzania for a $1,000,000. This MOU was then followed up with a second partnership between USAID/TZ and Touch from 2007 to 2010. USAID contributed $3,190,000 and Touch provided $6,626,010 to the 9 partnership. This was then followed up with a second partnership from 2010-2013, which was extended for an additional year to 2014. For this collaboration, USAID contributed $8,500,000 and Touch provided the same $8,500,000. The funding source for these three activities has primarily been the President's Emergency Plan for AIDS Relief (PEPFAR). Since 2007, the Touch-USAID partnership has focused on increasing the quantity and quality of the healthcare workforce in Tanzania. Initially, Touch focused on strengthening Bugando Medical Centre (BMC), the specialty referral hospital for the Lake Zone, and its affiliated medical university, the Catholic University of Health & Allied Sciences (CUHAS, formerly Weill Bugando University College of Health Sciences). In addition to providing direct operational financial support to both the university and the teaching hospital, Touch invested in infrastructure upgrades to enhance the learning and healthcare delivery facilities, served as strategic advisors to these institutions, and supported capacity building of key functions such as finance, management, faculty development and information technology. After seven years of support from OGAC and USAID (2004-2011) focused on securing a solid pipeline of new health workers at CUHAS and BMC and improving BMC infrastructure and operations, in 2011 Touch began to establish the Treat & Train program that extends the training of health workers to regional and district hospitals. While this expansion is aimed at improved quality of and greater capacity for health education provided by CUHAS and BMC, in particular clinical training for their students, it also represents a move towards strengthening the health system in the Lake Zone. How this is being done is explained in more detail below. Project objectives Touch’s work focused on five main objectives: 1. Increase the number of health students trained and improve the quality of education at CUHAS and BMC; 2. Establish CUHAS and BMC as self-sustaining institutions; 3. Strengthen the health system in the Lake Zone; 4. Enhance healthcare management across Tanzania and, in particular, in the Lake Zone 5. Create Touch Tanzania as the institutional platform for future local program implementation and scale-up. Increase the number of health students trained and improve the quality of education One of the project’s primary purposes was to contribute to the growth of CUHAS by providing operational grants to the institution. Initially, Touch covered direct student costs of accommodation, meals and supplies, which facilitated the enrollment of students who otherwise might not have entered the medical profession. The operational fund was also intended to improve the learning environment for students through the 10 establishment of book banks for the CUHAS basic science departments (anatomy, microbiology, physiology, pathology and biochemistry) and the upgrading of the computer laboratory. Additionally, in order to increase the quality of education, a capacity building component was built into the project to improve the instructional skills of faculty through training of faculty abroad or through facilitating mentoring relationships with U.S. faculty on short-term assignments at CUHAS and BMC. CUHAS and BMC’s increasing student enrolment produced the additional challenge of insufficient space at the BMC teaching hospital to accommodate clinical training. This led to the development of the Treat and Train program, which entails rotating medical students and faculty for their clinical training at regional institutions, Sekou Toure Regional Hospital and Sengerema Designated District Hospital. Establish CUHAS and BMC as self-sustaining institutions From an institution-building perspective, the ultimate vision of the partnership was that one day CUHAS-BMC medical complex would be a self-sustaining, self-sufficient center of excellence where continued external support would no longer be needed. In FY12, Touch restructured its grant agreement with CUHAS and BMC to reduce reliance on support for ongoing operations costs and to ensure their long-term sustainability. The agreement restricted part of the total grant to fund capital and special projects. The requirement for CUHAS and BMC to access these funds was changed. The institutions were requested to submit detailed business plans including the rationale for undertaking the project, a budget and implementation plan as well as a comprehensive operations and maintenance proposal. In order to strengthen the management of BMC, Touch also developed and implemented a fellows program, and selected key managers to participate in Continuing Professional Development courses. These CPD courses were intended to provide intensive training in hospital management topics the idea being that these fellows would come back, implement a quality improvement project. Strengthen the health system in the Lake Zone The launch of the Treat and Train program is the first step towards the building of health system in the Lake Zone down to the community level. In addition to providing students with professional orientation and skills needed to work in rural, often under-resourced environments, the Treat and Train program is intended to create a continuum of health delivery in the immediate vicinity of the hospitals. Enhance healthcare management across Tanzania and, in particular, in the Lake Zone The Healthcare Management Program was designed to accomplish two main objectives: to improve Mzumbe University’s master program in healthcare management, with support from Rush University Medical Center (RUMC), and to improve the overall management of BMC and other hospitals in the Lake Zone of Tanzania. BMC Fellows were paired with appropriate mentors from RUMC and engaged in quality improvement projects at BMC. This was supplemented by CPD short courses in various 11 organizational operations (finance, human resource management, department planning and the other yet to be determined) to strengthen the management skills. Create Touch Tanzania Touch has registered an affiliated organization in Tanzania, called Touch Tanzania Limited, referred to as "Touch Tanzania". The board of Touch Tanzania is currently comprised of seven members with a majority of the members also on the Touch U.S. board, thus eliminating it from being a candidate for USAID’s local solutions support. At the moment, Touch Tanzania is being used as the primary party in agreements with Tanzanian contractors and partners and Touch U.S. is the party in agreements with partners and vendors based internationally. In the long-term, however, the vision is that Touch Tanzania will be autonomous with an independent governing structure. TOUCH FOUNDATION LOGICAL FRAMEWORK The overall goal for Touch is to improve the health of the Tanzanian population through enhanced access to essential, quality healthcare. The conceptual framework (Figure 1) describes the causal pathway to achieve this goal (10). Touch aspires to do this by increasing the quantity and quality of the healthcare workforce and improving the healthcare delivery mechanisms to better enable provision of care. The approach emphasizes the horizontal strengthening of the overall health system rather than a vertical approach focusing on single diseases. The three primary elements of the approach are: Treat and Train (T&T), Health Care Management, and Knowledge Development and Dissemination. The T&T program is a comprehensive network of institutions providing healthcare and medical education across different levels of the health care system. Currently, this model extends the training of Assistant Medical Officers (AMO) from BMC and Medical Officers (MO) from CUHAS to rotate at peripheral sites for their clinical practicum. At the same time that students are being trained, students and specialist instructors are delivering health care to the populations of these peripheral sites. The Health Care Management component provides fellowships and practical application of quality improvement projects to strengthen management of BMC. Finally, the intent of the Knowledge Development and Dissemination is to conduct research on health worker training and health delivery in resource-constrained setting with the aim of sharing the model, as well as the lessons learned, with the broader health community to assist in replication and scale up. 12 Figure 1: Touch Foundation Logical Framework TREAT AND TRAIN FRAMEWORK The Treat and Train component of the Touch model is illustrated in Figure 2 with the top level consisting of institutional support for the flagship institutions CUHAS and BMC (11). This is through the financial support and enhancing the capabilities of CUHAS and BMC to provide quality education and healthcare. The next level is the network expansion at the hospital level through the strengthening of peripheral sites in infrastructure development and equipment upgrading, posting of rotating of specialist teams to provide clinical training, and provision of health care to local populations. The next level is the network expansion at the “village level” and is intended to reach Health Centers, or the next level of care. This would include updating essential infrastructure and equipment, training local community health workers, and integrating the program within the community. 13 Figure 2: Treat and Train Conceptual Framework EVALUATION METHODS & LIMITATIONS Evaluation type and methods The Scope of Work (SOW) (Annex II) called for an end-of-project performance evaluation focused on descriptive and normative questions to assess: 1) what the project achieved; 2) how it was implemented; 3) how it was valued and perceived by various stakeholders; and 4) whether expected results occurred. As a performance evaluation, no counterfactuals were established. Accordingly, the results do not address a cause and effect relationship. No baseline indicators were collected at the start of the project and therefore, no pre and post comparisons were made. Data from multiple sources were instead triangulated to demonstrate results achieved while considering other potential factors, which may have contributed to these results. The evaluation team used three primary data collection methods: 1) desk review; 2) key informant interviews of key stakeholders including the implementing partner, beneficiaries, and key stakeholders in the national and regional government as well as relevant associations; and 3) project data collected throughout the course of implementation. The team synthesized data from these numerous sources and made BUGANDO CENTER 9 Catholic University of Health & Allied Sciences (CUHAS) – Medical school 9 Bugando Medical Centre (BMC) – Tertiary hospital Sengerema District Hospital Sekou Toure Regional Hospital Shinyanga Regional Hospital Kagera Regional Hospital Tertiary hospital Regional & district hospitals Village level health facilities 14 corresponding conclusions and proposed recommendations accordingly. As much as possible, the influence of the Touch’s influence over institutional changes at the flagship institutions of CUHAS and BMC were explored in the context of other factors, which may have contributed, to these changes. Other factors, which were explored as potential causes of these changes, included other donors, national government support or intervention, and other internal institutional processes that may have led to these changes. Desk review documents were sent to the evaluation team prior to arrival in country. Interviews were conducted through site visits. The evaluation team members, as appropriate, visited Mwanza, where CUHAS and BMC are co-located, and two Treat and Train satellite training sites, are Sekou Toure Regional Hospital, also in Mwanza, and Sengerema Designated District Hospital, located in a rural area. The team also visited regional and national government representatives at their respective offices as well as associations and other relevant stakeholders. Desk review USAID/TZ and Touch provided the evaluation team with a core list and/or copies of the agreement, reports and other key documentation prior to the commencement of the evaluation. Touch provided internal reports, power point presentations, and other source documentation. The desk review was conducted of key project documents including quarterly reports, performance monitoring plan (PMP), work plans, Touch internal reports, national documents such as the Health Sector Strategic Plan (2009 – 2015) (12) and the Human Resources for Health Strategic Plan (2008 – 2013) (13), and external assessment reports such as the Private Health Sector Assessment (14). All of these documents provided critical context of the Human Resources for Health (HRH) situational analysis in the country as well as the project’s plans, objectives, and activities for implementation. These resources also served as key source documents for the development of relevant questions and interview guides for the stakeholder interviews. A list of the documents reviewed during the desk review can be found in Annex IV. Key informant interviews A core list of key informants to be interviewed was initially generated by USAID/TZ. Key informant interviews were selected based on their knowledge of the project from the perspective of implementer, beneficiary, government counterparts, and relevant national associations. Touch reviewed this list and added additional key personnel. Members of the evaluation team also made additional suggestions about national level associations and government officials, which should be included on this list. The final list of key stakeholders was finalized and approved by USAID/TZ. A total of 54 key informants were interviewed for the evaluation, spanning a very wide variety of stakeholders including: current students, alumnae, management and staff of both flagship institutions as well as both peripheral sites, government officials at the regional and national levels, members of various relevant associations, private sector 15 donors, expatriate doctors working on the project in a variety of different roles, and Touch staff from both the New York and Tanzania offices. The evaluation team conducted in-depth, face-to-face interviews with key stakeholders and partners at their own institutions whenever possible. In some instances, when this was not possible, the evaluation team conducted telephone interviews. A complete list of the key informants interviewed for the evaluation can be found in Annex V. Project data The team collected project level data from quarterly reports, power point presentations, excel spreadsheets, and budget data as provided by Touch staff either in-person or through correspondence before and after in-country data collection. Sampling A sampling of key stakeholders was selected to ensure a comprehensive assessment of perspectives from as many stakeholders as possible. While selecting alumnae and students, purposive sampling was implemented to ensure the capture of students from across the life-span of the program, from the first year as well as recent graduates. Informed consent Verbal informed consent procedures were followed as per the SOW. Interview subjects were provided with the context and the purpose of the evaluation, an introduction to the evaluation team members and their affiliation, and description of the risks and benefits that would arise from the interview, assurance of anonymity regarding specific responses provided, an explanation of the voluntary nature of the interview, as well as a description of how the evaluation findings would be used. Data collection and analysis The evaluation team developed semi-structured questionnaires (Annex III) during the planning and development stage of the evaluation design. These questionnaires were implemented during key informant interviews, asking only the pertinent questions per relevant stakeholder. Evaluation team members were able to probe and ask additional questions as needed. Written notes by all team members and electronic notes by a designated scribe were collected at each of the interviews. Each evening, the team reviewed their notes and filled in any blanks that were captured by the scribe in electronic format. These transcriptions were then compiled by day and by week and shared with the entire evaluation team for analysis. The team debriefed about their observations, perceptions, and insights each night and made note of any additional questions that need to be asked of the implementing partner who were interviewed again at the end of all of the interviews for clarification and additional questions. 16 Evaluation team members reviewed a number of interview transcriptions independently and generated a set of preliminary codes along emerging themes. The coding structure was then applied to all interview transcripts, with sub-themes developed and applied through iterative review and labeling. Each evaluation team member then used these codes linked to the evaluation question to which they were assigned and used the interview transcriptions to analyze and write up the findings, conclusions, and recommendations of their respective domains: effectiveness, scalability, and sustainability. Limitations There were several potential methodological limitations of this evaluation, starting with the evaluation team. This was an internal evaluation conducted by USAID Washington, USAID Tanzania, and OGAC staff. As members of the USG agencies and therefore the donors of the project, the potential for bias exists. Efforts were made to mitigate this bias by: 1) ensuring that evaluation team members reviewed the data and information collected with objectivity and substantiating all conclusions and recommendations with relevant findings; 2) selecting team members who had not been intimately involved in the procurement, management, or monitoring of the project; and 3) ensuring that a variety of stakeholders from many different perspectives were included as key stakeholders. As a result, we believe that we achieved a level of objectivity despite team members’ affiliations with the donor agency. Another methodical limitation was the potential for recall bias. Key informants were asked to recall past events, including alumnae of the program who were asked to share their experiences and observations from the earlier days of the program. Some were from the very first class at CUHAS, which was in 2004. To the extent possible, key informants were asked for their observations and perspectives about the program, and not specific details that may be difficult to remember. There is always potential for selection bias in performance evaluations that rely primarily on key informants. To address this, the team included multiple stakeholders, at different levels, and from institutions including national and local government, private sector, and national associations in addition to the beneficiary institutions and their staff. In addition to the sites, individuals, and stakeholders that were suggested initially, the Tanzanian Mission as well as the evaluators included additional stakeholders to ensure a breadth of perspectives on the Touch model. The evaluation team was in country for only two weeks and the team members were not able to dedicate full-time effort on the preparation, analysis, or report-writing phase, and were challenged with competing work demands upon completion of data collection in country. Despite competing demands, the team made substantial efforts to fulfill their respective obligations both, before, during, and after data collection to the extent possible. 17 It is important to note that the great majority of information collected was from the insights, perspectives, and thoughts of the stakeholders interviewed. No direct measures such as improvements in the quality of education were taken which would have required direct observation of classroom and clinical practicum teaching before and after the intervention. As such, faculty, students, and alumnae, through their descriptions of the quality of education they received, provided information on the perceptions of the quality of medical education they received. Those students and faculty who benefited from various capacity building efforts described the changes that they underwent and provided examples of the knowledge gained and how they were able to apply it to their departments (BMC) or their teaching (CUHAS). An important caveat to mention is that Touch raises funds not only from USAID, but also from other funding sources. It was not always clear to key informants which resources were used to implement various aspects of project, USAID or private donors. Key informants reported financial and technical inputs by the Touch, which may or may not have been USAID’s contributions. As a PPP, the goals is to leverage funds from the USG along with private sector funds to implement HRH interventions in Tanzania, so whether the dollar spent on a certain piece of equipment of staff was directly from USAID or from one of Touch’s other private donors, may or may not make a significant difference if the results that were achieved were successful. FINDINGS, CONCLUSIONS & RECOMMENDATIONS FINDINGS: EFFECTIVENESS Touch Foundation had the objective of increasing the quantity of medical graduates from CUHAS, of improving the quality of medical education, and improving the health care management at BMC. Below are our findings regarding achievements of these goals. Improving the quantity of health workers Since 2004, Touch has contributed to the rapid growth of CUHAS from 10 to more than 600 medical students, for a total of ~1800 (2013/14) students currently enrolled across 13 cadres, including specialist doctors, doctors, nurses, laboratory technicians, radiologists and pharmacists. Touch contributes to PEPFAR’s goal of training 140,000 new healthcare workers across Africa between FY10 – FY14 and to the Ministry of Health and Social Welfare (MOHSW) healthcare worker training targets as set forth in the Government of Tanzania. Achievements (past 10 years) 1. Approximately 900 graduates across several cadres (i.e. MDs, AMOs, nurses and nursing educators) have received HIV treatment & prevention training as an integral part of their academic curriculum, and are today well positioned to play their part in the on-going effort to curb the spread of HIV across Tanzania. 18 2. The inaugural class of 10 MD students in 2004 has grown to more than 600 MD students currently enrolled, with an average first year enrollment of 150 new MD students every year. Total enrollment at BMC and CUHAS across all cadres has also grown more than five times from 277 students in 2004 to nearly 1,600 students today. 3. CUHAS/BMC, with Touch support, has helped graduate 278 physicians (~10% of the country’s total), 70 specialists, and approximately 1,700 other health professionals, including 388 Assistant Medical Officers (AMOs), 376 nurses, 168 nurse anesthetists, 259 laboratory scientists and 249 pharmacists. 4. The total student population at both CUHAS and BMC grew more than 6-fold, from 277 in 2003/04 across 7 cadres to more than 1,800 in 2013/14 across 13 cadres. 5. Touch’s support to BMC and CUHAS has also contributed to the specific PEPFAR “Goal 5: Human Resources” established in the Tanzanian operational plan. 6. CUHAS is one of only seven medical universities in Tanzania, and its annual intake of 150 MD students comprises ~17% of MD training capacity nationally. 7. CUHAS and BMC have successfully scaled up their health worker education programs, and their annual graduation numbers are expected to stabilize at ~500 students per year by 2013/14, including ~150 MDs and 20 MMED, therefore increasing the number of health workforce in Tanzania. Challenges 1. The exponential growth of medical students from 2004 to 2013 has been astounding. However, this does place strains on the student/teacher ratios for both theoretical and clinical teaching, as well as infrastructure and equipment and has the potential to decrease the quality of education if not closely monitored. Focus on quantity may compromise focus on quality; 2. The rapid growth of students has not been aligned with a commensurate growth of faculty and staff, placing greater burdens on staff and potentially reducing their professional job satisfaction and increasing their work load; 3. The focus on increasing student enrollment as a means to increase income for the university places too many eggs in one basket. Other avenues of resource generation is needed to reduce the burden on the faculty, staff, infrastructure, and students; 4. Students expressed a great deal of anxiety and angst in the process of applying for and trying to garner funds to afford their education. Earlier classes of medical students were funded through a scholarship program funded by Touch, but when this was no longer available, they experienced extreme distress and anxiety trying to cover costs through other means; 5. As an FBO institution with higher fees than government institutions, students who chose to come to CUHAS must supplement the government loans that they receive to be able to attend. This may rule out some of the best and brightest 19 students, and the institution may receive students instead who can afford to supplement government loans instead. Innovation Though sending students on rotations to other hospital sites has been utilized in other medical universities in Tanzania such as the national university (Muhimbili University of Health and Allied Sciences), the Touch T&T model is unique. Through the T&T, AMOs rotated at one peripheral rural hospital (SDDH) for their clinical rotations with the intention of improving their quality of education. What is innovative about this program is not the concept of clinical rotations itself, but the cadres that are rotated - AMOs. Upon graduation AMOs are often placed in rural settings. Therefore, rural rotations during their pre-service training provide them with exposure to rural medicine, in a resource constrained setting, thus better preparing them for their careers in health centers in mostly rural areas in addition to their theoretical education. Improving the quality of education for AMO through Treat and Train (T&T) Our findings show that peripheral hospital rotations offer a rich learning experience for the AMO students, providing them with the opportunity to engage in hands-on experiences, and gain early exposure to healthcare delivery in rural settings. The T&T model is seen by the CUHAS and BMC management as an attempt to “decongest” BMC and provide a better quality of clinical training for students through better teacher/student ratios with the specialists. With the increasing number of students enrolled at CUHAS from 10 students in 2004 to 1,500 in 2014, clinical rotations at BMC became congested with more than 20 students per specialist, which reduced the effectiveness and therefore quality of the training. AMO students interviewed expressed a great degree of satisfaction with the T&T model and the benefits they have gained from these clinical rotations at the peripheral site (SDDH). One enthusiastic AMO described new knowledge and experiences possible at SDDH, which would not have otherwise been possible: “The delivery of babies is so difficult in the rural areas. Before we couldn’t participate in cesarean sections at Bugando (BMC). The AMOs who previously went to BMC don’t know how to do a C￾section. I’m so happy for the Touch Foundation to initiate this cause because myself, I already know how to conduct the C-section. When I get back to my community, I will be a good doctor.” AMOs also described access to different kinds of patients presenting with conditions they were not able to see at the tertiary institution: “We find the fresh cases in Sengerema, such as appendicitis. The doctors let you do it yourself and help you if you need. The exposure has been a good experience.” Another aspect with the potential to improve the health system of the Lake Zone is the concept that AMOs are usually located at the Health Center level or at hospitals in more rural areas. Having better trained AMOs may have the potential of reducing the number and type of referrals to tertiary hospitals if they are able to be handled locally. One AMO student noted: “This rotation has really improved my clinical aspect of my education. I’m sure when I go back there, I can do all necessary minor and small operations/surgeries without referring a patient to referral hospital. I recommend that it should be done to all 20 AMO students in Tanzania and not only us who are in BMC.” While the evidence to substantiate this assertion has not yet been documented, the improved quality of clinical training received by AMOs has the potential to, in the future, improve rural health care delivery and reduce the burden at tertiary hospitals for conditions, which can otherwise be handled closer to home. In addition to the descriptions of the AMO students who overwhelmingly expressed appreciation with the exposure, experiences, learning, and practice they are able to receive at the peripheral site, all staff, management, seconded expatriate specialists, and regional representatives echoed a great degree of satisfaction with the delivery of the T&T program in Sengerema. The key informants uniformly expressed a great degree of satisfaction and appreciation for the student housing built by Touch, the learning and knowledge gained as a result of the rotations, the benefits to the patients of improved access to specialist care, and the simultaneous benefits of the health care staff who learn from both the expatriate seconded specialist staff as well as the rotating specialists who come to teach the AMOs. The benefits and challenges of the clinical rotations of the AMO students to Sengerema are listed below. Achievements of the model (Sengerema) 1. Reducing the burden of practical training of students at BMC by rotating students to other facilities in peripheral sites; 2. Improving the teacher to student ratio at Sengerema where they rotate in small groups of between 5-10 students per specialist faculty as opposed to the (1/20) teacher to student ratios at BMC; 3. Exposing the students to rural medicine and different types of patients and cases; 4. Providing students with opportunities to practice in a limited resource setting; 5. Allowing students to practice and hone skills in procedures they otherwise would not able to perform at BMC; 6. Allowing transfer of learning from rotating faculty to health care staff at the peripheral sites; 7. Improving the access and quality of care received by the rural populations at these sites; 8. Potentially reducing the number of referrals sent to the tertiary hospitals if AMOs are well trained, working in rural areas, and able to deliver more services and procedures at the local level; 9. Secondment of three expatriate specialists in OB/GYN, Pediatrics, and Surgery at Sengerema has benefited the students, colleagues at the facility, and patients in the surrounding community. Challenges of the model (Sengerema) 1. Sengerema stakeholders expressed a desire to be more involved in the budgeting and planning of the project at their facility, and although they enjoy a 21 good collaboration with Touch, felt strongly that this would further strengthen the partnership. Improving the quality of education for medical students through clinical rotations While the findings regarding AMO clinical rotation and training at Sengerema was overwhelmingly positive, conversely, there was much more discord regarding the delivery of the CUHAS medical student rotations at Sekou Toure hospital. Faculty, students, management, and some expatriate staff expressed some degree of dissatisfaction with the way that the T&T was rolled out at this facility. Many key informants expressed the idea that the T&T concept originated from them and that they had worked on developing a concept paper for funding to roll this out. While they did not receive this funding, there is a perception by CUHAS management and staff that the idea was originally their own and was taken and implemented without their involvement in Sengerema. The initial disagreement about the readiness of the Sekou Toure facility to host clinical rotations led to divergent opinions. As a result, CUHAS began sending medical students to this site without the necessary infrastructure, equipment, and renovations that Touch felt were needed. What remained was a strong sentiment amongst CUHAS and Sekou Toure that funding and technical assistance was diverted to focus on AMO students at Sengerema at the expense of the medical student clinical rotations at Sekou Toure. Despite this difference of opinion, Sekou Toure stakeholders expressed appreciation for the support provided of the two expatriate staff who are helping to train the medical students and they expressed a desire for a stronger working relationship with Touch and a return to “the original plan” which had detailed infrastructure and equipment support that would allow the medical student clinical rotations to operate at a higher level. Echoing the sentiments expressed by management and staff at both CUHAS and Sekou Toure, medical students at CUHAS who are rotating at Sekou Toure expressed strong challenges with their rotations at this facility. While the medical students agree that clinical training is a good idea in theory to improve the quality of training provided, they acknowledge that rotating at Sekou Toure Hospital, a government facility unlike Sengerema and BMC (FBO), has many challenges. The benefits and challenges of the model are listed below. Achievements of the model (Sekou Toure) 1. The concept of clinical rotations for medical students in theory is a good one; 2. Medical students have the opportunity to practice in a resource constrained, urban, government facility without the benefits of the resources and equipment provided by the FBO BMC and gives them exposure to medicine in this constrained setting; 3. Medical students benefit from the two expatriate specialists teaching at Sekou Toure who are supported by Touch and other donors teaching pediatric and internal medicine (Baylor University). 22 Challenges of the model (Sekou Toure) 1. Pervasive opinion that CUHAS and Sekou Toure are implementing the T&T program for medical students at Sekou Toure without the benefits of the Touch funding and technical assistance as was originally intended/expected; 2. Lack of surgical equipment and supplies at Sekou Toure for students to practice procedures; 3. Routine failure of BMC clinical faculty showing up for their clinical rotation duties at Sekou Toure leaving medical students to stand around on their own and waste valuable time and energy; 4. Currently, there are two expatriate specialists seconded to Sekou Toure teaching pediatrics and internal medicine. The students, however, need to complete rotations in four areas, including surgery and OB/GYN. The lack of consistent and full time clinical teaching in these two areas leaves a large gap in the medical student clinical rotational experience. Improving the learning environment (Infrastructure and equipment) Touch has worked closely with the two flagship institutions, BMC and CUHAS, as well as one of the peripheral sites, Sengerema, investing in infrastructure upgrades to enhance the learning and teaching environment. Touch has upgraded basic hospital infrastructure and clinical equipment in the facilities both at the flagship institutions, and the peripheral sites (Sengerema more than Sekou Toure). These efforts have enhanced the working conditions in these facilities and have the potential to improve job satisfaction, patient care, and improve health worker retention. Significant infrastructure upgrades include improvements in the facilities themselves, as well as the critical need of construction of student dormitories at both CUHAS/BMC (private donor funds) and Sengerema (most recent USAID award). The following investments have been made in infrastructure upgrades at the following institutions: CUHAS, BMC, Sengerema and Sekou Toure; 1. Staff housing at Sengerema: 1 house for the rotating specialist from Bugando, and 1 house for the seconded expatriate specialist staff; 2. Student housing at Sengerema: 4 houses completed in August 2003. The first batch of students came in October 2013, from Bugando. The houses can accommodate 24 students. (About 20 students come at a time on rotation); 3. Equipment purchases and upgrades: surgical equipment, ultrasound, consumables, teaching aids (projectors/screens). Additional equipment are on the way, such as digital x-rays for Sengerema; 4. Student housing at CUHAS: Dormitories at CUHAS can accommodate 480 students, and this still remains as the only housing for students at CUHAS (earlier award-private non-USAID donations by private donors); 5. Staff housing at CUHAS: Construction of staff housing with funds from Citibank (private partnership with Touch). There are 47 flats for CUHAS staff; 6. Upgraded operating theaters at both BMC and Sengerema. 23 This infrastructure investment has enhanced the learning environment for students at both the flagship and tertiary institutions. With more students being housed near the university and hospital, it not only increases the motivation of staff and students but also enhances the quality of learning and teaching environment. The development of infrastructure at the peripheral site, not only improved the learning and teaching environment, but also allowed for the clinical practicum training to take place at Sengerema. Touch has also supported some infrastructure and equipment purchases for management capacity building such as information technology (IT) and finance through computer purchases, and the purchase of accounting software to assist with financial management. The evaluation team, however, did not have the opportunity to see the computers or accounting software purchased and is not able to verify their use. Improving access to care for rural populations Peripheral hospitals such as Sengerema Designated District Hospital in the rural district of Geita, has large patient loads, while simultaneously being severely under-resourced in staffing, equipment, supplies and infrastructure. SDDH’s patient load includes: 100,000 outpatients, 25,000 inpatients, and 9,000 deliveries per year. Before its inclusion in the Treat & Train Network, it had no physician specialists and only 2 generalist physicians serving the entire 300-bed hospital. Largely outdated or non￾functioning equipment, along with chronic stock-outs of basic medicine and supplies contributed to make the situation even more critical. Including SDDH in the T&T rotation of AMOs students has had many benefits including: improved infrastructure and equipment, seconded specialists transferring knowledge and skills to colleagues, and improved patient care in this rural setting. Improved infrastructure and equipment has been described above, and includes faculty housing, student housing, and the upgrade of infrastructure and equipment at the hospital. Transfer of knowledge and skills takes place through the T&T specialists based at the peripheral sites are sharing knowledge and skills with approximately 90 health workers in SDDH and approximately 150 in Sekou Toure. While the T&T is training students, the health workers at these peripheral institutions are gaining knowledge, capacity, and skills, which will in turn improve the quality of health care delivery to rural populations in a sustainable way. Visiting teams of specialists from BMC and the full time specialists and volunteers are providing patients with access to specialist care rarely available at the rural district level. As one example, since the posting of the surgeon specialist in Sengerema, 644 surgeries have been performed which would have otherwise been referred to BMC, hence bringing/improved accessibility of specialized health services close to the rural setting. Achievements 1. Currently, 19 specialists are rotated to Sengerema on a weekly basis in Surgery and OB/GYN, resulting in the potential transfer of knowledge, not only to students, but also to the facility faculty and staff; 24 2. Sengerema benefits from 3 specialists supported by Touch, one from the Baylor system, and 2 from Peace Corps, which means that rural populations have access to specialist care rarely seen in rural areas; 3. Improved infrastructure and equipment at Sengerema includes faculty housing, student housing, and the upgrade of infrastructure and equipment at the hospital; 4. Since the posting of the surgeon specialist in Sengerema, 644 surgeries have been performed which would have otherwise been referred to BMC, hence bringing/improved accessibility of specialized health services close to the rural setting, and reducing the referrals of patients to tertiary hospitals. Challenges 1. Recruitment and retention of specialist expatriate staff is not sustainable, can be expensive, and is not the silver bullet to bringing specialist care to rural populations; 2. Touch has achieved full implementation of T&T in Sengerema, partially to Sekou Toure, and not at all to the two other peripheral hospitals in their logic model (Shinyanga Regional Hospital and Kagera Regional Hospital). Sengerema is an FBO and runs under the management of a dynamic and powerful leader. The two remaining hospitals for expansion are government facilities and may be unlike Sengerema in their structure, uptake, motivation, and engagement. The successes of rolling out T&T at Sengerema must be tempered with the reality of rolling out the program at two additional, rural, government facilities; 3. Touch has developed significant experience at the tertiary (BMC), regional (Sekou Toure) and district (SDDH) hospital level, but has not yet began engagement of activities at the village level. The third component of the Touch Logic Model “village level” does not yet seem to be achieved after more than 10 years of implementation in the Lake Zone. Improved health care management Another objective of Touch is to improve the health care management (HCM) of healthcare institutions through training and capacity building. The Healthcare Management Program was designed to accomplish two main objectives: to improve Mzumbe University’s master program in healthcare management, with support from Rush University Medical Center and to improve the overall management of BMC and other hospitals in the Lake Zone of Tanzania. Achievements of HCM program 1. Touch sponsored 6 BMC staff to attend a two week training at RUMC in Chicago, USA which included attachments to departments at hospitals as well as lectures; 2. Fellowship students overwhelmingly enjoyed and gained from their two week excursion to RUMC; 3. The practical application of this learning is seen in the quality improvement projects that are selected by the students, implemented in their departments, and used to improve the quality of a problem area that they themselves have identified; 25 4. Many other faculty and staff are invited to participate in the short CPD short courses in topical areas such as: finance, human resource management, and department planning. These are offered periodically when faculty from RUMC come for short visits to the CUHAS; Challenges of HCM program 1. The initial plans were that RUMC would work with Mzumbe University to overhaul the Health System’s Masters Curriculum, and that Mzumbe University would then be capacitated to lead the production of a new cadre of hospital managers with business and organizational skills currently unavailable in the country. Little evidence was seen of the implementation of this original plan; 2. Many of the quality improvement projects that were described to us by the fellows were in initial stages of data collection or data had been collected but not yet analyzed. We were therefore unable to assess if these projects led to improvements in the respective departments; 3. The cost of each fellowship is a matter to be considered and alternatives for a more cost-effective model of improving health care management should be explored (will be described later). 4. The fellows who have completed the program are intended to train others in what they have learned, but it is not clear what the expectation is around this (cost, time, level of effort, target audience, etc.). Fellows don’t seem to be aware about this expectation to train others, other than the informal knowledge exchange and information sharing that would take place with their colleagues in their respective departments. Capacity building of CUHAS faculty Touch has supported the strengthening of CUHAS faculty in many ways. Exchange visits for residents between CUHAS and Cornell University in the area of Internal Medicine has facilitated a great exchange. CUAHS/BMC residents go to Cornell (New York, USA) for one month, and vice versa. CUHAS management has remarked great improvements in knowledge, practice, and motivation upon residents’ returning from their study abroad. Likewise, CUHAS faculty have been sent to New York for various capacity building activities, and this has also benefited both the individual faculty and the institution in terms of improvements in the curriculum and the ability of faculty to utilize problem-based teaching with their students. However, additional capacity building in writing research proposals, succeeding in obtaining them, implementing them, and presenting findings in conferences and publishing in journals is needed. Knowledge development and dissemination Touch has not yet created a significant amount of products in this area of knowledge development and dissemination. There are, however, three relevant documents worth noting. The first report was conducted in collaboration with McKinsey and Company and was published in 2006 titled: “Investing in Tanzanian Human Resources for Health”. This report was based on field research and advocated for the training of highly skilled health 26 workers to address public health challenges in the country. It also laid out a strategy that Touch would pursue for training skilled health workers. The second report was published in 2009, and is titled: “Action now on the Tanzanian health workforce crisis”. This laid out a roadmap of initiatives to increase health worker training capacity . Finally another joint McKinsey/Touch collaboration produced: “Catalyzing change; molecular strengthening of health system in the Lake Zone”, launched in July 2009. In early 2008, a joint Touch Foundation and McKinsey & Company team conducted a study in the Lake Zone and tried to identify key bottlenecks in the health system. Their fieldwork included over 200 interviews, 50 site visits to all levels of health facilities in the Lake Zone, and several workshops and focus group sessions with patients and health workers. This resulted in the development of a portfolio of practical initiatives designed to address them and informed the path of Touch for the past five years. In addition to the above-mentioned reports, Touch has also produced brief documents sharing their achievements and impact. In June 2013, Touch developed a report titled: “Achievements, Approach, and Path Forward” where the foundation describes their approach, programs, and achievements to date. In this document, Touch not only describes what they have achieved to date, but also explores Lake Zone health systems challenges, such as patient referrals, patient transportation, supplies and procurement, and also describe potential solutions to address them. In the same year, Touch released a document titled: “Touch Foundation Impact (2004 – 2012)” further describing their overall achievements and impact to date, as well as articulating their plans for the future. 27 TABLE 3: KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS (EFFECTIVENESS) FINDINGS CONCLUSIONS RECOMMENDATIONS Beneficiary institutions expressed a strong sentiment of lack of inclusion in the planning and budgeting processes. Limited collaborative planning and decision-making on the project’s activities has led to feelings of resentment and exclusion. Improve communications and joint planning between Touch and beneficiary institutional management and staff. Strong perception of improved quality of education for AMOs as a result of the T&T in Sengerema, but is less evident at STRH for various reasons. The clinical rotation at STRH provides a less conducive environment for medical student learning. Strengthen clinical rotation at STRH through infrastructure and equipment investments, and better scheduling and accountability of instructors. Sekou Toure has two expatriate specialists providing full time teaching in Pediatrics and Internal Medicine, but not in OB/GYN and surgery. There is a potential that students are receiving a higher quality of clinical experience in two departments and less so in the remaining two. Touch should consider placing two more specialists in the departments of OB/GYN and surgery at Sekou Toure with a preference for local specialists. There is a general lack of clarity or information sharing about the purpose, function, and roles of the “scribes”. Beneficiary institutions lack understanding as to what data the scribes, are collecting and how it is or will be used. The purpose, function, and roles of the “scribes” should be clarified and feedback shared with institutions. CUHAS/BMC, with Touch support, has graduated 278 physicians (~10% of the country’s total), 70 specialists, and 1,700 others including: 388 (AMOs), 376 nurses, 168 nurse anesthetists, 259 laboratory scientists and 249 pharmacists. There has clearly been a rise in the quantity of health care worker production through the rapid increase of enrollment and graduation of health students. It is, however, important to ensure that this rapid increase does not compromise quality of education provided to students. Continued vigilance is needed for balanced teacher/student ratios and maintained quality of medical training. 19 specialists are rotated weekly to Sengerema in Surgery and OB/GYN, resulting in the potential transfer of knowledge to students and facility staff. Sengerema, a rural district, benefits from 3 specialists supported by Touch. There is evidence of improved capacity to provide higher quality of services in Sengerema through provision of specialist care, as well as improvements in infrastructure and equipment availability. Though improved access to care for rural populations is evident through availability of specialists, improved data collection and analysis can further strengthen the evidence to support improved health outcomes as a result. Health care management fellows appreciate and have learned from their fellowship experience, quality improvement projects, and CPD short courses. The program has strengthened the capacity for health management at BMC for 6 fellows and participants of the CPD short-courses. Evidence could not be substantiated of results of QI projects. Analysis and documentation of the quality improvements resulting directly from fellows’ participation in the program can strengthen the evidence of the fellowship results. CUHAS faculty members have had various capacity building activities which has benefited both the individual faculty development and the institutional improvements in curriculum and teaching methods. Improved capacity of CUHAS faculty to design and teach medical students through problem-based learning approaches has likely resulted in an improved learning environment. CUHAS faculty and staff should continue to benefit from professional development opportunities to improve their capacity to utilize effective teaching methods and approaches, and additional capacity is needed in writing and implementing research. Touch has done a tremendous job of conducting data collection through innovative means to determine the job placement rates and geographic distribution of Bugando graduates showing high placement rates in the Lake Zone and in TZ. Graduate tracking has been achieved through Touch’s own human resources, efforts, and initiative. CUHAS/BMC has not yet institutionalized or fully owns a tracking system. Graduate tracking systems and procedures should be refined, transferred to local institutions, and institutionalized to facilitate graduate tracking and establish an alumnae network. 28 FINDINGS: REPRODUCIBILITY When assessing the evaluation’s questions of ‘to what extent is the model scalable beyond the Lake Zone to other regions of the country and what are the potential opportunities and obstacles to bringing the model to scale?”, the evaluation team followed the definition laid out in PEPFAR Core Elements for PPPS’s. This included looking at the potential for Touch to grow in order of magnitude within the next five to ten years, potential for regional expansion, and building local capacity to facilitate scale up. Touch staff, as expressed through interviews, envision four options for replicating the full Touch model: 1. Directly reproduce the full program in a different geographical area: In this instance the entire Touch Lake Zone model would be reproduced and implemented by Touch in other regions of the country. In this model, Touch would support multiple office locations to provide project management oversight—this oversight is especially essential during the dorm construction phase of launching the T&T program as well as for any structural facility upgrades. This possibility was the least likely model for scale up due to the organization’s size, staffing structure, and ability to raise sufficient funds for expansion. Currently, Touch does not plan to expand geographically outside of the Lake Zone. 2. Replication of parts of the program: In this scenario, Touch sees itself serving as an implementing partner to a larger organization to reproduce portions of their program. An alternative to this option presented by other non-Touch interviewees was for Touch to be partial implementers and hand over portions of their program to in-country stakeholders (other universities, district councils, other faith based health institutions). 3. Serve as technical experts /advisors to the MOHSW and other stakeholders: In this scenario, Touch would play an advocacy role at multiple levels of government and large stakeholders. This would shift Touch staff away from program implementation, towards program design. In this instance, Tanzanian partners would be the implementers. 4. Expansion to the village level: A fourth model presented in Touch’s “Achievements, Approaches and Path Forward June 2013” document is the future strategy to continue expansion at the hospital level then continue the network expansion to the village level (health center). In this version, Touch continues the existing T&T program, but expands the program to lower levels of the health system, thus expanding its reach within the Lake Zone. Reproducibility of Treat and Train 1. AMO cadre clinical rotations are highly replicable to other parts of the region and country. 2. There is potential to reproduce Treat and Train at government facilities, FBO facilities, and private sector facilities alike; 3. Local capacity to facilitate the replication and scale up T&T, to finance the expenses of running the program, to mobilize resource to fund it, and the 29 capacity to manage it is still in infancy and would require significant technical and financial support; 4. The continuation of the T&T at the two facilities where it is already operating is feasible; 5. It is acknowledged that expansion of T&T to the village level would be valuable, however, many questions remain about feasibility and use. One limiting factor identified by respondents was the limitation of the health centers for training purposes due to lack of a high volume of patients; 6. There was a general sense from key informants that the scribes would not be a cadre that would able to be sustained in government or FBO structures. To date, there has been little to no feedback to the institutions or their staff regarding the data collected, or how it can benefit students, staff, institutions or patients. 7. The ability to reproduce the T&T program would require galvanizing additional resources, particular those of local industries such as: breweries, mines, and telecommunications. Similarly, local councils should be approached to provide in￾kind support through identification of local housing for students; 8. One of the challenges to replicating T&T is the high startup costs associated with the construction of housing for students and rotating faculty. In Sengerema, it cost $215,000 USD. An additional $100,000 USD was spent to outfit the training site with medical equipment. Similar costs would be needed at new T&T sites; 9. While the current T&T model serves AMOs and MDs, the MOHSW sees the need to prioritize what they call “endangered cadres” such as: lab technicians, pharmacists, pathologists, anesthesiologists and, ophthalmologists. 10. Replication to other sites should consider the burdens to these sites of having students rotate including consumables such as: gloves, reagents, syringes and other medical supplies; 11. Currently Touch committed to covering the expenses of two physicians from Sengerema hospital to get specialist training – Masters in Medicine at a cost of about $7,000 – $8,000 USD per physician. The expectation is that both of these physicians will return to Sengerema to treat patients and train students. As T&T grows, it will be challenging to find sufficient skilled specialists to teach both medical students and AMOs; 12.Day-to-day oversight of the student and faculty housing project is a significant investment in not only funding, but human resource time to plan, procure, and manage the project; 13. The logistics of managing the T&T faculty schedules is considerably time consuming and must be taken into account in plans for replication and scale-up; Challenges to replicating Treat and Train 1. One of the challenges to replicating T&T is the high startup costs associated with the construction of housing for students and rotating faculty. As mentioned earlier, housing construction costs were close to $215,000 USD at Sengerema. An additional $100,000 USD was spent to outfit the training site with medical equipment. Similar costs would be needed at new T&T sites; 2. While the current T&T model serves AMOs and MDs, the MOHSW sees the need to prioritize what they call “endangered cadres” such as: lab techs, pharmacists, pathologists, anesthesiologists and, ophthalmologists. This has 30 potential to create a disconnect in priorities when identifying target cadres for the future role out of the T&T program; 3. Replication to other sites should consider the burdens to these sites of having students rotate including consumables such as: gloves, reagents, syringes and other medical supplies; 4. Currently Touch committed to covering the expenses of two physicians from Sengerema hospital to get specialist training – Masters in Medicine at a cost of about $7,000 – $8,000 USD per physician. The expectation is that both of these physicians will return to Sengerema to treat patients and train students. As T&T grows, it will be challenging to find sufficient skilled specialists to teach both medical students and AMOs; 5. Day-to-day oversight of the student and faculty housing project is a significant investment in not only funding, but human resource time to plan, procure, and manage the project; 6. The logistics of managing the T&T faculty schedules is considerably time consuming and must be taken into account in plans for replication and scale-up. Reproducibility of the Healthcare Management Fellowship 1. Fellows were grateful for the exposure provided through the international training experience gained through the fellowship, and were able to apply what they learned in their jobs. However, this model may not be the most efficient to improve health care management training. Strengthening of local institutions to conduct this training is more feasible, acceptable, scalable, and sustainable; 2. The time allocated by visiting professors to conduct the continuing education trainings was seen as too short for fellows to retain and synthesize the information that was imparted; 3. While Touch envisions utilizing the 6 fellows who were trained to in-turn teach this curriculum in the future, this was not understood or echoed as an expectation by the current fellows; 4. Touch plans to expand this program by “adding a few fellows from each of the four regional and district Treat and Train hospitals in the upcoming years.2 ” 5. Touch is collaborating with the Christian Social Service Commission (CSSC) to develop Continuous Professional Development modules; 6. The cost to run the fellowship program is approximately $250,000 USD per year, and may not be the greatest value for money for such a small number of fellows trained. Challenges to replicating the Healthcare Management Fellowship 1. In terms of pre-service training, there are not many options for collaboration. The healthcare management degree is only available at Mzumbe University in Dar es Salaam. It has been stated that RUMC evaluated the healthcare management curriculum at Mzumbe and will be helping to revise it. The evaluation team has not been provided any documentation regarding this. It was reported to us that the perception of the quality of the students who graduate from the Mzumbe program is low. Touch hopes that an improved curriculum and building the reputation of the Mzumbe program through its affiliation with RUMC will attract higher quality of students for the degree; 31 2. Manager positions at BMC do not require management training as prerequisite for the positions. Fellows who have returned to BMC state that if this program were to continue or be scaled up, it will be important to engage and train the hospital management staff that were not part of the fellowship to build a culture of management at BMC. While it seems to be an expectation that the existing fellows train others (formally), this understanding is not shared by the fellows themselves; 3. There may be other opportunities for in country collaboration. With support from Wharton School of Business, Muhimbili hosted a 3-day course in health administration. They invited 4th and 5th year students from Muhimbili, as well as students from other universities in Dar es Salaam, KCMC, Moshe and Bugando. Knowledge development and dissemination Though the evaluation focused mostly on Treat and Train, the team was able to make the following observations regarding Touch’s Knowledge Development and Dissemination. 1. There are three reports produced by Touch that have been published and disseminated on paper and on-line. These reports have articulated the diagnosis of the problem, an articulation of the Touch approach, achievements and successes to date, and plans moving forward; 2. These reports are of high quality and have been made widely available, via the Internet and some in-country distribution. Challenges for scale up: Knowledge development and dissemination 1. MOHSW stakeholders perceived that Touch progress reports, documents, and updates were not well disseminated throughout the ministry and that though they have the perception that Touch is doing good work, they have not received the evidence to support it; 2. There was a general perception of need for documentation of lessons learned by Touch to make information available about the elements of Touch programs that could be scaled up or reproduced by Tanzanian entities. The type of information that would be useful is a step-by-step, process-oriented document detailing both the processes and costs of starting and running their model. 32 TABLE 4: KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS (REPRODUCIBILITY) FINDINGS CONCLUSIONS RECOMMENDATIONS While there are other models of medical student rotation at other training institutions around Tanzania, Touch has been the leader in offsite clinical rotations for AMOs. T&T program scale up should include AMOs as well as MD students. The most readily scalable component of the T&T program for AMOs is the inclusion of all 4 clinical areas (OB/GYN, Pediatrics, Surgery, Internal Medicine) into clinical rotations for AMOs at Sengerema. Expansion of the T&T for AMOs should be considered at other medical colleges. Some suggestions include: Tanga, Mbeya and Kilimanjaro Christian Medical College (KCMC), Ifakara in Morogoro region, Songea (bordering Mozambique), and other schools affiliated AMOs. Medical student rotations at STRH have been less successful due to shifts in initial plans, lack of the necessary infrastructure and equipment upgrades, and expatriate staff in two (internal medicine and pediatrics) of the four clinical disciplines. STRH provides a less conducive environment for the medical student learning. Students may be getting high quality training in two departments and less so in the remaining two, due to the lack of dedicated clinical teaching staff. The inclusion of all 4 clinical areas (OB/GYN, Pediatrics, Surgery, Internal Medicine) into clinical rotations for medical students at Sekou Toure in addition to infrastructure upgrades is needed. T&T has been rolled out at just two sites, Sengerema (rural district) and Sekou Toure (urban district) in the Lake Zone. T&T is replicable, but expanding the program beyond the Lake Zone will require coordinated effort with in country partners. Touch should work with local partners to solicit in-kind contributions of land, housing, human resources, and local support from the community, private sector, and government. Touch’s core competency is working at the regional and the district level. In bringing the program to lower levels (village level), Touch will have to carefully consider its own operational capacity to support the program in remote areas, the reproducibility of the model at lower levels, and the feasibility of upgrading health centers. Health centers may also not have the volume of patients needed for clinical teaching. Touch should make a decision about expansion: I) Continue working with additional regional, district, and designated district hospitals, and consider scale up to both public and private health facilities in the region; 2) Reproduce the model to other regions, but remain at the district hospital level; OR 3) Reproduce the model at the health center level (village level) with modifications to align with this context. Touch brings specialists through partnerships with universities. Peace Corps clinicians are more cost effective way to continue to bring outside clinicians to Tanzania. Seconding expatriate specialists at T&T sites is neither scalable nor sustainable. Efforts should be made to find alternatives to the expatriate model by using local specialists, creating incentives for retention, and ensuring explicit roles in teaching students, facility staff, and treating patients. The Healthcare Management Fellowship is the least integrated of Touch activities and least likely to be replicable in its current form. The funds expended to send six fellows to Chicago may be better directed towards strengthening in country health management integration into curriculum, and building the capacity of in country academic teaching and training institutions. Touch should work more closely with other training institutes like KCMC, Muhimbili, and Mzumbe on healthcare management integration into medical curriculum. This should be coupled with professionalization of healthcare management. Touch has produced three reports: “Investing in Tanzanian Human Resources for Health” (2006), “Action now on the Tanzanian health workforce crisis” (2009), “Catalyzing change; molecular strengthening of health system in the Lake Zone” (2009). These reports and two additional documents describing their results and achievements are informative. However, these do not describe the Touch model in detail for purposes of replication. A step-by-step documentation of their model including costs and lessons learned will help inform and prepare in-country partners to assume ownership and be able to reproduce and expand it. 33 FINDINGS: SUSTAINABILITY Overview of Touch approach to sustainability Since inception in 2004, Touch has made important strides in increasing the supply of skilled new health professionals and in improving the skills of existing health workers. As the program shifts from an emergency-response mode to a sustainable platform for human resource development, strategies for maintaining and strengthening local ownership will continue to be critical to ensure the long term sustainability of Touch’s programs and interventions. Touch is in the process of transitioning from an emergency response to the HRH crisis in Tanzania, to a sustainable model for health delivery and education. Touch has designed their programs to help support Tanzanian ownership and long-term sustainability by focusing on four key elements: x Health worker training – Touch focuses on training local human resources responsible for supporting pre- and post- service trainings to ensure long-term clinical and managerial capacity of the system; x Program design – Touch designs solutions considering the limited financial resources of their Tanzanian partners, who will ultimately take ownership of the programs and run them alone. The programs entail substantial investments during the set-up and initial implementation stage, with gradual reduction of the expenses as Touch phases out. The on-going running costs are lower, enabling the Tanzanian owners to sustain the programs in the long run; x Co-implementation with local Tanzanian owners – Touch operates and undertakes the design and implementation of programs in collaboration with their local partners. This close cooperation enables Touch to build their partners’ capabilities in key areas such as management, finance and fundraising, supporting a long-term implementation of the programs once fully transitioned to Tanzanian ownership; and x Support from key Tanzanian stakeholders – Touch works in collaboration with national and local government to align and integrate their programs with the country’s long-term strategy. The President of Tanzania has been supportive of Touch’s work from the beginning and continues to endorse their programs. Going forward, Touch plans to further increase their focus on promoting program sustainability, to help ensure that activities will continue after Touch’s exit. In the long term, Tanzanian champion institutions are expected to provide financial support and technical expertise to further sustain activities that are considered critical to human resources development in the Lake Zone. Assessing sustainability across key dimensions of financial, managerial and country ownership Touch’s approach to program implementation has supported financial and management sustainability as well as country ownership of their programs by local partners. 34 FINDINGS: SUSTAINABILITY Financial sustainability: CUHAS & BMC 1. CUHAS has become increasingly self-sustainable, enabling Tanzanian ownership of the on-going initiatives Touch has developed and allowing a reduction of Touch’s involvement over time; 2. During the set up and initial implementation of this program, Touch covered the majority of the CUHAS costs, reaching a peak of approximately $3 million in 2008. Over time, Touch has been able to decrease their financial support to CUHAS down to ~$0.5 million USD in 2012; 3. CUHAS revenues from non-Touch sources has grown from $0.5 million in 2005 to approximately $3.5 million in 2012, more than offsetting Touch’s decrease in contribution and making CUHAS increasingly self-sustainable from a financial viewpoint (Figure 3); 4. Touch has supported BMC mainly on capital investments and capability building, fostering sustainability since the beginning of their partnership; 5. Touch’s financial support to BMC has been limited to capital investments, requiring low additional contributions from BMC (primarily for maintenance costs); 6. Beginning in FY2016, BMC will have full responsibility for covering the cost of the program at Sengerema. Out of a total running cost of $40,000 per year, however, $20,000 for student meals was already borne by BMC when students were receiving their clinical education at BMC. BMC will need to cover the incremental cost associated with physician outreach, transportation and student housing utilities, totaling up to another ~$20,000 per year. 7. The described costs are considered the “minimum” required to keep the program up and running successfully. However, during these last two years, Touch also pursued additional opportunities for hospital improvements, including the engagement of Canadian physicians to provide specialized maternal training to local staff, and collaborating with the Peace Corps to have two specialists posted at Sengerema on a yearly basis. Continuing to pursue such low-cost opportunities in the future will allow BMC to raise the quality of the program even further through sustainable public and private partnerships. 8. Costs of maintaining the programs are expected to decrease over time, after initial capital and start up investments, At Sengerema, the program cost is expected to decrease from an initial investment of ~$750,000 in FY2012 to ~ $48,000 per year from FY2016 (Figure 5). Challenges in financial sustainability 1. CUHAS has utilized student enrollment to increase resource mobilization and to compensate for the decreasing operational support that Touch provided in previous years. The rapid enrollment of students is currently the main source of revenue with limited alternative resources; 2. In a challenging environment with decreasing foreign aid (including PEPFAR), it is critical to diversify funding sources and not rely on one donor or one internal source of revenue. 35 Managerial sustainability: CUHAS & BMC 1. Touch’s partnership with BMC has facilitated capacity building in health care management through the healthcare management fellowship and CPD short￾courses; 2. Financial management capabilities have been enhanced by Touch-supported investment in QuickBooks for BMC and CUHAS; 3. Both BMC and CUHAS are required to submit proposals to access Touch’s capital and special project grants, the intention of which is to strengthen institutional grant-writing capabilities. Challenges in managerial sustainability 1. The health care management initiative is one effort to improving the health care management of BMC. However, currently, just 6 fellows from the hospital have benefited from the fellowship program and many others have benefited from short training. Expanding the reach of the initiative through strengthening of local institutions to provide this training is in its infancy and is the modality that would be most sustainable. Sustainability of Treat and Train 1. The T&T program is highly regarded and viewed as necessary and sustainable by stakeholders at the existing sites where it is currently operating. Challenges to sustainability of T&T 1. While the increasing enrollment of students at CUHAS is to be applauded, the shrinking student to faculty ratio may compromise the quality of education. In addition, BMC will not be able to cater for the clinical training of enrolled students, so T&T is not only an option, but a must; 2. While there is consensus that the T&T model has improved the quality of medical training, and there has been high placement of graduates throughout the country, placement/retention of graduates needs to be carefully monitored to ensure on￾going success. Placement rates of graduates has been good, but, there are on￾going challenges regarding retention of health workers, especially at rural posts; 3. Touch has worked with Sengerema and Sekou Toure in 2012, to prepare both hospitals to host and manage external clinical rotations of CUHAS and BMC students. While Sengerema has had significant infrastructure and equipment upgrades in addition to expatriate secondments, Sekou Toure benefited only from expatriate presence and has not had the requisite infrastructure and equipment upgrades; 4. Touch has supported significant up-front operational costs to engage international MD specialists at both Sengerema and Sekou Toure, in order to ensure a successful set-up of the T&T program. These expatriate specialists cannot be sustained without external support, and local solutions should be explored. One potential local solution is that one MD from Sengerema is being trained as a specialist and will return to the rural hospital to provide care as well 36 as teach students. This same model and others should be explored as a sustainable alternative to the expatriate model. 5. Touch has had limited interaction with the MOH PPP - TWG and could benefit from participation to streamline communications with national, regional, and local authorities around key financing and Human Resource Development polices; 6. Organizations such as the Association of Private Health Facilities of Tanzania (APHFTA) and the Christian Social Services Commission provide important opportunities and linkages to advocate and refine national policies for HRH as well as sensitization of PPPs to better plan for staff recruitment and retention. While there was good recognition of the Touch programs, association representatives suggested stronger participation and transparency of issues to promote local ownership of programs and sustainability. Country ownership Touch’s programs are grounded in close collaborations with local partner organizations that play a central role in program implementation. Touch has been transitioning the management of their programs to local partners (e.g. financial and HR management of Sengerema student rotations run by BMC staff, BMC healthcare management fellows independently running the development of their improvement projects under supervision of Touch and RUMC experts), as well as constantly seeking buy-in of their programs from both local (e.g. Regional Medical Officer, District Medical Officer) and national government (e.g. Chief Medical Officer, Director of Curative Services). While Touch has done a tremendous job of building country ownership by working with local partners, stronger participation especially in the planning stages as well as in replication and scale up is needed. In addition to the existing partners, there are greater opportunities for local engagement with additional private sector sources (mining, telecommunications, banking) as well as with local government (local councils) and national associations and TWG within the national government structures (MOHSW, PMORALG). 37 CUHAS’s non-Touch revenue sources have grown from $0.5 million USD in 2005 to approximately $3.5 million USD in 2012. Figure 3: Touch vs. non-Touch contribution to CUHAS to income CUHAS dependency on Touch’s contribution as a percentage of their total income has decreased from ~60% in 2005 to ~15% in 2012 (Figure 4). Figure 4: Touch contribution as percentage of total CUHAS total income Breakdown of total income at CUHAS: Touch Support vs. Other Income Source: Total Income from CUHAS Audits and Touch contribution from Touch Audits. CUHAS Fiscal Year Sept – Aug. Touch Fiscal Year July – June from 2005 to 2010, Oct – Sept 2011 to current - 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 2005 2006 2007 2008 2009 2010 2011 2012 Touch Contribution Other Than Touch Contribution Year Contribution (USD) Breakdown of total income at CUHAS: Touch Support vs. Other Income Source: Total Income from CUHAS Audits and Touch contribution from Touch Audits. CUHAS Fiscal Year Sept – Aug. Touch Fiscal Year July – June from 2005 to 2010, Oct – Sept 2011 to current 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2005 2006 2007 2008 2009 2010 2011 2012 Touch Contribution (as % of Total Budget) Other Than Touch Contribution (as % of Total Budget) Year Contribution (% of total budget) 38 At Sengerema, the program cost is expected to decrease from an initial investment of ~$750,000 in FY2012 to ~ $48,000 per year from FY2016 (Figure 5). Figure 5: Total cost of Treat & Train Network extension at Sengerema hospital The external rotation program at Sengerema Hospital will be owned and run by BMC with very limited financial burden on the hospital itself FY 2014 50 101 40 FY 2015 60 60 348 20 328 FY 2017 50 101 40 FY 2013 FY 2016 752 371 381 FY 2012 Touch-borne capital investments Touch-borne operational support BMC-borne operational cost Cost of external rotation program at Sengerema for AMO students ILLUSTRATIVE Annual cost in thousands of US dollars Cost comprised primarily of large up￾front capital (i.e. construction and clinical equipment) and operational (i.e. engagement of international MD specialists) investments No external rotation program The additional cost to BMC for AMO external rotations is only $20k out of a total operational cost of $40k ƒ $20k for student meals is already borne by BMC when rotations occur at BMC ƒ $20k for physician outreach and transportation and housing utilities is the only additional cost to be borne by BMC for external rotations at Sengerema 39 TABLE 5: KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS (SUSTAINABILITY) FINDINGS CONCLUSIONS RECOMMENDATIONS CUHAS revenues from non-Touch sources has grown from $0.5 million in 2005 to approximately $3.5 million in 2012, more than offsetting Touch’s decrease in contribution and making CUHAS increasingly financially self-sustainable. Much of the internal revenue sources are from increasing student enrolment and increasing revenues from student tuition fees. This increases the teacher/student ratios, has the potential for compromising the quality of education, and places much of the resource generation burden onto one revenue source. Increasing revenues from other sources should offset the heavy dependency on student tuition fees for internal revenue generation. This can include: fundraising efforts, other donors, and increased revenue from research grants. A focus on strengthening CUHAS’s ability to generate resources is needed to improve sustainability. Both BMC and CUHAS are required to submit proposals to access Touch’s capital and special project grants, the intention of which is to strengthen institutional grant-writing capabilities. Project proposals approved within the past year include ICT networking and infrastructure upgrade, new accounting software Sage Pastel to improve their financial operations, and elevator purchase and installation for MD student hostel. This shift towards requiring formal proposals for projects has improved the CUHAS/BMC’s abilities in grant writing and justifying activities and costs. Touch should continue to fund these institutions through formal grant requests thus increasing proposal writing capacity and ensuring projects driven by the institutions addressing their own needs and simultaneously promoting country ownership. This grant writing capacity should be used to garner resources outside of Touch. Touch has done a tremendous job of building partnerships with local institutions (CUHAS, BMC, Sengerema, Sekou Toure) as well as some private sector companies (African Barrick Gold). Touch has also built strong partnerships with some international institutions: Cornell University, Baylor University, McKinsey and Co., and U.S. Peace Corps, amongst others. While these partnerships have been fruitful and resulted in significant gains, additional partnerships will be needed to expand and sustain gains. Touch should continue to develop partnerships (both local and global) that provide shared value as well as technical specialist and financial support (grants, research). Touch should also work to facilitate CUHAS/BMC’s ability to mobilize resources, build its global network, and foster its own fruitful partnerships that can be sustained outside of Touch. CUAHS/BMC achieved high placement rates of their own graduates. Nationally, there are challenges to placement of medical school graduates within the MOHSW system. With an approximate 65% vacancy rate for funded positions in the public sector, the shortfall in health workers threatens to impede service delivery. Absorption of HRH will be a critical challenge going forward. Touch should liaise more closely with national private associations (APHFTA and CSSC), the PPP-TWG, and the MOHSW unit on HRH retention policies. Touch should work with government to develop strategies that can help strengthen polices for absorptive capacity and retention of health workers within the region. Touch should help to strengthen targeted recruitment and retention policies at the national and local (regional and district) levels. Touch has supported significant up-front operational costs to engage international MD specialists at both Sengerema and Sekou Toure, in order to ensure implementation of the T&T program. These expatriate specialists will not be able to be sustained without external support, and local solutions should be explored. One potential local solution is that one MD from Sengerema is being trained as a specialist and will return to the rural hospital to provide care as well as teach students. This same model and others are possible solution to the expatriate model. 40 The Touch program has made significant gains in increased health worker production, improved quality of education, and some progress on building capacity in health care management. Moving forward, there is a need to continue to focus on the long-term sustainability of programs by supporting the transition to local managerial, financial, and country ownership. Strengthen existing partners for the T&T program to maximize the impact of their investments to date by continuing to strategically support BMC and CUHAS while moving towards transition to local ownership. 41 CONCLUSIONS Effectiveness 1. Though CUHAS stakeholders acknowledge the tremendous support received by Touch, there is a perception of limited joint-planning and decision-making which has led to feelings of resentment and exclusion by some stakeholders; 2. Improved quality of education for AMOs as a result of the T&T program in Sengerema. However, less evidence was found on the improved quality of medical student education. The clinical rotation at Sekou Toure provides a less conducive environment for the medical student learning as described by current medical students; 3. Sekou Toure currently has two expatriate specialists in Pediatrics and Internal Medicine. Medical students also need clinical rotations in OB/GYN and surgery, so they may be getting high quality training in two departments and less so in the remaining two; 4. There is a general lack of understanding or information sharing about the purpose, function, and roles of the “scribes”. There is a lack of clarity as to what data is being collected and how it is or will be used; 5. Increased quantity of health care worker production through the rapid increase in enrollment and graduation of health students; 6. Improved capacity to provide higher quality of services in the peripheral hospitals through provision of specialist care, as well as improvements in infrastructure and equipment availability; 7. Improved capacity for health management at BMC through the implementation of the health care management fellowship and its subsequent quality improvement projects; 8. Improved capacity of CUHAS faculty to design and teach medical students through problem-based learning approaches; 9. Touch has done a tremendous job of conducting data collection through innovative means to determine the job placement rates and geographic distribution of Bugando graduates showing high placement rates in the Lake Zone and in TZ. This graduate tracking, however, has been achieved through Touch’s own human resources, efforts, and initiative and is not yet a tracking system that is fully owned by the institutions; Reproducibility 1. Treat and Train is highly replicable. While there are other models of medical student rotation at other training institutions around Tanzania, Touch has been the leader in offsite clinical rotations for AMOs. Bringing T&T to scale should include this cadre as well as others. The most readily scalable component of the T&T program for AMOS is the inclusion of all 4 clinical areas (OB/GYN, Pediatrics, Surgery, Internal Medicine) into clinical rotations for AMOS; 2. Expanding the T&T program beyond the Lake Zone will require coordinated effort with in country partners to identify geographical areas and sites for the program; 42 3. Touch’s core competency is working at the regional and the district level. In considering bringing the program to lower levels (village level), Touch will have to carefully consider its own operational capacity to support the program in remote areas; 4. In considering program expansion, it may make sense for Touch to continue working with additional regional, district, and designated district hospitals, but consider scale up to both public and private health facilities within the region; 5. In order to implement simultaneous expansion to the health center level (village level) and expanding T&T to other regions, Touch will have to explore co-implementation with Tanzanian partners (district health councils, corporate and social responsibility programs, Regional Administrative Secretaries (RAS)) to support the start-up costs of launching the T&T program as well as the costs of sponsoring doctors for specialist training to return to teach, train students, and treat patients; 6. While Touch may be using the data collected by the scribes, there is need for a closer look at the purpose, of the scribe position and the benefit to the health institutions in which they are placed; 7. Peace Corps clinicians were seen as a more cost effective and sustainable way to continue to bring outside clinicians (nurses and doctors) to Tanzania. It is clear that bringing expatriate specialists is neither scalable nor sustainable. Respondents encouraged Touch to make sure the expatriate specialists have both teaching and treating as part of their responsibilities; 8. There are several faith based catholic hospitals in the Lake Zone that in line to be upgraded to referral hospitals. This presents the opportunity for the Treat and Train program to be developed at these new facilities as they convert to referral hospitals and some of which may possibly become teaching hospitals; 9. The Healthcare Management Fellowship seemed the least integrated of Touch activities and least likely to be replicable in its current form. The funds expended to send six fellows to Chicago may be better directed towards strengthening in country health management integration into curriculum, and building the capacity of in country academic teaching and training institutes. Touch should work more closely with other training institutes like KCMC, which is a MEPI (Medical Education Partnership Initiative) school and Muhimbili on healthcare management integration into medical curriculum. This would also have to be coupled with an effort that looks at working with professional associations on the professionalization of healthcare managers; 10. It will be important for Touch to publish and widely disseminate documentation including steps to reproduce their model including costs, as well as lessons learned. This will help inform and prepare in-country partners to assume ownership and responsibility as elements of the Touch program are expanded and reproduced at other institutions. In addition, this will help contribute to the evidence base of the Touch health system strengthening approach. This will be critically important as Touch moves towards stepping out of the role of implementer. 43 Sustainability 1. While the Touch program to increase health worker production has achieved significant results, there is a need to continue to focus on the long-term sustainability of their programs by supporting the transition to local managerial and financial ownership; 2. An important focus for Touch will be to integrate strategies that can help strengthen polices for absorptive capacity and retention of health workers within the region. This will include informing government policies and driving the development of integrated, evidence-based public private partnership solutions to support the targeted placement of skilled health workers. It is critical to put appropriate incentives in place to ensure their future retention, particularly in rural areas in the Lake Zone; 3. An initial focus on the emergency response, which focused on increasing the supply of HRH provided support, through Touch, for CUHAS/BMC. As a result, the institutions have achieved a high rate of graduation, and the institutions were also able to achieve high placement rates. Nationally, however, there are challenges to placement of medical school graduates within the MOH system due to the growing shortage of available funded job openings. With an approximate 65% vacancy rate for funded positions in the public sector, the shortfall in health workers threatens to impede efforts to scale up and maintain care and treatment services3 . Absorption of HRH will be critical challenge going forward; 4. Touch should liaise more closely with national private associations (APHFTA and CSSC) and the PPP-TWG broader health systems gaps, including supply chain and HRH retention policies need to be addressed to ensure long-term satisfaction at post; 5. Given expansion of programs, including T&T to satellite facilities, there is need for the development of strong on-going tracking systems for current students, graduates, and in-service training to monitor quality of degree programs as well as retention at post. RECOMMENDATIONS In light of the evidence identified in the course of this evaluation, the team proposes the following recommendations for Touch in an effort to continue to improve the HRH project implemented by the foundation in order to leverage investments already made and create a path to meet the GoT’s goals of improving access to and the quality of health care for the population of Tanzania. EFFECTIVENESS 1. Improve communication and joint planning between Touch and CUHAS management and staff; 2. The clinical rotations at Sekou Toure, which currently provide a less conducive environment for the medical student learning, should be strengthened through infrastructure and equipment investments, better 44 collaboration and communication regarding scheduling of clinical instructors between Bugando and Sekou Toure, and greater accountability for clinical instructors in terms of their level of effort, showing up, and delivering high quality clinical education to medical students. While Touch’s focus moving forward seems to be on expanding T&T to additional regional sites, and then to lower levels (village), it would not be prudent to expand without achieving success in the original sites; 3. Touch should consider placing two more specialists in the departments of OB/GYN and surgery at Sekou Toure. Consideration should be placed at looking for local specialists who can be specifically employed for and committed to the T&T program. 4. There should be more information sharing about the purpose, function, and roles of the “scribes” to add clarity of what data was being collected, how it was going to be used, and how it would benefit students, faculty/staff, and patient care. Greater efforts also need to be placed to feed back collected and analyzed data to the institutions, or at the very least; 5. CUHAS/BMC have been able to rapidly increase health worker production. It is, however, important to ensure that the rapid increase does not lower the quality of education received by students and continued vigilance is needed for balanced teacher/student ratios and maintained quality of training for students. Greater connections could be made with the Tanzanian MEPI and use of innovative teaching modalities including distance learning and on-line methods to leverage technology to strengthen and improve learning; 6. While capacity for the provision of higher quality of services in the peripheral hospitals has been strengthened through provision of specialist care, as well as improvements in infrastructure and equipment availability, limited data is available to show evidence of this. Improved data collection and analysis can further strengthen the evidence for improved quality of care (which was beyond the scope of this evaluation); 7. While capacity for health management at BMC as been improved through the implementation of the health care management fellowship, evidence was not able to be substantiated of the results of improvements in health care management as a direct result of participation in this program. Analysis and documentation of the improvements in health care management resulting directly from fellows’ participation in the fellowship program can help to strengthen the evidence; 8. CUHAS faculty and staff should continue to benefit from professional development opportunities to improve their capacity to utilize effective teaching methods and approaches to continue to improve medical student training at CUHAS; 9. Touch’s own efforts to track graduates to better understand placement outcomes should be transferred to the local institutions so that they can have the capacity to implement graduate tracking, not only to determine where 45 graduates end up, but also to establish an alumnae network which can be tapped into for numerous purposes. REPRODUCIBILITY Treat and Train During the first seven years, Touch launched, implemented and grew the Treat and Train program. To continue to bring this to scale, Touch will have to continue to build its network of partners in the public, private and FBO health facilities and training institutions. To facilitate this, it is recommended that Touch complete the following activities in the next 1-2 years: 1. Identify and provide clear information on the core components of the Treat and Train Program and its key characteristics of practice. This should include program philosophy and values, and including guidance on strategies for integrating the program philosophy into existing health facility operations and teaching institution structures. There should also be explicit and clear descriptions of staff skills needed to run the program to ensure consistency across Treat and Train. This guidance should also identify the adaptable components of the Treat and Train program such as scribes, scholarships for specialists, Peace Corps volunteers, and international practitioners. Those components are ones that in country partners can choose to adapt or choose to not include in order to meet the unique needs of their target population. This guidance should be developed in coordination with Sekou Toure, Sengerema, CUHAS and BMC key stakeholders based on their experiences with Treat and Train. 2. Provide clear information on what it takes to implement the core components of Treat and Train. This should specify: x Cost of the program, including both implementation costs and ongoing operating costs; when possible consider high, medium and low costing options. For example with dormitory construction instead of building houses, one alternative would factor for the use of existing buildings and structures to serve as dorms or renting/leasing space versus constructing it; x Staff & specialist recruitment, selection criteria and retention incentives; x Training, coaching and mentoring for staff that serve as the Points of Contact (POC) for the Treat and Train program both at the training institution side as well is the clinical site; x Identify strategies to ensure the availability of the financial, organizational, and human resources required to support students and specialists; x Identify opportunities for resource mobilization, ideally using locally sourced resources and capitalizing on Corporate Social Responsibility (CSR) programs. 46 3. Set up a mechanism to provide technical assistance and consultation to those that want to reproduce Treat and Train. Touch would collaborate with those who want to implement Treat and Train in a new setting or with a new target population. This will ensure that stakeholders can observe an existing Treat and Train Program and understand programmatic implications. This should include South to South experience sharing between Sekou Toure, Sengerema, BMC, CUHAS and the new site. x Work with other academic institutions to identify sites that might be interested in replicating the T&T program in an added geographical region. Healthcare Management Fellowship 1. Consider redesigning the healthcare management fellowship to explore local solutions to deliver in-service management training. This could be through business schools and private training facilities in addition to continuing to build the capacity the six fellows to teach modules. 2. For pre-service Touch should also continue to pursue the relationship with Mzumbe University and explore other training institutions with whom it might be possible to advocate for healthcare management to be integrated in to MD, AMO, and nursing curriculum. Consider partnering with regional umbrella groups like AMREF (formerly the African Medical and Research Foundation), which have robust in person and virtual training curriculums. 3. Explore the possibility of short courses using in country professors (possibly from Dar es Salaam or Arusha). Knowledge Development and Dissemination 1. It is recommended that Touch continue producing high quality publications. Upcoming knowledge dissemination publications should focus on providing detailed information on the efficacy and effectiveness of the various program components (Treat and Train, Healthcare Management, Vodaphone project). This will enable the benefits and challenges of the Touch interventions to be known and programs to be adopted by other organizations 2. For dissemination program findings should be proactively communicated to MOHSW, donors, and other in country partners. SUSTAINABILITY General 1. Continue to demonstrate quantifiably how on-going investments will impact core PEPFAR care, treatment, prevention, and HSS goals. 2. Continue the shift from an emergency response focused on the increased supply of MDs and other cadres towards an integrated platform that strengthens targeted recruitment and retention policies at the national and local (regional and district) levels. 47 3. Integrate where possible innovative, cost-saving technologies and south-to￾south exchanges to leverage scarce resources. This would include linking tertiary level facilities with regional and district hospitals through telemedicine and e-learning solutions that will improve access to physicians and specialists from BMC as well as enhance the learning experience of the students through improved quality for training, supervision, and in-service care delivery. Managerial and Financial 1. Strengthen existing partners for the T&T program to maximize the impact of their investments to date by further improving the quality of education of CUHAS students, strengthening the skills of existing workers, and improving the ability of the participating hospitals to retain health workers. This would include continuing to strategically support BMC and CUHAS while finalizing transition to local ownership and institutionalizing incentives for staff to provide critical mentorship and supervisory assistance. 2. Continue to build local institutional capacity for CUHAS and BMC to develop partnerships (both local and global) that provide shared value as well as technical specialist and financial support (grants, research). 3. Continue to engage directly with local government, hospital management, and district communities to develop localized, cost-efficient, and sustainable strategies for a successful transition of Touch programs to local ownership. 4. Continue to build the management capabilities of their Tanzanian partners to complete the transfer of programs to the local institutions. This may include but not limited to provide further transparency and ‘how-to-guide’ regarding the ‘start-up’ and maintenance plans of program interventions. 5. Leverage the business-oriented skills of their core staff to engage private sector partners to explore synergies with private health institutions, local entrepreneurs and alternative sources of funding such as CSR funds from private-sector players with footprints in the region. Country Ownership 1. Create new and enhance existing linkages to local associations and MOH PPP TWG to help advance policies such as CSR and tailored posts. Touch should also liaise with the national level stakeholders including: the Prime Minister’s Office of Regional Administration and Local Government, (PMORALG), MOH, Ministry of Finance (MOF), and President's Office Public Service Management (POPSM). Relationships should also be strengthened at local levels to strengthen health sector financing, planning, and development. 2. In collaboration with national and regional decision makers, and by leveraging their partners’ experience on-the-ground, address and improve deployment and retention of healthcare workers within the Lake Zone and share best practices to support national policy development. 48 3. As needed support or conduct analysis and mapping of the existing HRD system for faith based and private in relationship to the government system and recommend streamlined deployment policies for health workers, including strategic deployment in high priority areas and harmonized procedures to tailor allocation of resources locally. 49 ANNEXES Annex I: Evaluation TEAM FULL BIOGRAPHIES Lily Asrat, Dr.P.H. (Evaluation Team Lead) Senior Evaluation Advisor United States Agency for International Development/Washington Dr. Asrat serves as the Senior Evaluation Advisor in the Strategic Planning, Evaluation, and Reporting Division of the Office of HIV/AIDS at the United States Agency for International Development (USAID)/Washington. In this capacity, she provides technical support to USAID Missions to plan, procure, and implement performance and impact evaluations of the President’s Emergency Plan for AIDS Relief (PEPFAR) funded projects. She also serves as a co-chair of the inter-agency Monitoring & Evaluation Technical Working Group and provides technical leadership to the Office of the Global AIDS Coordinator (OGAC) on issues of evaluation standards and compliance. Lily has over 12 years of experience working in international HIV and AIDS in the areas of monitoring, evaluation, reporting, operations research, and implementation science in East, West, and Southern Africa as well as the Caribbean. She was a Peace Corps volunteer in Namibia and has consulted for various academic institutions and multilateral organizations including: University of California, San Francisco, Columbia University, Pan American Health Organization, National AIDS Control Programs, and the Global Fund to Fight TB, HIV, and Malaria. Lily also worked at other US agencies including the Centers for Disease Control and Prevention, as well as with a PEPFAR implementing partner, University of Washington/International Training and Education Center for Health (I-TECH). Lily received a Master of Arts in International Affairs from Ohio University, a Master of Public Health from Tulane School of Public Health and Tropical Medicine, and a Doctorate of Public Health from University of California, Berkeley. Lily is fluent in French and Amharic. Temitayo Ifafore, M.P.H. Health Workforce Technical Advisor United States Agency for International Development/Washington Temitayo (Temi) Ifafore serves as the Health Workforce Technical Advisor in the Office of Population and Reproductive Health (PRH) at USAID/Washington. Temi provides technical assistance related to human resources for health, capacity building in leadership, management and governance (LMG), and systems strengthening in the Service Delivery Improvement Division’s (SDI) projects and activities. She currently provides project management support for the $198 million Leadership Management and Governance (LMG) Project and the $300 million CapacityPlus Project. 50 Prior to joining USAID, she provided administrative oversight over a public health workforce development program as National Program Director for Health Career Connection. Skilled in program management, hospital operations, quality improvement and health systems strengthening, Temi worked as Director of Operations for Johns Hopkins Medicine International in Panama and served as Regional Director for the William J. Clinton Foundation HIV/AIDS Initiative in Ethiopia. Temi received her Masters Degree in Public Health from the University of Michigan School of Public Health and a Bachelor of Arts (BA) in Socio-Cultural Anthropology from Yale University. Temi is fluent in Spanish, has working knowledge of Brazilian Portuguese, and basic proficiency in Tigrinya and Amharic. Jeffrey Blander, Sc.D. Deputy Director Private Sector Engagement Office of the Global AIDS Coordinator U.S. Department of State Dr. Blander serves as the Deputy Director of Office of Private Sector Engagement (PSE) in the Office of the Global AIDS Coordinator (OGAC). In this role, he support the development, implementation, and evaluation of policies, interventions, and strategies for public private partnerships (PPPs) by working closely with country teams, implementation partners, private sector organizations, and multilateral institutions to achieve the President’s Emergency Plan for AIDS Relief (PEPFAR). Jeff is a published author and experienced research scientist, non-profit foundation director, and industry executive specializing in areas of health systems strengthening, clinical research, business strategy, information systems, and microfluidic diagnostics. Prior to joining OGAC, Jeff served as the President of the Bienmoyo Foundation, a non-profit organization providing advisory services for the design of strategic partnerships focused on the adoption of point of care diagnostic technologies and integration of public and private provider networks to address the double burden of infectious and non-communicable diseases. Jeff held dual research appointments at the Brigham and Women’s Hospital and Harvard School of Public Health as well as served as the director for courses he co-founded on global health practice, business, and medical technology in the Division of Health Science & Technology (HST) at Harvard Medical School & Massachusetts Institute of Technology (MIT). Jeff has also served as a Volunteer Country Director for the Clinton Foundation and was awarded pre and post-doctoral Fogarty International Center/NIH Fellowships in Global Health and Clinical Research. Jeff held the positions of co-leader for the Technology Innovation Working Group for the Harvard Institute for Global Health and Chair of the Technology Committee for the Massachusetts Biotechnology Council. 51 Jeff received his Doctorate and two Masters Degrees from Harvard and his Bachelors of Science from the Wharton School of the University of Pennsylvania. Angela P. Mwaikambo, M.A. Monitoring and Evaluation Specialist United States Agency for International Development/Tanzania Angela serves as the Monitoring and Evaluation Specialist at USAID/Tanzania with the Strategic Planning and Program Support team and has been in this role since 2008. Her major role in this position is to ensure that effective performance management systems are developed and implemented across the Mission. She coordinates the strategic planning, monitoring, evaluating and reporting of performance results in collaboration with Mission technical teams and implementing partners. She also provides technical advice and support to teams on monitoring and evaluation as needed throughout the program cycle, from activity design to evaluation. Angela has 15 years of practical experience in community development work and acquired skills in designing, implementing, and performing a variety of monitoring and evaluation functions of development programs. Angela worked with United Nations Industrial Development Organization (UNIDO) as a program Coordinator for five years before joining Care International and International Labor Organization (ILO) in Tanzania for three and five years respectively in M&E positions. While working with ILO, Angela facilitated the development of Monitoring and Evaluation systems for a Public Private Partnership in solid waste management in Dar Es Salaam, Mwanza and Arusha, as well as cities in Somaliland. While working with Care International and ILO, Angela conducted a number of baseline studies and performance evaluations of projects focusing on income generation, health, agriculture, and environment as a member of evaluation teams. Angela holds a Masters Degree in Community Economic Development from the Southern New Hampshire University in collaboration with Open University of Tanzania (2005). Angela is a native speaker of Kiswahili. Regine Jean-Francois, M.P.H. Analyst United States Agency for International Development/Washington Regine serves as the Analyst for the Information Strategic Planning and Evaluation (ISPE) team in the Strategic Planning, Evaluation, and Reporting (SPER) Division of the Office of HIV/AIDS at USAID/Washington. She also serves as the Strategic Information (SI) Advisor for Haiti. In her role as Program Analyst, Regine backstops the team in the area of Strategic Information, Monitoring and Evaluation, and Health Information System. In her role as a SI Advisor, she supports the Operating Unit in the planning, implementation, monitoring, and reporting of PEPFAR programs. 52 Prior to joining USAID, Regine served as a Research Analyst for the Federal Emergency Management Agency assessing the Public Alert and Warning Systems throughout the United States. Regine received her Master’s degree in Public Health from George Mason University. Regine is a native speaker of French and Haitian Creole. 53 Annex II: Evaluation SCOPE of Work I. Project Information Project Title: "Strengthening the Tanzanian Health System by Increasing Number and Quality of Health Workers and Improving the Health Delivery Mechanisms in the Lake Zone of Tanzania” Project Number: No. 621-A-00-11-00002-00 Project Dates: October 6, 2010 – September 30, 2014 (the period of performance was extended by nearly one year on September 11, 2013) Project Funding: USAID’s investment in the public-private partnership is $8.5 million and Touch’s investment is $8.5 million. Implementing Partner: Touch Type of evaluation: End of Project Internal Performance Evaluation AOR: Gene Peuse II. Performance Period The evaluation is estimated to take a total of five weeks which will consist of two weeks of preparation, two weeks of in country data collection, and one week for report write up and submission. The current tentative dates for the data collection are January 13 – 24th with a first draft submission of February 3rd. III. Funding Source This will be an internal evaluation with team members from USAID/Washington, USAID/Tanzania, and the Office of the Global AIDS Coordinator (OGAC). The Mission will not allocate any funds for this internal evaluation. IV. Evaluation Purpose/Rationale This performance evaluation comes towards the end of the second public-private partnership between USAID/Tanzania and Touch. The overall purpose of this evaluation is to provide information that will be used by the Mission to inform future programming and that can provide specific feedback and recommendations regarding sustainability and taking the intervention to scale. The primary users of the evaluation findings are USAID/Tanzania, other PEPFAR/Tanzania agencies, the Office of the U.S. AIDS Coordinator, the government of United Republic of Tanzania, and Touch. V. Project Overview: A Series of Partnerships A current Cooperative Agreement (Co-Ag) between USAID/Tanzania and Touch was signed in October 7, 2010 for a three-year period ending on October 6, 2013. On September 11, 2013, the current agreement was extended for nearly one-year to September 30, 2014 because the Treat and Train component required more time to establish than anticipated. The current public private partnership (PPP) with USAID is a follow-on to a previous partnership between USAID and Touch from November 1, 2007 to October 31, 54 2010. USAID contributed $3,190,000 to this first partnership and Touch provided $6,626,010 to the partnership. Prior to engagement with USAID, Touch signed a Memorandum of Understanding (MOU) with the Office of the Global AIDS Coordinator and the Office of the President, Tanzania for a $1,000,000 partnership. All of these partnerships have followed the vision of strengthening the Lake Zone health system, using as the base of operations the (now) Catholic University of Health and Allied Science (CUHAS) and the Bugando Medical Center (BMC), which are located adjacent to each other in Mwanza. VI. Project Description Introduction The current agreement with Touch is the second partnership with USAID/Tanzania. The first partnership from 2007-2010 was followed immediately by the current partnership. Since 2007, Touch-USAID partnership has focused on increasing the quantity and quality of the healthcare workforce in Tanzania and improving the healthcare delivery systems to enable provision of care by the health workers. Initially, Touch focused on strengthening Bugando Medical Centre (BMC), the specialty referral hospital for the Lake Zone, and its affiliated medical university, the Catholic University of Health & Allied Sciences (CUHAS, formerly Weill Bugando University College of Health Sciences). In addition to providing direct operational financial support to both the university and the teaching hospital, Touch has invested in infrastructure upgrades to enhance the learning and healthcare delivery facilities, served as strategic advisors to these institutions and supported capacity building of key functions such as finance, management, faculty development and information technology. After seven years of support from OGAC and USAID (2004-2011) focused on securing a solid pipeline of new health workers at CUHAS and BMC and improving BMC infrastructure and operations, in 2011 Touch began to establish the Treat & Train program that extends the training of health workers to regional and district hospitals. While this expansion is aimed at improved quality of and greater capacity for health education provided by CUHAS and BMC, in particular clinical training for their students, this also represents a move towards strengthening the health system in the Lake Zone. How this is being done is explained in more detail below. Touch’s current work focuses on five main objectives, which are linked and reinforcing, to: 1) Increase the number of health students trained and improve the quality of education at CUHAS and BMC 2) Establish CUHAS and BMC as self-sustaining institutions 3) Strengthen the health system in the Lake Zone 55 4) Enhance healthcare management across Tanzania and, in particular, in the Lake Zone 5) Create Touch Tanzania as the institutional platform for future local program implementation and scale-up 1) Increase the number of health students trained and improve the quality of education Since 2004 Touch has contributed to the rapid growth of CUHAS from 10 to 587 medical students, for a total of ~1400 students currently enrolled across 13 cadres, including specialist doctors, doctors, nurses, laboratory technicians, radiologists and pharmacists. Over the past 8 years, Touch has helped graduate 143 physicians and over ~1300 allied healthcare professionals. In this way Touch contributes to PEPFAR’s goal of training 140,000 new healthcare workers across Africa between FY10 – FY14 and to the Ministry of Health and Social Welfare (MOHSW) healthcare worker training targets as set forth in the Government of Tanzania’s (GOT) Primary Health Services Development Programme. Through operational grants Touch has covered direct student costs of accommodation, meals and supplies, which has facilitated the enrollment of talented students who otherwise might not have entered the medical profession. The operational fund has improved the learning environment for students through the establishment of book banks for the CUHAS basic science departments (i.e. anatomy, microbiology, physiology, pathology and biochemistry) and the upgrading of the computer laboratory. Instructional skills of faculty have been enhanced through the training of faculty abroad or through mentoring relationships with U.S. faculty on short-term assignments at CUHAS and BMC. A Careers Office was created to assist in the job placement, retention and tracking of graduates. By counseling students on post-graduation opportunities, introducing them to prospective institutions, ensuring more satisfactory placements and providing ongoing support through clinical supervisors beyond graduation, the Careers Office is expected to significantly reduce career attrition. Touch’s success in greatly increasing the enrolment of students has resulted in an anticipated challenge—insufficient space at the BMC teaching hospital to accommodate clinical training for CUHAS students. This led to the development of the Treat and Train program, which entails having medical students and faculty rotate to participate in clinical training at Sekou Toure Regional Hospital and Sengerema Designated District Hospital. Collection and analysis of process and outcome data in each Treat & Train hospital has been established to track progress and assess the impact of the program on the student educational experience. Approximately 90 metrics have been established to monitor “quantity” (e.g. presence rate, specialists attendance, teaching session frequency, student to teacher ratio) and “quality” (e.g. program student evaluation, student success rate at examination) as students engage across their core activities - morning reports, ward rounds, teaching sessions, clinical conferences, etc. Early anecdotal evidence suggests that 56 students appreciate the greater hands-on experience in a setting that more closely resembles the kind of environment in which they likely will be working in the future. 2) Establish CUHAS and BMC as self-sustaining institutions From an institution-building perspective, the ultimate vision of the partnership is that one day CUHAS-BMC medical complex will be a self-sustaining, self-sufficient center of excellence where continued external support is no longer needed. In FY12 Touch restructured its grant agreement with CUHAS and BMC to reduce Bugando’s reliance on support for ongoing operations and to ensure their long-term sustainability. The agreement restricted part of the total grant to fund capital and special projects. The requirement for CUHAS and BMC to access these funds is to submit a detailed “business plan” including the rationale for undertaking the project, a budget and implementation plan as well as a comprehensive operations and maintenance proposal. Project proposals approved within the past year include Information Communication Technology (ICT) networking and infrastructure upgrades, new accounting software Sage Pastel to improve their financial operations and elevator purchase and installation for MD student hostel. Going forward, Touch will continue to fund these institutions by increasingly focusing on strategically important capital projects and gradually decreasing the focus on operational expenses. Touch is helping CUHAS expand its own Development Office, sharing fundraising skills and facilitating valuable long-term partnerships. Touch is also helping to develop other sources of income, such as faculty and student research, student tuition and government support. At BMC Touch will continue exposing staff to new hospital management tools. As noted in section 4 below, during the next year five individuals, designated as BMC Fellows, will participate in Continuing Professional Development (CPD) courses. These CPD courses will provide intensive training in hospital management topics and will be required to mentor their co-workers. 3) Strengthen the health system in the Lake Zone The launch of the Treat and Train program is the first step towards the building of health system in the Lake Zone down to the community level. In addition to providing students with professional orientation and skills needed to work in rural, often under￾resourced environments, the Treat and Train program is creating a continuum of health delivery in the immediate vicinity of the hospitals. By working hand in hand with the local staff in the hospitals, the Treat & Train specialists are expected to improve the skills of approximately 90 health workers in Sengerema Designated District Hospital (SDDH) and approximately 150 in Sekou Toure Regional Hospital (STRH). The presence of both Treat & Train faculty teams and students will greatly increase the capacity of these hospitals to attend to an increasing number and diversity of patients. It is anticipated that over the next year 55,000 patients in Sengerema and 55,000 patients in Sekou Toure will be seen by Treat & Train faculty teams and students receiving clinical training. In addition, students will engage in public health related projects in communities around Treat & Train hospitals, 57 contributing to community based solutions to problems affecting these health care facilities (e.g., pediatric malnutrition prevention and care). The program will be limited to improving the most urgent clinical infrastructure necessary to provide an appropriate learning environment for the students. This will entail the construction of a sterilization room and equipment procurement for surgical theaters and OB/GYN facilities (e.g. surgical instrument trays, washing stations) at SDDH, and instituting a safe surgery program there as well, which includes refresher training for anesthetists, refresher surgical training for AMO staff, development of an infection control program and institution of a surgical safety checklist, and a morbidity and mortality conference. Touch’s longer-term vision for a follow-on partnership is to build a Treat and Train network of at least six regional and district hospitals within the Lake Zone, modeling how this scale-up might be reproduced across the country. 4) Enhance healthcare management across Tanzania and, in particular, in the Lake Zone The Healthcare Management Program has been designed to accomplish two main objectives: to improve Mzumbe University’s master program in healthcare management, with support from Rush University Medical Center (RUMC), and to improve the overall management of BMC and other hospitals in the Lake Zone of Tanzania. During the next year, five BMC Fellows will be paired with appropriate mentors from RUMC and will engage in improvement projects at BMC. This will be supplemented by four CPD short courses in various organizational operations (finance, human resource management, department planning and the other yet to be determined) to strengthen the management skills of sixteen healthcare managers including at least ten from BMC (including the Fellows), two faculty from Mzumbe University, two faculty from St. Augustine’s University of Tanzania, and two from other regional and district hospitals in the Lake Zone. The five BMC Fellows will participate in a two week on-site training at RUMC in Chicago. Similar opportunities will be given to Mzumbe University faculty, and RUMC will assist in overhauling Mzumbe’s Department of Health System’s Masters curriculum. Building this capacity at Mzumbe will lead to the production of a new cadre of hospital managers with business and organizational skills currently unavailable in the country. 5) Create Touch Tanzania Touch has registered an affiliated organization in Tanzania, called Touch (Tanzania) Limited (referred to as "Touch Tanzania"). The board of Touch Tanzania is currently comprised of seven members with a majority of the members also on the Touch US board, thus eliminating it from being a candidate for USAID’s local solutions support. At the moment, Touch Tanzania is being used as the primary party in agreements with Tanzanian contractors and partners and Touch US is the party in agreements with partners and vendors based internationally. In the long-term, however, the vision is that Touch Tanzania will be autonomous with an independent governing structure. 58 EVALUATION QUESTIONS: There are three broad themes that should be explored for this evaluation, which are effectiveness, sustainability and scale. The evaluation team should review, analyze, and evaluate the project with these themes in mind: Effectiveness 1. How effective has the intervention model developed by the Touch been? 1a. What has been the impact of the model on the quality of medical education provided to CUHAS and BMC students? 1b. How has Treat and Train program enhanced access to care for the more rural population? Reproducibility 2. To what extent is the model scalable beyond the Lazke Zone to other regions of the country and what are the potential opportunities and obstacles to bringing the model to scale? 2a. After seven years, what more is needed in the Touch’s approach to continue to advance this work and/or bring it to scale? 2b. Which components have been the most difficult to make progress on and are not possible or advisable to reproduce or scale-up? 2c. Which components of the model have the greatest potential for replication? Sustainability 3. What is needed to ensure that the intervention model developed by the Touch and its’ institutional partners is sustained beyond the life of the project? 3a. How successfully has Touch managed to reduce CUHAS and BMC dependency on Touch operational support, utilize the more strategic/capital projects support, and move toward self-sustaining, private institutions? 3b. Are there things that can allow for the project to have more sustainable impact? 3c. How and how long would it take to make the model self-sustaining without external assistance in the form of grants from the donor community? EVALUATION DESIGN AND METHODOLOGY Evaluation design The evaluation design will use mostly qualitative methods to gather information from a variety of sources that will be analyzed by the evaluation team to come to some conclusions. As a performance evaluation, no counterfactuals have been established and therefore, the results will not address a cause and effect relationship through rigorous methods. However, the project, has collected significant baseline 59 information and has tracked progress throughout the intervention. Comparisons between project baseline data will be compared to end of project to assess changes in institutional capacity and outcomes. As much as possible, the influence of the Touch’s influence over these changes will be explored in the context of other factors which may have contributed to such changes. Data collection methods The primary data collection methods used for this evaluation will be: desk review of existing documents, key informant interviews of relevant stakeholders, and participant observations of relevant instruction and/or rotations at the two institutions. Desk Review: USAID/Tanzania will provide the evaluation team leader with a core list and/or copies of the agreement, reports and other key documentation before the evaluation begins. Touch will provide the internal evaluation report and source documentation. The evaluation team leader will be responsible for expanding this background documentation as appropriate, and for reviewing, prioritizing and distributing it to other evaluation team members for their review. All evaluation team members will review relevant documentation before their initial team meetings. A complete list of data sources are listed below. Key informant interviews: The evaluation team will conduct qualitative, in-depth interviews with key stakeholders and partners. Whenever possible, the evaluation team should conduct face-to-face interviews with informants. When it is not possible to meet with stakeholders in person, telephone interviews should be conducted. The evaluation team will have interviews with the following (not exhaustive): Ȉ Relevant USAID offices and other USG offices in Tanzania; Ȉ Touch Implementing partner representatives in Tanzania; Ȉ Touch Board members and Touch Donors to the partnership Ȉ Key Government of Tanzania representatives at both national and local levels A list of suggested key informants are attached to this SOW. Participant observations: Evaluation team members, as appropriate, will visit Mwanza, where CUHAS and BMC are co-located, and two Treat and Train satellite training sites, which are Sekou Toure Regional Hospital, also in Mwanza, and Sengerema Designated District Hospital, which is approximately one hour outside of Mwanza. VI. EXISTING DATA Document Review A broad range of background documents in addition to project documents will be provided to the evaluation team. USAID and the Touch staff will provide the evaluation team with access to materials, including: 60 x Project progress reports, program description, close-out report for the first partnership with USAID, and other related documents; x USAID/Tanzania CDCS, HSS Strategy, HSS PAD, Strengthening Public Private Partnership Capacity in Tanzania PAD, and other related documents; x Human Resource for Health Strategic Plan 2008-2013, Ministry of Health and Social Welfare, January 2008 (http://www.unfpa.org/sowmy/resources/docs/library/R223_MOHTanzania_2008_ HRH_Strategic_Plan_2008_2013.pdf) x Acting Now to Overcome Tanzania’s Greatest Health Challenge: Addressing the Gap in Human Resources for Health (http://www.touchfoundation.org/resources/publications.html?page=2&filter=repor ts) x Investing in Tanzanian Human Resources for Health: An HRH Report for the Touch (http://www.touchfoundation.org/resources/publications.html?page=2&filter=repor ts) x Catalyzing Change: Molecular Strengthening of the Health System in the Tanzanian Lake Zone (http://www.touchfoundation.org/resources/publications.html?page=2&filter=repor ts) x Action Now on the Tanzanian Health Workforce Crisis” Expanding Health Worker Training: The Twiga Initiative (http://www.touchfoundation.org/resources/publications.html?page=2&filter=repor ts) x Touch Achievements, Approach and Path Forward (Internal Evaluation, 2013) x Touch Impact (2004-2012) (Internal Evaluation, 2013) x Data Analysis x Strengths/limitations Deliverables The evaluation team will be responsible for preparing the following deliverables: Draft and final work plan, research design, and data collection instruments: The internal evaluation team will finalize the SOW and work-plan/timeline. The team will work virtual, and in-person when possible, to finalize the SOW and seek feedback/input from relevant stakeholders. Following consensus of the SOW and work plan/timeline, the team will develop and finalize data collection tools. Debrief with Tanzania Mission: The team will discuss preliminary findings with USAID/Tanzania at the end of the in-country data collection period. It is understood that this will be preliminary findings. As such, the team will need to be analyzing data captured as they go along in order to provide preliminary findings prior to departure from the country. 61 Draft evaluation report: The team will prepare and submit a draft report to the Tanzania Mission two weeks after departure from Tanzania. The report should clearly describe methods, findings, conclusions, and recommendations. The draft report will be disseminated to relevant USAID/Washington and USAID/Tanzania staff for feedback and input. Final draft evaluation report: Upon receiving comments and feedback, the Team Leader will revise and submit a final draft evaluation report to the Tanzania Mission one week after receipt of feedback. Evaluation Report The evaluation team leader will be ultimately responsible for delegating responsibility, putting together the final evaluation report, and incorporating feedback. The USAID Evaluation Report template will be used and all required elements within, including Annexes, should be included. Team Composition The evaluation team will consist of four members that have collective knowledge, experience, and context in evaluation methods, HRH, PPP, and local context. The team composition will include: Senior Evaluation Advisor - USAID/Washington The evaluation advisor will have knowledge and experience in evaluation design and implementation, qualitative research methods, and knowledge of both USAID and PEPFAR evaluation requirements and standards of practice. She will serve as a member of the evaluation team and participate in all aspects of evaluation design, data collection, analysis, and report write up. Deputy Director Private Sector Engagement – OGAC The Deputy Director of Private Sector Engagement will have technical knowledge of private public partnerships and will contribute to the team through participation in reviewing and providing input into the design and data collection tools, participating in some or all of the data collection, and contributing towards the analysis and report write up. Program M&E Specialist – USAID/Tanzania The Health M&E Specialist will serve as a member of the evaluation team to provide local knowledge about the context within which the project operates. He will participate in all aspects of the evaluation design, data collection, analysis, and report write up. HRH Technical Expert The Technical Expert will serve as a member of the evaluation team and provide HRH technical knowledge and insights. He/she will participate in all aspects of the evaluation design, data collection, analysis, and report write up. 62 Estimated Level of Effort (in days) Activity Team Leader Team members (4) Total LOE Desk review (remotely) 5 20 25 Team planning meeting (DC) 1 4 5 Finalization of work plan/timeline and tools, interview lists, data sources, 5 20 25 Interviews and participant observations/site visits 10 40 50 Data analysis 3 12 15 Debrief to TZ Mission 1 4 5 Development and submission of draft report 5 20 25 Incorporation of feedback from the Mission on draft report 1 0 1 TOTAL 31 120 151 Annex III: Data Collection Instruments Semi-structured questionnaire Key informants: CUHAS management, staff, expatriate specialists Institutional Strengthening/Capacity Building (management and staff) 1. What kind of support (technical assistance, budget support, other) has Touch Foundation provided in the establishment of CUHAS? 2. What have been Touch foundation’s biggest contributions to your institutions? 3. What are some of the faculty skills/areas that still need to be developed at CUHAS? 4. How is it decided which faculty/students have the opportunities to attend international rotations (residents), and training opportunities for fellows and faculty? 5. Can you tell us more about the recruitment process and retention of faculty at the university? 6. How long have you been at the University in your current role (and previous roles)? 63 7. How is the medical training at CUHAS different from other medical training institutions (public, private, FBO)? 8. In the process of changing curriculum, what role did Touch Foundation play? 9. What is different at CUHAS if the Treat and Train model is the same elsewhere? 10. What keeps you here at CUHAS? 11. How do you engage CUHAS graduates to continue to engage and contribute back to the University once they are out in the workforce? 12. What were the selecting criteria for the scholarship at the beginning? 13. Do recent graduate leave because they cannot find a job? 14. Based on what you know about Touch foundation, what recommendation would you make? Treat and Train (management and staff) 1. What do you know about the Treat and Train program? 2. What existed before Treat and Train? How do government training health training institutions implement clinical rotations students (if they do)? 3. What have you heard from your colleagues about the benefits/challenges institutions face in receiving students for trainings? 4. What are the benefits received and burdens to other hospitals that receive medical and AMO students for clinical rotations? 5. In terms of HR, do hospitals (where students rotate) have to provide their own supervision and consumables? 6. Touch Foundation proposes that this program is able to retain more graduates in the Lake Zone area. Did you find that to be true? 7. What could be improved with Treat and Train? 8. What have you heard from your colleagues about the benefits/challenges institutions face in receiving students for trainings? 9. How has the Treat and Train improved the quality of education the students are receiving? 10. Are you seeing any differences in student’s knowledge/competence? 11. Who is training the AMOs at Sengerema? 12. Who initiated the idea of Treat and Train? How was it conceptualized, planned, implemented, and monitored? 13. Would you have been able to send specialist to Sekou Toure if Touch Foundation was not involved? 14. When did you start sending students to the peripheral institutions? 15. Can you talk about government involvement/ownership of the program? 16. Are there formal reports about the Treat and Train program? How is information going out about the successes of the program? 17. Have you ever done an assessment or evaluation of the Treat and Train model to have more evidence? 18. If there were something you wanted to improve about the collaboration with Touch or the model, what would it be? 19. If medical students join AMOs in Sengerema, how will you address the faculty/student ratio? 64 20. Do you know how you were selected for this program? 21. Have you received any training in teaching skills through your experience? 22.Do other universities, such as the one you received your medical education training in or others you are aware about, have a rotation program similar to Treat and Train? 23. What are the advantages of the Treat and Train in the peripheral hospitals? 24.In your role as a resident, alongside the specialist, how do you conduct your day? 25. Talk about the faculty/students ratio at BMC vs. Sengerema? 26. At Sekou Toure, are there students from other schools? 27. You were at Sengerema for five years before the program started, can you let us know of any changes/differences noticed before the rotations started and after in regards to quality of education? 28. How do you know the students are ready to go to the field? 29. Do you have other recommendations to improve what Touch Foundation is doing? 30. In your role of training students, what could be improved or changed? 31. How has the ratio of faculty/student changed from the time you were getting trained to now? 32. Do you have anything else to add in terms of suggestions and recommendations? 33.Would you agree that Touch Foundation’s contributions have improved the quality of care improved of patients? 34. What were the fruits of the collaboration with Touch Foundation? 35. Was the first class of 10 funded through the government grant or through Touch Foundation? 36.Did you feel that throughout that the University was consulted throughout the planning process? 37. Can the AMOs and medical students be trained at the same time? 38. If the AMOs and medical students cannot be trained together, what about alternating the two groups? 39.Currently medical students are doing rotations at Sekou Toure and BMC. What benefits would they get by going to Sengerema and Shinyanga? 40. Do you think the rotation at different locations has helped the quality of education of the students? 41. When AMOs and Medical Students are mixed, are the AMOs sideline Institutional strengthening and T&T (expatriate specialists) 1. How long have you been here? 2. In what capacity are you here? 3. What are Cornel’s activities at CUHAS/BMC? 4. What is Cornell’s long-term commitment/investment in this institution? 5. How many faculties from CUHAS have gone for training to New York? 6. Who comes here from Cornell come here, students, faculty, management? 7. Are medical students also coming here in a teaching role? 8. You’ve been here for 7 years, and have seen the institutions growth? What have you observed regarding this growth? 65 9. What have been the challenges in the growth of CUHAS? 10. What has been Touch Foundation’s role in your being here? 11. Would all his Cornell/CUHAS collaboration be possible if your position was not here? 12. How do you identify the needs of the University and how do you identify the resources? 13. Touch Foundation, as you know, has developed the Treat and Train program. Do you feel that the Treat and Train has improved the quality of education of medical students? 14. How do you know that the quality has improved? 15. Are students getting different experiences at Sekou Toure than at BMC? 16. Are students getting different experiences at Sengerema than at BMC? 17. Is it possible to train the two cadres together? 18. What other contributions has Touch made? 19. What are your recommendations/Suggestions in terms of Touch’s collaboration with these institutions? Semi-structured questionnaire Key informants: Health Care Management Fellows (BMC) Health Care Management Fellowship recruitment 1. How much do you know about Touch Foundation and how has it benefited you? 2. How are individuals selected to participate in the Health Management fellowship? 3. What motivated you to apply for this fellowship? 4. When you went to Chicago, who covered your responsibilities in your current position at BMC on your behalf? Health Care Management Fellowship training learning environment 1. What is covered in the initial training of the fellowship? 2. What were the topics covered during the short workshops after the initial visits? 3. During the 2 years, how often have you interacted with the teaching staff from RUMC in Chicago? 4. How do the 6 fellows interact with each other? Quality improvement project 1. How do the fellowship topics assist you in the quality improvement of your department? 2. Can you talk about one of the projects in quality improvement that you have developed and implemented as a result of the fellowship, and what were the results? 3. Did the training received from the fellowship help you resolve problem you identified in your department at BMC? What did you learn at RUMC? 66 4. How do you know your intervention/quality improvement project achieved the results you were expecting? Application of learning 1. How do you anticipate passing on the knowledge, experiences, and learning that you gained through this fellowship with your colleagues and department? 2. What was the biggest thing you learned while at RUMC University in the specific department that you visited, and how were you able to apply it in your department at BMC? 3. What did you learn at RUMC that you can implement at BMC? How did you apply this learning? 4. As fellows, how have you been able to improve leadership and management? 5. The Touch Foundation’s program is meant to built health systems, do you think it did that? Benefits of fellowship 1. What happens after two years of fellowship? 2. How can completing this fellowship program benefit you in the future? 3. What could be improved with the fellowship program? 4. What is your biggest gain from the fellowship program? 5. How has the mentorship with RUMC faculty helped you to improve your knowledge and skills in health care management? 6. What were the biggest challenges from this experience, and in what areas can the fellowship program be improved? 7. Do you have any final recommendations or suggestions in regards to the future of Touch Foundation? Semi-structured questionnaire Key informants: Peripheral Sites (Management, staff, and expatriate specialists at Sengerema and Sekou Toure) Institutional strengthening/capacity building (management and staff) 1. What do you feel about the level of joint collaboration and planning of activities, between Touch Foundation and your institutions? 2. What have been the greatest contributions from Touch Foundation? 3. In terms of infrastructure, can you tell exactly what has been done through Touch Foundation? 4. Is Touch Foundation helping you track expenses? 5. How much does the government at the district level know about the Treat and Train program? 6. In your perspective, what you would you say is Touch Foundation biggest contribution to your institution and to the health system in the community, and in the Lake Zone? 7. How does all of this fit in with all the goals for the region as per government (local, regional, national)? 67 8. Have you been able to collaborate or make recommendation to Touch Foundation? Have you been involved in all the planning? 9. How do you monitor the progress of the program and it’s benefits to your institutions? Treat and Train (management and staff) 1. What are both the burdens and benefits of Treat and Train program on your institutions? 2. Any challenges in having the AMOs at Sengerema, in terms of cost/supply? 3. In terms of the quality of the education of the students, can you describe how that has or is improving as a direct result of the Treat and Train program? 4. Is there any connection/benefit from the interaction of AMOs and COs? 5. Do the AMOs have a log book or procedure manual they need to follow? 6. How you envision the AMO and medical students working together, if in-fact in the future, both cadres are sent to your institutions for clinical training simultaneously? 7. What are the contributions of the scribes? Have they changed anything in terms of monitoring clinical training, changes in health service monitoring or delivery, or anything else? 8. Is there something formal within the AMO curriculum about the number of procedures they need to complete? Sekou Toure (management and staff) 1. In terms of what Touch Foundation has contributed, what do you think has been achieved? 2. Is the presence of the scribes making a difference? 3. Is the Treat and Train different than how they do medical education in other parts of the country? 4. How do you think the rotations in other hospitals have made a difference in the education of the students? 5. At Sekou Toure would students be able to see different types of population? 6. In the Lake Zone, do you have any specialists that are not brought by outside donors? 7. How much added responsibility is it for you, as a Medical Doctor, to teach? 8. Of the other doctors in the department, how many are teaching only? 9. How many other teachers have a relationship with CUHAS are at Sekou Toure with the same type appointment? 10. Are there other specialists who come on rotation at Sekou Toure? 11. Do you see any different in the teaching methods of medical doctors that come to your department? 12. What are the cadres that come for rotations? Do you envision the AMOs rotating with medical students? 13. Are you providing anything to Touch Foundation for the students? 14. What is your expectation in regards to how long the specialists would be here? 68 15. What changes have you seen as result of the specialist being at Sekou Toure? 16. What are some of the recommendations the specialist made? 17. What is being done in the area of operational research? 18. Are students not able to get the required rotations required for OB/GYN? Is this due to the limited number of specialists? 19. Does a specialist from Sengerema come to teach at times at Sekou Toure? 20. Are you currently working in the ways you expected, as per your scope of work? 21. In terms of the curriculum and clinical teaching, what is you perception? 22. What happens when the students are left alone? 23. What do you know about the other staff that have received an academic responsibility? 24. Were you asked to provide input on the Kindle Application? Institutional strengthening, T&T, and sustainability (expatriate specialists) 1. How long have you been here and what have you been doing here? 2. What was the selection recruitment, and how you ended up here? 3. Did you have a job description before arriving? And has it changed? 4. Do you have to change your approach when teaching the different cadres (CO, AMOs, MD)? 5. In terms of benefits, who are the different people benefiting from your presence here? 6. How many patients do you see per day? 7. Going back to the patients benefiting, how do you evaluate patient care? 8. In terms of the program, Touch’s contribution, has the quality of care improved? Do you agree? 9. Do you see the T&T model as being sustainable? Why or why not? 10. So do you think that it will be a problem when the medical students come here? 11. What recommendations would you make for the future of Touch? 12. How do you retain a specialist in this environment/future? Semi-structured data collection tool Key informants: AMO students and medical students at CUHAS/BMC Treat and Train (AMO students) 1. What other contributions has Touch Foundation made? 2. Are any of you, AMO, benefiting from Touch Foundation? 3. How far are you into the AMO program? 4. How many people are in a group? 5. As an AMO, do you have any interaction with Clinical Officers? 6. What are the benefits of working in Sengerema instead of other places? What are some challenges? 7. Sengerema has a form on OB/GYN, how do you feel about that form/format compared to the log-book? 69 8. What is your interaction with the doctors from the US at Sengerema? 9. What is your interaction with the specialist from BMC who travel with you to Sengerema to teach? What about the specialist from BMC who come with you to teach? 10. What opportunities do you have to give feedback to the lecturers? 11. Do you have the opportunity to provide written evaluation of lecturers at BMC? 12. Can another AMO who has been working longer than you teach you? 13. When you graduate and return to the rural health centers, how would you like to keep in touch with your peers? What is your responsibility to the school or BMC? 14. You all had such good experiences at Sengerema, would you go back to work as an AMO? Treat and Train (Medical Students) 15. As Medical Students, how much do you know about the Treat and Train programs? 16. Are you the beneficiary of the bedside teaching program? and how is it done? 17. As Medical Students, how do you compare the quality of education now and before the TNT program? 18. As Medical Students, how do you compare the rotation at BMC and at Sekou Toure? What are the challenges? Semi-structured data collection tool Key informants: Tanzanian private sector Areas of past collaboration 1. What collaboration have you had with Touch Foundation? 2. What types of activities are you supporting through Touch Foundation? 3. What have you contributed to at Sengerema? 4. Why didn’t you consider direct support through the ministry or the local government? 5. Do you get annual reports from Touch Foundation? Does it go into details in the types of indicators you are interested in? 6. How have you seen your investment facilitate placement or retention in the community? 7. At this point, are these folks getting placement in the lake zone area? 8. In terms of your mines and business interest, how do you cover the workers? Do they have their national insurance plan, or you have them through the business health plan. Can they choose where they go? 9. Do you cover the families of mine employees? 10. Do you invest in just private sector institutions? 11. Can you describe what you have seen of Touch’s Public Relation efforts with respect to garnering resources from the various private sector players in Tanzania? What can they do to improve this effort, how, and with which sectors and companies? 70 Future collaboration 1. Where do you see your future collaboration with Touch Foundation going? 2. What is it that Touch can do better to improve the performance of their programs? 3. In regards to sustainability, how do you decide on an investment in Touch or CUHAS? 4. Can you name other companies Touch could get support from? 5. Based on your knowledge, do you foresee touch being to reproduce their model? 6. Is it off the table to support Touch for future construction? 7. Final recommendations in regards to the future collaboration and the future and vision of Touch? Semi-structured data collection tool Key informants: Touch Foundation staff (NY, TZ) 1. How do you select institutions for expansion of this model? 2. From everything accomplished, do you envision this type of model to be reproduced in government institutions? 3. In regards to sustainability, is the vision to demonstrate that the model is successful here and can be reproduced elsewhere, but not necessarily to reproduce it yourself? 4. Please describe Touch’s relationship and engagement with government (local, regional, national)? 5. How does the kindle application M&E project using the scribes emerged? In other words, where do the ideas for activities come from? How do you assess the institutional needs both at the flagship institutions (CUAHS/BMC) and the peripheral sites (Sengerema/Sekou Toure)? 6. Can you tell us about the beginning of the Treat and Train Program and how is it innovative? How has it improved the quality of education? 7. How are the patients benefiting? 8. Recognizing that capacity building is needed, are there incentives currently being explored? What are the policy discussions and opportunities? 9. When students graduate, do you know where they are generally employed (government or private)? 10. Do you have a sense if most of the graduates stay in the lake zone? 11.How was the decision made to shift from directly funding students to supporting infrastructure and capital projects? What was the level of involvement from CUHAS in decision-making? 12. How would the process used at CUHAS work at Shinyanga or Kagera? 13. In terms of the supervision, the supplies and the incentives (separately), it seems that you’re selecting institution that have an infrastructure. Can you tell us more about how institutions are selected? 14. Can you talk about the decision to shift support from Sekou Toure to Sengerema, and from focusing on one cadre to another (from medical 71 students to AMOs)? 15. The difference between Sekou Toure and Sengerema is very visible. Is the relationship with Sekou Toure more representative of the relationship with government entities as you expand to Shinyanga or Kagera? 16. What are people’s perceptions in Tanzania of the Touch Foundation’s brand? 17. What is Touch’s contribution to the development of guidelines? 18. How much does it cost to maintain the international specialists? 19.Are there any other physicians being funded on the short-term or long-term (i.e. when Cornell send specialist here)? 20. What percentage of the organizational expenses does the physician portion represent? 21. We know that not all specialists/doctors are good teachers or want to or know how to teach. Is there any preparation of the specialists before they start teaching the AMOs? 22. It doesn’t seem that there is a common understanding of the role of the scribes or of their purpose. What kind of data is being collected and what is the purpose for the data being collected by the scribes? Sustainability (across various key informants) 1. What is the sustainability of Teach and Train? 2. What relationship does Touch Foundation have with the Government? 3. In regards to sustainability, is the vision to demonstrate that the model is successful here and can be reproduced elsewhere, but not necessarily to reproduce it yourself? 4. What’s your feeling about retention of graduates in the country? 5. Thinking about the Health Care Management training model, with students going to the US (2 weeks) and coming back to implement their quality improvement projects, do you see this being sustainable? 6. What will happen when CUHAS starts sending medical students to train with AMOs at Sengerema? Do you foresee any challenges? Will there be additional teachers to supervise the students? 7. What would you see/envision the relationship between BMC/Touch be in the future? 8. As your position of chairman of the board at BMC, what are the greatest needs for the institution to stand on its own? 9. Now that a few people have a good understanding of the program, would you be able to continue the same and do what it takes to get the resources needed? 10. What’s the vision for BMC and CUHAS in the future? 11. Will you train both AMOs and medical students together in the future? 12. When talking about the future and sustainability, how will that function operationally (i.e. salary support for specialists)? 13. Do see the treat and train program as a sustainable one? 14.Do you think it will be problematic when medical students come to Sengerema, in regards to sustainability? 15. How do you retain a specialist in this environment/future? 72 16. One of the goals of Touch was to make CUHAS a first class and self￾sustaining institution, in terms of strengthening it in different areas. What do you feel has been accomplished in terms of institution strengthening? 17. What is your ability to raise resources, network for resource mobilization.? Anything has been done to boost the resources of the institution? 18. It seems that you are aware that Touch foundation may not always be there, are you taking the necessary steps to take continue with the program? 19. If we think about moving forward and looking at the future, what are your suggestions or recommendations? 20. What are your recommendations or suggestions in terms of Touch’s collaboration with international medical institutions? 21. Some of the activities can be expensive. What do you think about sustainability of the Treat and Train program? 22. What is the government’s vision in being able to absorb the cost/salaries of specialists? 23. What is the government thinking in terms of how to recruit and retain specialists in the peripheral district hospitals? 24.Would the government be able to continue the Treat and Train program if Touch is no longer there? 25. Are there any final recommendations or suggestions? 26. Of the patients that come to Sekou Toure, how many are exempt from pay? 27. How do you mobilize resources? 28. Is there a way monitoring the cost sharing? 29. Does the government pay hospitals back for the services given to those exempt from pay? 30. Do you keep the profit from the ECG machine? 31. Would a Tanzanian specialist want to be at Sekou Toure? 32. In regards to sustainability, how would you work the structure? 33.What would it take to keep an international specialist at Sekou Toure? 34. It’s seems that you’ve been satisfied about your interaction with Touch Foundation. Where do you see this collaboration going? 35. In regards to sustainability, how do you decide on an investment in Touch Foundation or CUHAS? 36. How have you seen your investment facilitate placement or retention in the community? 37. At this point, are these graduate getting placements in the lake zone area? 38. Can you mention other companies Touch Foundation could get support from? 39. Would you be comfortable making a pledge in front of other colleagues/businesses? 40. Is it off the table to support Touch Foundation for future construction? 41. As an investor, do you have final recommendations in regards to the future and vision of Touch Foundation? 42. Once the international specialists are replaced with local specialist, whom will they sign the agreement with? 43. What are your strategies moving forward? 44. What are the opportunities for influencing retention of graduates in the area? 73 45.Can you talk about the plan for the next five years? 46. When you look at the financing, it is important to consider the mix of support, especially in terms of contributions. Where do you see that going? Also, can you talk about potential partnerships that might fill the gap. 47. How are other donors/potential sponsors hearing about you? 48. When we’re talking about partnerships, South Africa has a rule about all companies must give a certain percentage of their profits to CSR, and then there are a pre-selected list of organizations that have been vetted that they can give to. What are the opportunities in terms of policy dialogues around CSR? And what other things could OGAC do to help facilitate and convene stakeholder meetings? In terms of the new plan vs. the the past 7 to 8 years, what can you do around maintenance of programs that you have invested in? 49. We know that sometimes we invest in programs, and when we pull out, it falls apart. How will you re-distribute your resources and at the same time maintain past investments. 50.What are the trade-offs? 51. What is the fallback strategy? In essence, what is the minimum package? 52. Where do you see the opportunities for Touch Foundation? 53. What are the opportunities for future collaboration based on the gaps? 54. Across the years, how has funding changed? Is there a document available that can show this? 55. When looking at the decreasing levels of funding, what do these conversations look like with the institutions - especially at the beginning? 56. What are the incentives to retain specialists? What are the drivers for retention? 57. Touch Foundation is sponsoring students to go to CUHAS for their MMED. What are the number of students sponsored and what are the costs? 58. How much is the funding per sponsorship for the MMED students? 59. What is the cost of the Health Care Management fellowship training? 60. How will the institutions maintain the high tech that you have established, such as the kindle app? 61. If USAID Tanzania’s funding were to be cut by 75%, would you be able to stand alone as an organization? How would Touch be able to maintain the offices? 62. One of the model aspects include bringing in expatriate specialist, and up front investments (i.e. housing), do you think that can be sustained? 63. Touch has constructed housing and has come supported international specialists to teach. In terms of sustainability, if Touch Foundation were no longer able to provide financial support, would the institutions be able to continue the Treat and Train program? Scalability (across various key informants) 1. From everything accomplished, do you envision the treat and train model replicable in government institutions? 2. Has there been any instance where a hospital approached Touch Foundation based on what they heard about the treat and train program? 74 3. How can the Treat and Train model be generalized in other parts of the country or other country? 4. What are the opportunities for scaling up initiatives from fellowship to hospital? 5. Touch Foundation would like for other hospitals to benefit from this program, they also want to decrease their involvement overtime, what would it take for CUHAS to successfully run these programs on its own? 6. What is the vision for CUHAS? 7. Has there been any conversation about extending the treat and train to the health center level since the AMOs will go back to work at that level? 8. That model has been described as an innovative model/approach with the potential of extending to other areas of the lake zone or other regions of the country, what are your thoughts on scaling it up? 9. Do you have any recommendations and suggestions for Touch Foundation in regards to scalability? 10. When you talk about expanding the program, it requires housing. How will the government address this? 11. As the program expands to other sites, would you be able to train the AMOs and medical students together? 12. What would be some challenges? 13. What do you think is needed to expand the program? 14. What would be the best way to plan for an extension of the treat and train? 15. Is the Teach and Train replicable? 16. How do you select institutions where to expand? What are the criterias? 17. Are there other leverage points in terms of contributions? What are other ways that the government is contributing? How are they demonstrating country ownership? 18. What is Touch’s marketing and branding strategy around disseminating the work done? 19. Which institutions would you recommend as having potential for expansion? 20. Do you feel comfortable to approach Touch Foundation to work on priorities of the Ministry? 21.How do graduates get placed in their position? 22. What do the start up costs look like, what does it cost to get a program off the ground? 23. Are there any possibilities of rotating lower level cadres in private institutions? 75 Annex IV: DOCUMETS REVIEWED Name of Document Year Produced Organization Purpose of document Attachment B- Program Description (from Cooperative Agreement) FY 2013 USAID/TZ Provides Historical View Performance Management Plan (PMP) FY 2010 USAID/TZ M&E Plan FY11 Q2 Quarterly Report FY 2011 USAID/TZ Progress Report FY11 Q3 Quarterly Report FY 2011 USAID/TZ Progress Report FY11 Q4 Quarterly Report FY 2011 USAID/TZ Progress Report Touch Foundation Work plan October 31 2010 FY 2011 USAID/TZ Work plan FY12 Q1 Quarterly Report FY 2012 USAID/TZ Progress Report FY12 Q2 Quarterly Report FY 2012 USAID/TZ Progress Report FY12 Q3 Quarterly Report FY 2012 USAID/TZ Progress Report Touch Foundation Work plan Revised October 1, 2011 FY 2012 USAID/TZ Work plan FY13 Q1 Quarterly Report FY 2013 USAID/TZ Progress Report FY13 Q2 Quarterly Report FY 2013 USAID/TZ Progress Report FY13 Q3 Quarterly Report FY 2013 USAID/TZ Progress Report Work plan September 30, 2012 Revised FINAL Jan 17 2013 FY 2013 USAID/TZ Work plan Work plan FY14 September 30, 2013 FY 2014 USAID/TZ Work plan Touch Foundation Impact (2004-2012) 2013 Touch Foundation Internal Evaluation Report Touch Foundation Achievements, Approach, & Path Forward 2013 Touch Foundation Internal Evaluation Report Achieving Better Health: Partnerships & Programs in Africa 2009 Touch website Report Action Now: Expand Health Workforce Training - the Twiga Initiative 2009 Touch website Assessment Catalyzing change: Molecular strengthening of the health system in the Tanzanian Lake Zone 2009 Touch website Assessment Supply Chain Strategic Review - 2013 USAID Deliver Review 76 Technical Report April 2013 Project Cost Delivering Health Services in TZ February 2013 2013 Oxford Policy Management Costing exercise Health Sector Strategic Plan III: July 2009-June 2015 2009 MOHSW Plan Mid Term HSSP III debriefing PPT 2013 MTR Steering Com Review Tanzania [Health] Availability Service Mapping: 2005-2006 2007 MOHSW & WHO Service mapping Overworked? The relationship between workload and Health Worker performance in rural Tanzania 2009 Chr. Michelsen Institute Assessment In-Depth Assessment of the Medicines Supply System in TZ 2008 MOHSW Assessment Primary Health Service Program 2007-2017 2007 MOHSW Plan TZ Private Health Sector Assessment 2013 USAID SHOPS Project Assessment The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania (2008 – 2015) 2008 MOHSW Plan HRH Strategic Plan 2008-2013 2008 MOHSW Strategic plan TZ Health System Assessment 2010 Report 2011 USAID Health 20/20 Project Report HRH Recruitment Process PPT 2013 Recruitment framework Vodafone Program Description 2013 USAID/TZ Private sector program description Catholic University of Health and Allied Sciences- BUGANDO, Five Year rolling Strategic Plan CUHAS Plan Catholic University of Health and Allied Sciences (CUAHS) Prospectus for Academic Year 2013/14 CUHAS Prospectus 77 Annex V: KEY INFORMANTS Key informant Organization/Affiliation Title Touch Foundation Alex Rabadziska Touch Foundation, NY Managing Director Steve Justus Touch Foundation, NY Chief Medical Officer, Senior VP Noah Leff Touch Foundation, NY Director of Finance Massimiliano Pezzoli Touch Foundation, Tanzania Country Director Valerio Parisi Touch Foundation, Tanzania Program Manager Renae Stafford Touch Foundation, Tanzania Academic and Clinical Services Deo Rweyemamu Touch Foundation, Tanzania Finance & Procurement Associate Vince Jeong Touch Foundation, Tanzania Project Manager Catholic University of Health & Applied Sciences (CUHAS) Bishop Augustine Shao CUHAS Board of Directors Bishop of Zanzibar and Chairman of the Board of CUHAS Father Charles Kitima CUHAS Board of Directors Member of the board of CUHAS and Head of Department of Health in Tanzania Prof. Mange Manyama CUHAS Head Anatomy Department Prof. Stephen Mshana CUHAS Head Microbiology Department Dr. William Mahalu CUHAS Deputy Vice Chancellor Academics & Research Dr. Stella Mongella CUHAS Department Pediatrics Prof. Erasmus Kamugisha CUHAS Senior Rector Dr. Ladius Rudovick CUHAS Dr. Matiko Mwita CUHAS Head of Psychiatric Department Dr. Derick David CUHAS Head of Emergency Department Prof. Jacob Mtabaji CUHAS Vice Chancellor Esther Mosh CUHAS Assistant Medical Officer (AMO) Chars Mirambo CUHAS Assistant Medical Officer (AMO) Benadeta Mosha CUHAS Assistant Medical Officer (AMO) Venance Mgaiga CUHAS Current Medical Students Obeid Kabinza CUHAS Current Medical Students Sailus Gerold Komba CUHAS Current Medical Students Dr. Isidore Ngayomela CUHAS Orthopedic Surgeon Dr. Rob Peck CUHAS Pediatrics Specialist from Weill Cornell Bugando Medical Center (BMC) Archbishop Thaddeus BMC Board of Directors Chair & Archbishop Mwanza 78 Ruwa'ichi Prof. Charles Majinge BMC Director General Dr. Alphonse Chandika BMC Coordinator Treat & Train Program Francisco Chibunda BMC Healthcare Management Fellows Dr. Godfrey Kasanga BMC Healthcare Management Fellows John Pemba BMC Healthcare Management Fellows Elizabeth Chiwamba BMC Healthcare Management Fellows Sengerema Sr. Dr. Marie Jose Voeten Sengerema District Hospital Medical Officer in Charge Dr. Charles Mguta Sengerema District Hospital Dr. Simplice Harusha Sengerema District Hospital Dr. David Reis Peace Corps/Global Health Service Partnership Internal Medicine and Pediatrics Dr. Maureen Reis Peace Corps/Global Health Service Partnership Obstetrics/Gynecologist Sekou Toure Dr. Onesmo Rwakendera Sekou Toure Medical Officer in Charge Dr. Ariadne Lie CUHAS, Baylor Clinical instructor (expatriate) Dr. Jessica Bradford CUHAS/Baylor Clinical instructor (expatriate) Others Ronald Massawe TEC Diocese of Geita Health Secretary Bishop Damian Dallu TEC Bishop of Geita Bruno Ursprung InterTeam Dr. Ndaro Kulwijila Government of Tanzania Acting Regional Secretary for Mwanza Region Dr. Yusef Boure Sekou Toure Acting Regional Medical officer Dr. Steve Kisakye African Barrik Gold Community Relations Dr. Joseph Kavit Association of Private Medical Training Institutions (APHECOT) General Secretary Dr. Adeline Kimambo Public Health Association Dr. Mariam Ongara Ministry of Health & Social Welfare (MOHSW) National Health PPP Coordinator Andy O’Connell Ministry of Health & Social Welfare (MOHSW) PPP Advisor Martin Steven Mapunda Ministry of Health & Social Welfare (MOHSW) Assistant Director of HRH planning and policy 79 Renatus Mashawuli Ministry of Health & Social Welfare (MOHSW) Health Quality Assurance in Training Department Dr. Samwel Ogillo Association of Private Health Facilities of Tanzania (APHFTA) CEO 80 REFERENCES 1. Regional Strategy for Human Resources for Health – World Health Organization 2. Guidance on the Definition and Use of the Global Health Programs Account, A Mandatory Reference for ADS Chapter 200http://inside.usaid.gov/EE/po/pdfs/ghcs_guidance.pdf 3. MOSW Human Resources for Health Strategic Plan 2008 – 2013 4. PEPFAR Technical Considerations - FY 2014 COPS and ROPS, October, 2013 5. PEPFAR Blueprint: Creating an AIDS-free Generation, 2012 6. Health Sector Strategic Plan III July 2009 – June 2015, “Partnership for Delivering the MDGs”, The United Republic of Tanzania Ministry of Health and Social Welfare 7. White, James, Barbara O‘Hanlon, Grace Chee, Emmanuel Malangalila, Adeline Kimambo, Jorge Coarasa, Sean Callahan, Ilana Ron Levey, and Kim McKeon. January 2013. Tanzania Private Sector Assessment. Bethesda, MD: Strengthening Health Outcomes through the Private Sector Project, Abt Associates 8. Achievements, Approaches and Path Forward, June 2013 9. Achievements, Approaches and Path Forward, June 2013 10. Health Sector Strategic Plan (2009 – 2015) 11. Human Resources for Health Strategic Plan (2008 – 2013) 12. Private Health Sector Assessment, SHOPS 13.MOHSW-Joint Annual Sector Review 81 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 82