USAID/SOUTH AFRICA: INTEGRATED PRIMARY HEALTH CARE PROJECT END OF PROJECT PARTICIPATORY EVALUATION NOVEMBER 2010 This publication was produced for review by the United States Agency for International Development (USAID). It was prepared by William Emmet, Lindsey Carpenter Toomey, and Swati Sadaphal through the Global Health Technical Assistance Project. Cover Photo: William Emmet, Moretele Clinic, Bojanala District, North West Province, October 2010 USAID/SOUTH AFRICA: INTEGRATED PRIMARY HEALTH CARE PROJECT END OF PROJECT PARTICIPATORY EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 10-01-429) is available in printed and online versions. Online documents can be located in the GH Tech website library at resources.ghtechproject.net. Documents are also made available through the Development Experience Clearing House (dec.usaid.gov). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION i ACKNOWLEDGEMENTS The GH Tech IPHC Evaluation Team worked in participation with Malik Jaffer (USAID), Mariah Boyd-Boffa (MSH), and Tendani Muthambi of The Integrated Primary Health Care Project (IPHC), and thanks the United States Agency for International Development in South Africa for commissioning the evaluation of the Integrated Primary Health Care Project and for selecting us to undertake this challenging task. The time, patience, goodwill, and depth of knowledge of the 132 persons interviewed as part of this evaluation made it possible for the team to understand the significance of the many and diverse reports and technical interventions associated with IPHC’s multiple technical initiatives. The willingness of IPHC staff to provide us with a full array of project documentation and to share with us their thoughts on the project’s development and their perspectives regarding the IPHC’s initiatives was of significant importance in assisting us to understand the complex nature of the challenges associated with meeting IPHC objectives. The importance of the contribution of Ms. Anita Sampson, USAID’s COTR for the IPHC to this evaluation in terms of framing its technical methodology, cannot be overstated. Finally, the evaluation team greatly appreciates the time set aside by the representatives of the National Department of Health and of the districts and sub-districts in the five IPHC provinces to meet with us and discuss the progress of the IPHC. The evaluation team is especially appreciative of the time spent with facility and OVC program staff to discuss the effectiveness and sustainability of IPHC initiatives, in seeking to respond to the challenge of HIV and AIDS through integration of primary health care services. The six members of this evaluation team acknowledge, with thanks and appreciation, the entire staff of the IPHC for having assisted us in the many logistical and administrative details associated with this evaluation. While many IPHC staff have contributed to this effort, we would like to particularly thank Dr. Tracey Naledi, IPHC Chief of Party, and Ms. Tendani Muthambi for their invaluable role in securing and scheduling our interviews as well as ensuring that we met with the wide range of participants and stakeholders of the IPHC project. Finally, the evaluation team expresses its appreciation to the staff of GH Tech, most especially Ms. Taylor Napier, for having provided us with administrative support throughout this interesting and challenging assignment. ii USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION iii CONTENTS ACRONYMS .......................................................................................................... vii EXECUTIVE SUMMARY ..................................................................................... ix I. INTRODUCTION ......................................................................................... 1 II. BACKGROUND ............................................................................................ 3 Overview of PHC in South Africa..................................................................................................3 USAID Strategies and Priorities......................................................................................................3 Description of IPHC Program .........................................................................................................3 IPHC Organizational Structure .......................................................................................................4 IPHC Financial Data .............................................................................................................................4 III. METHODOLOGY ......................................................................................... 7 IV. FINDINGS BY STRATEGIC PERFORMANCE AREA (SPA) ................... 9 SPA 1: Maternal Health and Family Planning.............................................................................9 SPA 2: Child Health ......................................................................................................................... 11 SPA 3: Youth Programs.................................................................................................................. 14 SPA 4: HIV and AIDS ...................................................................................................................... 16 SPA 5: PHC Systems and Services............................................................................................. 20 SPA 6: Orphans and Vulnerable Children............................................................................... 24 V. BEST PRACTICES....................................................................................... 27 VI. STRENGTHS AND WEAKNESSES .......................................................... 29 IPHC Strengths................................................................................................................................... 29 Weaknesses.......................................................................................................................................... 31 VII. CONSTRAINTS........................................................................................... 33 VIII. LESSONS LEARNED .................................................................................. 35 IX. DISCUSSION ............................................................................................... 37 Alignment Between Donor/Contractor/DOH ...................................................................... 37 Human Resources.............................................................................................................................. 37 Management and Leadership Capacity ...................................................................................... 38 A Nationwide Culture of Entitlement and Dependency.................................................... 38 Cultural and Religious Beliefs Present a Significant Barrier to Fully-integrated PHC ........................................................................................................................ 38 X. RECOMMENDATIONS .............................................................................. 39 Improve the Use of Data for Decision-making Purposes.................................................. 39 Improve System-wide Accountability......................................................................................... 40 Improve the Uptake And integration of FP/RH into PHC Services............................... 40 Improve the Quality of Senior-level Technical Supervision.............................................. 41 Improve the Effectiveness of Supportive Supervision.......................................................... 41 Address Human Resource for Health Challenges................................................................ 42 iv USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Remove Barriers to the Effectiveness of the PHC Review Process.............................. 43 Enhance the Synergy among all Levels: National to Community.................................... 43 Enhance the Practical Application of the DHER and the DHP........................................ 44 Strengthen Effectiveness and Sustainability of Donor Assisted Initiatives................... 45 ANNEXES ANNEX A. SCOPE OF WORK .......................................................................... 47 ANNEX B. IPHC EVALUATION CONTACTS ............................................... 63 ANNEX C. IPHC KEY BACKGROUND DOCUMENTS REVIEWED MSH CONTRACTS......................................................................... 71 ANNEX D. IPHC EVALUTION SCHEDULE ................................................... 73 ANNEX E. INTEGRATED PRIMARY HEALTH CARE PROJECT (IPHC) END-OF-PROJECT PARTICIPATORY EVALUATION ............. 79 ANNEX F. IPHC: INTERVIEW SUMMARY ..................................................... 83 ANNEX G. MSH/IPHC EXTENSION PROPOSAL ......................................... 85 TABLES Table 1. PHC Summary Financial Data (in USD) ........................................ 4 Table 2. IPHC Project Evaluation Interview Respondent Affiliations and Totals............................................................................................ 8 Table 3. Progress on USAID/IPHC Indicators for SPA 1. Maternal Health and Family Planning ......................................................................... 10 Table 4. Progress on National Department of Health Indicators for SPA 1. Maternal Health and Family Planning ........................................... 10 Table 5. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 11 Table 6. Progress on USAID/IPHC Indicators for SPA 2. Child Health ... 12 Table 7. Progress on National Department of Health DHIS Indicators for Spa 2. Child Health ......................................................................... 12 Table 8. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 14 Table 9. IPHC SPA 3. Youth Program: Progress on Selected Indicators 15 Table 10. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 16 Table 11. Progress on USAID/IPHC Indicators for SPA 4. HIV/AIDS ........ 17 Table 12. Progress on National Department of Health DHIS Indicators for SPA 4. HIV/AIDSs ............................................................................ 18 Table 13. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 19 Table 14. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 20 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION v Table 15. Progress on USAID/IPHC Indicators for SPA 5. PHC Systems and Services ...................................................................................... 22 Table 16. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 24 Table 17. Progress On USAID/IPHC Indicators for SPA 6. Orphans and Other Vulnerable Children ............................................................. 25 Table 18. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives ..... 26 FIGURES Figure 1. IPHC Provinces and Districts ............................................................... 3 Figure 2. PSP-E Organization Chart .................................................................... 5 vi USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION vii ACRONYMS AIDS Acquired immune deficiency syndrome ANC Antenatal care ARV Antiretroviral ART Antiretroviral therapy AZT Zidovudine CBO Community-based organization CD4 Helper T cell CHC Community health center CHW Community health worker C&T Counseling and testing DHER District Health Expenditure Review DHIS District Health Information System DHP District Health Plans DHS District Health System DOH Department of Health DOTS Directly Observed Therapy Short-Course EPI Expanded Program on Immunization EOP End of project FP Family planning GH Tech Global Health Technical Assistance Project HAST HIV, AIDS, STIs and TB HBC Home-based care/caregiver HCT HIV counseling and testing HIV Human immunodeficiency virus HRH Human resources for health HST Health Systems Trust IMCI Integrated management of childhood illnesses IPHC Integrated primary health care project IR Intermediate results IDU Intravenous drug user IUD Intrauterine contraceptive device KAP Knowledge, attitude, and practice KPA Key performance area LDP Leadership Development Program MCH Maternal and child health MCWH Maternal, child, and women’s health MSH Management Sciences for Health NDOH National Department of Health MOU Memorandum of understanding viii USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION NGO Non-governmental organization OVC Orphans and vulnerable children PCR Polymerase chain reaction test PHC Primary health care PMTCT Prevention of mother-to-child transmission QA Quality assurance RH Reproductive health SAG South African Government SOP Standard operating procedures SPA Strategic performance area STI Sexually transmitted infection TA Technical assistance TASCII Technical Assistance Support Contract II TB Tuberculosis TOT Training of trainers URC University Research Corporation USG U.S. Government USAID United States Agency for International Development YFS Youth-friendly services USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION ix EXECUTIVE SUMMARY The Integrated Primary Health Care Project (IPHC) in South Africa is a collaborative project between the South African National Department of Health (NDOH); eight selected districts from five provincial Departments of Health from Eastern Cape, KwaZulu-Natal, Limpopo, Mpumalanga and North West; and the United States Agency for International Development in South Africa (USAID/SA). IPHC began in July 2004 and will end December 30, 2010, with Management Sciences for Health (MSH) as the prime contractor. In October and November 2010, the Global Health Technical Assistance Project (GH Tech) was commissioned to undertake an IPHC end-of-project evaluation. Designed by USAID/SA as a participatory evaluation, the evaluation team was comprised of three persons from GH Tech Team, one person from USAID/SA, and two from MSH. When available, representatives from the NDOH joined the evaluation team during site visits to the IPHC-assigned districts. The evaluation focused on: Review of project results (outputs and outcomes) in relation to the project’s six strategic performance areas (SPAs); Assessment of the project’s strengths, weaknesses, gaps in service delivery, and any constraints to successful implementation; and Identification and documentation of best practices, lessons learned, and recommendations to inform future activities focusing on sustainability. DESCRIPTION OF THE INTEGRATED PRIMARY HEALTH CARE PROGRAM IPHC builds upon the initiatives and achievements of USAID/SA’s EQUITY Project (1995–2004), focusing on supporting the work of the Department of Health and local partners. The project’s purpose was to improve access to integrated primary health care services, with an emphasis on strengthening management systems in planning, financial management, human capacity development, and quality assurance (QA); ensuring that the achievements of the EQUITY Project are sustained. It is also a priority that long-term improvements continue under local management through collaboration with local stakeholders. In responding to the project’s purpose, IPHC was designed to provide technical assistance (TA) focused on six strategic performance areas (SPA): Maternal health and family planning (FP) (SPA1), Child health (SPA 2), Youth programs (SPA 3), HIV and AIDS (SPA 4), Primary health care (PHC) systems and services (SPA 5), and Orphans and vulnerable children (OVC) (SPA 6). Operating in the five of the nine South provinces (Eastern Cape, KwaZulu-Natal, Limpopo, Mpumalanga, and North West), the project has provided direct support to nearly 70 health facilities as well as to the management teams of eight districts selected in collaboration with the NDOH and the targeted provinces. x USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION EVALUATION METHODOLOGY Description of Process The evaluation, scheduled during October 10 to November 12, 2010, was conducted using participatory approaches and methods and employed a combination of the following qualitative techniques: Review of relevant documents: In preparation for the evaluation and during the evaluation itself, the team reviewed project-related documents supplied by MSH and by USAID/SA. Team planning and review meetings: At the start of the evaluation in South Africa, the GH Tech team met to agree upon the evaluative documents and process. In the process of the evaluation itself, the team held five team planning meetings that focused on summarizing the interim results of the evaluation and on the preparation of the evaluation report. Briefing meetings: At the beginning of the evaluation, the team met with relevant staff of NDOH, USAID/South Africa, and technical and management staff of MSH to discuss and reach agreement on the evaluative process and expected deliverables. Thereafter, the team met twice with the same individuals, first to provide them with a preliminary briefing during the third week of the evaluation, and then as a final briefing at the end of the evaluation to present findings incorporated in the first draft report. Interviews and site visits: The six-person team was organized into two teams that conducted simultaneous field trips to maximize exposure to field activities. Key informant interviews were conducted with senior provincial and district DOH staff, health facility staff, local non-governmental organizations (NGOs), and current IPHC staff. All interviews were conducted using a standardized interview template that, when practical, was emailed to respondents in advance of the interviews. A total of 132 respondents in 31 venues were interviewed during the course of the evaluation. At the completion of each day’s interviews, members of each of the teams met to summarize their findings, once again using a standardized interview summary template. Analysis and report writing: The evaluation team used interview and site visit results, document reviews, and other relevant sources to obtain a comprehensive and in-depth understanding of the IPHC project, in order to support sound analyses, arrive at inferences, and make actionable recommendations. SUMMARY OF FINDINGS ACROSS STRATEGIC PERFORMANCE AREAS (SPA) The evaluation team assessed that IPHC achieved significant success in building upon the programmatic foundation established under the Equity project, and in the effectiveness of its introduction of new interventions, tools, and training programs to strengthen facility-level management and service delivery within each of the six SPA areas. However, due to the lack of an exit plan complete with specific deliverable milestones specified at the project’s onset, IPHC has generally failed to ensure a measureable level of sustainability for interventions introduced during the course of the project. As an aid to summarizing key findings associated with each SPA, a dashboard was developed by the evaluation team to illustrate the effectiveness and sustainability of key IPHC initiatives within each SPA. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION xi BEST PRACTICES ―Best Practice‖ is defined as: A clearly defined intervention that is known to provide near optimum results every time it is correctly implemented. Accordingly, based on this definition, the following IPHC applications, interventions, and tools are best practices within the South African health context. Cross-cutting IPHC Project Best Practices IPHC staff have nurtured and enhanced a sense of accountability with reference to the provision of technical assistance to all IPHC-supported sites by IPHC staff. Tools such as the Primary Health Care (PHC) Review and Supervision Guide have been instrumental in promoting effective supervision leading to integrated PHC services. IPHC’s approach to training facility staff in data management has led to enhanced understanding and proactive use of data for decision-making purposes. Specific Best Practices The extent to which district and facilities have effectively institutionalized application of the PHC review process stands out as one of the principle accomplishments of the Equity Project and of IPHC. IPHC’s training and promotion of facility mapping of their catchment areas has led to IPHC￾assisted facilities’ ability to more fully identify and respond to the needs of their communities. IPHC training in the development of clinic health committees has led to increased, effective collaboration between the facilities and the communities they serve. IPHC’s ability to work within the existing structures of orphans and vulnerable children (OVC) programs toward improvements in administrative and data management and marketing has enabled these programs to increase their ability to develop themselves as viable and sustainable entities. IPHC’s work with facilities and communities on the establishment of HIV and AIDS support groups has enhanced an environment in which people living with the disease can more effectively use services offered by the communities’ facilities. STRENGTHS AND WEAKNESS In assessing IPHC’s strengths and weaknesses, the evaluation team focused on those elements of implementation that appeared to contribute or detract from the project’s effectiveness and sustainability. Accordingly, IPHC’s major strengths were the project’s effectiveness in training; in particular: Introducing innovative management and supervision processes; Enhancing the ability of facility staff to understand and creatively use facility data for management purposes; Establishing effective collaboration and communication between facilities and the communities they serve; Nurturing, through its technical assistance management style, a positive and enabling environment between IPHC and facility staff. xii USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Conversely, IPHC’s major weaknesses were the project’s absence of an exit plan focused on sustainability; the abrupt cessation of support for IPHC initiatives; the lack of objective, on-going analytical tools and processes to measure the project’s progress toward sustainability; and the project’s limited attention to ensuring the relevance of IPHC project indicators with reference to monitoring and evaluation. CONSTRAINTS In assessing constraints associated with IPHC’s implementation, the evaluation team focused on those factors associated with IPHC’s implementation that appeared to be beyond the project’s ability to control, but nevertheless, that negatively impacted its ability to effectively implement a sustainable project. External factors or constraints impacting IPHC’s effectiveness and sustainability included South Africa’s acute deficit in human resources for health; its limited management, leadership, and primary health care (PHC) expertise within NDOH; a culture of dependency on donor assistance; and cultural and religious beliefs that limit the willingness of facility staff to provide reproductive health services as provided for in government policy. LESSONS LEARNED Lessons learned in IPHC’s implementation fall into the five thematic areas and can found in the sections below. Theme 1: Management and Leadership Management and leadership skills are essential for the sustainability of primary health care programs. If facility staff are to be effective, they must have ready access to qualified mentors. Skills gained through training dissipate if not regularly practiced. To remain current and motivated, facility staff must have access to scheduled in-service training. Teamwork and communication are key to effective provision of primary health care. Theme 2: Data Management A national integrated information system is essential to the monitoring and evaluation of the nation’s health care system. For the purposes of effective decision-making, data must be accurate and timely. Agreement on core national health indicators assists program managers and health care providers to focus and monitor health priorities. Excessive amounts of data are a constraint to effective decision-making. Theme 3: Integrated PHC Services Traditional emphasis on ―silos‖ is an ineffective approach to primary health care delivery. Integrated services are the most effective means of providing comprehensive care and addressing the likelihood of missed opportunities in regards to access to and provision of care. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION xiii Theme 4: Facility/Community Engagement Community involvement enhances the ability of communities to work together with facilities to respond to health care priorities. The development of ―youth friendly‖ services and youth mentors has been an effective means of reaching out to youth and encouraging them to utilize facility health services. Use of community health workers (CHWs) extends the ability of facility health care staff to provide quality and timely health care to the community. Cultural and religious beliefs present constraints to facility staff provision of reproductive health care as provided for in national health policy. Support groups for HIV and AIDS have been effective in responding to the needs of people with the disease and provide a model for other disease-specific support groups to emulate. Theme 5: Government and Donor Issues Collaboration between the government, donor agencies, and implementing contractors in the design of projects is essential to the sustainability of donor interventions. The design of exit plans for all donor initiatives, complete with measurable and deliverable milestones, is key to the sustainability of donor interventions. The ―disconnect‖ between the various levels of the DOH in terms of a thorough understanding of each level’s needs and priorities is a significant constraint to effective management and delivery of health care. Health facility employees, especially those working within rural areas, require an enhanced employment package to address inequalities and constraints associated with living conditions, access to training, and educational opportunities for themselves and their families. RECOMMENDATIONS Documentation associated with each of the following 10 recommendations in the body of this report includes the recommendations themselves; the context in which the recommendations are offered; and, if applicable, a discussion regarding the technical assistance required to implement a specific recommendation. For the purpose of this executive summary, only the recommendations are presented. Improve the Use of Data for Decision-making Purposes Recommendation The validity and reliability of the District Health Information System (DHIS) should be examined and an action plan to address the identified weaknesses should be implemented. Improve System-wide Accountability Recommendation A system-wide organizational development study should be commissioned to assess and document current health system organizational strengths and weaknesses, and to develop a strategy and action plan to address identified weaknesses. It should be focused on the standardized application of national policies and guidelines. xiv USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Improve the Integration of Reproductive Health/Family Planning (RH/FP) into PHC Services Recommendation An RH/FP promotion communication strategy should be integrated into PHC services. The long-term focused strategy should be responsive to individual behavior change needs, and should maximize the potential for change on a broader societal level. Improve the Quality of Senior-level Technical Supervision Recommendations An accredited and accelerated master’s level curriculum with an emphasis on PHC management (including statistics and epidemiology), should be developed within South Africa’s academic institutions to address the need for professional training for all PHC managers at the level of sub-district and above. All current PHC managers at the level of sub-district and above (including national level) should be scheduled to undertake the accelerated curriculum. Successful completion of the master’s level PHC management curriculum should be a pre￾requisite for promotion of DOH employees to supervisory and managerial positions. This would apply to supervisors and managers who work outside the PHC setting, as PHC is the backbone of South Africa’s health care delivery system. Improve the Effectiveness of Supportive Supervision Recommendations The DOH should implement a standardized Leadership Development Program (LDP) based on the MSH model. MSH should design a LDP curriculum to be accredited in South Africa for developing a cadre of LDP facilitators in South Africa. Although MSH has recently trained 20 LDP trainers, the current MSH training of trainers (TOT) process was not designed for long-term post￾training support (at least 12 months) and for the South African certification of those who were trained. The DOH should introduce the LDP nationally at the sub-district and facility levels. District health management teams, provincial program managers, and national directors should all receive orientation to the LDP. Following orientation to the PHC review process, national level directors should conduct quarterly supportive supervision visits to facilities, and provincial senior managers should conduct monthly supportive supervision visits to facilities. As a KPA (key performance area) indicator, the sub-district manager should be responsible for ensuring that all facility staff are trained in technical protocols, and for implementing changes in a timely manner. All new health-service delivery staff should participate in a standardized orientation program that addresses all aspects of health service delivery at the facility level. As a KPA indicator for the facility operations manager, consistent provision of a standardized orientation program should be administered to all staff. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION xv Address Human Resources for Health Challenges Recommendations Review and standardize post-specific job descriptions for all levels. Job description review should be focused on clarification of responsibilities and on harmonization (e.g., removing duplications and overlap) among posts. Develop training courses for operations managers to build their capacity to ―work smart‖ that is, to work effectively with the limited time and resources available. Develop a simple handbook of standard operating procedures (SOP) based on an analysis of key tasks, responsibilities, authority and resources for district and sub-district managers as a daily reference guide. Develop a long-term strategy and action plan to rationalize current DOH staffing patterns at all levels. Remove Barriers to the Effectiveness of the PHC Review Process Recommendation A computerized program should be developed for the production of summary ―dashboards‖ linked directly to PHC review process data. Enhance the Synergy among all Levels: National to Community Recommendations The NDOH should undertake an exercise to map all PHC programs and focus on the development of a strategy to identify and reduce program overlap (harmonization). By utilizing gap analysis, the NDOH should also identify and respond to the needs of underserved areas. As a KPA indicator, central and provincial level staff program managers should be trained in PHC and use of the supervisory manual as a guide should be required to undertake quarterly (for central staff) and monthly (for provincial staff) supportive supervisory visits to health facilities. As part of their ―Work Smart‖ training course, facility operations managers should be provided with an orientation on the linkage between service delivery and national policy on PHC. The NDOH should develop and ensure the application of an operations manual for facility￾level catchment area mapping to include the location of villages and OVCs; the availability of caregivers, CHWs, and NGOs; and current PHC response priorities such as low immunization rates, prevalence and incidence of tuberculosis (TB), HIV, AIDS, diarrheal outbreaks, and absence of clean water supply. Enhance the Practical Application of the District Health Expenditure Review (DHER) and the District Health Plans (DHP) Recommendation The DHP should be reduced to essential action-oriented interventions complete with measurable indicators with a clear linkage to the DHER and available resources. xvi USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Strengthen Effectiveness and Sustainability of Donor-assisted Initiatives Recommendations The NDOH and all provinces included in a project should agree to all initiatives within donor-assisted projects as part of the project procurement process. All contractors should be required to develop an exit plan with clearly defined milestones within the initial three-month project implementation phase. All contractors should be required to prepare and present a quarterly review of progress against established milestones to the government and to USAID. All contractors should be required to prepare an action plan to respond to those areas in which identified milestones are behind schedule. At the highest level of project implementation, a specific government official should be identified and actively engaged as a project implementation counterpart to participate in monthly project reviews and in quarterly reviews noted above. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 1 I. INTRODUCTION The Integrated Primary Health Care (IPHC) project in South Africa is a collaborative project between the South African National Department of Health (DOH); eight selected districts from five provincial departments of health from Eastern Cape, KwaZulu-Natal, Limpopo, Mpumalanga and North West; and the United States Agency for International Development (USAID) in South Africa. IPHC began in July 2004 and will end on December 30, 2010, with Management Sciences for Health (MSH) as the prime contractor. PURPOSE The main objectives of the evaluation are to: Review of project results (outputs and outcomes) in relation to the project’s strategic performance areas (SPAs); Assess the project’s strengths, weaknesses, gaps in service delivery, and any constraints to successful implementation; and Identify and document best practices, lessons learned, and recommendations to inform future activities focusing on sustainability. AUDIENCE The three principal audiences for this evaluation are USAID South Africa, the South African National Department of Health (NDOH), and MSH. This evaluation is also intended to provide lessons learned and recommendations for future implementing partners working to strengthen Primary Health Care (PHC) and HIV and AIDS systems and services in a sustainable manner in South Africa. SYNOPSIS OF TASK In responding to the above evaluation objectives, all three principal audiences requested evidence of program impact at district and facility levels, and specific examples of IPHC interventions that have been institutionalized. USAID South Africa and NDOH both requested information on ways the IPHC project supports the South African Government’s (SAG) re￾engineering and revitalization of PHC services. Additionally, the evaluation’s three audiences requested that the evaluation address the following specific issues: USAID expressed an interest in lessons learned under the Equity Project and to what extent IPHC built on those lessons and profited from them; USAID requested information on linkages the IPHC project developed between communities and the health system; NDOH would like to learn of IPHC interventions related to information systems and the use of data that were sustainable; MSH requested information on the success of the partnership with the government; and MSH requested information on project challenges and how they have been addressed. 2 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 3 II. BACKGROUND OVERVIEW OF PHC IN SOUTH AFRICA Primary health care (PHC) is seen as a cornerstone in the South African health care system. The DOH’s Strategic Plan 2010– 2013 outlines 10 key priority areas of which PHC features in four areas: (1) overhauling the health care system and improving its management by refocusing the health system on PHC; (2) improving the functionality and management of the health system by decentralizing management and training managers in leadership, management, and governance; (3) improving human resources planning, development, and management with a focus on training of PHC personnel and mid-level health workers; and (4) accelerated implementation of the HIV and AIDS strategic plan and the increased focus on tuberculosis (TB) and other communicable diseases as part of an approach to integrated comprehensive service delivery at a level that is closest to the community. USAID STRATEGIES AND PRIORITIES To address South Africa’s major health issues, the U.S. Government (USG) partners with SAG to ensure that assistance contributes to the SAG’s strategic health plans. This partnership also works with non-government, faith-based, private, and grassroots organizations to address the impact of HIV/AIDS, TB, maternal and child health (MCH), and family planning and reproductive health (FP/RH) for the delivery of quality health care in South Africa. USAID South Africa supports activities in all nine provincial regions. The USAID health mission supports the SAG to build local capacity, strengthen health systems, establish and foster key partnerships, provide health care and treatment, and support innovation in the development of state-of-the-art health technologies. DESCRIPTION OF IPHC PROGRAM The IPHC project under TASCII is funded by USAID and charged with providing technical assistance to strengthen the district health system (DHS) in five provinces of South Africa. IPHC builds on the successes of the EQUITY Project, focusing on supporting the work of the DOH and local partners. The project aims to improve access to, and emphasis on strengthening management systems in planning, financial management, human capacity development, and quality assurance of the DOH and local partners; ensuring that the achievements of the EQUITY Project are sustained. It is also a priority that long-term improvements continue under local management through collaboration with local stakeholders. Figure 1. IPHC Provinces and Districts 4 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION The IPHC Project works to improve access to and use of child health, reproductive health, and HIV and AIDS services. Operating in five provinces (Figure 1), the project provides direct support to nearly 70 health facilities as well as to the management teams of eight districts, contributing to effective decentralization and focusing on local capacity, ultimately resulting in stronger community-based approaches to health, especially in HIV and AIDS services and support. This includes the roll-out of antiretroviral (ARV) drugs, the introduction of new approaches to performance-based management in the public sector, the establishment of a monthly PHC review system and the creation of district-to-district mentoring and support systems to ensure improved data management and quality care. IPHC focuses on supporting the TB program, HIV, AIDS, MCH, FP and orphans and vulnerable children (OVC). MSH has played the role of lead partner in the IPHC Project with the Health Systems Trust (HST) and the University Research Corporation (URC) as collaborating partners up until the scheduled end of the project in September 2008. The project was extended until 2010, under MSH without the partnership of HST and URC. The project works to strengthen and improve health management systems at a district level. A supportive supervision system and improved use of data are the two key factors in achieving this. Improved data management facilitates good planning processes which are promoted and supported. Linkages between health facilities and the communities they serve are encouraged. Special attention and support is given to youth mentors, as they play an important role in the way that health providers are able to interact with their clients in a clinical setting. The IPHC￾supported facilities offer a comprehensive package of services in a full ―supermarket‖ approach. Positive management styles and strong leadership play a central role in good district performance. It is necessary to strengthen these management attributes. IPHC ORGANIZATIONAL STRUCTURE In response to its extension in 2008, to the concomitant reduction in funding and the closure of its partnership with URC and HST, the IPHC management structure was reorganized to reflect the new funding realities (See Figure 2). IPHC FINANCIAL DATA Table 1 represents the current financial status of the IPHC through its scheduled closure. Based on information supplied by MSH, the project is scheduled to effectively close as of December 16, 2010. Table 1. IPHC Summary Financial Data (in USD) Total Contract Amount (USD) 25,902,737 Expended as of 9/30/2010 24,466,422 Accruals as of 11/01/2010 355,040 Balance as of 10/01/2010 1,081,275 Current Burn Rate 302,000 Anticipated Expenditures through 12/31/10 906,000 Estimated Balance as of 12/31/10 175,275 USAID/SOUTH AFRICA: IPHC PROJECT END OF PROJECT PARTICIPATORY EVALUATION 5 Figure 2. PSP-E Organization Chart COP Dr. Tracey Naledi Director Finance Carole Evans HIV/AIDS Vacant Trainer Dr. Almakio Phiri MNCH& Youth Dr. Nombulelo Skeyile MNCH Coordinator Nomvuyo Shongwe Engcobo TA Andile Lesele TB/HIV Vacant M&E Tendani Data Capture Vacant Provincial Coordinators EC Nondumiso EC Ntuthu KZN Bongi KZN Gugu Limpopo Ogrinah ART TA Alude Dube North West Vacant Mpumalanga Vacant Finance Enid Finance Lloyd Logistics Nadia PA Mandisa Receptionist Keketso IT Pule 6 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 7 III. METHODOLOGY DESCRIPTION OF PROCESS The evaluation was conducted using participatory approaches and methods. The evaluation team consisted of three international consultants with expertise in HIV and AIDS care and treatment, MCH, health systems management, and community systems of care; one MSH headquarters staff member; one representative from USAID/South Africa; and a local MSH Monitoring and Evaluation Officer. The use of the word ―facility‖ in this report refers to both the primary health care clinic and the community health center. The evaluation was scheduled during October 10 to November 12, 2010. The evaluation methodology used a combination of the following qualitative techniques: Review of Relevant Documents The evaluation team spent the initial three days (October 10–13) reviewing existing key project data and reports (Annex C). MSH internal qualitative assessments of its activities served as background to the evaluation. Additional documents were requested from MSH for information with reference to the IPHC’s achievement of outputs by SPAs as specified by the MSH 2008 Extension Proposal. (Annex G). Team Planning Meeting A detailed agenda of visits to various sites as well as evaluation instruments were developed by the team before the start of site visits (See Annex D). Initial Briefing Meetings These meetings with relevant staff of NDOH, USAID/South Africa, and technical and management staff of MSH enabled the evaluation team to understand the project activities and evaluation expectations, and to finalize the approach and activities for the evaluation. As a result of these initial briefings, a final workplan and methodology was developed and approved by USAID’s contracting officer technical representative. Interviews and Site Visits The six-person team was organized into two teams that conducted simultaneous field trips to maximize exposure to field activities. The teams spent two weeks (October 19 to November 1) visiting program sites in eight IPHC supported districts in five provinces—Eastern Cape, KwaZulu-Natal, Mpumalanga, Limpopo, and North West. Key informant interviews were conducted with senior provincial and district DOH staff, health facility staff, local NGOs, and current IPHC staff (See Table 2). Each team used a standardized interview and site visit guideline (Annex E) to ensure that the teams addressed the same issues. Depending on their availability, two representatives from the NDOH joined the teams during the field visits. Each facility visit or district office meeting took about two to three hours. Following each day of interviews, key points, issues, and observations were summarized using an interview summary form for each interview and site visited. After each week of the interviews and site visits, both teams met in Pretoria to compare notes and summarize findings. 8 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 2. IPHC Project Evaluation Interview Respondent Affiliations and Totals Province Number Province Number District Representatives Community-based Organizations Limpopo 8 Limpopo 8 Eastern Cape 6 Eastern Cape 0 North West 17 North West 0 KwaZulu-Natal 13 KwaZulu-Natal 0 Mpumalanga 7 Mpumalanga 0 TOTAL 51 TOTAL 8 Health Services Personnel NDoH & Health Systems Trust Limpopo 24 Dr. Yogan Pillay, NDoH 1 Eastern Cape 10 Dr. R. Morewane, NDoH 1 North West 9 Mr. Bennett Asia, NDoH 1 KwaZulu-Natal 23 Dr. Tim Wilson, NDoH 1 Mpumalanga 2 Dr. P M Matse, HST 1 TOTAL 68 TOTAL 5 TOTAL RESPONDENTS 132 Analysis and Report Writing The evaluation team used interview and site visit results, document reviews, and other relevant sources to obtain a comprehensive and in-depth understanding of IPHC project to support sound analyses, to arrive at inferences, and to make actionable recommendations. Debriefing Meeting This meeting was organized with USAID, NDOH, and MSH staff to present the preliminary findings and recommendations. The analysis and final draft report writing were completed during the last two weeks (November 2–12) of the evaluation. The final draft report was submitted on November 12, 2010 to USAID/South Africa for comments. CONSTRAINTS AND GAPS The respondents and sites were not chosen randomly by the evaluation team. These sites were suggested by and agreed upon with the USAID and MSH staff. It is possible that only better￾performing sites were visited. Further, non-IPHC supported provinces or districts were not visited for a comparative analysis. Quantitative internal assessment report or data on impact indicators was not available to the team before the site visits. This information might have helped the evaluation team to contextualize and probe during the qualitative interviews. Perspectives of former partners of IPHC (URC and HST) on project activities, progress, and achievements were not obtained. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 9 IV. FINDINGS BY STRATEGIC PERFORMANCE AREA (SPA) SPA 1: MATERNAL HEALTH AND FAMILY PLANNING With reference to the MSH Extension Proposal (September 2008—December 2010), the IPHC’s interventions in maternal health and family planning focused on one principal objective: Improve women’s and maternal health in order to effectively reduce maternal complications and deaths. Interventions In addressing the above objective during the extension period, IPHC was contractually obligated by USAID to report on four maternal health and family planning indicators, including the following: Number of antenatal care (ANC) visits provided by skilled providers from USG-assisted facilities, and Number of people trained in FP/RH with USG funds. In addition, IPHC increased monitoring of the District Health Information System (DHIS) population-based indicators during the extension period for maternal health and family planning indicators, including: ANC coverage; ANC coverage < 20 weeks; and Women Year Protection Rate. Progress on Selected Indicators IPHC data (Tables 3 and 4) indicates that the number of ANC visits increased 22%, and the number of people trained in FP/RH increased 4% during the last four years of the project. When these indicators are compared to set targets, however, achievement against targets is reported at 129% for the number of people trained and 307% for the number of ANC visits. For both indicators, targets for FY 2009/2010 were set significantly below what was achieved for those indicators in FY 2006/2007, particularly in the case of the number of ANC visits. Consistently setting targets below program achievements raises questions about the quality of IPHC program monitoring by MSH and USAID. Improvements in maternal health and FP indicators are supported by DHIS data for ANC coverage: less than 20 weeks (30% coverage increase), and women-year protection rate (14% coverage increase). 10 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 3. Progress on USAID/IPHC Indicators for SPA 1. Maternal Health and Family Planning Indicator FY 2006/2007 FY 2009/2010 % Increase FY 2009/2010 Target % achievement of target # of antenatal care (ANC) visits provided by skilled providers from USG￾assisted facilities 88,000 107,452 22% 35,000 307% # of people trained in RH/FP with USG funds 186 193 4% 150 129% Table 4. Progress on National Department of Health Indicators for SPA 1. Maternal Health and Family Planning Indicator 2004 2009 % Increase Indicator 1: ANC Coverage 96% 90% (-6%) Indicator 2: ANC Coverage < 20 weeks 26% 34% 30% Indicator 3: Women-year protection rate 23% 26% 14% Achievements Based on reported progress on selected project indicators and DHIS data, positive progress was made on maternal health and family planning at IPHC supported sites, and in IPHC-supported sub-districts. Additionally, qualitative data collected during interviews with district and facility staff highlighted the following findings. Facilities reported that IPHC training and support encouraged continuity of care. Before IPHC support, it was common for cases to be treated in isolation. Facility staff says that now, if patients come for sexually transmitted infections (STIs) treatment or FP services, they also use the opportunity to take a pap smear. Pap smear statistics have increased due to this continuity of care approach, and also due to giving women more information at clinics and at community education campaigns. Despite religious and cultural beliefs that do not support family planning in many rural areas, there have been positive impacts. Many more women now see the importance of family planning and the dangers of not using it. Cultural beliefs also prevent many women from disclosing that they are pregnant early enough to receive timely ANC services. Through the use of clinic health committees, IPHC training and coaching, and educational campaigns, more ANC bookings are being made earlier. This allows HIV testing and, if necessary, treatment of ARV to reduce mother to child transmission of HIV. Despite these achievements, facilities report that more progress is needed in the areas of disclosing pregnancy and termination of pregnancy. Many clinics reported an increase in the number of women and teens referred for family planning, and starting a family planning method. This achievement is particularly notable given the R250 per child offered by the Department of Social Development to support mothers, which has been a perverse incentive for teens and women to have children. Both the IPHC project and USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 11 the South African national ―Love Life‖ initiative have contributed to the increase in uptake of FP methods. However, more progress is needed, particularly in the area of teen pregnancy. Many clinics reported that the use of tools and systems introduced by IPHC to support maternal health and FP have improved delivery of these services. They also report that this progress can be sustained, since the tools and systems are now embedded in facility operations. Effectiveness and Sustainability Based on interviews with respondents and on field observations, Table 5 summarizes the effectiveness and sustainability with reference to two key indicators associated with IPHC maternal health and FP interventions. Table 5. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 1: Maternal Health and Family Planning Less than 40% 40–80% Greater than 80% (less than40%: Shaded Red 40-80%: Shaded Yellow greater than 80%: Shaded green Initiative 1: Integrated Services Effectiveness Five green bars Sustainability Four yellow bars Initiative 2: Family Planning Effectiveness Three yellow bars Sustainability Two red bars SPA 2: CHILD HEALTH With reference to the MSH Extension Proposal (September 2008—December 2010), the IPHC’s interventions in child health focused on one principal objective: Improving child survival and reducing infant and child mortality Interventions In addressing the above objective during the extension period, the IPHC was contractually obligated to report on eight child health indicators, including the following: Number of people trained in child health care and child nutrition through USG-supported programs; Number of people trained in maternal/newborn health through USG-supported programs; and Number of children < 1 year of age fully immunized. In addition, IPHC increased monitoring of DHIS population-based indicators during the extension period for selected performance areas. DHIS child health indicators included: Primary health care utilization rate for < 5 years Fully immunized < 1 year rate 12 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Progress on Selected Indicators IPHC data (See Table 6) indicates that progress declined between 41% and 73% on child nutrition training, maternal/newborn health training, and immunization for children less than 1 year during the last four years of the project. When these indicators are compared to set targets, however, achievement against targets is reported between 93% and 154%. These reported declines in training and immunization are not corroborated by interviews conducted with district and facility staff. Conversely, in the case of immunization for children less than 1 year old, DHIS data (Table 7) reports an increase in coverage in IPHC-supported sub-districts from 75% in 2004, to 85% in 2009. DHIS data also showed an increase in the PHC utilization rate for children under five. Setting USAID/IPHC targets well below reasonable program expectations coupled with IPHC/DHIS/qualitative interview data inconsistencies raise questions about the quality of IPHC program monitoring by MSH and USAID, and about the quality of program data reported. Achievements Based on IPHC reported progress on selected indicators and on DHIS data, progress on child health initiatives declined at IPHC supported sites. However, progress improved overall in the sub-districts where IPHC was working. In reviewing IPHC documentation such as semi-annual and annual reports, it was not possible to find evidence that would explain the inconsistencies between IPHC’s reporting on indicators and that of the DHIS. However, based on the following Table 6. Progress on USAID/IPHC Indicators for SPA 2. Child Health Indicator FY 2006/2007 FY 2009/2010 % Increase FY 2009/2010 Target % Achievement of Target Number of people trained in child health care and child nutrition through USG-supported programs 360 194 (-46%) 125 93% Number of people trained in maternal/newborn health through USG￾supported programs 213 58 (-73%) 50 154% Number of children <1yrs fully immunized 18,677 10,966 (-41%) 10,000 151% Table 7. Progress on National Department of Health DHIS Indicators for Spa 2. Child Health Indicator 2004 2009 % Increase Indicator 2: Primary health care utilization rate for < 5 years 3.5% 4.8% 38% Indicator 3: Fully immunized < 1 year rate 75% 85% 14% USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 13 findings associated with the evaluation team site visits to the 18 sites included in the evaluation, it would appear that significant progress has been achieved on the single child health objective specified for SPA 2. With training provided by IPHC, community-based caregivers are now able to track children lost to follow-up. The use of home visits to check ―Road to Health‖ cards has resulted in fewer immunization defaulters and an increased focus on growth monitoring and evaluation and on children at risk. In selected provinces (e.g. North West and Mpumalanga), IPHC’s 2010 introduction of the Leadership Development Program (LDP) appears to have assisted health center staff in identifying priorities. These priorities include: the need to conduct catch up immunization, to design comprehensive immunization registers, to train nurses in drug supply management, and to train nurses and data clerks on data management. Integrated management of childhood illnesses (IMCI) focused checklists have been introduced by the project and embedded in clinic operations. For example, while Expanded Program on Immunization (EPI) protocols existed prior to IPHC interventions, there appears to have been limited staff compliance. By contrast, with the introduction of the IMCI checklist, staff compliance with these protocols in some facilities visited has approached 100%. Similarly, in IPHC assisted facilities, use of the checklist has resulted in malnutrition now being rare in patients treated at the clinic and in a significant reduction in life-threatening instances of diarrhea. Under IPHC guidance, training and supportive supervision are centered on integrated services. Facility staff now focus on ensuring that mothers are counseled and, if appropriate, tested for HIV/AIDS, TB and STIs, and are also counseled on family planning and immunization compliance. As stated by one clinic manager, IPHC’s training and supervision has re-enforced the concept that: ―To build a healthy nation, the testing of all mothers will help the next generation be free of HIV/AIDS.‖ With training and supportive supervision assistance provided by IPHC, lower categories of nurses have been trained and can now weigh and monitor the growth of infants and children under five. This simple but effective intervention has freed up higher category nurses for more complex clinical work. Under IPHC training and supportive supervision in many of the facilities visited community￾based caregivers now ensure that mothers breastfeed within 72 hours and that they maintain post-natal monthly visits to their facilities. In the event that there are medical issues, the caregivers are trained to refer their clients to the facility and, if an appointment to a clinic is missed, the mothers are visited at home to encourage compliance with the scheduled visit. Effectiveness and Sustainability Based on interviews with respondents and on field observations, Table 8 summarizes the effectiveness and sustainability with reference to two key indicators associated with child health interventions. 14 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 8. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 2: Child Health Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 1: Community Tracking of Patients Effectiveness Six green bars. Sustainability Four yellow bars. Initiative 2: IMCI Integration Effectiveness Five green bars. Sustainability Four Yellow bars. SPA 3: YOUTH PROGRAMS With reference to the MSH Extension Proposal (September 2008—December 2010), the IPHC’s interventions in the youth programs (SPA 3) focused on one principal objective: Increasing youth participation in the promotion and provision of youth-friendly services in order to reduce reproductive health problems among adolescents. Interventions In addressing the above objective during the extension period, the IPHC proposed to continue or initiate the following interventions: Support the implementation of the Adolescent Health Policy at district and facility levels to increase uptake of RH services (including HIV and AIDS services); Establish and improve linkages between health facilities and youth in the clinic catchment areas and capacitate the community youth to support implementation of youth friendly services (YFS); Create support for a youth presence in health facilities and involve youth in service provision with the aim of integrating and sustaining youth mentors through programs of the DOH or Department of Social Development; Institutionalize the YFS strategy at the district and provincial levels to ensure its continuation (including placing these activities in the District Health Plans [DHP]); Focus youth mentor activities on achieving the 10 YFS standards; and Foster and support acceptance of the youth program as a cross-cutting strategy in health facilities and districts by ongoing monitoring, support, and mentoring of youth volunteers in health services. Progress on Selected Indicators Data indicates that the number of youth receiving HIV tests and STI treatment increased in facilities and communities served by IPHC-trained YFS staff and youth mentors (See Table 9). The number of youths tested for HIV increased 12% (approximately 1,500 additional youth reached), and youths receiving STI treatment increased 2% (approximately 200 additional youth reached). The number of youth provided with STI information decreased drastically by the end of the project. This may be explained by the fact that support for youth programs was USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 15 withdrawn, with the exception of youth mentor stipends, for the two-year extension period. It is particularly remarkable that progress in HIV testing and STI treatment was improved, given the decrease in program support. This fact points to the success of the training and skills put in place before funding for those activities ceased. Table 9. IPHC SPA 3. Youth Program: Progress on Selected Indicators Indicator FY 2006/2007 FY 2009/2010 End line % Increase Number of individuals who tested for HIV 13,341 14,895 12% Number of individuals given STI information 263,776 93,772 (-64%) Number of individuals who received STI treatment 12,014 12,213 2% Achievements In most cases, achievements directly associated with the above six interventions were not explicitly addressed in the IPHC’s annual and semi-annual reports. However, during site visits to facilities and sub-district offices in the five targeted IPHC provinces, significance of IPHC youth interventions was noted. Based on interviews, IPHC working with facility staff has successfully developed a corps of youth mentors whose door-to-door work in communities, schools, clinics, and churches has demonstrably resulted in an increased willingness of young people to utilize facility based YFS (also enhanced through IPHC training). All facilities visited during the evaluation with YFS, and with youth mentors supported by IPHC, reported increases in youth utilization of information, counseling, and testing services for HIV/AIDS/STIs, and RH issues. Again, based on interviews with health facility staff and with youth mentors themselves, it appears that efforts of youth mentors, in coordination with facility-based YFS introduced by the IPHC, have resulted in increased numbers of youth taking advantage of clinic-based services. This has led to a decrease in teen pregnancy, an increase in youth willing to be tested for HIV/AIDS and STIs, and in an increase in the use of contraceptives, including emergency contraceptives, among the under-25 population. Thanks to the IPHC-initiated youth mentors’ training and their subsequent interaction with their peers, the environment at IPHC-sponsored health clinics has been significantly improved so that youth are now aware of the services provided by health facilities and the benefits associated with their utilization. Effectiveness and Sustainability Based on interviews with respondents and observations in the field, Table 10 summarizes the effectiveness and sustainability with reference to two key indicators associated with IPHC youth interventions. 16 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 10. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 3: Youth Program Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 1: Youth Mentors Effectiveness Six green bars. Sustainability One red bar. Initiative 2: Youth-Friendly Services Effectiveness Six green bars. Sustainability Five green bars. SPA 4: HIV AND AIDS During the extension period (October 2008 to December 2010), IPHC focused on achieving the following objectives with the purpose of reducing the impact of HIV and AIDS: Strengthen the integration of counseling and testing (C&T) into routine PHC activities; Strengthen and expand a comprehensive prevention of mother-to-child transmission (PMTCT) program; Strengthen ARV treatment service delivery to those who are infected with HIV; and Strengthen palliative care in the PHC setting; integrate palliative care for TB and HIV. Interventions The project aimed to achieve the above objectives by implementing interventions under five HIV and AIDS activity sub-categories of SPA 4: (4.1) counseling and testing, (4.2) (PMTCT), (4.3) HIV and AIDS treatment, (4.4) palliative care (basic), and (4.5) palliative care (TB/HIV). IPHC was contractually obligated to report on thirteen indicators for monitoring these activities, including those in Table 11. In addition, IPHC increased monitoring of DHIS population-based indicators during the extension period for selected HIV/AIDS indicators (Table 12). Progress on Selected Indicators Impressive progress has been made on nearly all HIV and AIDS objectives. The only exception in the IPHC data (Table 11) is in the area of counseling and testing, and in receiving HIV test results. However, this decrease was not supported by DHIS data or by qualitative interviews, both of which show significant improvements in this area. Most progress was made in the area of integrated HIV/TB testing, with an increase from 6% to 32%. Training for treating TB in HIV￾infected patients increased more than ten-fold, and exceeded the project target set for this indicator. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 17 Table 11. Progress on USAID/IPHC Indicators for SPA 4. HIV/AIDS Indicator FY 2006/2007 FY 2009/2010 % Increase FY 2009/2010 Target % Target achievement 4.1 Counseling & Testing: # of clients who received counseling and testing and received their test results (excl. antenatal) 45,310 40,532 (-11%) 40,000 101% 4.2 PMTCT: # of pregnant women provided with a complete course of antiretroviral prophylaxis in a PMTCT setting 2,788 3,330 19% 5,000 67% 4.3 Treatment: Indirect # of individuals receiving antiretroviral therapy at the end of the reporting period 22,172 31,977 44% 20,000 160% 4.4 Palliative Care (Basic): # of individuals provided with HIV￾related palliative care (including TB/HIV) 2,651 5,153 94% 5,000 106% 4.5 Palliative Care (TB/HIV): # of individuals trained to provide clinical prophylaxis and/or treatment for TB to HIV￾infected individuals (diagnosed or presumed) 3 319 10,533% 300 106% 18 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 12. Progress on National Department of Health DHIS Indicators for SPA 4. HIV/AIDSs Indicator 2004 2009 % Increase Indicator 3: HIV testing rate 57% 83% 46% 22%* Indicator 6: HIV-positive clients screened for TB 6% 32% 462% Indicator 8: ANC clients tested for HIV 50% 90% 81% Indicator 11: Nevirapine uptake among HIV-positive pregnant women 46% 68% 48% *For Indicator 3: HIV testing rate, Uthungulu sub-district in Kwazulu Natal reported an increase of 0% to 82.6%. This was an extreme outlier compared to the other rates reported. It is highly unlikely that HIV testing was at 0% in 2004. This drastic reported uptake in testing significantly changes the average district HIV testing rate. When Uthungulu’s data is removed, the average percent increase is 22% (less than half of 46%). Achievements The project conducted several training workshops for health professionals and lay counselors with a focus on integrating HIV, TB, HIV counseling and testing (HCT), PMTCT, ANC, and FP services. Technical assistance was complemented by monthly support visits for mentoring and coaching staff on activities including patient chart review, data management, catchment area mapping, and use of supervision checklists. Sub-district clinic supervisors disseminated innovative tools and systems from IPHC-supported facilities to unsupported facilities. Facility staff reported confidence in using these tools and systems, and cited them as playing a key role in integrating and improving HIV and AIDS services. For example, facility staff said that the use of tick registers have enabled easy tracking of client movement between different health facility units, as well as tracking of adherence to ARV treatment. IPHC supported the transition to provider-initiated HIV testing in line with the national HCT policy. To support HIV testing services, IPHC encouraged the use of rapid HIV test kits, significantly reducing the time lapse between testing and issuing of results, resulting in an increase in the number of clients receiving their results. The use of clinic health committees to clarify HIV and AIDS confidentiality policy has increased community HCT uptake. Youth mentors were repeatedly mentioned as a factor in increasing HCT uptake among youth. Additionally, as a result of IPHC support, ANC HIV testing increased from 80–90% to 95–100% in the majority of facilities. With the integration of HIV, TB, HCT, PMTCT, ANC, and FP services, HIV-positive mothers are given dual therapy according to new PMTCT guidelines, and pregnant women are referred internally for HIV/AIDS, TB, and STI counseling and services. Facilities emphasize early ANC booking and HIV testing at the first visit, with some facilities even promoting retests (at least twice during pregnancy). Staff recognize that early bookings for ANC allow testing and, if necessary, early initiation of ARV treatment to reduce transmission from mother to child. Facilities reported an approximately 20% increase in the number of HIV-negative babies tested at six weeks. In Madibeng sub-district in North West Province, sub-district staff used skills they learned in MSH’s Leadership Development Program (LDP) to develop a comprehensive PMTCT register to address the problem of PMTCT patient tracking complicated by multiple register use for USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 19 different services. Their improved data review process has led to increased enrollment in the PMTCT program following testing— and immediate follow-up with patients with low helper T cell (CD4) counts rather than waiting for them to come for the next appointment. Polymerase chain reaction test (PCR) rates for infants have increased and CD4 testing has also increased. IPHC assisted in accreditation for ARV services and training of staff at most facilities visited. Currently, treatment is initiated at hospitals, and patients are down-referred to clinics for ongoing management and treatment. IPHC assisted facilities by providing mentoring support to integrate anti-retroviral therapy (ART) with other services and to improve down-referral systems that decentralize services and lighten patient load at hospitals. Collaboration and cross-screening of HIV and TB services has improved the management of patients with one or both diseases. IPHC introduced, trained and institutionalized the use of a TB screening tool for HIV-positive clients, and cross-screening of all HIV and TB patients is now routinely done as a result of IPHC support. Proactive management of TB at the community level, through use of home-based care in the provision of directly observed therapy short￾course (DOTS), has helped to reduce the TB defaulter rate. One facility reported a rise in TB cure rate from 18% in 2004 to 81% in 2010. Another facility reported a 0% TB defaulter rate in the last month. Home-based caregivers also provide palliative care to HIV-positive babies in orphanages; care for sick patients; and administer medicines, nutritional supplements and fresh vegetables from clinic gardens. Effectiveness and Sustainability Based on interviews with respondents and on field observations, Tables 13 and 14 summarize the team’s assessment of the effectiveness and sustainability with reference to four key indicators associated with IPHC HIV and AIDS interventions. Table 13. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 4: HIV and AIDS Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 1: Integrated Services Effectiveness Six green bars. Sustainability Four yellow bars Initiative 2: PMTCT Effectiveness Five green bars. Sustainability Four yellow bars 20 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 14. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 4: HIV and AIDS Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 3: ARV Effectiveness Five green bars. Sustainability Four yellow bars Initiative 4: TB and HIV Effectiveness Five green bars. Sustainability Four yellow bars SPA 5: PHC SYSTEMS AND SERVICES With reference to the MSH Extension Proposal (September 2008–December 2010), the IPHC’s interventions to strengthen PHC systems and services (SPA 5) focused on three principal objectives: Contribute to service quality improvements through selected tools such as the Primary Health Care Supervision Manual and structures such as clinic health committees; Improve the knowledge and use by service staff and managers of health information of acceptable quality for planning, management and monitoring; and Support the development of district health plans, operational plans, operational reviews, and the conducting of District Health Expenditure Reviews (DHER). In addition, the IPHC sought to build on the following products developed previously under the MSH Equity Project: Clinic Supervision Manual The purpose of this manual is to provide a set of flexible, adaptable tools, and guidelines to support supervisors in their role of improving the quality of care in the clinics. It is especially helpful for focusing managers on the key elements of integrated primary health care as they simultaneously integrate new interventions for HIV/AIDS, TB, and malaria. District Health Information System (DHIS) The DHIS, developed to collect aggregated routine data from all public health facilities in a country, is intended to support decentralized decision-making and health service management, and allows health care workers to analyze their levels of service provision, predict service needs, and assess performance in meeting health-service targets. District Health Expenditure Review (DHER) A DHER presents a clear picture of funding, distribution, and use of health resources within the district. This is an important foundation for planning and helps to restructure district health services and manage resources more effectively to meet the needs of communities. A DHER focuses on financial data and links this to other resources such as staff, as well as to service and USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 21 population data. These are then analyzed according to performance measures for the various costs in the district. District Health Plans (DHP) From a health-planning viewpoint, it is necessary to describe the relationships with other government-funded health services in the plans and reports. Relationships with private health providers should also be described, particularly if public/private partnerships are planned or in place. District health planning should also serve to integrate the plans of different health programs, different management structures, and different disciplines. District health plans and reports aim to cover all aspects of health care to ensure that resources are used equitably, effectively, and efficiently. Primary Health Care (PHC) Review Process The PHC Review process is a methodology and means for staff to be involved in monthly joint performance monitoring at the facility level and in the promotion and support of periodic PHC review meetings at sub-district and district levels. Action plans and lessons learned are shared during these review meetings to encourage use of best practices. Use of the Clinic Supervision Manual and data management skills are integral to the process. Other tools and interventions used include: District Monitoring, Economic, Social and Human Resources (MESH) The MESH tool was utilized as a checklist to compare district level performance between those districts receiving support from IPHC and those with no direct support, in the same provinces. Leadership Development Program (LDP) The LDP helps organizations develop managers to lead with a vision of a better future. The program has three major learning objectives: (1) learn the basic practices of leading and managing so that managers are capable of leading their workgroups to face challenges and achieve results; (2) create a work climate that supports staff motivation; and (3) create and sustain teams that are committed to continuously improving client services. Introduced during the project’s last year of operations, the LDP program trained participants from North West Province and from Mpumalanga Province. Interventions In responding to the above objectives during the extension period, indicator tracking was weak, and meaningful targets were not set for all indicators. Quality assurance (QA) indicators were not tracked at all. The following two indicators were reported: DHIS: number of facilities with updated graphed indicators that are displayed; and District Health Systems: Number of districts with completed DHPs and DHERs. Progress on Selected Indicators Tracking the facilities that graph indicators and publicly displaying them appears to have started during the last year of the project. Since data is only available for one year, it is not possible to calculate a percent increase over time for this indicator. The target set for FY 2009/2010 was 10, and 68 facilities achieved this indicator. All eight IPHC-supported districts had completed DHPs and DHERs by FY 2006/2007. 22 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 15. Progress on USAID/IPHC Indicators for SPA 5. PHC Systems and Services Indicator FY 2006/2007 FY 2009/2010 % Increase FY 2009/2010 Target % Target Achieved 5.2 District Health Information System: Number of facilities with updated graphed indicators that are displayed -- 68 -- 10 680% 5.3 District Health Systems: # of districts with completed DHPs and DHERs 8 8 N/A 8 100% Achievements Improved management systems, including data management and supervision tools introduced during the EQUITY Project and strengthened during IPHC, formed the foundation from which health services were improved. Qualitative data collected during interviews with district and facility staff highlighted the following significant systems and services achievements: District Health Information System/Data Management Training in the DHIS has been conducted for IPHC technical and provincial staff. Sub-district and facility staff have gained awareness, through the IPHC project, of the value and power of data management and exhibit an increased commitment to the data management process. There has been improved data management process at district, sub-district and facility levels. IPHC has supported district health teams with the establishment of a routine process for DHIS data review, analysis, and feedback to facilities and sub-districts regarding the identification of trends and gaps in performance. The design and establishment of the data review process is an achievement. The next big step is the consistent implementation of this process; staff understanding and value of the process at the facility, sub-district and district levels; and ultimately, the institutionalization of the routine data review process. Joint visits by the IPHC team in collaboration with district and provincial supervisors were conducted in districts and facility clusters to support health service program reviews with the utilization of DHIS facility data. District Health Systems The IPHC Project has put systems in place to facilitate a patient-centered approach, increase access to services, and increase integration of PHC interventions. This is a significant achievement. At the same time, it is recognized that interventions often still exist within vertical program structures, and that as the systems facilitating integration are increasingly institutionalized, interventions will also become increasingly interconnected in clinic operations, resulting in improved integration of PHC delivery. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 23 PHC Review Process Consistent, institutionalized implementation of the PHC review process will take some time to achieve, but staff awareness and training in the process is a critical and significant first step. It is an achievement of the IPHC Project that facility and sub-district staff are familiar with and trained in the use of the IPHC tools (e.g., Clinic Supervision Manual, data management process, and PHC review process). Institutionalization of PHC review process tools will require ongoing support to staff as they encounter the inevitable challenges that tool utilization will present. With continued, accessible technical support, staff skills in tool utilization will mature and institutionalization of the tools will, in turn, take place. IPHC provided technical support to the district DOH to strengthen the alignment of district and provincial annual plans and budgets. Alignment efforts were also directed at the DHP and the DHER. With regard to the strengthening of the community component of the district health system, YFS have had a noteworthy, positive impact on the provision of reproductive health education to their youth peers in the community, bringing about increased numbers of youth presenting to facilities for care. Similarly, the OVC program has in its brief time, increased linkages between the community and the facility, community health center (CHC) and district hospital. Quality Assurance Training in QA techniques and strategies has been provided to staff at the district, sub-district, and facility levels. Consistent application of QA techniques is impacted by facility capacity: (e.g., human resources, material resources, and infrastructure), highlighting the need for continued efforts in these capacity-building areas. District management teams (DMTs) have been trained in the processes of supervision and monitoring of health services quality. QA capacity at the sub-district and facility levels must be further developed and alignment of the facility/sub-district/district levels requires further improvement before QA techniques and strategies can be institutionalized. Quality assurance capacity has increased as the data management and leadership capacity has increased at the sub-district and facility levels. This is an ongoing effort requiring ongoing technical support for the near-to-medium term. The experiential learning process of the LDP has introduced QA concepts and built capacity to utilize QA tools at the facility and sub-district level. The quarterly review process of routine DHIS data at the district and sub-district levels has been the focus of IPHC technical support, with the goal of achieving improved utilization and analysis of routine DHIS data. In turn, analysis of data is to be utilized to review progress with annual plans, identify performance gaps, and identify interventions for performance improvement. Effectiveness and Sustainability Based on interviews with respondents and field observations, Table 16 summarizes the effectiveness and sustainability with reference to four key indicators associated with PHC systems and services interventions. 24 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 16. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 5: PHC Systems and Services Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 1: Integrated Services Effectiveness Five green bars. Sustainability Four yellow bars. Initiative 2: PHC Review Process/Quality Assurance Effectiveness Six green bars. Sustainability Three yellow bars. Initiative 3: DHP/DHER Effectiveness Three yellow bars. Sustainability Two red bars. Initiative 4: LDP/Management Training Effectiveness Six green bars. Sustainability Three yellow bars. SPA 6: ORPHANS AND VULNERABLE CHILDREN With reference to the MSH Extension Proposal (September 2008—December 2010), the IPHC’s interventions for the benefit of orphans and vulnerable children (OVC–SPA 6) focused on one principal objective: Strengthen community-based organizations (CBOs) and networks to enhance the welfare of children affected by HIV and AIDS mortality in their families. Interventions In responding to the above objective during the extension period, IPHC tracked the following indicators: Number of OVCs served—three or more services Number of OVCs served—two or less services Number of providers/caregivers trained in caring for OVCs Progress on Selected Indicators Data collected for these indicators indicates significant progress in the number of OVCs receiving services during the IPHC project. OVCs receiving three or more services reached 13,062 in the last year of the project, nearly doubling the 7,156 OVCs receiving services four years earlier (See Table 17). Additionally, 6,134 OVCs received at least two services in the last year of the project. An area needing improvement appears to be training for OVC service providers and caretakers. Less than half of the target set for this indicator was achieved, and the USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 25 number of people trained decreased by nearly 50% over four years. This is likely due in part to the decrease in project funding for OVC activities during the two-year project extension period. Table 17. Progress On USAID/IPHC Indicators for SPA 6. Orphans and Other Vulnerable Children Indicator FY2006/2007 FY2009/2010 % Increase FY2009/2010 Target % Target Achieved Number of OVC served - 3 or more services 7,156 13,062 83% 15,000 87% Number of OVC served - 2 or less services 4,930 6,134 24% 5,000 123% Number of providers/caretakers trained in caring for OVC 477 242 (-49%) 500 48% Achievements Despite the fact that IPHC’s active involvement in the OVC programs ceased (with the exception of stipends through March 2010) prior to the extension of the program in October 2008, IPHC’s interventions are significant. Based on interviews with staff attached to OVC program sites visited, it appears that the IPHC interventions enhanced the OVC programs’ capacity to develop and manage the database and to maintain financial and administrative records. In turn, the development of OVC databases and IPHC assistance in the development of proposals has enabled OVC programs to successfully solicit government support. Staff from one of the OVC sites visited stated: ―Thanks to [IPHC] assistance, we now know who to approach and how to do it.‖ The IPHC-sponsored vocational training program appears to have been effective in building the income-generating capacity of OVCs to start their own businesses and become financially independent as young adults. The computer training, financial management skills, and entrepreneurial skills received through the vocational training program provided OVCs with the foundation to be financially secure. Community members, particularly parents and police, have been pleased with the changes seen in their communities as a result of IPHC-supported OVC programs. They volunteer for night duty and OVC events, and engage with problem-solving as needs arise. This community involvement and support has been encouraging for those delivering OVC services, and has strengthened their commitment to creating a better future for OVCs—―tomorrow’s leaders, nurses, teachers and pastors.‖ Effectiveness and Sustainability Based on interviews with respondents and field observations, Table 18 summarizes the effectiveness and sustainability of two key indicators associated with IPHC interventions for OVCs. 26 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Table 18. IPHC South Africa Evaluation: Assessment of Effectiveness and Sustainability of Strategic Performance Area Key Initiatives SPA 6: OVC Less than 40% 40–80 % Greater than 80% Less than40%: shaded red 40-80%: shaded yellow. Greater than 80%: shaded green. Initiative 1: Organizational Development Effectiveness Six green bars. Sustainability Five green bars. Initiative 2: Vocational Training Program Effectiveness Six green bars. Sustainability Three yellow bars. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 27 V. BEST PRACTICES In its review of IPHC interventions, the evaluation team has defined a ―best practice‖ as: A clearly defined intervention that is known to provide near optimum results every time it is correctly implemented. Accordingly, the team considers the following IPHC applications, interventions and tools to have met the above best practice criteria within the South African health context. CROSS-CUTTING IPHC PROJECT BEST PRACTICES Supportive Accountability Provided to all IPHC Supported Sites by IPHC Staff One high-value, transformative, capacity-building approach that IPHC staff provided DOH staff throughout the project was consistent, empathetic, positive, technical assistance. Coupled with this support was accountability. The linkage of positive support and accountability created an environment within which learning could occur, changes could be undertaken, and staff￾empowerment could be nurtured. The linkage of positive support with accountability motivated staff to complete initiatives, as they knew they were being held accountable to do so, and their work was verbally appreciated by IPHC staff. Tools to Guide the Implementation and Supervision of Integrated PHC Services IPHC provided a full range of tools to support and guide the complex process of PHC service integration at the facility level. As staff capacity was developed in the utilization of these tools, facility staff experienced the tools’ positive impact upon the provision of an integrated PHC package to the patient, and in turn attached increasing value to the use of IPHC tools. Data Management Throughout the project, IPHC staff provided training and ongoing technical support in data capture, reporting, and analysis for use in decision-making. The importance of accurate, timely data is now better understood by facility and sub-district staff. This is a significant step forward in the strengthening of health services and the provision of targeted health services. SPECIFIC BEST PRACTICES PHC Review Process The PHC review process is the foundation of health systems strengthening and quality improvement of PHC services. The PHC review process provides the structure for systematic data analysis. This, in turn, strengthens change implementation and monitoring as well as evaluation of the initiative, and improves the quality of the PHC services received by the patient. Mapping of Facility Catchment Area Facility staff has been trained and is now creating hand-drawn maps of their catchment areas, identifying the citing of resources, ―hot spots‖ of disease, and natural geography (rivers, mountains, etc). This visual aid assists planning of health interventions in the communities they serve. 28 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Clinic Health Committees The project has clarified clinic health committee members’ roles and responsibilities for both the community and the facility staff. This has empowered committee members to become more active liaisons between the community and facility, and has strengthened facility health outreach initiatives and community understanding of health facility resources. Technical Assistance Provided to OVC Programs The best-practice component of the project’s support of OVC programs was that support was directed at existing programs and structures. This approach strengthens existing networks and linkages; a more sustainable development strategy than the traditional supply-driven ―silo‖ structure of many development initiatives. The community-facility linkages were strengthened, OVC management staff was capacitated, and youth-friendly services, community health workers (CHWs), home-based caregivers (HBCs) and youth mentors were supported with training, financial and material support. The OVC programs deserve commendation for their effective efforts in working toward ―an HIV and AIDS-free generation.‖ Establishment of Support Groups for HIV and AIDS Patients This intervention has facilitated information sharing around mutual problems, boosted patient morale, and reduced the isolation of HIV and AIDS patients. Community-facility relationship was strengthened, with each having a better understanding and appreciation of the other. The improved community-facility relationship, in turn, increased community utilization of facility services and support of health campaigns in the community. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 29 VI. STRENGTHS AND WEAKNESSES As required under this evaluation’s scope of work, the evaluation team has examined the strengths and weaknesses associated with the IPHC’s implementation of the program. Accordingly, the following paragraphs represent our assessment of the IPHC’s strengths, broken down into six discrete categories, and of the IPHC’s weaknesses, broken down into an additional five categories. IPHC STRENGTHS Strengthening of Management and Supervision In all district and sub-district offices, hospitals, community health centers, PHC clinics and OVC community organizations visited as part of this evaluation, respondents said IPHC’s greatest strength is the provision of management structures and tools to manage health care delivery, and the provision of clinical guidelines, checklists and registers to guide and track service delivery. As reported by the evaluation’s respondents, standardized application of the clinic supervision manual, initially developed under the Equity Project and modified under IPHC, has been a major force in assisting health care providers in maintaining a sustained and focused approach to the integrated management and provision of HIV, AIDS, TB, PMTCT, STIs and IMCI. At all facilities visited by the evaluation team, standardized use of registers has now become the norm rather than the exception. The IPHC has ensured that proper recording of health services delivered within the district is performed in a timely, quality manner. Indeed, a number of staff from different facilities cited their learning from IPHC, saying ―…if it was not recorded, it was not done.‖ In addition, introduction of the use and application of the PHC Review Process— from the level of the facility up to and including district and sub-district levels—has resulted in PHC being recognized as a vital part of district health service delivery responsibilities. Finally, in response to high staff turnover within many of the facilities visited, the IPHC was able to assist in getting new staff trained, mentored, and functioning effectively while still managing to assist facilities in meeting their technical targets. Training In addition to its work with staff on institutionalizing the PHC review process, IPHC’s introduction of the LDP was cited as a significant IPHC strength. Although regrettably introduced during the last year of the project’s implementation, the LDP was cited by course participants as having a significant impact in terms of building their confidence and empowering them in their ability to manage programs and bring about effective change. In recognizing facilities’ needs for capacity-building in HCT, IPHC instituted an on-the-job training program focused on ensuring that all facility nurses are able to independently and effectively carry out counseling and testing. In addition, IPHC training support on HIV, AIDS, STI, and TB (HAST) has led to widespread integration of HAST and FP diagnosis and treatment within those clinics supported by IPHC. Finally, IPHC’s training in the management and administration of OVC initiatives has resulted in the OVC program’s ability to manage their administration and to successfully solicit financial and material support for their programs. As reported by one of the OVC program managers, ―Thanks to their [IPHC] assistance we now know who to approach and how to do it.‖ 30 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Data Management In assisting supported facilities with training and implementation associated with the clinic supervision manual and with the PHC review process, IPHC devoted a significant amount of effort in developing the capacity of facility to understand and work with data associated with these two initiatives. Based on the evaluation team’s assessment, it was evident that a significant number of facility staff ―own‖ their data, are able to analyze it, and to develop initiatives and action plans that are directly associated with the data they have captured and reported. In addition, staff at facilities noted that with IPHC assistance, the time devoted to the completion of DHP-related data has been reduced from as much as three months for the creation of a DHP to less than three days. Establishing Linkages with the Community One of the hallmarks of the IPHC is its success in developing effective linkages between the facilities and the communities they serve. For example, IPHC guidance in the implementation of clinic health committees was often cited as leading to a marked improvement in community relations and to an increased knowledge-based role for the community in assisting facilities respond to health care priorities. Establishment of the committees led to the formation of community-supported home-based TB DOTS programs and to more effective integration of HIV and AIDS community support groups within the community and with the facilities themselves. Through the IPHC project, community health workers were trained to follow-up with PMTCT for every child at risk in the community. Traditional healers were trained in the promotion of immunizations, oral rehydration, and identification of childhood illnesses requiring clinic referrals. Finally, the IPHC’s introduction, training, and support of youth mentors led to a marked increase in the willingness of young people to be tested and counseled on HAST, and to seek out information related to FP/RH. IPHC Specific Impact Given that the project was called upon to develop initiatives in a limited number of facilities within a limited number of districts, and given that due to confounding variables it is difficult to identify the true impact of IPHC interventions, even within their assigned facilities, any statement of facility specific impact would be questionable. However, the IPHC did succeed in developing individual ―pockets of excellence‖ within the districts to which it was assigned. In addition, facility records indicate that following IPHC interventions, facility staff recorded IPHC￾related improvements in selected indicators. For example, following IPHC training, one clinic noted an increase in counseling and testing in ANC sessions from 62% to 96%. In another clinic, ANC visits increased from 94% to 100%, and PHC visits reached 100% among HIV-positive mothers following IPHC’s training interventions. In a third clinic, CD4 counts are now taken regularly, enabling staff to identify patients due for ARV and Zidovudine (AZT). Finally, following IPHC management and administration training, one of the OVC projects was successful in securing a grant of R450,000 from the Embassy of Japan for a new vocational training center. Technical Assistance Management Style In assessing the process by which IPHC staff provided technical assistance, the evaluation team was impressed by respondents’ appreciation for the passion and technical competence of IPHC staff as they sought to strengthen clinic staff technical capacity both in management and delivery of services. Consistently, clinic staff noted the significance of IPHC staff competence and dedication to the achievement of results centered on the value of positive reinforcement and a focused and informed approach to problem-solving. Accordingly, the IPHC’s passion for TA coupled with IPHC’s technical competence was crucial to all success achieved. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 31 WEAKNESSES Questionable Sustainability Despite the IPHC’s demonstrated and documented success in introducing effective health system strengthening interventions, the evaluation team was concerned about the lack of planning and action to strengthen the sustainability of IPHC initiatives prior to the project’s scheduled December 2010 termination. While this weakness will be addressed in the report’s recommendations, the evaluation team believes that the absence of an exit plan, including specification of milestones along the path toward project completion, has severely compromised the ability of the DOH to sustain IPHC initiatives. In addition, the absence of an exit plan led to an abrupt cessation of support for IPHC initiatives, such as those associated with the youth mentor program and the OVC programs. It also led to termination of activities essential to the project’s success, especially those focused on quality improvement under URC and on systems development under HST. Poor Analytical Base In its review of project documentation, the evaluation team could find no objective analytical base upon which to evaluate the effectiveness or impact of the project’s initiatives. For a project focused on health systems strengthening, the evaluation team expected an operations research component or independently commissioned qualitative evaluations by which to measure the importance, if not the impact, of IPHC interventions. The reports that did exist were generally self promotional in nature and lacking in objectivity. Finally, project indicators were generally without meaning, as they focused on outputs rather than outcomes and appeared to have low achievement expectations in order to guarantee ―success.‖ Limited Provincial or District Engagement: The evaluation team acknowledges that MSH had limited control over the project’s facility-based focus. However, in focusing explicitly on facility interventions, the project’s design was flawed in that it neglected the importance of ensuring that district and sub-district program managers were fully engaged in the project’s activities and had a vested interest in working toward the success and sustainability of IPHC initiatives. 32 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 33 VII. CONSTRAINTS The evaluation team recognizes that issues beyond the IPHC project’s control have constrained its progress and the effectiveness and sustainability of its initiatives. The following paragraphs identify and discuss the five key constraints that appear to have had the most significant impact on its implementation. Human Resources South Africa struggles with a chronic deficit in human resources for health, and this deficit impacts all aspects of integrated primary care delivery in the country. One strategy for responding to this constraint is to move from facility-based PHC delivery to community-based PHC delivery with facility support. This change in service delivery model holds potential for increased access to health care for the community and redistributes available human resource capacity for greater impact. Another strategy to address this constraint is to gradually move toward a rationalization of the current DOH personnel ―pyramid‖ by reducing the top-heavy structure at the top of the pyramid (province and national) in order to provide more (and less costly) personnel at the facility level. Increased numbers of trained facility staff are required for the delivery of integrated PHC services throughout the country. Limited Program Management Expertise As a result of its project design, IPHC focused more on operations and management systems at the facility level than on strengthening the capacity of DOH leaders at all levels to manage and direct improvement of health services and health service delivery systems. As a result, program managers at the district and sub-district level have struggled to fully understand the management requirements of IPHC initiatives. Provincial and national DOH leaders have not been as engaged with the project as needed for institutionalization and sustainability of achievements. However, while not part of the project’s design or its workplan, MSH’s recent introduction of the LDP appears to have been highly successful in equipping a small but motivated cadre of health managers with basic skills in management and leadership. Institutionalization of an accredited LDP training program for health service managers may mitigate this constraint of limited management and leadership capacity within the DOH. Staff Turnover If the IPHC’s activities (or any development initiatives) are to be sustained at the facility level, chronic staff turnover must be addressed. One strategy to address this constraint is to develop an enhanced employment package that provides housing, schooling, and other essential quality of life resources for DOH staff, especially in rural communities which struggle to recruit and retain skilled facility staff. Culture of Dependency As expressed by facility staff, the willingness of trained staff to accept responsibility for sustaining activities introduced by IPHC is limited—due in part to a donor/recipient culture that anticipates that when one donor leaves, another donor will appear. This cultural mindset applies equally to managers at all levels of the government. One strategy to address this constraint is to insist that all development projects, at their onset, have a SAG-approved exit plan based upon the premise that, as projects implement their TA initiatives, government counterparts assigned to the project are being proactively mentored and capacitated to assume responsibility at the completion of the project. 34 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Cultural and Religious Constraints There is a reluctance of some facility staff to offer certain health care services such as family planning or termination of pregnancies, due to conflicts with their cultural and/or religious beliefs. One possible approach to alleviating this is to strengthen the quality and frequency of supportive supervision to ensure that operations managers and supervisors are trained to provide counseling to staff to address cultural and religious constraints, in the hopes of achieving greater alignment between personal values and personal health needs. This task is a delicate undertaking but one that is required in the interest of ensuring that clients are offered and receive a truly integrated package of PHC services as provided for in government policy. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 35 VIII. LESSONS LEARNED Management and leadership are essential skills for successful implementation and sustainability of PHC initiatives Tools, protocols, guidelines and training itself cannot be implemented effectively and sustained without skilled management providing consistent and supportive accountability. Leadership is essential to motivate and sustain staff commitment and energy. Ongoing clarification regarding how initiatives are tied to achievement of the integrated PHC vision is also essential. Ready access to a mentor is important for capacity development and skills transfer. People need role models that reflect, in their mindset and behavior, the values that the DOH aspires to are a commitment of the individual to the betterment of the community; commitment to one’s work within the DOH in order to improve PHC quality and access for all. Mentoring and role modeling starts with each manager at the national level and impacts every manager at every level below. Individual discipline and accountability matters. Regular use of skills is required to retain those skills. Training is an ongoing process. Without regular use of skills and ongoing, frequent ―in-servicing‖ and development of skills, capacity will dissipate over time and sustainability will be compromised. Team work, communication, and feedback are key. These three things are key to maintaining good quality systems and services. Vertical or ―silo‖ efforts are not effective. Everyone must understand his/her own role and responsibilities and those of every team member, in order to value and support the efforts of each member. Communication and feedback regarding actions taken, challenges encountered, and strategies utilized are all essential if the team is to be successful. Information sharing is empowering to all. Alignment and linkages between all levels of the DOH are essential for nationwide integrated PHC that is sustainable. Without alignment, each level of the DOH is moving in a different direction rather than supporting, understanding and valuing each level’s critical role in the building of an integrated PHC delivery system. Critical importance of accurate, timely data for utilization in informed decision￾making. Effective decision-making cannot occur without accurate, timely data. Likewise, monitoring and evaluation of actions taken cannot occur, and ultimately improvements in service delivery cannot be sustained without accurate, timely data providing feedback to inform staff of progress. Information regarding the ―why‖ of problem solving is required to inform decision-making based on evidence, to guide ongoing efforts to achieve and sustain quality services. An integrated information system for use by all levels of the DOH nationwide is essential to all health service provision, including integrated PHC service provision. Lack of standardized key indicators and DOH training at all levels has produced poor data. Therefore, quality data management is constrained and its use in decision-making weakens the quality of health care services. 36 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Community involvement is a key to the successful development, implementation and sustainability of integrated PHC service. Weak linkages between the facility and the community inhibit the provision of quality PHC services. Youth respond to their peers. With this understanding, the utilization of youth mentors and YFS is an effective strategy. Access to PHC services is constrained by the lack of peer outreach to provide health education and linkage to the clinic for diagnosis and treatment of HAST. Health staff seek quality of life for themselves and their families. Accordingly, attention to employment packages that address the housing, education, infrastructure, and other needs of health staff and their families strengthens recruitment and retention. This is especially true in under-resourced rural areas. For integrated PHC to occur, facilities and community health workers must be equipped to provide integrated education, diagnostic and treatment services for patients. The success of cross-screening of HAST and PMTCT and the resultant reduction in ―missed opportunities‖ is a dramatic example of the significance of this lesson learned. Health initiatives that work with existing organizations and structures, rather than creating “silos” of effort, strengthen the communities and patients they serve. Cultural and religious beliefs have an impact on the provision, perception and sustainability of care. Those providing effective PHC interventions must be aware of cultural and religious beliefs and work with community leadership to facilitate quality care. Support groups are effective structures in the provision of health education, problem solving, reduction of stigma, morale-boosting and nurturing of hope and a sense of well-being. HIV and AIDS support groups are testaments to the effectiveness and value of such groups. It is essential that project design and exit plans, with identified funding and human resources, be signed off on by the funder, contractor, and DOH before project approval. Failure to provide for a systematic, planned exit of a donor-supported program is the greatest constraint on sustainability and raises doubt as to the value of a donor’s intervention. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 37 IX. DISCUSSION Over the course of the IPHC external evaluation, team members noted ―overarching issues‖ that impacted the entire breadth and depth of the IPHC project, and indeed impact health development efforts throughout South Africa. These issues are identified below, with the hope that progress within the DOH, as well as with future health development projects, will be aided by awareness of these larger challenges so that targeted interventions can be developed to effectively address them. ALIGNMENT BETWEEN DONOR/CONTRACTOR/DOH DOH policies and priorities should guide the design of all development projects based on demand generated by the DOH, not supply generated by the donor and contractor (e.g., youth mentors introduced without DOH ―buy-in‖ to absorb/fund/sustain role). Thorough, specific, exit strategies within an agreed-upon timeframe should be a requirement of all project design. The donor, contractor and DOH must agree to all aspects of the exit strategy before a project is approved. Operational research needs to be integrated into all project design to ensure that relevant and accurate data is obtained. Well-designed operational research initiatives implemented throughout the project will ensure meaningful data analysis of interventions and their impact at critical stages and at the project’s final assessment. Targeted TA should be secured through donors and contractors, with awareness that, although the ―building blocks‖ of the last 16 years have indeed created the foundation for integrated PHC (e.g., tools, manuals, processes), TA requirements must take on a new dimension focused on sustained development of interventions. Targeted TA that develops capacity at all levels of DOH to effectively implement, manage, and utilize the interventions of the past 16 years is currently needed and must be reflected in all future project designs. Alignment of technical assistance with specifically-named DOH recipients should be considered a priority before the launch of a development initiative or the assignment of technical advisors. In addition to strong technical skills, strong negotiation skills and emotional intelligence are all essential for effective TA provision. HUMAN RESOURCES NGO Resources In order to rationalize and strategically target development assistance, the NDOH should first identify all organizations providing health development assistance in the country. Currently, hundreds of NGOs are providing assistance throughout South Africa. As part of the rationalization process, the DOH should analyze the resources provided by all organizations supporting it and identify and eliminate overlaps. During this process, the DOH can identify gaps in resources required to facilitate implementation of its policy and develop a strategic plan to be utilized by the DOH, donors, and contractors as a common source for identifying where targeted technical assistance can best be applied. DOH Human Resources Inequities exist in human resources allocation throughout all levels of DOH. There is a need to harmonize human resources nationwide to achieve effective policy implementation. 38 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION DOH’s human resources allocation profile is top-heavy. It has a ―V‖ shape, with excessive staff at the top of the staff pyramid and insufficient staff and capacity at the sub-district and facility level to implement the initiatives intended to realize the health policies crafted by higher levels. Job descriptions are often inaccurate and out-of-date. In addition to delineation of tasks, job descriptions should clearly state the position’s authority and the resources allocated to the position to support the successful conduct of responsibilities. MANAGEMENT AND LEADERSHIP CAPACITY Targeted TA in management and leadership skills is a significant need in developing DOH staff and their capacity to implement and sustain the key ―building blocks‖ of the last 16 years. Although district health plans, DHERs, the clinic supervision manual, PHC review process, and data management (to name a few) are all clearly delineated, DOH at all levels does not have the full complement of management and leadership skills to help staff take these tools and processes and implement them effectively, monitor implementation, and sustain impact. Guiding and supporting staff to implement, monitor, and sustain the tools and processes requires the ability to exercise effective management skills such as team development, motivation, delegation, supportive accountability, and constructive criticism grounded in positive supervision as opposed to negative fault-finding. Positive, skilled managerial role models are needed. Managers at all levels currently struggle with a lack of self-discipline and accountability. This is a negative model for others. Academic institutions need to develop a standardized, accredited health management curriculum and degree program that provides comprehensive management and leadership skills along with statistics and epidemiology, and produces professionally trained health managers for the district, provincial and national levels. The LDP is an effective, ongoing management capacity-building tool for use at the facility and sub-district levels that emphasizes the team over the individual and helps individuals move from a mindset of dependency to empowerment. Basic management skills are provided in an experiential setting to strengthen retention of management capacity. A NATIONWIDE CULTURE OF ENTITLEMENT AND DEPENDENCY People in different settings noted that some South Africans are losing values and lessons learned from the struggle against apartheid. This is especially true for values that included a social contract amongst citizens to improve the lives of all. Increasingly the perspective is moving from the ―good of all‖ to the betterment of self. Health development depends on a shared vision and commitment to ensure quality services for all. Teamwork is essential for success. Respondents also noted that follow-on support was expected from donors. CULTURAL AND RELIGIOUS BELIEFS PRESENT A SIGNIFICANT BARRIER TO FULLY-INTEGRATED PHC Family planning is believed to be a crime or, at the very least, is often not supported in more conservative areas by the community or by clinic staff. There is a need for targeted, collaborative TA between the departments of social development and health to address this challenge. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 39 X. RECOMMENDATIONS The IPHC Project and the Equity Project, upon which it builds, have together achieved significant progress over the more than 12 years of USAID technical assistance to the SAG. The projects’ overarching technical focus been the introduction of initiatives designed to strengthen the nation’s PHC system in response to the many challenges associated with HIV and AIDS. The recommendations proposed by the IPHC external evaluation team are presented here as actionable interventions that the DOH can utilize to build upon the effectiveness and sustainability of IPHC interventions. Each recommendation below is placed in context of the issue being addressed. Where applicable, each recommendation includes suggestions regarding ways in which targeted TA can be applied to facilitate the achievement of a specific recommendation. IMPROVE THE USE OF DATA FOR DECISION-MAKING PURPOSES Recommendations District Managers should examine the validity and reliability of DHIS and develop and implement an action plan to address identified weaknesses. Current registers should be consolidated. Development of electronic registers at all levels should be a medium-term (five-year) development goal with IT systems developed at the facility level; Within the five-year development period, the government should budget for and recruit data capturers to provide technical support for all facilities. Current health indicators should be rationalized to ensure relevance to management, monitoring and evaluation needs. Training in the health information systems should be included in the curriculum of all health service providers including, but not limited to, that of doctors and nurses. Context Use of facility-level data was observed through the display of catchment area maps drawn and wall-posted graphs and charts for various service-delivery indicators. However, in MSH’s quantitative internal assessment report, flaws in data extracted from DHIS system were noted. Since record keeping at the facility level is paper-based and transfer of data from the facility to the district level DHIS system occurs manually, errors are possible in data transfer. This is due to either inadequate data quality checks or lack of coordination between facility and district/sub￾district office. Additionally, at the facility level there are too many registers for a particular service, leading to having patient’s information at different locations. This makes collation of data difficult, including tracking of patients for follow-up. Recommended Technical Assistance TA should be directed toward (i) a data-quality audit of the DHIS and data-collection and transfer methodology, including an examination and rationalization of current health indicators and should be undertaken to inform the actions required to increase validity and reliability of data; (ii) the development and implementation, initially for use in a pilot, of a framework for a comprehensive electronic patient and facility records management system with unique patient identification for improved patient tracking and follow-up; (iii) the development of job descriptions for facility-level data capturers; and (iv) a review and upgrading of current entry- 40 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION level curricula for health providers to include training materials focused on the health provider role with regards to the maintenance of health information systems. IMPROVE SYSTEM-WIDE ACCOUNTABILITY Recommendation A system-wide organizational development study should be commissioned to assess and document current health system organizational strengths and weaknesses and develop a strategy and action plan to address weaknesses focused on the standardized application of national policies and guidelines. Context Currently, the health care system is dysfunctional in the sense that, from province to province and from district to district, implementation of national policy and guidelines and standardized application of decentralized management is not universally applied. The result is a nationwide lack of a systematic approach that utilizes standardized strategies and tools to effectively address the challenge of HIV and AIDS within South Africa. Recommended Technical Assistance Conduct an organizational development study and develop a concomitant strategy and action plan to address identified DOH organizational gaps and weaknesses. IMPROVE THE UPTAKE AND INTEGRATION OF FP/RH INTO PHC SERVICES Recommendation An FP/RH promotion communication strategy should be integrated into PHC services. The long￾term focused strategy should be responsive to individual behavior change needs and should maximize the potential for change on a broader societal level. Toward that end, family planning should be made one of the entry points for HIV prevention. Context The uptake of cervical cancer screening and modern contraceptive methods, particularly the intrauterine contraceptive device (IUD), is very low. Cultural and religious norms and beliefs are reported as a major hindrance to uptake. It is important to understand both the client and health provider related factors that affect acceptance and refusal of FP/RH services. Recommended Technical Assistance Knowledge, attitude, and practices/usage (KAP) surveys among clients and service providers, including the role of religion and culture in the use of FP/RH services, should be carried out to understand the myths, misconceptions, and barriers related to the uptake of FP/RH. Using KAP survey results, targeted communication messages should be designed to increase the uptake of FP/RH services and methods. This survey should also include health system issues, skills, and competencies of community health workers as well as an assessment of knowledge regarding the linkage between FP and HIV and AIDS. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 41 IMPROVE THE QUALITY OF SENIOR-LEVEL TECHNICAL SUPERVISION Recommendations An accredited and accelerated master’s level curriculum, with an emphasis on PHC management (including statistics and epidemiology), should be developed within South Africa’s academic institutions to address the need for professional training for all PHC managers at the level of sub-district and above. All current PHC managers at the level of sub-district and above (including the national level) should be scheduled to undertake the accelerated curriculum. Successful completion of the post-graduate (NQF Level 6) level PHC Management curriculum should be a prerequisite for entry of DOH employees to supervisory and managerial positions. This would apply to supervisors and managers who work outside the PHC setting, as PHC is the backbone of South Africa’s health care delivery system. Context Staff responsible for supervising and managing the nation’s PHC program frequently lack the necessary management and technical qualifications and orientation to effectively supervise PHC initiatives being implemented at facility level. Recommended Technical Assistance TA should be directed toward (i) an assessment of the current level of PHC training among existing PHC managers to identify current gaps in training; (ii) a review of the extent to which current academic curricula provide PHC training and have the capacity to address gaps identified; ( iii) an engagement with selected academic institutions to develop an accredited PHC management curriculum to respond to the training needs of PHC managers; and (iv) the development of a long-term (five-year) plan to train existing managers and to provide training for employees as they are nominated for PHC management positions IMPROVE THE EFFECTIVENESS OF SUPPORTIVE SUPERVISION Recommendations: The DOH implement a standardized LDP based on the MSH model with the LDP being adapted to the specific levels and needs of managers. MSH designs a LDP curriculum to be accredited in South Africa and utilized to develop a cadre of LDP facilitators in South Africa. Although MSH has recently trained 20 LDP trainers, the current MSH TOT process was not designed for long-term post-training support (at least 12 months) and for the South African certification of those who were trained. The DOH introduces the LDP nationally at the sub-district and facility levels. District health management teams, provincial program managers and national directors should all receive orientation to the LDP. Following orientation to the PHC review process, national and provincial-level directors and managers should conduct quarterly supportive supervision visits to facilities. Directors and managers of both levels should utilize PHC review process guidelines during their supervisory visits. As a KPA indicator, the sub-district manager should be responsible for ensuring that all facility staff are trained in and implement changes in technical protocols in a timely manner 42 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION All new health service delivery staff should participate in a standardized orientation program that addresses all aspects of health service delivery at the facility level. New managers should receive LDP training within six months of their appointments. As a KPA indicator for the facility operations manager, consistent provision of a standardized orientation program should be administered to all staff. Context Supervision by staff from sub-district level and above is frequently negative and heavy-handed in its nature, and thus ultimately counter-productive to sustained improvement in staff performance. DOH staff at all levels have developed a ―culture of dependency‖ based on their reliance on outside TA to provide the effective management and leadership skills so essential to sustained quality improvements. Once institutionalized and rolled-out nationally by certified trainers, the recommended LDP training should significantly reduce DOH reliance on outside TA to provide effective supportive supervision, management, and leadership. In addition, sub￾district managers are not sufficiently engaged currently to ensure that facility staff stay up to date on best practices for the provision of integrated PHC, and that newly-assigned facility staff are adequately oriented to technical guidelines. Recommended Technical Assistance Given the specific nature and unique strengths of LDP training, MSH should be engaged to work with the SAG to develop a certified LDP training. MSH should be further engaged to ensure that a sufficient quantity of LDP trainers is developed to roll out LDP training nationwide. TA should assist the DOH in the development of a standard orientation and training package for newly￾appointed facility staff. ADDRESS HUMAN RESOURCE FOR HEALTH CHALLENGES Recommendations Review and standardize post-specific job descriptions for all levels. Job description review should be focused on clarification of responsibilities and on harmonization (e.g., removing duplications and overlap) among posts. Develop training courses for operations managers to build their capacity to ―work smart,‖ that is, to work effectively with the limited time and resources available. Develop a simple handbook of standard operating procedures based on an analysis of key tasks, responsibilities, authority, and resources for district and sub-district managers as a daily reference guide. Develop a long-term strategy and action plan to rationalize current DOH staffing patterns at all levels. Standardize organograms for each specific establishment level. Develop system-wide job-specific academic and competency requirements for DOH staff. Develop human resource incentive packages targeted to rural-area services. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 43 Context Human resources for health (HRH) is a complex issue and the DOH is currently engaged in a long-term re-engineering process, partially addressing this challenge. The current top-heavy staffing pyramid must be addressed in order to align the allocation of staff according to need at each level of the DOH. Re-allocation of staff is also required for improved fiscal responsibility. Recommended Technical Assistance TA should be directed toward: (i) review and modification of current job descriptions; (ii) development and implementation of a ―work smart‖ training curriculum for facility operations managers; (iii) performance of a job function review for district and sub-district managers with a focus on developing a standard operating procedures handbook for use by these managers; (iv) design of developmental competency guide and checklist for use by supported facilities in moving toward ―graduation‖ from one competency level to the next; and (v) development of a forum for learning for operational managers to extend learning to other sub districts and districts, to facilitate leveraged learning and cross-pollination. REMOVE BARRIERS TO THE EFFECTIVENESS OF THE PHC REVIEW PROCESS Recommendation A computerized program should be developed for the production of summary ―dashboards‖ linked directly to PHC review process data. Context As currently implemented, the presentation of exhaustive amounts of data during the PHC monthly review process makes it difficult for participants to identify areas of concern that require action. Linking the initial input of data to the production of summary dashboards will enable participants to focus on areas of concern while identifying for special notice those facilities that have achieved quality results on established targets. Recommended Technical Assistance TA should be directed toward: (i) developing and piloting a standardized PHC Review process; and (ii) producing computerized PHC review-linked summary dashboards. ENHANCE THE SYNERGY AMONG ALL LEVELS: NATIONAL TO COMMUNITY Recommendations The NDOH should undertake an exercise to map all PHC programs in South Africa, focused on the development of a strategy to identify and reduce program overlap (harmonization) and, by utilizing gap analysis, to identify and respond to the needs of underserved areas. As a KPA indicator, national and provincial level staff program managers should be trained in PHC. Using the supervisory manual as a guide, they should also be required to undertake quarterly (for national and provincial staff) and monthly (for district and sub-district staff) supportive supervisory visits to health facilities. 44 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION As part of their ―work smart‖ training course, facility operations managers should be provided with an orientation on the linkage between service delivery and national policy on PHC. The NDOH should review, adapt, and apply the MSH-developed operations manual for facility-level catchment area mapping. This should include the location of villages; availability of care-givers, CHWs and NGOs; the location of OVCs; and current PHC response priorities such as low immunization rates, prevalence and incidence of TB. HIV, AIDS, diarrheal outbreaks and the absence of a clean water supply. The adapted manual should be included as part of the standardized training and orientation program for all new employees. NDOH and PDOH should have a standardized guide for providing support to the district level. This sub-recommendation includes a focus on an integrated planning process between national and planning level authorities. Context At all DOH levels there is a pervasive lack of synergy regarding a common understanding and commitment to health policies and priorities. Without a common vision supported by a unified understanding of the policies to support that vision, initiatives focused on health systems strengthening will be severely constrained. Recommended Technical Assistance TA should be directed toward: (i) assisting with harmonization and gap analysis and with the development of a strategy and action plan to rationalize gaps; (ii) inclusion of an orientation for staff that links policy with service delivery in the ―work smart‖ training program for facility operations managers; (iii) development of a standardized ―mapping‖ manual for facility operations managers; and (iv) development of an NDOH and PDOH standard guide regarding support to district level that includes guidelines for developing annual plans for district-level visits aligned with all strategic programs as part of the annual planning process. ENHANCE THE PRACTICAL APPLICATION OF THE DHER AND THE DHP Recommendation The DHP should be reduced to essential action-oriented interventions complete with measurable indicators with a clear linkage to the DHER and available resources. Context For planning purposes, there is minimal linkage between the DHER and the DHP. From the standpoint of operations applicability, the DHP is not viewed as a useful operational tool, as it frequently does not include input from or participation of international development partners. Recommended Technical Assistance TA should be directed toward: (i) an analysis regarding DHER and DHP development and utilization and (ii) development of an action plan to improve the utility and application of both documents. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 45 STRENGTHEN EFFECTIVENESS AND SUSTAINABILITY OF DONOR ASSISTED INITIATIVES Recommendations The NDOH and all provinces included in a project should agree to all initiatives within donor-assisted projects as part of the project procurement process. Development partners and government authorities should conduct needs and gap analyses to determine where available resources can be most effectively utilized. The practicality of developing memoranda of understanding (MOUs) between international development partners and provincial and district-level authorities should be explored, supported by all parties, and as developed, reviewed annually by all signatories. All contractors should be required to develop an exit plan with clearly defined milestones within the initial three-month project implementation phase. All projects should be required to develop baseline data for use in periodic and end-of￾project assessments and evaluations. All contractors should be required to prepare and present to the government and to USAID a quarterly review of progress against established milestones. All contractors should be required to prepare an action plan to respond to those areas in which identified milestones are behind schedule. At the highest level of project implementation, a specific government official should be identified and actively engaged as a project implementation counterpart to participate in monthly project reviews and in quarterly reviews noted above. Context During the design phase of the project, little attention was focused on the importance of developing an exit plan to ensure that the government agreed to all IPHC initiatives and that initiatives would be sustained following the project’s closure. As a result, the great majority of IPHC initiatives, while largely effective and well-executed, hold little promise of sustainability. 46 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 47 ANNEX A. SCOPE OF WORK Global Health Technical Assistance Project GH Tech Contract No. GHS-I-00-05-00005-00 SCOPE OF WORK (Revised 9-27-10) I. TITLE Activity: USAID/South Africa: Integrated Primary Health Care Project End of Project Evaluation Contract: Global Health Technical Assistance Project (GH Tech), Task Order No. 01 II. PERFORMANCE PERIOD O/a October 11th, 2010- o/a December 31st, 2010 III. FUNDING SOURCE Mission IV. PURPOSE The Health Office of USAID/Southern Africa requests technical assistance to implement an end of project evaluation of the five-year Integrated Primary Health Care (IPHC) Project. V. BACKGROUND The Integrated Primary Health Care (IPHC) project in South Africa is a collaborative project between the South African National Department of Health (NDoH); five provincial Departments of Health (DoH), Eastern Cape, KwaZulu Natal, Limpopo, Mpumalanga, and North West; and eight selected districts of these provinces; and the United States Agency for International Development (USAID) in South Africa. This task order under the TASC II IQC began in July 2004 and currently has an end date of 30 December 2010 with Management Sciences for Health (MSH) as the prime contractor. Partners over the life of the project included Health Systems Trust (HST) and University Research Corporation (URC). IPHC is designed to improve access to and use of child health, reproductive health and HIV/AIDS services with an emphasis on improving the management systems at the district level and in selected facilities in those districts. The project provides direct support to nearly 70 health facilities and to the management teams of the eight districts; to 14 hospitals and three community health centers accredited for provision of anti-retroviral (ARV) therapy; and to CBOs for support of orphans and other vulnerable children (OVC). A two-year project extension, awarded in 2008, continued the focus on the original six strategic performance areas of maternal health and family planning, child health, youth program, HIV & AIDS, primary health care services and systems, and OVCs; and added the dimensions of consolidation, integration, and institutionalization to these initiatives while also encouraging expansion and replication of project successes in other districts, facilities, and communities. 48 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION VI. OVERVIEW OF THE IPHC PROJECT MSH undertook an internal assessment of the IPHC project in January 2008. The report of the assessment will be provided as an important background document as well as a performance baseline for this end-of-project (EOP) evaluation. IPHC has also undertaken an internal qualitative evaluation, interviewing provincial, district, sub-district and facility staff. This will also be provided to the evaluation team. Below is the vision that reflects the main performance areas of the agreement with USAID. The IPHC project envisioned that as a result of its efforts, the emergence of functional district health systems capable of providing accessible, high-quality HIV & AIDS interventions that are fully integrated into primary health care services for the benefit of disadvantaged communities, thus offering previously disadvantaged South Africans a better quality of life. The technical interventions of the project were designed to contribute to the above goal by addressing the following intermediate results (IRs): IR 1 Strengthened HIV and AIDS prevention measures IR 2 Increased availability of quality STI services IR 3 Improved treatment of TB and AIDS IR 4 Expanded HIV/AIDS care and support IR 5 Expanded systems and services of selected primary health care systems. The IPHC Project reports on the following strategic performance areas (SPA): SPA1: Improve Maternal Health and Family Planning, with Emphasis on Youth SPA2: Improve Child Survival, Health, and Nutrition SPA3: Increase Youth Participation in Promotion and Use of Youth-Friendly Services SPA4: Reduce the Impact of HIV & AIDS – SPA4.1: Counseling and Testing (C&T) – SPA4.2: Prevention of Mother-to-Child Transmission – SPA4.3: Comprehensive Care, Management, and Treatment – SPA4.4: Palliative Care in the OVC Setting; Integration of TB and HIV SPA 5: Strengthening Primary Health Care Systems and Services – SPA 5.1: Quality Assurance, Clinic Supervision and District Development – SPA 5.2: District Health Information System – SPA 5.3: District Health Systems – SPA6: Strengthen Community Support and Participation for OVC. VII. SCOPE OF WORK The main purpose of this EOP evaluation will focus on assessing the effectiveness, efficiency, and quality of the IPHC project interventions at the facility and district level; identifying what has been successfully incorporated into the DOH’s ongoing programs and what challenges remain; establishing evidence of project results and impact, and providing lessons and recommendations for the planning and management of future projects that focus on health system strengthening. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 49 The objectives for this evaluation will be to: Review project results (outputs and outcomes) in relation to the project’s strategic performance areas (SPA) and baseline status or comparison areas/facilities/organizations for each SPA (where available) and identify operational constraints encountered; Assess the project’s strengths, weaknesses, gaps in service delivery, and any constraints to successful implementation; and Identify and document best practices, lessons learned, and recommendations to inform follow-on activities focusing on sustainability. To achieve these objectives, USAID/Southern Africa requires a team of three people to conduct a EOP evaluation of the IPHC project. The evaluation will assess the contribution of the IPHC project to improving the quality of HIV/AIDS care and support and treatment services at the facility and district level and building the capacity of the health system. VIII. METHODOLOGY The following data collection methods will be used: 1. Document Review – Review key project documents prior to arrival in country; 2. Team Planning Meeting – Orientation and planning meeting to produce a workplan, timeline, interview instruments and draft outline of the report. Initial briefing meetings will be held with IPHC staff, USAID, and DOH to review finalize the approach and activities for the evaluation; 3. Interviews and Site Visits – Key informant interviews to include USAID Mission Health Office staff, National and Provincial Department of Health staff and IPHC staff; – Visits to provinces and districts to observe the project in action and to collect evaluation data (eight selected districts in five provinces will be visited in the Eastern Cape, KwaZulu, Natal, Limpopo, Mpumalanga, and North West); – At the discretion of the Team Leader, MSH and DOH staff may be asked to excuse themselves from certain interviews to allow the team to collect the necessary data; – Prepare and present summary findings and recommendations; write and submit final evaluation report. IX. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORT The Review Team requires a team of three international consultants with expertise in HIV/AIDS care and treatment, maternal and child health, health systems management and community systems of care. Combined, the team should have expertise in monitoring and evaluating large￾scale programs, reproductive health, and maternal and child health programs and health systems strengthening. The three team members will be joined by MSH headquarters staff member with expertise in HIV and AIDS program management and a local MSH staff member for the local project perspective for each of the site visits. Two representatives from the National Department of Health and one representative from USAID/South Africa will join the team during field visits to provide an in-country perspective. 50 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION The Team Leader will be an international consultant with extensive experience in HIV/AIDS, prevention, care and treatment expertise. The Team Leader will hold conference calls with the other two core team members, key representatives from USAID/South Africa Health Office, the National Department of Health, and the Management Sciences for Health South Africa staff prior to in-country arrival. The Team Leader will: Finalize the workplan for the assignment; Establish assignment roles, responsibilities and tasks for the members of the team; Ensure that the logistics arrangements in the field are complete with assistance from the local Management Sciences for Health office; Facilitate the Team Planning meeting; Take the lead on preparing, coordinating team member’s input, submitting, revising, and finalizing the assignment report; Manage the process of writing the final report; Manage team coordination meetings in the field; Coordinate the workflow, team tasks and ensure that the team schedule works; and Ensure that the team field logistics are arranged. The Maternal and Child Health Expert and the Health Systems and/or Community Program Expert should each have an advanced degree in health and five years experience in their specialties. In addition, each team member should have, at minimum, the following skills and experience: 1. Demonstrated skill in written and oral communication: 2. Demonstrated knowledge of international HIV/AIDS mitigation approaches, including strategies for health systems strengthening and promoting host-country ownership of programs; 3. Ability to work effectively in, and communicate with, a diverse set of professionals; and 4. Excellent English language skills (both written and verbal). USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 51 Sample LOE Table Task Team Leader Team Member-Lindsey Toomey Team Member￾Swati Sadaphal Background Document Review 3 3 3 International Travel Days 3 3 3 Team Planning Meeting 1 1 1 Meetings and Interviews with Key Stakeholders and Field Visits 14 14 14 Analysis and Writing Draft Report 3 3 3 Debrief 1 1 1 5 5 0 Finalizes Report for Submission to USAID 3 1 1 TOTAL 33 days 31 days 26 days A six-day work week is approved for in-country work. X. LOGISTICS The evaluation will be conducted over a 32-day period with a start date in-country of on/about October 17, 2010. The evaluation team, in collaboration with the staff of Management Sciences for Health in South Africa, National Department of Health, and USAID/South Africa will arrange all of the meetings, interviews, site visits, in briefing and out briefing in advance. South Africa logistical support will be arranged by IPHC/Pretoria office. GH Tech will provide all logistical arrangements such as flight reservations, country cable clearance, in-country travel, airport pick-up, lodging, and supplies as necessary for the evaluation team. XI. DELIVERABLES AND PRODUCTS 1. Pre-trip Briefing: Prior to arrival, the Team Leader and evaluation team will review all relevant documentation and schedule a conference call with USAID/Southern Africa, National Department of Health, and in-country Management Sciences for Health team members. 2. Team Planning Meeting and Workplan: After the Team Leader’s arrival in country, a workplan will be developed during the team planning meeting and briefings with USAID/Southern Africa Health Office, National Department of Health, and in-country Management Sciences for Health team members. The workplan should include should include, but not be limited to, the following items: a. Milestones and deliverables with due dates clearly established b. Key interview questions, methods, and tools c. Parameters for secondary analyses of existing data d. Timeline for key activities, including preparatory activities (e.g., literature review) 52 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION e. Product due dates f. Schedule of in-briefing and formal debriefing presentations g. Tentative schedule for informant interviews h. Tentative schedule of travel to field sites i. Timeline for drafting the assessment report, requesting feedback, and finalizing the final report 3. Debriefing: Prior to departure, the evaluation team will make a presentation to USAID/Southern Africa Health Office, National Department of Health, and in-country Management Sciences for Health team members. 4. Draft Report: Prior to departure, the Team Leader will submit a draft evaluation report to USAID/Southern Africa Health Office and the National Department of Health— one hard copy and one electronic copy on a CD Rom or flash drive. The report (not including attachments) will be no longer than 30 pages with an Executive Summary, Introduction, Methodology, Findings, Lessons Learned, Conclusions, and Recommendations. 5. Final Report: After the evaluation Team departs, USAID/South Africa has 14 working days to review the draft report and provide one single set of comments. The Team Leader will submit the final report to USAID/Southern Africa, Health Office within one week of receiving comments from USAID/Southern Africa and the National Department of Health. GH Tech will provide the edited and formatted final document approximately 30 days after USAID provides final approval of the content. USAID/South Africa requests both an electronic version of the final report (Microsoft Word 2003 format) and five hard copies of the report. The report will be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the GH Tech project web site www.ghtechproject.com). XII. RELATIONSHIPS AND RESPONSIBILITIES (USAID AND CONSULTANTS) GH Tech will conduct and manage the assessment and will undertake the following specific responsibilities throughout the assignment: Recruit and hire the three-person evaluation team; Make logistical arrangements for the consultants, including travel and transportation, country travel clearance, lodging, and communications; and Respond to all points included in the SOW, including the submission of the final report. MSH will provide all compensation and travel costs for their staff and any DOH staff. They will also be responsible for logistics, including reserving vehicles and hotel bookings for the site visits. USAID/South Africa will provide overall technical leadership and direction for the evaluation team throughout the assignment and will undertake the following specific roles and responsibilities: USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 53 Before In-Country Work: Consultant Conflict of Interest. To avoid conflicts of interest or the appearance of a COI, review previous employers listed on the CV’s for proposed consultants and provide additional information regarding potential COI with the project contractors or NGOs evaluated/assessed and information regarding their affiliates. Documents. Identify and prioritize background materials for the consultants and provide them, preferably in electronic form. Local Consultants. Assist with identification of potential local consultants and provide contact information. Site Visit Preparations. Provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line items costs. Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in￾country travel (i.e., car rental companies and other means of transportation) and identify a person to assist with logistics (i.e., visa, letters of invitation, etc.) if appropriate. During In-Country Work: Mission Point of Contact. Throughout the in-country work, ensure constant availability of the Point of Contact person(s) and provide technical leadership and direction for the team’s work. Meeting Space. Provide guidance on the team’s selection of a meeting space for interviews and/or focus group discussions (i.e., USAID space if available, or other known office/hotel meeting space). Meeting Arrangements. While local consultants typically will arrange meetings for contacts outside the Health Office, support local consultant(s) in coordinating meetings with stakeholders. Formal and Official Meetings. Arrange key appointments with national and local government officials and accompany the team on these introductory interviews (especially important in high-level meetings). Other Meetings. If appropriate, assist in identifying and helping to set up meetings with local professionals relevant to the assignment. Facilitate Contact with Partners. Introduce the Evaluation Team to implementing partners, local government officials, and other stakeholders, and where applicable and appropriate prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. After In-Country Work: Timely Reviews. Provide timely review of draft/final reports and approval of the deliverables XIII. MISSION AND/OR WASHINGTON CONTACT PEOPLE/PERSON Mission Contact for this Assignment: Anita Sampson, Health Office, USAID/Southern Africa asampson@usaid.gov +27 12 452 2236 +27834436614 IPHC Contact in South Africa: Tracey Naledi, Management Sciences for Health, Chief of Party, IPHC Project. MSH House, Block 6, Phase 4 Boardwalk Office Park; Haymeadow Street; Faerie Glen; Tel: 012 9913559; Fax: 012 991 2714; Mobile: 0832687310; Email: traceyn@msh.co.za 54 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Final report to be sent to: Anita Sampson, Health Office, USAID/Southern Africa asampson@usaid.gov XIV. COST ESTIMATE (TO BE PROVIDED BY GH TECH) XV. REFERENCES (PROJECT DOCUMENTS) Reviewers will be provided with the following background documents in preparation for the assignment: Key Resource Documents: Original (2004) and Extension (2008) Proposals, IPHC/South Africa Project. MSH. IPHC Project routine (semi-annual and annual) reports to USAID. Youth Services Health Services Review: A Baseline Assessment. Health Systems Trust, June 2007. The Integrated Primary Health Care (IPHC) Project/South Africa. (Mid-term) Internal Assessment. January 2008. District Health Barometer; 2007/08. Health Systems Trust. July 2009. Workplans 2005–2009 Field Trip Reports 2005–2009 IPHC Monitoring and Evaluation Data 2005–2009 IPHC Four-year report 2005–2009 IPHC Internal Evaluation Reports (2008 and 2010) IPHC communication documents and other internal reports XVI. ADMINISTRATIVE AND LOGISTICS FUNCTIONS GH Tech will make every effort to support the successful completion of assignments. However, GH Tech does not have sufficient staff available to provide full-time administrative/clerical services to consultants and teams. Consultants are expected to undertake the following tasks independently of GH Tech assistance (unless otherwise stated in the scope of work): maintain individual calendars, set appointments, take notes, send emails, make phone calls, do photocopying, and other administrative functions necessary to implement the assignment. Team leaders are additionally responsible for maintaining the schedule and workplan for the team and for making local logistical arrangements (in-country travel, meeting rooms, appointments) when overseas, if the USAID mission or CAs are not providing such arrangements. If USAID approves and the assignment budget allows it, GH Tech may authorize the team leader to hire a local logistics assistant in country. GH Tech provides administrative and logistical support in the following specific areas: Providing instruction in completing required forms (expense report, invoice, etc.); Providing GH Tech office space for DC-based work and assisting in set-up of space (e.g., IT/equipment technical support and instructions, office supplies); Support for DC-based team planning meetings (facilitation, printing background materials, set-up, food, typing of notes if specifically authorized by GH Tech Project Director); Arranging travel in the U.S. and from the U.S. to overseas assignment location (country clearance, visa, plane tickets, hotel reservations, processing travel advance and expenses). USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 55 Consultants are responsible for arranging in-country travel while overseas and ground transportation in the U.S.; Facilitating contact with USAID staff; Instruction and/or assistance with formatting charts, graphs, and tables and PowerPoint slides; and Arranging for editing/layout of final report All other tasks required to complete the scope of work will be done by the consultant, except where the scope of work designates specific tasks for USAID, GH Tech, or another organization. Where other specific GH Tech assistance is desired, consultants will make a request to the GH Tech Project Director, who may provide staff if the request is deemed appropriate and staff is available. XVII. CONSULTANT COMMUNICATION WITH GH TECH AND USAID The consultant/team leader reports to the GH Tech Project Director or designee and is required to keep GH Tech informed of any relevant updates, including deliverables, changes to schedule, and constraints/concerns in implementing the assignment. If questions, problems, or concerns arise during the course of the assignment, the consultant will discuss those issues with the GH Tech Project Director or other GH Tech staff, who will communicate them to USAID as appropriate. If USAID requests the consultant to make any changes to the scope of work or undertake any activities that are outside of the scope of work, the consultant will ask the GH Tech Project Director or designee for authorization. GH Tech staff will ask periodically for updates on assignment status or a debriefing at the end of an assignment; the consultant will reply promptly and with the level of detail requested. When traveling overseas, the consultant will inform the GH Tech assignment manager of their arrival in country, and consultants will keep GH Tech informed about their location and travel plans (hotel room number, local travel arrangements, etc.). The consultant is required to submit a copy of all deliverables to GH Tech, unless informed otherwise by GH Tech. XVIII. METHODOLOGY, MILESTONES, AND TIME-LINES 1. Document Review: Review key project documents prior to arrival in country (October 10–13); 2. Team Planning Meeting: Orientation and planning meeting to produce a workplan, timeline, interview instruments, and draft outline of the report (October 17, 2010); 3. Initial Briefing Meetings: Briefings from IPHC staff, USAID and DOH to review and finalize the approach and activities for the evaluation (October 18, 2010); 4. Interviews and Site Visits (October 19–November 1, 2010): 5. Key informant interviews to include USAID Mission Health Office staff, National and Provincial Department of Health staff and IPHC staff; 6. Visits to provinces and districts to observe the project in action and to collect evaluation data (eight selected districts in five provinces will be visited in the Eastern Cape, KwaZulu Natal, Limpopo, Mpumalanga, and North West). 7. Prepare summary findings and recommendations ( November 2–4) 8. Provide preliminary briefing to USAID, DOH and MSH Staff (November 5) 9. Prepare Final Draft Report (November 6–11) 10. Present and Deliver Final Draft Report (November 12) 11. USAID Review of Draft Report (November 12–December 6) with comments to evaluation team o/a December 7 56 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 12. Evaluation team response to USAID Review (December 8–December 14) Evaluation Team Composition During the interviews and site visits, the evaluation team will be divided into two teams—A and B. Team A William Emmet, GH Tech Team Leader and Team Leader of Team A. (GH Tech) Swati Sadaphal (GH Tech) Malik Jaffer (USAID) Team B Lindsey Toomey, Team Leader of Team B (GH Tech) Mariah Boyd-Boffa (MSH) Tendani Muthambi (MSH) Key Interview Questions, Methods, and Tools The evaluation team will use a respondents' evaluation tool as a guide for discussions with key respondents. At the discretion of the Team Leader, MSH and DOH staff may be asked to excuse themselves from certain interviews to allow the team to collect the necessary data. Following each day of interviews the two teams (A and B), will meet separately to summarize key points and issues introduced during the interviews. For this purpose, the team will use an interview summary form for each interview and site visited. At the end of each week of site visits, the team will meet together to summarize findings associated with their site visits and to work on the rough draft of the final report. Team review meetings are scheduled for Saturday, October 23rd and Saturday, October 30th. The draft report will be delivered to USAID/SA prior to close of business on November 12. Parameters for Secondary Analyses of Existing Data For the purposes of the evaluation, the evaluation team will review existing data and reports and will request MSH for information with reference to the IPHC’s achievement of outputs by strategic performance areas (SPA) as specified by the MSH 2008 Extension Proposal. Information contained in these reports will be summarized in the report. XIX. EVALUATION ISSUES AS PRESENTED AND APPROVED AT THE OCTOBER 18TH GH TECH—USAID INITIAL BRIEFING MEETING Issue 1: The GH Tech Team would request USAID concurrence that the scope of work’s call for an analysis of impact should focus on district-level impact rather than national-level impact. Issue 2: The GH Tech Team would request confirmation from USAID that the team is not expected to examine data relating to those sites not supported by the IPHC. Issue 3: The GH Tech Team has worked with MSH to revise the schedule for field visits, especially those during Week 2 to Kwazulu Natal and Eastern Cape. The Team would request USAID concurrence with the adjusted schedule. Issue 4: Documentation from USAID refers to the evaluation as an ―End of Project Participatory Evaluation.‖ Team would request USAID’s clarification of the meaning of ―participatory‖: If the term ―participatory‖ means that all partners in the evaluation team (GH Tech, USAID, MSH, DOH) participate in all meetings, then all members of Team A and Team B take part and would not be asked to recluse themselves in the event that it was felt that respondents might be less than frank in responding to GH Tech questions. Alternatively, if USAID would like GH Tech to be in a position to decide on instances where ―closed‖ interviews would contribute to more open responses to questions or issues, then the GH USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 57 Tech team would decide on an interview-by-interview basis whether USAID/MSH/DOH participants should be asked to recluse themselves if and as appropriate. GH Tech is open to either interpretation of the meaning ―participatory‖ and would welcome USAID’s clarification on this issue. Issue 5: Paragraph VI of the scope of work states that one of the main purposes of the evaluation is to focus on ―…recommendations for the planning and management of future projects that focus on health system strengthening.‖ Given that it is understood that no additional funding is available for a future project, the GH Tech team requests a clarification whether USAID is interested in such recommendations. If not, what should be the team’s focus in the section on recommendations? Issue 6: Paragraph VI of the scope of work includes seven points to be covered in the evaluation: – Assessment of effectiveness, efficiency, and quality of the IPHC project interventions at the facility and district level; – Identification of what has been successfully incorporated into the DOH’s ongoing programs; – What challenges remain; – Establishment of evidence of project results; – Establishment of evidence of impact; – Discussion of lessons learned; and – Recommendations for the future with a focus on health-system strengthening. The GH Tech Team would appreciate USAID guidance on the weight which the team should attach in its report to each of the above points. Issue 7: In its preparation of the final report, the GH Tech Team would appreciate USAID’s guidance on the mission’s intended use of the final report: Who is the audience and should the report’s findings be directed toward a specific audience? Issue 8: In preparing its debriefings, the GH Tech Team would request USAID guidance on whether the DOH and/or MSH will be included in all debriefings or whether separate presentations should be prepared for the three separate audiences. Issue 9: In addressing the preparation and delivery of the draft report on November 12th, the GH Tech Team would appreciate USAID’s guidance on whether the draft report in its entirety should be provided to both MSH and the DOH for their comments and feedback to the GH Tech Team. Issue 10: With reference to the calendar following the team’s delivery of the draft report on November 12th, the GH Tech Team would appreciate USAID’s concurrence with the fact that, following USAID’s review and that of MSH and the DOH within 10 business days after November 12th, USAID can expect GH Tech’s response to comments no later than December 14th. (This last request is due to the Team Leader’s inability to respond to comments prior to December 8th.) 58 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION XX. RESPONDENT’S DISCUSSION GUIDELINES Integrated Primary Health Care Project (IPHC) End-of-Project Participatory Evaluation Discussion Guidelines for Field Visits and Interviews The Integrated Primary Health Care (IPHC) project in South Africa is a collaborative project between the South African National Department of Health (NDoH); five provincial Departments of Health (DoH), Eastern Cape, KwaZulu Natal, Limpopo, Mpumalanga and North West, and eight selected districts of these provinces; and the United States Agency for International Development (USAID) in South Africa. The project began in July 2004 and currently has an end date of 30 December 2010 with Management Sciences for Health (MSH) as the prime contractor. Partners over the life of the project included Health Systems Trust (HST) and University Research Corporation (URC). IPHC is designed to improve access to and use of child health, reproductive health, and HIV/AIDS services with an emphasis on improving the management systems at the district level and in selected facilities in those districts. By 2010, the project is expected to meet the following objectives: 1. Improved maternal health and Family planning, with emphasis on youth, 2. Improved child health and nutrition, 3. Increased and improved participation of youth in advocacy for reproductive health and sexuality and solutions to gender violence, 4. Reduced transmission and impact of HIV and AIDS, 5. Strengthened primary health care systems and service delivery, and 6. Strengthened community support for OVCs. In meeting these objectives, The IPHC project reports on the following Strategic Performance areas (SPA): SPA1: Maternal Health and Family Planning SPA2: Child Health SPA3: Youth Program SPA4: HIV and AIDS – SPA4.1: Counseling and Testing (C&T) – SPA4.2: Prevention of Mother-to-Child Transmission – SPA4.3: HIV and AIDS Treatment (ARV) – SPA4.4: Palliative Care (Basic) – SPA4.5: Palliative Care (TB and HIV) SPA 5: PHC Systems and Services – SPA 5.1: Quality Assurance – SPA 5.2: District Health Information System – SPA 5.3 District Health Systems SPA 6: Orphans and Vulnerable Children Under a contract with USAID/South Africa, the GH Tech Project is undertaking an end-of project evaluation whose purpose is to assess the effectiveness, efficiency and quality of the IPHC project interventions at the facility and district level; to identify what has been successfully incorporated into the DOH’s ongoing programs and what challenges remain; to establish evidence of project results and impact; and to provide lessons and recommendations for the USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 59 planning and management of future projects that focus on health system strengthening. In responding to the evaluation’s purpose, the evaluation team will focus on the following objectives: Review project results (outputs and outcomes) in relation to the project’s strategic performance areas (SPA) and baseline status or comparison areas/facilities/organizations for each SPA (where available) and identify operational constraints encountered; Assess the project’s strengths, weaknesses, gaps in service delivery, and any constraints to successful implementation; and Identify and document best practices, lessons learned, and recommendations to inform follow-on activities focusing on sustainability. Based on the respondent’s experience and knowledge of the IPHC, in working with MSH and its partners, and with reference to IPHC objectives and SPAs, the evaluation team’s interviews will focus on the following questions: 1. What is the respondent’s assessment of IPHC progress achieved to date? The respondent should be encouraged to consider and comment on IPHC’s contribution to improved management systems focused on: a. Maternal health and family planning b. Child health c. Youth program d. HIV and AIDS – Counseling and Testing (C&T) – Prevention of Mother-to-Child Transmission – HIV and AIDS Treatment (ARV) – Palliative Care (Basic) – Palliative Care (TB and HIV) e. PHC Systems and Services – Quality Assurance – District Health Information System – District Health Systems f. Orphans and Vulnerable Children 2. What is the respondent’s assessment of IPHC progress achieved to date? The respondent should be encouraged to consider and comment on IPHC’s contribution to improved health services focused on: a. Maternal Health and Family Planning b. Child Health c. Youth Program d. HIV and AIDS – Counseling and Testing (C&T) – Prevention of Mother-to-Child Transmission – HIV and AIDS Treatment (ARV) – Palliative Care (Basic) – Palliative Care (TB and HIV) e. PHC Systems and Services 60 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION – Quality Assurance – District Health Information System – District Health Systems f. Orphan and Vulnerable Children 3. What is the respondent’s assessment of best practices‖ instituted by the IPHC in addressing management systems and health service delivery of the above strategic priority areas. What’s new and what’s working? 4. What is the respondent’s assessment of IPHC strengths and weaknesses associated with management systems and health service delivery of the above strategic priority areas? If something worked well, why did it work well? If something did not work well, why not? 5. What is the respondent’s assessment of constraints associated with IPHC’s efforts to improve management systems and health service delivery of the above strategic priority areas? – Human Resources – Geography – Time – Funding – Community/Facility Interface – USAID/MSH/DOH Interface – Government Policy Facility Policy Leadership 6. Sustainability a. What IPHC interventions are not sustainable and why do you think so? b. 6.2 What IPHC interventions are sustainable and what actions or interventions would the respondent recommend to build upon and improve the sustainability of management systems and health service delivery of the above strategic priority areas ? 7. What is the respondent’s assessment of lessons learned with reference to IPHC’s efforts to improve management systems and health service delivery of the above strategic priority areas? XXI. DAILY INTERVIEW SUMMARY TEMPLATE Evaluation Team Interviewer: Respondent Name: Respondent Title and Affiliation: Interview Location: Date: Respondent Focus: SPA1: Maternal Health and Family Planning SPA2: Child Health SPA3: Youth Program SPA4: HIV and AIDS USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 61 – SPA4.1: Counseling and Testing (C&T) – SPA4.2: Prevention of Mother-to-Child Transmission – SPA4.3: HIV and AIDS Treatment (ARV) – SPA4.4: Palliative Care (Basic) – SPA4.5: Palliative Care (TB and HIV) SPA 5: PHC Systems and Services – SPA 5.1: Quality Assurance – SPA 5.2: District Health Information System – SPA 5.3: District Health Systems SPA6: Orphan and Vulnerable Children Interview Summary: 1. Progress of the IPHC Project in Improved Management Systems: 2. Progress of the IPHC Project in Improved Health Services: 3. IPHC ―Best Practices‖: 4. IPHC Strengths/Weaknesses: 5. IPHC Constraints: 6. Lessons Learned: 7. Sustainability: 8. Recommendations: 9. Comments/ Observations: 62 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 63 ANNEX B. IPHC EVALUATION CONTACTS Name Organization Position Location Date LIMPOPO: District Representatives Maponya, R.L. University Research Council District Coordinator: Waterberg District Capricorn District 19th October 2010 Mothemi, M. Capricorn District Information Manager Capricorn District 19th October 2010 Mokgoba, T.J. Limpopo Provincial Health Transformation and Governance Capricorn District 19th October 2010 Lukhele, Z.P. Limpopo Provincial Health CFO Office Capricorn District 19th October 2010 Malumane, N. Capricorn District Capricorn District 19th October 2010 Chuene, S.E. Capricorn District PHC Manager Capricorn District Office 20th October 2010 Phosa, M. Capricorn District MCWHYN Manager Capricorn District Office 20th October 2010 Morewane, M. Greater Sekhukhune District PHC Manager Pretoria East 22nd October 2010 EASTERN CAPE: District Representatives Kizza, N.M. Chris Hani District District Manager Queenstown 25th October 2010 Sixam, N. Chris Hani District Quality Assurance Manager Queenstown 25th October 2010 Openshaw, M. Chris Hani District Information Manager Queenstown 25th October 2010 Mkabile, N. Chris Hani District HIV/AIDS Programs Queenstown 25th October 2010 Philaphi, N.C. Chris Hani District All Programs Manager Queenstown 25th October 2010 Shibani, N.O. Chris Hani District Planning Officer Queenstown 25th October 2010 NORTHWEST: District Representatives Moromole, Dineo KS Hospital Complex Acting CEO Bojanala District Office 19th October 2010 Boloyi, D.E. Moretele Sub-District Sub-District Manager Bojanala District Office 19th October 2010 Boikanyo, K.S. Moses Kotane Sub-District Sub-District Manager Bojanala District Office 19th October 2010 64 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Name Organization Position Location Date Bolokwe, M.E. Bojanala District Office District Director, District Health Services Bojanala District Office 19th October 2010 Matjila, N.N. Bojanala District Office District Coordinator Bojanala District Office 19th October 2010 Tlhowe, Lawrence Rustenburg Sub-District Sub-District Manager Bojanala District Office 19th October 2010 Diratsagae, B.P. Moses Kotane Sub-District Assistant Manager Nursing Moses Kotane Sub-District Office 20th October 2010 Kgatlhante, A.S.M. Moses Kotane Sub-District Assistant Manager Nursing Moses Kotane Sub-District Office 20th October 2010 Moabi, S.S. Moses Kotane Sub-District Assistant Manager Nursing (PHC) Moses Kotane Sub-District Office 20th October 2010 Mogupi, T.P. Moses Kotane Sub-District Assistant Manager Nursing Moses Kotane Sub-District Office 20th October 2010 Moloi, I.M. Madibeng Sub-District Sub-District Manager Madibeng Sub-District Office 21st October 2010 Molefe, M.C.B. Madibeng Sub-District Ikhutseng Assistant Manager Nursing (PHC) Hebron Clinic 21st October 2010 Mogotsi, L.S.F. Madibeng Sub-District Bapong Assistant Manager Nursing (PHC) Hebron Clinic 21st October 2010 Mangezi, M.D. Moretele Sub-District Assistant Manager Nursing (PHC) Moretele Sub-District Office 22nd October 2010 Madia, M.J. Moretele Sub-District Assistant Manager Nursing (PHC) Moretele Sub-District Office 22nd October 2010 Seqwai, K. Moretele Sub-District Acting Clinical Manager Moretele Sub-District Office 22nd October 2010 Sentle, M.P. Moretele Sub-District Assistant Manager Nursing (PHC) Moretele Clinic 22nd October 2010 KWAZULU NATAL: District Representatives Dube, N. Uthungulu District Office OMN Uthungulu District Office 24th October 2010 Cabeichulu, S.M. Uthungulu District Office OMN PHC Uthungulu District Office 24th October 2010 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 65 Name Organization Position Location Date Dube, J.Y. Uthungulu District Office Deputy Director Manager Clinical Uthungulu District Office 24th October 2010 Mpupole, Isiphile Uthungulu District Office OMN MCWH Uthungulu District Office 24th October 2010 Ntuli, Bongi Uthungulu District Office PHC Supervisor Uthungulu District Office 25th October 2010 Khumalo, M.H. Sisonke District Office OPM IPC Sisonke District Office 28th October 2010 Mpongoma, S. Sisonke District Office District Information Officer Sisonke District Office 28th October 2010 Langa, Londa Sisonke District Area 2 AGM Office Prinicpal Technical Advisor IAP Sisonke District Office 28th October 2010 Nwme, V.V. Sisonke District Office OPM PHC Sisonke District Office 28th October 2010 Msami, T.L. Sisonke District Office STA PMTCT Sisonke District Office 28th October 2010 Mkluze, B.A. Sisonke District Office DDM Sisonke District Office 28th October 2010 Mokgalapa, Yvonne NDOH: DHS Deputy Director Sisonke District 28th October 2010 Makhaye, B.H.S. Sisonke District Office Deputy Manager Sisonke District Office 28th October 2010 MPUMALANGA: District Representatives Ngaleka, N. Ermelo District Office Operational Manager: Dun Donald Clinic Ermelo District Office 1st November 2010 Mabande, K. Ermelo District Office MNCHW Program Manager Ermelo District Office 1st November 2010 Luthulu, T. Ermelo District Office TB Program Manager Ermelo District Office 1st November 2010 Dhlahla, S. Ermelo District Office Health Promotion Program Manager Ermelo District Office 1st November 2010 Khumalo, T. Ermelo District Office Health Information Program Manager Ermelo District Office 1st November 2010 Makhanya, S. Ermelo District Office NGO Coordination Ermelo District Office 1st November 2010 Dlamini, S. Ermelo District Office Clinic Supervisor Ermelo District Office 1st November 2010 LIMPOPO: Health Services Personnel Segorela, K. Moletjie Clinic Clinic Youth Mentor Moletjie Village 19th October 2010 Mokwatlo, M. Moletjie Clinic General Nurse Moletjie Village 19th October 2010 66 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Name Organization Position Location Date Moabelo, I. Moletjie Clinic Clinical Nurse Practitioner Moletjie Village 19th October 2010 Mabotja, J. Moletjie Clinic Clinical Nurse Practitioner Moletjie Village 19th October 2010 Selepe,W. Moletjie Clinic Clinic Committee Chairperson Moletjie Village 19th October 2010 Setati,P. Moletjie Clinic Cleaner Moletjie Village 19th October 2010 Maubane, M. Moletjie Clinic Lay Counselor Moletjie Village 19th October 2010 Mamabolo, P.J. Moletjie Clinic Registered Nurse Moletjie Village 19th October 2010 Masedi, F.A. Moletjie Clinic Data capturer Moletjie Village 19th October 2010 Magotlane, M.M. Moletjie Clinic Enrolled Nurse Moletjie Village 19th October 2010 Sekgala, R.C. Moletjie Clinic Enrolled Nurse Moletjie Village 19th October 2010 Manoko, M.S. Moletjie Clinic Moletjie Village 19th October 2010 Mofepi, J.J. Moletjie Clinic Moletjie Village 19th October 2010 Motloutsi, S.M Moletjie Clinic Moletjie Village 19th October 2010 Mphahlele, M.J. Lebowakgomo Hospital Occupational Health Care Unit Manager Lebowakgomo 20th October 2010 Teffo, R.G. Lebowakgomo Hospital TB Clinic Manager Lebowakgomo 20th October 2010 Mahlatji, R.D Lebowakgomo Hospital Quality Assurance Manager Lebowakgomo 20th October 2010 Hika, K.M. Lebowakgomo Hospital Lay Counselor Lebowakgomo 20th October 2010 Ramphaka, M. Lebowakgomo Hospital Dietician Lebowakgomo 20th October 2010 Mabena, N.L Lebowakgomo Hospital Lay Counselor Lebowakgomo 20th October 2010 Mailula, M.M. Lebowakgomo Hospital Lay Counselor Lebowakgomo 20th October 2010 Ralithi, L.P. Lebowakgomo Hospital Lay Counselor Lebowakgomo 20th October 2010 Phasha, M.J. Lebowakgomo Hospital ARV Unit Manager Lebowakgomo 20th October 2010 Laka, M. Lebowakgomo Hospital Out Patient Unit Manager Lebowakgomo 20th October 2010 NORTHWEST: Health Services Personnel Legotlo, L. Tlhabane cluster Assistant Manager Nursing (PHC) Tlhabane CHC 19th October 2010 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 67 Name Organization Position Location Date Lesejane, N.J. Tlhabane Health Center Operational Manager Tlhabane CHC 19th October 2010 Mataboge, M.J. Tlhabane Health Center Information Officer Tlhabane CHC 19th October 2010 Motlhabi, M.J. Tlhabane Health Center Communication Officer Tlhabane CHC 19th October 2010 Masilo, N.E. Bakubung Clinic Operational Manager Bakubung Clinic 20th October 2010 Letlape, N.H. Hebron Clinic Operational Manager Hebron Clinic 21st October 2010 Phajane, T.A. Oukasie Maternity Operational Manager Hebron Clinic 21st October 2010 Malemane, S.M. Motholung Operational Manager Hebron Clinic 21st October 2010 Tlhake, M.J. Moretele Clinic Operational Manager Moretele Clinic 22nd October 2010 KWAZULU NATAL: Health Services Personnel Khanyile, Sizakele Chwezi Clinic Operational Manager Chwezi Clinic 25th October 2010 Mgenge, Takhona Inkosinathi OVC Project Data Capturer Inkosinathi OVC Project 26th October 2010 Ndlovu, Nondumiso Inkosinathi OVC Project Finance Officer Inkosinathi OVC Project 26th October 2010 Liman Linda, S.S. Inkosinathi OVC Project Project Manager Inkosinathi OVC Project 26th October 2010 Ndlovu, Reginah Inkosinathi OVC Project Community Caregiver Inkosinathi OVC Project 26th October 2010 Ndlovu, Thoko Inkosinathi OVC Project Community Caregiver Inkosinathi OVC Project 26th October 2010 Peter Inkosinathi OVC Project Facilitator Inkosinathi OVC Project 26th October 2010 Shahdu, Musa Nseleni CHC Pharmacy Assistant/Former Youth Mentor Nseleni CHC 26th October 2010 Mngadi, Khanyisile Nseleni CHC Pharmacy Assistant/Former Youth Mentor Nseleni CHC 26th October 2010 Mhgonezulu, Gugu Nseleni CHC Operational Manager IMCI Nseleni CHC 26th October 2010 Ntuli, Thembi Nseleni CHC Operational Manager ARV Nseleni CHC 26th October 2010 Doke, M.P. Nseleni CHC Youth Friendly Services Chair Nseleni CHC 26th October 2010 Mavundla, B.L. Nseleni CHC Nursing Manager Nseleni CHC 26th October 2010 Mthabela, Mrs. King Dinizulu Clinic Nursing Manager King Dinizulu Clinic 27th October 2010 68 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Name Organization Position Location Date Magubane, Sister King Dinizulu Clinic Operational Manager King Dinizulu Clinic 27th October 2010 Zondi, T.C. Pholela CHC Clinical Nurse Practitioner Pholela CHC 28th October 2010 Willie, N. Pholela CHC OMN Pholela CHC 28th October 2010 Sikhakhane, T.G.O. Pholela CHC OMN Pholela CHC 28th October 2010 King, B.C. Pholela CHC Acting Facility Manager Pholela CHC 28th October 2010 Maphanga, B.M. Pholela CHC PHC Supervisor Pholela CHC 28th October 2010 Klumalo, Nokuthumla Hlokozi Clinic Clinical Nurse Practitioner Hlokozi Clinic 29th October 2010 Dlamini, Thandekhe Jolivet Clinic Operational Manager Jolivet Clinic 29th October 2010 Jwora, Lethani Jolivet Clinic Clinic Committee Representative Jolivet Clinic 29th October 2010 EASTERN CAPE: Health Services Personnel Riwica, N.V. Frontier Regional Hospital ARV Doctor Queenstown 25th October 2010 Chitha, W.W. Frontier Regional Hospital Queenstown 25th October 2010 Phakade, N. Frontier Regional Hospital ARV Coordinator Queenstown 25th October 2010 Tywati, E.S. Frontier Regional Hospital COO Queenstown 25th October 2010 Mbontsi, K. Inxuba Ye Themba LSA Clinic Supervisor Thornhill Community Health Center 26th October 2010 Notshe, N. Inxuba Ye Themba LSA HIV/AIDS/STI and Prevention Manager Thornhill Community Health Center 26th October 2010 Spenxe, S. Thornhill CHC ARV Coordinator Thornhill Community Health Center 26th October 2010 Tsheko, N.A. Inxuba Ye Themba LSA Operations Manager Thornhill Community Health Center 26th October 2010 Mntambo, N.C. Inxuba Ye Themba LSA All Programs Manager Thornhill Community Health Center 26th October 2010 Spelman, K. Thornhill CHC ARV Doctor Thornhill Community Health Center 26th October 2010 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 69 Name Organization Position Location Date MPUMALANGA: Health Services Personnel Makhubu, T. Dundonald Clinic Professional Nurse Dundonald Clinic 1st November 2010 Mahla, M. Dundonald Clinic Professional Nurse Dundonald Clinic 1st November 2010 LIMPOPO: Community Based Organizations Legodi, S. Bahlaloga Community Home Based Care Manager Moletjie Village 19th October 2010 Boshamane, E. Bahlaloga Community Home Based Care Caregiver Moletjie Village 19th October 2010 Matlala, M. Makotse Women's Club Manager Mokotse Village 21st October 2010 Mphahlele, B. Direlang Project Board Member Lenting Village 21st October 2010 Maphuthi, A. Direlang Project Project Manager Lenting Village 21st October 2010 Tshebesebe L. Direlang Project Financial Officer Lenting Village 21st October 2010 Aphane, J. Direlang Project Monitoring and Evaluation and Reporting Officer Lenting Village 21st October 2010 Lekgau, C. Direlang Project OVC Coordinator Lenting Village 2st October 2010 National Department of Health & Health Systems Trust Matse, P.M. Health Systems Trust Project Manager Madibeng Sub-District Office 21st October 2010 Dr. Pillay, Yogan NDoH Deputy Director General NDoH 5th November 2010 DrMorewane, R. NDoH Chief Director NDoH 18th October 2010 Asia, Bennett NDoH Director NDoH 18th October 2010 Dr. Wilson, Tim NDoH Consultant NDoH 31st October 2010 70 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 71 ANNEX C. IPHC KEY BACKGROUND DOCUMENTS REVIEWED MSH CONTRACTS Original (2004) Proposal, IPHC/South Africa Project. MSH. Extension (2008) Proposal, IPHC/South Africa Project. MSH. Project Management Documents Contract Performance Matrix. MSH Performance Monitoring Plan. Work Plan, 2005. Work Plan, 2006. Work Plan, 2007. Work Plan, 2008. Work Plan, 2009. Project Financial Documents MSH Accruals MSH Funds by Activity Area Reports and Analyses The Integrated Primary Health Care (IPHC) Project/South Africa. Mid-term Internal Assessment. January 2008. IPHC Internal Evaluation Report, 2010. IPHC Four Year Report, 2005–2009. IPHC Monitoring and Evaluation Data, 2005–2009. Youth Services Health Services Review: A Baseline Assessment. Health Systems Trust, June 2007. District Health Barometer; 2007/08. Health Systems Trust. July 2009. Internal Assessment of the IPHC Project, 2004-2009. October 2010. IPHC Project routine (semi-annual and annual) reports to USAID. PHC Review Manual. Summary Statistics extracted from Human Resources for Health: A Needs and Gaps Analysis of HRH in South Africa, November 2009. 72 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION Trip reports Field Trip Reports 2005–2009 Communications Documents Helping Others Help Themselves: A Community-led Response to Healthcare in Mbabakazi. July 2010. LDP Brochure. MSH, Undated. NDOH Documents National Department of Health Strategic Plan, 2010/11–2012/13. PHC Clinic Supervision Manual, 2009. USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 73 ANNEX D. IPHC EVALUTION SCHEDULE DATES TIME ACTIVITY VENUE DRIVING DISTANCE Monday, 18th October 2010 10:00 Meeting with USAID USAID Offices, Groenkloof Pretoria 7 km from hotel (VILLAS) 11:00 Meeting with DOH Representatives 14:00 MSH Briefing MSH Office Faerie Glen Pretoria and VILLAS 16:00 Team Meeting VILLAS 16:30 Team A departs for Limpopo Tuesday, 19th October 8:30 Meeting with District Representatives Polokwane - Capricorn District Office 4 km from hotel (Masana/Victoria Place/ 14:00 Moletjie Clinic Moletjie 60 km from Lonsdale toward Polokwane Wednesday, 20th October 8:00 Lebowakgomo Hospital Lebowakgomo 57 km from Polokwane 12:00 Unit R Clinic Lebowakgomo Township 5 km from Lebowakgomo Hospital 16:00 Travel to Groblersdal Sleep at Guinea Feathers - 0823301916 or Lion's Guesthouse (013) 262 2268 119 km Thursday, 21st October 9:00 Makotse Women's Club Makotse Village 68km from Polokwane 13:00 Direlang OVC Project Lenting Village 27km from Makotse Village 15:30 Travel to Groblersdal Sleep at Loskop Lodge B&B 119 km Friday, 22nd October 9:00 Klipsruit Clinic Lessofontein 55km from Groblersdal 17:00 Meeting with Greater Sekhukhune Representative Pretoria East - Woodlands 74 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION DATES TIME ACTIVITY VENUE DRIVING DISTANCE Saturday, 23rd October 09h –13h00 Evaluation Team Meeting: Week Recap VILLAS Tuesday, 19 October 8:30 Meeting with the Bojanala district management team Bojanala district office 122km from Pretoria. Team could also stay overnight in Rustenburg 12:00 Visit to Thlabane CHC Thlabane 5.6km Wednesday, 20 October 8:30 Meeting with the Moses Sikotane sub￾district management team Moses Sikotane sub district office Villas - Pretoria, 122 from Rustenburg 13:00 Visit to Bakubung clinic Bakubung Thurday, 21st October 9:00 Meeting with the Madibeng sub-district manager PHC Review Meeting Madibeng sub district office 16:00 Friday, 22nd October 9:00 Hebron clinic Hebron 45 km from Pretoria Visit to Moretele clinic Moretele Saturday, 23rd October 09h –13h00 Evaluation Team Meeting: Week Recap VILLAS Sunday, 24th October 10:00 Depart Durban for uThungulu District Empangeni 172km from Durban 15:00 uThungulu District: Meeting with district representatives Empangeni 172km from Durban Monday, 25th October 9:00 Nponjwana Clinic Nomponjwana Village 12:00 King Dinizulu 27km from district office Tuesday, 26th October 9:00 Nseleni Nseleni 13:00 Inkosinathi OVC CBO Wednesday, 27th October Drive to Sisonke USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 75 DATES TIME ACTIVITY VENUE DRIVING DISTANCE Thurdsday, 28th October 8:30 Sisonke District Meeting Ixopo 13:00 Pholela CHC Pholela Village 108km from Ixopo Friday, 22nd October 8:30 Jolivet Clinic Jolivet 53km from Ixopo toward Durban 13:00 Hlokozi 30 km from Jolivet toward Durban 17:40 Fly from Durban to Johannesburg Sunday, 24th October 10:00 Travel from Johanesburg to East London 12:00 Drive from East London to Queenstown 200km from East London Monday, 25th October 8:30 Chris Hani District Office Queenstown 13:00 Frontier Hospital Queenstown 5km from district office Tuesday, 26th October 9:30 Glen Grey Hospital 50 km from Queenstown 13:30 Askeanton 15 km from Glen Grey Hospital Wednesday, 27th October 8:00 Travel from Queenstown and Kokstad Ngcobo 11:00 All Saints Gateway Ngcobo 358km from Queenstown Thursday, 28th October 8:30 Meeting with district representatives Kokstad - Alfred Nzo 13:00 Madzikane Ka Zulu Friday, 29th October 9:00 Mt Ayliff Hospital Mt Ayliff 12:00 Bonukhanyo Youth Organization Mt Ayliff 45km from Mt Frere 14:00 Drive from Kokstad to Durban 17:40 Fly from Durban to Johannesburg 76 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION DATES TIME ACTIVITY VENUE DRIVING DISTANCE Saturday, 30th October 8:30 Meeting with Dr. Tim Wilson Parktown Johannesburg 65 km from Villas Saturday, 30th October 09h –13h00 Evaluation Team Meeting: Week Recap VILLAS Sunday, 31st October Report Writing Monday, 1st November 6:00 Travel to Mpumalanga Ermelo Tendani, Mariah and Swati 8:30 Gert Sibande District: Meeting with district representatives Ermelo 14:00 Dun Donald CHC Dun Donald Village 150km from Ermelo Monday, 1st November 08:30 – 17:30 Evaluation Team meeting: Briefing Prep and Recommendation writing VILLAS Bill, Lindsey and Malik Tuesday, 2nd November 09:00 –17:00 Evaluation Team meeting: Briefing Prep and Recommendation writing VILLAS Wednesday, 3rd November 09:00 –11:30 13:00–18:00 Evaluation Team Meeting: Week Recap and Briefing Prep. Tracey to provide a presentation on Re-engineering PHC Report Writing MSH Offices - Faerie Glen Pretoria and VILLAS Thurday, 4th November 08:30 –16:30 Evaluation Team Meeting :Briefing Prep VILLAS Friday, 5th November 11:00 — 13:00 13:30 – 16:00 USAID Offices: Debriefing Meeting at MSH Offices Full Team Recap following USAID Meeting NDOH offices - Civitus Building Pretoria Saturday, 6th November 09h –18h00 Report Preparation: Emmet and Toomey VILLAS Sunday, 7th November 09h –18h00 Report Preparation: Emmet and Toomey VILLAS Monday, 8th November 09h –18h00 Report Preparation & Preparation for USAID / DOH Review Meeting: - Emmet and Toomey VILLAS USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 77 DATES TIME ACTIVITY VENUE DRIVING DISTANCE Tuesday, 9th November 09:00–18:00 Report Preparation & Preparation for USAID / DOH Review Meeting: Emmet and Toomey VILLAS Wednesday, 10th November 09h –18h00 Report Preparation: Emmet and Toomey VILLAS Thursday, 11th November 09h –18h00 Report Preparation: Emmet and Toomey VILLAS Friday, 12th November 10:00 Final Report Production and Delivery of Report to USAID by Close of Business USAID Offices 7 km from VILLAS 78 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 79 ANNEX E. INTEGRATED PRIMARY HEALTH CARE PROJECT (IPHC) END-OF-PROJECT PARTICIPATORY EVALUATION DISCUSSION GUIDELINES FOR FIELD VISITS AND INTERVIEWS The Integrated Primary Health Care (IPHC) project in South Africa is a collaborative project between the South African National Department of Health (NDoH); five provincial Departments of Health (DoH), Eastern Cape, KwaZulu Natal, Limpopo, Mpumalanga and North West, and eight selected districts of these provinces; and the United States Agency for International Development (USAID) in South Africa. The project began in July 2004 and currently has an end date of December 30, 2010 with Management Sciences for Health (MSH) as the prime contractor. Partners over the life of the project included Health Systems Trust (HST) and University Research Corporation (URC). IPHC is designed to improve access to and use of child health, reproductive health, and HIV/AIDS services, with an emphasis on improving the management systems at the district level and in selected facilities in those districts. By 2010, the project is expected to meet the following objectives: 1. Improved maternal health and family planning, with emphasis on youth 2. Improved child health and nutrition 3. Increased and improved participation of Youth in Advocacy for Reproductive Health and Sexuality and solutions Gender Violence 4. Reduce transmission and impact of HIV and AIDS 5. Strengthened primary health care systems and service delivery 6. Strengthened community support for OVCs In meeting these objectives, The IPHC project reports on the following strategic performance areas (SPA): SPA1: Improve maternal health and family planning, with emphasis on youth SPA 2: Improve child survival, health, and nutrition SPA 3: Increase youth participation in promotion and use of youth-friendly services SPA 4: Reduce the impact of HIV & AIDS – SPA 4.1: Counseling and testing (C&T) – SPA 4.2: Prevention of mother-to-child transmission – SPA 4.3: Comprehensive care, management, and treatment – SPA 4.4: Palliative care in the OVC setting; integration of TB and HIV SPA 5: Strengthening primary health care systems and services – SPA 5.1Quality assurance, clinic supervision and district development – SPA 5.2: District health information system – SPA 5.3 District health systems SPA 6: Strengthen community support and participation for OVC. Under a contract with USAID/South Africa, the GH Tech Project is undertaking an end-of￾project evaluation whose purpose is to assess the effectiveness, efficiency, and quality of the IPHC project interventions at the facility and district level; to identify what has been successfully incorporated into the DOH’s ongoing programs and what challenges remain; to establish evidence of project results and impact; and to provide lessons and recommendations for the 80 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION planning and management of future projects that focus on health system strengthening. In responding to the evaluation’s purpose, the evaluation team will focus on the following objectives: Review project results (outputs and outcomes) in relation to the project’s strategic performance areas (SPA) and baseline status or comparison areas/facilities/organizations for each SPA (where available), and identify operational constraints encountered; Assess the project’s strengths, weaknesses, gaps in service delivery, and any constraints to successful implementation; and Identify and document best practices, lessons learned and recommendations to inform follow-on activities focusing on sustainability. Based on the respondent’s experience and knowledge of the IPHC, in working with MSH and its partners, and with reference to IPHC objectives and SPAs, the evaluation team’s interviews will focus on the following questions: 1. What is the respondent’s assessment of IPHC progress achieved to date? The respondent should be encouraged to consider and comment on IPHC’s contribution to improved management systems focused on: 1.) Maternal Health and Family Planning 2.) Child health 3.) Youth program 4.) HIV & AIDS with reference to: – Counseling and testing (C&T) – Prevention of mother-to-child transmission – Comprehensive care, management, and treatment – Palliative care in the OVC setting; – Integration of TB and HIV 5.) Primary health care systems and services with reference to: – Quality assurance, clinic supervision and district development – District health information system – District health systems (district management teams, district health plans, expenditure review) 6.) Orphans and vulnerable children 2. What is the respondent’s assessment of IPHC progress achieved to date? The respondent should be encouraged to consider and comment on IPHC’s contribution to improved health services focused on: 1.) Maternal Health and Family Planning 2.) Child health 3.) Youth program 4.) HIV & AIDS with reference to: – Counseling and testing (C&T) – Prevention of mother-to-child transmission – Comprehensive care, management, and treatment – Palliative care in the OVC setting; – Integration of TB and HIV USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 81 5.) Primary health care systems and services with reference to: – Quality assurance, clinic supervision and district development – District health information system – District health systems (district management teams, district health plans, expenditure review) 6.) Orphans and vulnerable children 3. What is the respondent’s assessment of IPHC progress What is the respondent’s assessment of ―best practices‖ instituted by the IPHC in addressing management systems and health service delivery of the above strategic priority areas. What’s new and what’s working? 4. What is the respondent’s assessment of IPHC strengths and weaknesses associated with management systems and health service delivery of the above strategic priority areas? If something worked well, why did it work well? If something did not work well, why not? 5. What is the respondent’s assessment of constraints associated with IPHC’s efforts to improve management systems and health service delivery of the above strategic priority areas? – Human Resources – Geography – Time – Funding – Community/Facility Interface – USAID/MSH/DOH Interface – Government Policy – Facility Policy – Leadership 6. What IPHC interventions are not sustainable and why do you think so? 7. What IPHC interventions are sustainable and what actions or interventions would the respondent recommend to build upon and improve the sustainability of management systems and health service delivery of the above strategic priority areas? 8. What is the respondent’s assessment of lessons learned with reference to IPHC’s efforts to improve management systems and health service delivery of the above strategic priority areas? 82 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION 83 ANNEX F. IPHC: INTERVIEW SUMMARY Evaluation Team Interviewer: Respondent Name: Respondent Title and Affiliation: Interview Location: Date: Interview Summary: 1. Progress of the IPHC Project: 2. IPHC Strengths: 3. IPHC Weaknesses/Challenges: Other Comments and Observations: 84 USAID/SOUTH AFRICA: IPHC END OF PROJECT PARTICIPATORY EVALUATION MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 1 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. CONTENTS Contents .......................................................................................................................................... 1 Acronym List .................................................................................................................................. 2 Executive Summary ........................................................................................................................ 4 I. Background .................................................................................................................................. 6 II. Project Performance to Date ...................................................................................................... 7 A. SPA 1: Maternal Health and Family Planning ....................................................................... 7 B. SPA 2: Child Health ............................................................................................................... 8 C. SPA 3: Youth Program ........................................................................................................... 8 D. SPA 4: HIV & AIDS .............................................................................................................. 8 E. SPA 5: PHC Systems and Services ........................................................................................ 9 F. SPA 6: Orphans and Other Vulnerable Children .................................................................. 10 III. Gaps and Opportunities for Improvement .............................................................................. 10 A. Gaps in Services for HIV & AIDS, STIs, and TB ............................................................... 11 B. Human Resource Constraints ............................................................................................... 11 C. Data Collection, Analysis, and Use ...................................................................................... 11 D. Coordination and Other Challenges ..................................................................................... 11 E. General Cross-Cutting Challenges and Gaps ....................................................................... 12 IV. Project Results Expected for the Extension Period ................................................................ 12 A. Objectives by SPA ............................................................................................................... 13 B. Key Performance Targets ..................................................................................................... 13 V. Overarching Strategies ............................................................................................................. 14 A. A General Principle: Consolidation, Integration, and Institutionalization ........................... 14 B. Provincial/District Support and Oversight ........................................................................... 14 C. Strengthened Links with the National and Provincial Departments of Health .................... 14 D. Project Database Development and Use .............................................................................. 15 E. District Data Management .................................................................................................... 15 F. Service Performance Reviews .............................................................................................. 16 G. Facility Staff and Skills Audits ............................................................................................ 16 H. Identification and Documentation of Best Practices and Models of Success ...................... 16 I. Procedures for Referring and Tracking Patients .................................................................... 17 VI. New Technical Approaches and Activities by SPA ........................................................... 17 SPA 1: Maternal Health and Family Planning .......................................................................... 17 SPA 2: Child Health.................................................................................................................. 18 SPA 3: Youth Program ............................................................................................................. 18 SPA 4: HIV & AIDS................................................................................................................. 18 SPA 5: PHC Systems and Service ............................................................................................ 19 SPA 6: Orphans and Other Vulnerable Children ...................................................................... 20 VII. Project Management .............................................................................................................. 21 VIII. Monitoring and Reporting .................................................................................................... 21 IX. Summary of Resource Requirements ..................................................................................... 23 Annexes......................................................................................................................................... 24 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 2 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ACRONYM LIST AIDS Acquired immune deficiency syndrome ANC Antenatal care ART Antiretroviral therapy ARV Antiretroviral CBO Community-based organization CLO Community liaison officer COP Country operating plan CT Counseling and testing DAC District AIDS Council DHER District health expenditure review DHIS District health information system DHMT District Health Management Team DHP District health planning DHS Demographic and Health Survey DISCA District STI Quality of Care Assessment DMT District Management Team DOH Department of Health DOTS Directly observed treatment short-course FP Family planning HAST HIV, AIDS, STI, TB HISP Health Information System Project HIV Human immunodeficiency virus HST Health Systems Trust IMCH Integrated maternal and child health IMCI Integrated Management of Childhood Illness IPHC Integrated Primary Health Care [Project] IR Intermediate result KZN KwaZulu Natal MCH Maternal and child health MDR Multidrug resistant M&E Monitoring & evaluation MESH Management, Economic, Social Cohesion, Community, and Human Resource [Tool] MSH Management Sciences for Health NDOH National Department of Health NGO Nongovernmental organization OI Opportunistic infection ORT Oral rehydration therapy OVC Orphans and other vulnerable children PCR Polymerase chain reaction PEPFAR President’s Emergency Plan for AIDS Relief PHC Primary health care PMTCT Prevention of mother-to-child transmission RA Responsibility area RH Reproductive health RTC Regional Training Centre SAG South African Government SPA Strategic performance area MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 3 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. STI Sexually transmitted infection STTA Short-term technical assistance TA Technical assistance TALI Tool to Assess the Level of Information Utilization TB Tuberculosis URC University Research Corporation USAID United States Agency for International Development VCT Voluntary counseling and testing YFS Youth-friendly services MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 4 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. Executive Summary For over a decade, Management Sciences for Health (MSH) has partnered with the US Agency for International Development (USAID), South Africa’s National Department of Health (NDOH) and other agencies to strengthen primary health care (PHC) systems and services for the people of South Africa. First through the EQUITY Project and, more recently, through the Integrated PHC Project (IPHC) which works in eight districts in five provinces, MSH provides technical assistance and other support aimed at achieving following vision: The IPHC project expects, as a result of its efforts, the emergence of functional district health systems capable of providing accessible, high-quality HIV & AIDS interventions that are fully integrated into primary health care services for the benefit of disadvantaged communities, thus offering previously disadvantaged South Africans a better quality of life. In this proposal for the extension to its current IPHC contract, MSH outlines its expectations of building upon past experience and lessons learned to bring integrated PHC services to a new level of success, and ultimately to sustainability. During the extension period (October 2008 to December 2010), IPHC will continue to focus its efforts on achieving objectives in six strategic performance areas (SPA) as follows: SPA Objectives 1. Maternal Health and Family Planning ƒ Improve women's and maternal health complications and deaths in order to effectively reduce maternal 2. Child Health ƒ Improve child survival and reduce infant and child mortality 3. Youth Program ƒ Increase youth participation in the promotion and provision of youth friendly services in order to reduce reproductive health problems among adolescents 4. HIV & AIDS ƒ ƒ ƒ ƒ … Strengthen the integration of counseling and testing into routine PHC activities Strengthen and expand a comprehensive PMTCT program Strengthen ARV treatment service delivery to those who are infected with HIV Strengthen palliative care in the PHC setting; integrate palliative care for TB and HIV all with the purpose of reducing the impact of HIV & AIDS 5. PHC Services and Systems ƒ Contribute to service quality improvements through selected tools such as Primary Health Care Supervision Manual and structures such as clinic committees ƒ Improve the knowledge and use by service staff and managers of health information of acceptable quality for planning, management, and monitoring ƒ Support the development of district health plans, operational plans, operational reviews, and the conducting of DHERs …in order to strengthen functional district health systems 6. OVC ƒ Strengthen CBOs and networks AIDS mortality in their family to enhance the welfare of children affected by HIV & Proposed activities in these six SPAs will be supported by key overarching and cross-cutting strategies focused on a general principle of consolidation, integration, and institutionalization of all project initiatives. The extension period will allow for consolidation through better field coordination, both to encourage integration and as a cost-saving measure. The IPHC team proposes institutionalization efforts that will focus on achieving sustainability of the initiatives MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 5 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. introduced through MSH’s technical assistance over the years of its partnership with USAID and the NDOH. This proposal describes an approach designed to convert the project’s activities into sustainable approaches not only in those districts and facilities that are directly supported by IPHC but also ensuring the expansion and replication of these successes in other districts, facilities, and communities Operationally and managerially, the extension of the IPHC Project will begin with existing staff who are under employment contracts currently scheduled to end largely between June and September 2008. The proposed approach to consolidation of provincial initiatives will result in revised provincial team structures as current employment contracts come to an end. Current subcontracts with Health Systems Trust and University Research Corporation will not be renewed when they come to an end in September 2008 and resulting staff vacancies will be filled as required. The project will continue to work out of its present location in Pretoria with provincial teams working virtually from their home bases in project provinces. During the extension period, MSH will devote additional attention to documenting and communicating IPHC’s results, success stories, and lessons learned. The communication plan (Annex 5) is designed to provide the NDOH, at all levels, with necessary information to extend IPHC’s approaches, tools, and activities to other provinces, districts, and facilities which are not directly partnered with IPHC. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 6 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Integrated Primary Health Care Project (TASC II) I. BACKGROUND The Integrated Primary Health Care (IPHC) Project under TASCII is funded by the United States Agency for International Development (USAID) and charged with providing technical assistance to strengthen the district health system in five provinces of South Africa. IPHC has been implemented by a consortium led by Management Sciences for Health (MSH), in partnership with Health Systems Trust (HST) and the University Research Corporation (URC). IPHC contributes to the achievement of USAID/South Africa’s goal to help the South African Government (SAG) reduce the impact of HIV & AIDS and provide better health care for historically disadvantaged South Africans. The IPHC Project has focused much of its efforts on strengthening the district health systems as a vehicle for primary health care (PHC) service delivery. By targeting impact at the facility level, IPHC is building a cadre of health care workers who are competent in planning, implementing, and evaluating comprehensive, high-quality, integrated PHC in a sustainable manner. The IPHC Project has focused capacity building in eight targeted districts in the five provinces; these were chosen after a national mapping activity. Selection of the target provinces and districts was based on criteria for support provided by the National Department of Health (NDOH), and in consultation with USAID and the provinces and districts themselves. These provinces and districts, which will remain the focus of support during the extension period, are: ƒ Chris Hani and Alfred Nzo districts, in the Eastern Cape Province; ƒ Sisonke and Uthungulu districts in KwaZulu Natal (KZN) Province; ƒ Capricorn and Sekhukhune districts in Limpopo Province; ƒ Gert Sibande District in Mpumalanga Province; ƒ Bojanala District in North West Province. The technical interventions of the IPHC Project were designed to support USAID/South Africa’s health goal of “reduced impact of HIV & AIDS, and improved health care for all South Africans” by addressing the following intermediate results (IRs): IR 1 Strengthened HIV & AIDS prevention measures IR 2 Increased availability of quality STI (sexually transmitted infection) services IR 3 Improved treatment of tuberculosis (TB) and AIDS IR 4 Expanded HIV & AIDS care and support IR 5 Expanded systems and services of selected PHC systems In October 2005, USAID issued its new Health Sector Strategic Objective: “Strengthened capacity to deliver sustainable and integrated PHC and HIV & AIDS services.” In consultation with the South African NDOH, it was agreed that the IPHC Project would focus on the following objectives: Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 7 ƒ Improved maternal health and family planning, with emphasis on youth ƒ Improved child survival, health, and nutrition ƒ Increased and improved participation of youth in advocacy for reproductive health and sexuality and solutions for gender violence ƒ Reduced transmission and impact of HIV & AIDS ƒ Strengthened PHC systems and service delivery ƒ Strengthened community support for orphans and other vulnerable children (OVC) II. PROJECT PERFORMANCE TO DATE IPHC’s work is organized into six strategic performance areas (SPAs) with related sub￾components in some SPAs. These SPAs are aligned with the project objectives noted above. Project interventions are currently undertaken in a number of facilities, districts, and communities as indicated in Table 1. Table 1. Current Number of Sites Directly Supported, by Strategic Performance Area Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. MCH/ FP Child Health Youth Programs HIV & AIDS PHC Services and OVC VCT PMTCT ARVs* Palliative, (CBOs) Basic Palliative, HIV/TB Systems (districts) 69 69 58 69 69 12/59 62 62 8 23 *First number refers to sites initiating ARVs; second number refers to feeder sites. A summary of key activities being carried out under each SPA follows and the quantitative targets and results for related indicators during the 2006–2007 project year are provided in Annex 1. IPHC considers many of these activities to be foundational, and expects to continue many as well as build upon them to further advance the project’s impact during the extension period. The project will also undertake initiatives during the extension period to address the sustainability of these activities. A. SPA 1: Maternal Health and Family Planning IPHC implements many initiatives to improve the integration and quality of reproductive health (RH) services, all of which are designed to build the capacity of the District Management Team (DMT) to supervise these services. Training of service providers, according to national guidelines, aims at improving knowledge and capacity in family planning (FP). Joint supervisory visits by project staff and provincial and district program managers are aimed at improving technical capacity of managers through in-depth reviews of FP and antenatal care (ANC) using the Primary Health Care Supervision Manual. During these visits, emphasis is placed on improving the quality of data recording and information reporting in order to improve the monitoring and evaluation (M&E) of ANC and FP services. All facilities are supported to ensure routine voluntary counseling and testing (VCT) of all ANC clients with provision of test results and referral for treatment as required. Also, quality improvement plans are developed at the facilities to increase the uptake of VCT among pregnant women. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 8 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. B. SPA 2: Child Health IPHC’s support for child health services is focused on nutrition and growth monitoring, immunization, and Integrated Management of Childhood Illness (IMCI) for children under five. The project’s technical advisors work with district program managers and clinic supervisors to conduct in-depth reviews of child health services in project facilities, using the standard checklist from the Supervisor’s Manual. Immunization is supported by giving special attention to the availability of equipment and supplies, along with an estimation of the number of newborns in the catchment area, monthly target-setting, and monitoring of cohort coverage. A system for tracking drop-outs has been introduced, and quality improvement plans have been formulated to include facility- and community-level mentoring in integrated maternal and child health (IMCH). In pursuit of the NDOH target (60% of facilities covered by trained nurses), facility nurses have been trained in IMCI. Additionally, IMCI registers have been introduced to reinforce proper IMCI clinical tasks and recording thereof in the facilities; IMCI guidelines were distributed and monitored for use; and strategies such as establishment of oral rehydration therapy (ORT) corners in the clinics have been pursued. While IPHC has provided hands-on support to selected facilities in each target district, the project also supports the planning and implementation of subdistrict- and district-level PHC reviews in which progress and problems are monitored and actions planned for all facilities in the district. C. SPA 3: Youth Program IPHC has established the Youth Friendly Services (YFS) Program in 58 facilities where facility managers and providers are mentored on YFS standards in compliance with the Youth and Adolescents Policy. This program began in 2006 and addresses reproductive health and family planning, antenatal care, voluntary counseling and testing, sexually transmitted infections, and gender-based violence education in addition to clinic testing and treatment services. A baseline assessment of the current status of youth access and utilization of health services was implemented by HST, a project partner. A youth-centered approach was designed to involve young people at all levels of service planning and delivery. Youth groups are identified, workshops are conducted, and youth peer mentors have been placed in each facility. Project advisors assist to create, strengthen, and maintain linkages and working relationships between the youth mentors (96 are currently active) and clinic staff. Youth mentors also participate with facility staff in school health visits. D. SPA 4: HIV & AIDS SPA 4.1: Counseling and Testing. The project supports 69 facilities to increase the uptake in HIV counseling and testing, through three kinds of interventions: improving quality of VCT, coaching and mentoring of professionals, and increasing youth access to VCT services. The project also focused on the integration of HIV & AIDS, TB and STI (HAST) services through introduction of the HAST approach and related operational plans. In-depth reviews of TB and STI programs were conducted to assess adherence to national guidelines. Compulsory counseling and routine offering of testing was reinforced for all STI, TB, FP, and ANC clients. Health care providers are mentored in the integration of VCT services along with proper record-keeping and data flow during facility visits. STI reviews are conducted using the District STI Quality of Care Assessment (DISCA) tool. To encourage VCT uptake, IPHC supports health education talks, contacts with local communities, and support groups. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 9 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. SPA 4.2: PMTCT. The project approach focuses on improving the quality of maternal and child health (MCH) services, the capacity and skill of health care providers, and the full integration of prevention of mother-to-child transmission (PMTCT) of HIV & AIDS into the full package of PHC services. In-depth clinic reviews are the backbone of this quality improvement effort with the specific aim of identifying any gaps in service. HIV counseling for groups and individuals is provided by lay counselors to increase testing rates of all new ANC visitors. National PMTCT protocols provide guidance for administration of nevirapine to babies of HIV-positive mothers within 72 hours of birth, and five-day PMTCT training is provided in partnership with HAST teams. District and subdistrict meetings and PHC reviews are also used to review progress and problems, through the use of routine district health information system (DHIS) data. The involvement of district supervisors in the clinic reviews is intended to enhance their monitoring and supervision of PMTCT, including support for the use of facility data for self-monitoring. SPA 4.3: HIV & AIDS Treatment (ARV). The project provides a consultant physician to support the management of antiretroviral (ARV) treatment in 12 hospitals. Other IPHC technical advisors support the feeder clinics which refer clients for treatment. Staging of patients, the feedback of results, and the management of referrals for treatment are important elements of IPHC’s support in this sub-component. The emphasis on assisting project-supported facilities to achieve accreditation to provide ARV services is also very important. SPA 4.4 & 4.5: Palliative Care. The two palliative care subcomponents focus on basic care and support for HIV patients as well as integrated testing and treatment for those affected by both HIV and TB. As one important element of providing appropriate palliative care is the proper clinical staging of HIV-positive cases using CD4 cell counts, this procedure receives emphasis in training and mentoring. Project focuses also include the proper maintenance of clinic-level client records (with effective feedback of test results from the testing facilities), and the HAST protocol for integration of services and record linkage. Within-clinic referral procedures are emphasized, including testing of TB and STI patients and an integrated treatment process. The project provides much of this mentoring through direct facility visits, but subdistrict and district reviews also offer opportunity for assessment and reinforcement of standard procedures and practices. Review of HAST registers are used to discover and then address gaps in performance. E. SPA 5: PHC Systems and Services SPA 5.1: Quality Assurance. A variety of related interventions have been delivered by the project including promotion of clinic quality improvement committees and the increased frequency of facility supervision visits, training, and supervision using the Primary Health Care Supervision Manual. Project technical advisors and provincial staff are actively involved in joint performance monitoring at the facility level and in promotion and support of periodic PHC review meetings at subdistrict and district levels. Action plans and lessons learned are shared during these review meetings to encourage use of best practices and scaling up. The project has introduced defaulter tracing through use of facility tracing registers; coordinated transport schedules to facilitate lab specimen collection and transfer; and the provision of national policies, standards, and guidelines documents. IPHC technical advisors have encouraged community participation by strengthening the role of clinic committees in support of clinic services. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 10 SPA 5.2: District Health Information System. The project has supported implementation of the new version of the DHIS, including the introduction of new indicator data sets. Workshops were held in conjunction with district health expenditure review (DHER) and district health planning (DHP) activities to introduce the use of information for decision-making and action; to this end, the Tool to Assess the Level of Information Utilization (TALI) was introduced in some districts to assess the flow and use of data. Data analysis and review activities were conducted in some districts in conjunction with “road show” visits on data use to selected facilities. Facilities have been supported to produce and display data graphs and maps. SPA 5.3: District Health Systems. The project is working with district health staff to raise awareness of the importance of planning and reviewing expenditures within the context of the National Planning and Budgeting Cycle and provincial planning, giving special emphasis to supporting the annual planning and DHERs. The project has also supported district health staff to undertake data audits and analysis, and to write reports. Municipality reviews have been held prior to district reviews in some instances and, where they are functional, District Health Councils are also brought into the health planning process. F. SPA 6: Orphans and Other Vulnerable Children The primary strategy for supporting the protection and care of OVC is to strengthen community￾based organizations (CBOs) by providing grants as well as on-site coaching and mentoring, the provision of informational materials, and related consultations. IPHC developed and disseminated a poster that classifies the various types of activity and support that CBOs may choose to organize. Project efforts are also extended to communities to strengthen their support to families and children in need. The CBO grants are providing several types of support to OVC including the facilitation of NGO collaboration with the Department of Social Development in leveraging food parcels, general education, legal assistance, health care, and psychosocial and financial support. The recruitment and training of home caregivers is also a growing element of this program, as is access to ARV treatment and follow-up. The latter requires good links among caregivers, CBOs, and treatment facilities. Target-setting based on estimates of OVC in each CBO’s catchment area is being promoted, and CBOs have also been given support to improve their record-keeping through the introduction of new intake registers. With a focus on sustainability, all CBO grantees are being supported to seek other sources of financial support. III. GAPS AND OPPORTUNITIES FOR IMPROVEMENT Two sources were drawn on for this section: the self-assessment of continuing challenges by the IPHC team (as presented in the project’s 2006–2007 annual report), and the January 2008 internal project assessment conducted by the MSH/IPHC team. While these gaps and opportunities will be the focus of much of the work of the extension period, it is important to recognize that some of these challenges are beyond the direct control of the project. It is anticipated that, by naming them here, some challenges will be elevated for increased attention both by the project and by its partners at district and facility levels during the extension period. Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 11 A. Gaps in Services for HIV & AIDS, STIs, and TB These include challenges related to the need to increase VCT for pregnant women and to attracting more men for VCT. Data and information gaps result in delays in initiation and proper PMTCT protocols. ARV activities are negatively affected by long and difficult accreditation process for referring facilities, turnover in staff qualified to provide ART and low treatment rates among children. In STI services, a low proportion of client partners return for testing and treatment services and some STI clients are not managed through the syndromic approach. Counseling TB clients to obtain HIV tests could be improved. For all these diseases, stigma constrains clients from seeking palliative care services. B. Human Resource Constraints Staff turnover and vacancies impact the work of the IPHC Project because of the need for continuity among trained staff to implement services at the facilities and trained counterparts in the DHMTs. Equally, if not more importantly, this issue also negatively impacts health service delivery throughout South Africa. In particular, attrition has negatively affected delivery of IMCI, VCT, and PMTCT services, and has required continuous in-service training by the project. Service staff and managers have limited understanding of how to work effectively with youth or of the importance of providing YFS. Vacancies for information officers at the district and facility levels have had a negative impact on the ability to collect, analyze, and use data effectively—both for the project and for improved service delivery. C. Data Collection, Analysis, and Use At the facility level, standardization and maintenance of clinic registries lead to difficulties in data collection and use. Different versions of the DHIS are used in project districts and this challenge is further complicated by the use of an entirely different information system in KwaZulu Natal. The lack of a national OVC database and lack of computer equipment among CBOs to record data also constrains proper data collection and use in OVC activities. Referral systems at service facilities and links with home care workers are weak and require attention. At the district level, operational plans are not aligned with provincial budgets or annual performance plans and there is also inconsistency in understanding of the system to be used for DHERs. Feedback of DHIS data summaries and performance comparisons from districts to facilities is very limited and is inadequate in terms of population projections for facility catchment areas. Further use of the DHIS by districts and facilities can be reinforced by IPHC and used—with minor supplemental data collection for certain PEPFAR indicators—for its reporting to USAID. Project staff requires training to use Health Information System Project (HISP) software that is tailored for project use. The project strategy for supporting data and service performance reviews can be made more efficient, effective, and sustainable by shifting to a team approach and working with clusters of facilities rather than individual facilities. D. Coordination and Other Challenges Lack of coordination among partners at the district level has led to a shift of which facilities receive visits for technical support as well as the double-counting of clients served by the Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 12 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. partners. Abolishment of the PHC coordinator post has led to inadequate coordination in preparing for and conducting district PHC review meetings. Particular challenges are presented in the project’s attempts to strengthen the District Health System. These include the absence of or non-functional District Health Councils in some project districts, vacancies in district health manager posts, and a general lack of leadership at the district level with inexperienced junior staff assigned to important management tasks. E. General Cross-Cutting Challenges and Gaps ƒ Sustainability. There is a great need to prepare an exit strategy for the project that institutionalizes the main performance improvement and integration interventions at all levels and includes the youth mentors and OVC special program activities. ƒ Scaling up. A means is needed to share interventions and best practices throughout the participating districts and provinces as well as with the NDOH. ƒ Structural issues and barriers. Limited interactions and poor linkages (e.g., disconnections in planning between district and higher levels of government) need to be addressed through improved contacts and communications with the national and provincial levels. ƒ Supervision. While progress has been made, methods of supervision need to be made more efficient and shifted to the district and provincial offices. ƒ Use of data for everyday management. Considerable improvement is called for in completeness, accuracy, use, reporting, and feedback of service data. ƒ Alignment issues. A particular challenge exists with regard to alignment of district and provincial plans and budgets. ƒ Links between facilities and communities. While these are improving—especially with the support of home-based or community-based care workers—the existence and functioning of clinic committees is variable. ƒ Supervisory guidelines and other tools. Significant needs and opportunities exist for routinely incorporating data in supervision visits and program performance reviews, including use of the District Management, Economic, Social Cohesion, Community and Human Resource (MESH) tool to assess district-level performance. IV. PROJECT RESULTS EXPECTED FOR THE EXTENSION PERIOD The IPHC Project Vision The IPHC Project expects, as a result of its efforts, the emergence of functional district health systems capable of providing accessible, high-quality HIV & AIDS interventions that are fully integrated into primary health care services for the benefit of disadvantaged communities, thus offering previously disadvantaged South Africans a better quality of life. To ensure the achievement of this vision, the guiding principle during the extension period is that all activities carried out under each SPA are undertaken with an increased focus on the progressive institutionalization of all tools, methods, and processes within the District Health Services. This institutionalization is absolutely essential if the work of the IPHC is to be sustained. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 13 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. A. Objectives by SPA For the remainder of the IPHC Project, each IPHC SPA will be aligned with the achievement of one or more development objectives as indicated in Table 2. Table 2: Objectives by Strategic Performance Area SPA Objectives 1. Maternal Health and Family Planning ƒ Improve women's and maternal health complications and deaths in order to effectively reduce maternal 2. Child Health ƒ Improve child survival and reduce infant and child mortality 3. Youth Program ƒ Increase youth participation in the promotion and provision to reduce reproductive health problems among adolescents of YFS in order 4. HIV & AIDS ƒ ƒ ƒ ƒ … Strengthen the integration of counseling and testing into routine PHC activities Strengthen and expand a comprehensive PMTCT program Strengthen ARV treatment service delivery to those who are infected with HIV Strengthen palliative care in the PHC setting; integrate palliative care for TB and HIV all with the purpose of reducing the impact of HIV & AIDS 5. PHC Services and Systems ƒ Contribute to service quality improvements through selected tools such as Primary Health Care Supervision Manual and structures such as clinic committees ƒ Improve the knowledge and use by service staff and managers of health information of acceptable quality for planning, management, and monitoring ƒ Support the development of district health plans, operational plans, operational reviews, and the conducting of DHERs …in order to strengthen functional district health systems 6. OVC ƒ Strengthen CBOs and networks AIDS mortality in their family to enhance the welfare of children affected by HIV & The activities and initiatives in these six SPAs will be supported by important overarching and cross-cutting strategies focused on the integration, performance improvement, district leadership development, documentation, and sustainability of all initiatives. B. Key Performance Targets The IPHC Project has been given or has defined for itself a total of 120 performance indicators. Forty-one of these indicators have been used to set annual performance targets; 32 apply to PEPFAR indicators and appear in the Country Operational Plan (COP). The remaining nine targets have been set by IPHC. All targets are updated annually; Annex 4 lists the objectives and targets with baseline values for each SPA during the extension period. The method of capturing and monitoring all indicators against baseline values and targets is described in Section VIII. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 14 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. V. OVERARCHING STRATEGIES The IPHC team has identified the following overarching strategies as its response to identified gaps and opportunities. These strategies will be pursued alongside the technical assistance and other project activities dedicated to the specific SPAs (covered in Sections II and VI of this proposal). The overarching strategies—including the intention to better document and communicate what the project is doing and its results—are presented below. A. A General Principle: Consolidation, Integration, and Institutionalization To the extent possible, all facility and district training, facilitation, and technical support will be designed and carried out in a consolidated and integrated manner to address several SPAs at the same visit. All field initiatives will also be coordinated to ensure, to the extent possible, full provincial and district staff participation. A continuing theme of this approach is to ensure that all client needs are attended to in an integrated fashion and across the SPAs when project staff members visit facilities. At the same time, training and other facilitation visits to districts, subdistricts, and facilities will be planned so that teams of IPHC technical advisors work together in a coordinated fashion. Guidelines and procedures for the above activities will be modified to enable non-specialist IPHC team members to carry them out when necessary for efficiency. B. Provincial/District Support and Oversight Job descriptions and functions of project provincial coordinators will be reviewed and revised as necessary to ensure that they provide appropriate oversight of these cross-cutting strategies and activities. The cross-cutting strategies will be examined for each province and prioritized, based on gaps and opportunities identified in each location. Assumed new or additional activities at the provincial and/or district levels might include better networking among offices, subdistricts, clusters of facilities, facilities, NGOs, and CBOs. These communications links and collaboration will be confirmed in writing. Communications will be improved to ensure that provincial and national departments of health are fully informed of the project’s activities, best practices, and lessons through regular reports and presentations. IPHC district-specific annual plans will be worked out in collaboration with the DHMT and included in district operational plans. Also supported will be routine district and facility data analysis (DHIS) and problem identification; district and facility staffing inventories and skill audits; and monitoring of the preparation of district training schedules, plans, and budgets. Provincial coordinators will also ensure the timely submission of monthly USAID indicator data to the national IPHC Project office and database (from both the DHIS and IPHC data capture); oversee the scheduling; and conduct a results analysis of the various program review processes at facility, cluster, and subdistrict levels. C. Strengthened Links with the National and Provincial Departments of Health Recognizing the need for better informing national and provincial health officials about the activities, lessons, and best practices of the IPHC Project, project staff will plan and implement several activities during the extension period to ensure these individuals and units are kept up to date on the progress of the project, priority service performance, and disease trends: MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 15 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ƒ Participation in regularly scheduled national and provincial meetings, and planning opportunities for special project presentations. ƒ Shared project reports, including USAID annual reports. ƒ Joint design and implementation of special activities with relevant offices and programs. ƒ Participation in national calendar events related to the project’s SPAs and provision of technical guidance on these events, as required. ƒ Identification of, and ongoing communication with key health contact persons in important places/roles such as the President’s Office, mayors, District AIDS Councils (DACs), community liaison officers (CLOs), and other local bodies and persons. Communication will include information about relevant project activities and results. ƒ Identification and sharing of models of success among districts and clinics, and among partner CBOs. D. Project Database Development and Use Recognizing the continuing challenge of managing data in IPHC Project sites, and the need to collect and report reliable information on a regular basis, the project seeks the support of HISP to prepare a modified HISP data platform in the central IPHC office for receiving, analyzing, and reporting the data summaries required by USAID. This will necessitate the following: ƒ Confirming the data elements and indicators obtainable through the DHIS from district databases, and those indicators which must be captured from the facility registers; ƒ Devising monthly formats and procedures for project provincial teams to capture and compile the data; ƒ Setting up and modifying as necessary the HISP data platform for IPHC purposes; ƒ Designing the required report formats and content; ƒ Training project staff in data cleaning, entry, analysis, and report generation; ƒ Undertake the extraction of DHIS data for project facilities to construct a clinic data baseline for the project, with support from HST; ƒ Develop a training database to track and report the various training activities supported by the project to USAID; ƒ Participate in the consortium of organizations endeavoring to create a common OVC data warehouse. E. District Data Management During the extension period, the project will work to strengthen the routine use of the DHIS at the district level for data analysis and generation of reports (including routine feedback to facilities) and, most importantly, the use of the DHIS data and analysis by district health officials for level monitoring of disease, service performance and coverage, and resource allocation and use in support of annual planning and budgeting, operational planning, and targeted performance improvement efforts to subdistricts and facilities. Some specific needs for and examples of district level data use include: ƒ Confirmation of facility responsibility/catchment areas and estimation of total and target group populations within each responsibility area (RA) to better enable facility monitoring of service; MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 16 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ƒ Selected indicator analysis for district and subdistrict monitoring and interpretation to enhance PHC reviews; ƒ Identification of high risk areas (villages and wards) for disease detection, prevention, and management across the district; ƒ Identification of high-performing facilities or subdistricts (whose achievements can be used for success stories and as examples of sound management) and of low-performing facilities or subdistricts (that deserve more attention for uncovering and overcoming constraints). Using this information, the project might go on to include mentoring and coaching by high-performing facilities and subdistricts for low-performing ones. F. Service Performance Reviews The project team will devise a modified approach for conducting program performance reviews for use at subdistrict or cluster levels. This will require that the existing supervisory checklists be modified for guiding joint reviews of clusters of facilities for performance in selected service subjects with the incorporation of critical data items and indicators drawn from registers and reports that facility representatives bring to the cluster review. Participants in cluster reviews will be facilitated by small teams of IPHC technical staff along with district and provincial supervisors to confirm their performance with the use of selected indicators, and then to jointly discuss how to improve performance in areas of common difficulty. These reviews will generally focus on several project SPAs and the related health services which have strong natural linkages and interdependencies (e.g., ANC, VCT, FP, and PMTCT) to support their integration. Cluster reviews will also reinforce tools and methods for facility analysis and display of important trend data, particularly the self-assessment of critical service tasks and recording, and monitoring of coverage across the responsibility area to identify pockets of underserved communities. Supervisory and performance review visits will continue to be made to selected individual facilities that most need such attention, but these are expected to decline in favor of the more efficient and effective cluster reviews. G. Facility Staff and Skills Audits In support of district training for improved planning and budgeting, IPHC will develop and apply a procedure for conducting periodic audits of staff in the project districts to determine who is currently posted in PHC facilities, and their current skills in terms of post-basic training received. Included in this effort will be support for maintaining staff database inventories at district and provincial levels, including the skill inventory. This audit effort will link directly to the project’s provincial coordinators’ role of supporting the district process of planning annual in-service training programs and the application for in-service training seats in the various courses scheduled in the Regional Training Centre (RTC). It will also serve to gradually shift the training budgets from the IPHC Project to the districts themselves to sustain the training program implemented by IPHC. H. Identification and Documentation of Best Practices and Models of Success The cluster performance review process described above will be the primary means to identify well-performing facilities and clusters of facilities across the SPAs addressed by this project. To aid this process, the project will define criteria for identifying best practices and a means to MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 17 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. substantiate exceptional facility performance through the use of data and assessment results. Case studies will be conducted for these facilities to develop descriptions of the management characteristics and practices being employed by these facilities. Assigned focal points within IPHC, with the support of provincial coordinators, will continuously seek potential best practices and success models and undertake investigations for developing descriptive material. The development of such documentation will be part of the overall communication plan for this project, described in Annex 5. I. Procedures for Referring and Tracking Patients A common difficulty in insuring the integration and continuity of care are gaps in the referral and feedback communications taking place within facilities, as well as between facilities and higher level referral sites and laboratory services. Examples include the linkages required for screening and care of OVC, HIV-positive pregnant women, at-risk children, and TB and STI patients. The NDOH has initiated a project to improve the referral system in all facilities. The IPHC Project will link to this process to strengthen this system within the facilities that we support. The HAST approach offers one opportunity to improve these linkages and will be further developed during the extension period. In addition, IPHC proposes to undertake a special effort with selected district supervisors, facility managers, and CBOs to review current clinical procedures, registers, records, and referral slips, and to devise steps to prevent patients from missing necessary testing, higher-level diagnosis and treatment, and referral back to the primary facility and community￾based organizations for treatment continuation and follow-up. The ultimate objective of this effort is to develop proven procedures for insuring continuous tracking of all infectious disease patients and maternal cases to prevent drop-outs from the care process and to confirm the outcome of treatment. The improved process will be documented, built into procedures manuals and performance review processes and shared with the relevant national health programs. VI. New Technical Approaches and Activities by SPA As noted in Section II, many current activities have been identified by IPHC for continuation. In addition, a number of complementary new activities are proposed for initiation during the remainder of the current project year (March–December 2008) and into the extension period. These new activities are presented below and elucidated in the project work plan (see Annex 2). Some of these activities will be combined with or supported by the overarching strategies introduced in Section V. SPA 1: Maternal Health and Family Planning ƒ Capacitate district coordinators on managing and supporting implementation of the Reproductive Health Policies and Guidelines at the facility level. ƒ Support the review of RH services at facility and district levels to improve quality of care. ƒ Strengthen the district perinatal review process with a focus on the inclusion of PHC facilities. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 18 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. SPA 2: Child Health ƒ Review and address the availability of nurses trained in IMCI by district and within facilities. ƒ Provide special support to IMCI supervision through local surveys of performance and standards adherence. ƒ Support facilitation of the IMCI complementary course to improve the management of ARV and treatment of OIs in HIV-positive children under 5 years of age. ƒ Ensure integration of PMTCT management in child health program. ƒ Capture and analyze child health indicators from the DHIS to identify areas for improvement. SPA 3: Youth Program ƒ Support the implementation of the Adolescent Health Policy at district and facility levels to increase uptake of RH services (including HIV & AIDS services). ƒ Establish and improve linkages between health facilities and youth in the clinic catchment areas and capacitate the community youth to support implementation of YFS. ƒ Create support for a youth presence in health facilities and involve youth in service provision with the aim of integrating and sustaining youth mentors through programs of the DOH or Department of Social Development. ƒ Institutionalize the YFS strategy at the district and provincial levels to ensure its continuation (including placing these activities in the DHP). ƒ Focus youth mentor activities on achieving the 10 YFS standards. ƒ Foster and support acceptance of the youth program as a cross-cutting strategy in health facilities and districts by ongoing monitoring, support, and mentoring of youth volunteers in health services. SPA 4: HIV & AIDS SPA 4.1: Counseling and Testing ƒ Focus on provider-initiated VCT with emphasis on STI, TB, ANC, and high-risk groups. ƒ Establish internal referral procedures and registers at facilities to insure testing of these risk groups. ƒ Promote the use of supervisor checklists and program review protocols (e.g., the DISCA Tool) for use by clinic managers supported by the prescribed use of register and report data. ƒ Focus on STI partner identification, notification, and testing through improved recording of tracing, testing, and counseling. ƒ Monitor and improve the quality of counseling of high-risk clients. SPA 4.2: PMTCT ƒ Provide additional training in PMTCT in collaboration with RTCs, district training plans and budgets, and the NDOH (HIV Cluster). ƒ Improve recording and reporting for PMTCT to ensure the proper management of mothers and babies at risk. ƒ Support the implementation of the NDOH’s dual therapy policy in the management of PMTCT. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 19 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ƒ Encourage the “full supermarket” approach to ANC at all facilities to support opportunities to promote and provide PMTCT. ƒ Improved recording and reporting of CD4 cell count data by monitoring the feedback of lab results. ƒ Develop improvement strategies with the clinic managers for monitoring feedback and recording of lab results. ƒ Strengthen the strategies for increasing the uptake of nevirapine and PCR (polymerase chain reaction) testing. SPA 4.3: HIV & AIDS Treatment (ARV) ƒ Form a link with the director of ARV services to keep the NDOH informed of the project’s efforts, and to keep abreast of new standards and policies. ƒ Develop a quality database showing facility performance to maintain the data required for the USAID PEPFAR data warehouse. ƒ Take action to insure that ARV quality assurance and training management is are within district and provincial DOHs. SPA 4.4: Palliative Care (Basic) ƒ Provide training in the full range of case situations and disease progression to identify HIV-positive clients; this includes record-keeping, data analysis, and facility performance self-monitoring and improvement, means. ƒ Establish clinic-based HIV support groups and encourage greater promotion, training, and use of “treatment buddies.” ƒ Focus on better confirmation of HIV status of all health facility clients, identification and management of OIs, and insuring the full package of HIV palliative care. ƒ Support the facilities to better monitor the recording of cotrimoxazole to HIV-positive clients who have initiated ARV treatment. SPA 4.5: Palliative Care (TB/HIV) ƒ Collaborate with the TB TASC team to resolve the indicator discrepancies between facility registers and district TB electronic registers. ƒ Reinforce the DOTS support system. ƒ Support the enhancement of integrated TB/HIV case identification and management through performance assessments and improved procedures. ƒ Carry out focused training on integrated TB/HIV case management. ƒ Improve the management of TB in support of the multidrug resistant (MDR) TB program. SPA 5: PHC Systems and Service SPA 5.1: Quality Assurance ƒ Provide training in practical quality assurance techniques. ƒ Involve DMTs in service quality monitoring and supervision processes. ƒ Support the district and subdistrict quarterly review process enabling better analysis and use of routine data (DHIS) for reviewing annual plan progress, identifying and defining service performance gaps and problems, and devising performance improvement interventions. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 20 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ƒ Strengthen the links between health facilities and communities by fostering active participation of clinic committees. SPA 5.2: District Health Information System ƒ Support the District Health Office to establish a routine process of DHIS data review, analysis, and feedback of performance trends and gaps to the facilities. ƒ Plan and carry out joint visits by the IPHC team in collaboration with provincial and district supervisors to districts and clusters of facilities to support service program reviews using DHIS facility data. ƒ Train all IPHC technical and provincial staff in the use of the DHIS. SPA 5.3: District Health Systems ƒ Enhance and periodically apply the MESH tool for assessing performance of the District DOH. ƒ Assist the District DOH to improve the alignment between provincial and district annual plans and budgets, and district costing of the Annual Performance Plan and Operational Plan, along with the Medium-Term Expenditure Framework. ƒ Using the above and other methods, strive to institutionalize the use of IPHC tools, such as the Primary Health Care Supervision Manual and checklists. SPA 6: Orphans and Other Vulnerable Children ƒ Address the improvement of OVC case referral for care, including procedures and feed￾back to OVC caregivers, CBOs, and community caregivers. ƒ Strengthen the case and service reporting to IPHC for onward reporting to USAID with emphasis on shifting this reporting to responsible health and social service officers at district and provincial levels. ƒ Utilize Child Care Forums at district and ward levels to strengthen CBO linkages with local leaders. ƒ Further clarify and define CBO catchment/responsibility areas for improving estimation of numbers of OVC, setting targets, determining budgets, and allocating grants. ƒ Develop a strategy and additional means to enhance CBO independence and phase-out of IPHC grants. ƒ Identify especially high-performing CBOs and groups of community caregivers in order to define models of success; develop and share success stories through the IPHC communication plan. ƒ Establish links with officials and political leaders, such as the President’s Office, as targets of project communications on the community care component. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 21 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. VII. PROJECT MANAGEMENT From March 2008, the IPHC project will reduce the current listed positions and staff but will continue to honore URC and HST subcontract positions until the end of these subcontracts on 30 September 2008 . In the same way current employment contracts for IPHC staff run for several more months (terminating between June and late-September 2008); those end dates also will be honored (see Annex 3). Some changes include: ƒ Some current job descriptions will be revised so that the positions can better match IPHC’s needs as outlined in this proposal (e.g, human resource manager, grants manager, and logistics manager). ƒ As current employment contracts end, the provincial teams will be reduced to two staff members in total (including the coordinator). Some existing vacancies and vacancies created by departing partner staff will need to be filled, but the total number of provincial team staff will be reduced. The quality assurance technical advisor position will become a less-specialized and more multi-purpose position of program manager. ƒ Each Provincial Team will continue to be led by a coordinator who, in keeping with the new thrusts of this proposal, will begin to focus more attention on activities at the District Health Office level, and on communications and linkages with the Provincial Health Department. Such activities include the project’s support to expenditure reviews, annual planning and annual budget formulation, along with the strengthening and focusing of district and sub-district program review processes. This will reduce—but not eliminate— the visits by the coordinator to the facilities and cluster program review activities. VIII. MONITORING AND REPORTING For the extension period, the IPHC Project process for assembling data for monitoring and reporting against defined targets and indicators is being modified to rely more on the routine DHIS, which will reduce the amount of indicator data to be captured during contacts by project advisors at each facility. This approach has several advantages and some challenges. The project will be able to rely on data routinely reported to districts through the DHIS, and at the same time will devote some facilitation to district-level data analysis and feedback to the facilities. As such, it is expected that data entry into the HISP data platforms at district level will be timelier and the production of appropriate comparative reports for feedback to the facilities will begin to take place. This approach will also help reduce redundancy and inconsistency between the data captured by project staff and that reported up through the DHIS. While beginning to make greater use of DHIS information for reporting, the IPHC provincial teams will still be required to capture PEPFAR data and indicators which are not currently reported within the DHIS. This process will be streamlined by building data capture into the cluster program review processes. Participating facilities will be required to bring the relevant registers and records to these cluster review meetings for use in the program review process. At the same time, project technical advisors will be able to extract the required monthly PEPFAR indicators. MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 22 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. The project’s provincial teams will be required to transmit the new DHIS client counts along with the PEPFAR indicators to the central project office for entry into the modified HISP database. The data platform will be set up with HISP staff support to enable IPHC to maintain the full set of indicator data across all SPAs and participating districts, and to generate the required data summaries for reporting to USAID on a quarterly basis (ARV figures) and semi￾annually and annually for all other indicators. ƒ resources ƒ The project vision ƒ SPA objectives ƒ SPA targets and indicators ƒ SPA activities: products and milestones ƒ Critical A broad project monitoring framework has been defined to cover all the basic levels of project planning and monitoring: The conceptual framework encompassing these levels of planning and monitoring is presented in Annex 4. It is proposed that SPA managers populate all columns in this framework and update them quarterly. Further guidance for the monitoring of performance targets is provided in Annex 3.2 which lists all SPAs with their objectives, indicators, related target achievement, the data source, and any special data requirements (e.g., gender disaggregation). The annex also provides for the indicator targets, 2007–2008 baseline values, and 2008–2009 target values where available. The aggregation of indicator and activity data will be undertaken quarterly by SPA managers and submitted to the M&E officer, with the exception of the client counts which are provided through the DHIS and facility contacts, as described above. Reports for USAID will be compiled as scheduled by USAID (quarterly for ARV client data, semi-annually and annually for all required indicators). MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 23 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. IX. Summary of Resource Requirements The complete budget for IPHC activities proposed for the 27-month period (October 2008 to December 2010) is provided as a separate document. Cost savings from March 1 2008 until September 30, 2008, will result from significant changes in the IPHC approach which include a slightly reduced field staff combined with a focus on consolidating and integrating field activities. In addition, there will be an increasing focus on institutionalization and sustainability of key project interventions as the project moves into its final two years, including assisting with the hand over of youth and OVC efforts for funding from other sources. MSH will maintain its home office support for technical, finance and administrative activities, but at a reduced level of effort. Short-term technical assistance (STTA) proposed will focus on follow-up on recommendations from previous interventions and from recent data quality and impact assessments. Technical areas addressed through STTA will include strengthening the project’s monitoring and evaluation systems. MSH plans to undertake an assessment of its work in these selected critical SPAs during the first half of 2009 and will also provide related STTA. To ensure a very high quality final contract report, IPHC will have STTA from MSH home office communications staff and from an AIDS/PEPFAR specialist to ensure that all PEPFAR results are captured and summarized for the life of the project.    MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 24 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ANNEXES 1. Summary of Quantitative Results, by SPA (Project Year 2006–2007) 2. Work Plan 3. Staffing List 4. Indicator List and M&E Framework 5. Communication Plan MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 25 Use or disclosure of data contained on this sheet is subject to the restriction on the title page of this proposal. ANNEX 1. Summary of Quantitative Results by SPAs (Project Year 2006–2007) Output Indicators Target Total Achieved SPA 1: Maternal Health and Family Planning No. total antenatal visits 10,000 88,008 No. of 1st ANC visits 26,064 Average no. ANC visits per client 4 No. Pregnant women tested for HIV 19,560 No. Pregnant women who tested positive 5,353 SPA 2: Child Health No. of children fully immunized 18,677 No. of children given vitamin A at 6–12 months 19,875 No. children <12 months who received DPT3 1,500 16,227 No. nurses trained in IMCI 200 275 No. CBHWs trained in IMCI home care 85 No. of children weighed 644,723 No. children underweight 1,179 SPA 3: Youth Program No. youth given family planning and RH education 49,315 No. youth receiving HIV testing 8,035 Increase in youth utilization of RH services 45% No. of youths counseled on FP and provided FP methods 800 4,414 SPA 4: HIV & AIDS No. clients receiving counseling and testing and their test results 35,000 45,310 No. of new STI clients 33,542 No. clients not accessing testing 13,127 No. and % of STI client partners responding for testing and treatment 5,730/21% No. STI clients syndromically treated 28,492 No. and % of first ANC visits who were tested for HIV 20,225/ 88.9% No. of facility staff supported in data management and data use on palliative care 863 No clients receiving palliative care including HIV/TB 35,000 20,034 No. TB clients tested for TB 5,000 2,462 No. HIV patients treated for TB 5,000 1,206 Facilities supported in ARV treatment 12 hospitals 59 Feeder clinics Clients provided with ARV treatment in supported facilities 17,000 22,712 SPA 5: PHC Services and Systems % of facilities having quality improvement teams 100% 92% SPA 6: OVC and Community Support Networks Number of NGOs receiving small grants 23 up from 6 Number of OVC served 10,000 12,086 up from 6,208 Community OVC caregivers trained 400 477 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 26 26 ANNEX 2. Workplan MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 27 27 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 28 28 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 29 29 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 30 30 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 31 31 MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 32 32 ANNEX 3. Staffing List CURRENT IPHC STAFF February 2008 NAME TITLE MSH Staff Agherdine, Nadia LOGISTICS MANAGER Brown, Muriel BOOKKEEPER Combrink, Gert Cornelius DEPUTY CHIEF OF PARTY Dhlamini, Nontuthuzelo PROVINCIAL COORDINATOR, KWAZULU NATAL Evans, Carole FINANCIAL MANAGER Jaxa, Nozipho ART TECHNICAL ADVISOR, EASTERN CAPE Hlabano, Vusi NATIONAL HIV COORDINATOR Klaas, Nondumiso Primrose HIV TECHNICAL ADVISOR, EASTERN CAPE Mabusela, Mmaphohla Emily YOUTH ADVOCACY TECHNICAL ADVISOR Manzini, Khanyisa M&E MANAGER Mazaleni, Nomathemba CHIEF OF PARTY Mudzunga, Gloria DATA CAPTURER Muthambi, Tendani PROGRAM ASSOCIATE Ngomane, Sharon DEPUTY DIRECTOR TECHNICAL PROGRAMS Pataki, MS ARV TECHNICAL ADVISOR, LIMPOPO Pitsi, Moipone HUMAN RESOURCES MANAGER Radebe, S GRANTS MANAGER Sefularo, Kgomotso HIV TECHNICAL ADVISOR, NORTHWEST Setshotlo, Pule INFORMATION TECHNOLOGY OFFICER Shamu, Rodwell OVC MANAGER Sokhela, GP HIV TECHNICAL ADVISOR, NORTHWEST Taole, Khetisa DISTRICT HEALTH PLANNING AND FINANCE TECHNICAL ADVISOR Thebela, Theresa Mmalego RECEPTIONIST Thela, Samuel PROVINCIAL COORD., MPUMALANGA Currently vacant HIV TECHNICAL ADVISOR, NORTHWEST Sub Contractors’ Staff Nonceba, Languza (HST) PROVINCIAL COORDINATOR, EASTERN CAPE Ogrinah, Nogoveni (HST) PROVINCIAL COORDINATOR, LIMPOPO Currently vacant (HST) PROVINCIAL COORDINATOR, NORTHWEST Currently vacant (HST) HEALTH INFORMATION OFFICER Currently vacant (HST) CLINIC SUPERVISOR Timbela, Damane (URC) QUALITY ASSURANCE TECHNICAL ADVISOR, EASTERN CAPE Maponya, Luisa (URC) QUALITY ASSURANCE TECHNICAL ADVISOR, LIMPOPO Ndlela, Bongoli (URC) QUALITY ASSURANCE TECHNICAL ADVISOR, KZN Lubisa, Judith (URC) QUALITY ASSURANCE TECHNICAL ADVISOR, MPUMALANGA Currently vacant (URC) QUALITY ASSURANCE TECHNICAL ADVISOR, NORTHWEST MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Annex 4_Monitoring and Evaluation Framework_Page 33 of 42 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal ANNEX 4 Indicator List and M&E Framework Monitoring and Evaluation Framework Levels of Planning and Monitoring Vision: The IPHC Project expects, as a result of its efforts, the emergence of functional district health systems capable of providing accessible, high quality HIV & AIDS interventions that are fully integrated into primary health care services for the benefit of disadvantaged communities, thus offering previously disadvantaged South Africans a better quality of life. SPA Objectives Performance Targets Activities Critical Products (Milestones) Critical Resources 1. MH & FP Improve women's and maternal health maternal complications and deaths in order to effectively reduce 2. Child Health To improve child survival and reduce infant and child mortality 3. Youth Increase youth participation in the promotion and provision of YFS in order to reduce reproductive health problems among adolescents 4. HIV & AIDS Reduce the impact of HIV & AIDS 4.1 CT Strengthen the integration activities of counseling and testing into routine PHC 4.2 PMTCT Strengthen and expand a comprehensive PMTCT program 4.3 ARV Strengthen the ARV treatment service delivery to those who are infected with HIV 4.4 PC-Basic Strengthen palliative care in the PHC setting; integrate palliative care 4.5 for TB and HIV PC-TB/HIV 5 PHC Ser/Sys Strengthen functional district health systems 5.1 Qual Assur Contribute to service quality improvement through selected structures tools and 5.2 HIS Improve the knowledge and use by service staff and managers of health information 5.3 DHS Support the development of district health plans, operational plans, operational reviews, and the conducting of DHERs 6. OVC Strengthen CBOs and networks to enhance the welfare of children affected by HIV & AIDS mortality in their family MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 34 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal Annex 4.1: Integrated Primary Health Care Project – Extension Period SPA Objectives, Indicators, and Targets SPA 1: Maternal Health and Family Planning Objective: Improve women's and maternal health in order to effectively reduce maternal complications and deaths Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of antenatal care (ANC) visits provided assisted facilities. by skilled providers from USG- 88,000 15,000 DHIS # of USG-assisted services delivery points services providing FP counseling or 58 65 DC Tool # of individuals counseled on FP and provided with FP methods 4,414 4,500 PHC register Gender # of people trained in RH/FP with USG funds (health professionals, primary health care workers, community health workers, volunteers, non-health personnel) 186 220 Training db Gender SPA 2: Child Health Objective: To improve child survival and reduce infant and child mortality Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of IPHC-assisted facilities offering IMCI services in accordance with the national standard treatment guidelines for IMCI services 68 * DC Tool Province # of people (health professionals, primary health care workers, community health workers, volunteers, non-health personnel) trained in child health care and child nutrition through USG-supported programs 360 * Training db Gender # of children under 5yrs fully immunized 18,677 * DHIS % of children fully immunized under 1yr, for the month 90% PHC register # of children given Vitamin A at 6 -11 months 19,875 * DHIS # of children under 12 months who received DPT3 in a given year from USG￾supported program 16,227 * DHIS PHC register Gender # of children under 5yrs weighed * DHIS MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 35 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal SPA 3: Youth Program Objective: Increase youth participation in the promotion and provision on YFS in order to reduce reproductive health problems among adolescents Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of individuals given information VCT information 286,230 * DC Tool Gender # of individuals who received pre-test counseling 20,212 * DHIS Gender # of individuals who tested for HIV 13,341 * DC Tool Gender # of individuals given STI information 263,776 * DHIS Gender # of individuals who received STI treatment 12,014 * DC Tool Gender # of STI clients who tested for HIV 4,904 * DHIS Gender # of individuals given FP information 230,915 * DC Tool Gender # of individuals provided with FP methods (subset of SPA 1) 64,210 * DHIS Gender SPA 4: HIV & AIDS Component 1: Counseling and Testing Objective: Strengthen the integration of counseling and testing into routine PHC activities Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of service outlets providing counseling and testing according to African or international standards South 69 80 DC Tool Province # of clients who received pre-test counseling (excl. antenatal) 20,415 * DHIS PHC register Gender # of clients who received counseling and testing and received their test results (excl. antenatal) 45,310 40,000 DHIS PCH register Gender # of newly identified HIV-infected individuals who were screened for TB 3,664 * DHIS PHC register Gender # of individuals trained in counseling and testing according or international standards. to South African 349 300 Training db Gender Indirect # of individuals who received counseling and testing for HIV and received their test results 27,554 85,000 Attendance register & DHIS Gender MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 36 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal Component 2: PMTCT Objective: Strengthen and expand a comprehensive PMTCT program Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of service outlets providing the minimum package of according to South African or international standards PMTCT services 69 80 DC Tool Province # of pregnant women who received HIV and received their test results counseling and testing for PMTCT 20,225 19,000 DHIS # of pregnant women provided with a complete course of antiretroviral prophylaxis in a PMTCT setting 2,788 5,000 DHIS # of health workers trained in the provision of PMTCT services according to South African or international standards 220 350 Training db Gender Indirect number of pregnant women who received HIV counseling for PMTCT and received their test results and testing 36,789 30,000 Attendance register & DHIS Indirect number of pregnant women provided with a complete course of antiretroviral prophylaxis in a PMTCT setting 5,203 5,000 Attendance register & DHIS Component 3: HIV & AIDS Treatment (ARV) Objective: Strengthen the ARV treatment service delivery to those who are infected with HIV Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of service outlets providing ARV therapy 71 15 DC Tool Province # of HIV-infected individuals provided reporting period with ARV treatment at the end of the 4,850 Hospital register/DHIS Gender # of individuals who started ARV therapy during the reporting period 2,600 Hospital register/DHIS Gender # of individuals who ever received ARV therapy period (cumulative) by the end of the reporting 2,600 Hospital register/DHIS Gender Total # of health workers trained to deliver ART services, according to national and/or international standards 186 400 Hospital register/DHIS Gender Indirect # of individuals receiving antiretroviral therapy at the end of the reporting period 22,172 20,000 Attendance register & DHIS Gender MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 37 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal Component 4: Palliative Care – Basic Objective: Strengthen palliative care in the PHC setting; integrate palliative care for TB and HIV Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of service outlets providing HIV-related palliative care (including TB/HIV) 62 80 DC Tool Province # of TB/HIV) individuals provided with HIV-related palliative care (including 20,024 25,000 PHC register Gender # of individuals receiving post exposure prophylaxis (PEP) [initiated and referred at this PHC facility] 90 250 DHIS Gender # of family members of HIV-infected individuals who received HIV-related palliative care 2,651 5,000 PHC register NGO register Gender # of HIV-infected individuals provided with the basic care package (incl. TB/HIV) 10,000 PHC register NGO register Gender # of HIV-infected individuals who received cotrimoxazole prophylaxis 3,715 6,000 DHIS Gender # of individuals trained to provide HIV-related TB/HIV) palliative care (including 13 400 Training db Gender Indirect # of individuals with facility-based, community-based, and/or home￾based HIV-related palliative care including those HIV-infected individuals who received treatment for TB 22,418 25,000 Attendance register & DHIS Gender Component 5: Palliative Care – TB/HIV Objective: Strengthen palliative care in the PHC setting; integrate palliative care for TB and HIV Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of service outlets providing treatment for tuberculosis (TB) to individuals (diagnosed or presumed) in a palliative care setting HIV-infected 62 80 DC Tool Province # of HIV-infected clients attending receiving treatment for TB disease HIV care/treatment services that are 1,296 8,000 PHC register Gender # of TB patients tested for HIV 2,463 8,000 DHIS Gender # of individuals trained to provide clinical prophylaxis and/or treatment for TB to HIV-infected individuals (diagnosed or presumed) 3 300 Training db Gender Indirect # of HIV-infected clients attending HIV care/treatment services that are receiving treatment for TB disease 891 10,000 Attendance register & DHIS Gender MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 38 SPA 5: PHC Systems and Services Component 1 : Quality Assurance Objective: Contribute to service quality improvement through selected tools and structures Indicator 2007–08 Baseline 2008–09 Target 200–-09 Achievement Data Source Disaggregated by # of supervisory visits to the facility 1 visit/month/ facility DC Tool Facility # of clinic committee meetings with records/minutes & action plans * # of facility staff, supervisors and program coordinators trained using clinic supervision tools 160 Gender Component 2 : District Health Information Objective: Improve the knowledge and use by service staff and managers of health information of acceptable quality for planning, management, and monitoring Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of facilities with updated catchment population been posted for all ages? * # of facilities with updated graphed indicators that are displayed * # of facilities with that have implemented the TALI tool 10 Component 3: District Health System Objective: Support the development of district health plans, operational plans, operational reviews, and the conducting of DHERs Indicator 2007-08 Baseline 2008-09 Target 2008-09 Achievement Data Source Disaggregated by # of districts with completed DHPs per year 8 # of districts with completed DHER reports 8 # of facilities where the MESH assessment tool has been administered * SPA 6: OVC and Community Support Network Objective: Strengthen Community HBC Organizations and Networks for enhancing the welfare of children affected by HIV mortality in their family Indicator 2007–08 Baseline 2008–09 Target 2008–09 Achievement Data Source Disaggregated by # of OVC served - 3 or more services 7,156 10,000 Grantee register Gender Service type # of OVC served - 2 or less services 4,930 5,000 Grantee register Gender Service type # of providers/caretakers trained in caring for OVC 477 500 Grantee register Gender Number of OVC served by OVC (indirect) 2,591 * DC Tool Gender Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 39 ANNEX 5 Project Communications Strategy Introduction and Situation Analysis By offering a two-year cost extension to the IPHC project, the mission and the South African National Department of Health (NDOH) have indicated their confidence in the project and effectively reiterated their understanding that HIV & AIDS services are only as effective as the underlying primary health care system. This extension also reconfirms USAID’s and NDOH’s joint commitment to reducing the impact of HIV & AIDS and bettering health care for historically disadvantaged South Africans, through the sustainable strengthening and integration of services delivered by primary health care facilities and managed by District Health Offices. That being said, the wealth of epidemiological and service performance information being generated through the facilitation processes at the community, facility, and district level needs to be assembled, packaged, and delivered to the higher levels of the health system, especially the Provincial Health Departments, the NDOH, and the managers of the national health programs. This extension advances a long-term commitment of the US and South African governments to the people of South Africa. It has never been more important than now to effectively communicate this commitment and its life-saving impacts to the people and governments of South Africa and the US. We are proposing proactive, creative, and engaging ways to tell these stories, to highlight the results and achievements of the unique partnership among the Integrated Primary Health Care (IPHC) Project, USAID/South Africa, and the NDOH. The IPHC team, with strategic capacity￾building support from the MSH home office, will use all appropriate means to achieve the project’s communications goals. Those goals and objectives are designed to complement the technical work and accomplishments of IPHC, USAID/South Africa, and the NDOH. Our plan uses proven and consistently implemented communication techniques to ensure the high visibility of USAID/South Africa and the NDOH. We will work in partnership with the USAID mission in South Africa and in accordance with the norms and standards established by USAID. Upon award, we will work closely with stakeholders at the NDOH and USAID/South Africa to continually hone tactics and messages; we will also check in with these stakeholders periodically to ensure that the plan and the targets remain relevant and effective. Special effort has been and will continue to be made to ensure that this plan is realistic and operational; implementation of this plan will be made an overarching project activity to which each SPA manager and the project leadership will contribute. GOAL & OBJECTIVES The overall goal of the communications plan is to increase and reinforce awareness—especially among our targeted audiences (see below)—of IPHC’s positive impact on individuals and communities in the targeted districts. We will do so by capturing and disseminating the project’s successes and lessons learned. Key communications objectives that support and drive us toward this overarching communications goal are: ƒ Wide dissemination of accurate and relevant information about the project’s efforts and impact; Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 40 ƒ Demonstrations of the project’s effectiveness, namely the positive impact of integrating services and strengthening primary health care service delivery (thus evidence of well￾spent US taxpayer money); ƒ Creation, cultivation, and maintenance of effective information-sharing with stakeholders to demonstrate that integrated health programs are working to mitigate the impact of HIV & AIDS and opportunistic infections and to improve health overall. TARGET AUDIENCES Our primary audience is the Government of South Africa, the NDOH, USAID/South Africa, and other in-country USG partners. Members of the Government of South Africa will: ƒ be better informed of health trends, resource and performance gaps, and effective improvement strategies, helping them to make informed funding and policy decisions (e.g., government leadership); ƒ understand lessons and better practices to enhance better replication and scale-up across other provinces and programs (e.g., national-level program clusters); ƒ benefit from “positive peer pressure” and be encouraged to take pride in their own successes through the publication and dissemination of project impact and results (e.g., provincial-, district-, and community-level stakeholders and actors). Communications targeting USG partners in South Africa will encourage and enable the sharing of interventions, best practices, and lessons learned; communications activities will include technical seminars, dissemination workshops, and broad distribution of annual technical reports and relevant portions of other reports. The secondary audience for this project’s communications is the people of South Africa. As a result of IPHC communications activities, they will come to know that the American people and the Government of South Africa are committed to reducing the impact of HIV & AIDS and providing better health care for historically disadvantaged South Africans. South Africans will also understand the importance of strong, integrated primary health care programs in reducing the impact of HIV & AIDS in South Africa. Furthermore, the people of South Africa will be reminded of the impact and availability of essential health care offered by district health services and community-based entities. We will reach this audience primarily through proven USAID public affairs tools such as broadly disseminated Success Stories that strike an individual-level chord with the South African people. The tertiary audience is the American people and decision-makers in Washington, DC, (including the US Congress). We will communicate effectively to demonstrate the positive impact of their tax dollars allocated to foreign assistance. Buy-in from these audiences will help to ensure ongoing financial and political support for foreign assistance, specifically for effective, efficient, and integrated health programming. Proven USAID public affairs tools like Success Stories disseminated through USAID/South Africa, USAID/Washington, and (when appropriate) OGAC will be very effective means to reach Americans and American decision-makers. Non-technical materials like the “Voices of the Children” mentioned as a project deliverable are also important in reaching this audience. Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 41 Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal KEY MESSAGES Key messages will be a part—overtly or subtly—of all communication activities. Key messages might require modification through the course of the cost extension, but those currently appropriate for most audiences include the following: ƒ The US government and the South African NDOH are committed to reducing the impact of HIV & AIDS and improving primary health care services for historically disadvantaged South Africans. ƒ IPHC and the NDOH—partnering with district- and facility-level public health actors—are making a tangible difference in the lives of South Africans by reducing the impact of HIV & AIDS by improving primary health care services for historically disadvantaged South Africans. ƒ A strengthened and integrated primary health care system in South Africa is enabling the delivery of effective services to reduce the impact of HIV & AIDS and other related diseases. ƒ IPHC technical advisors are working with local health professionals, clinic services, and at the community level to fight and reduce the impact of HIV & AIDS in their districts. ƒ IPHC and the NDOH—partnering with provincial and district public health actors—are building a cadre of health workers to ensure sustainable primary health care service delivery. ƒ IPHC is a USAID project supported by the American people. OPERATIONAL COMMUNICATIONS This type of communication will cover issues which mainly focus on service delivery, best practices, encouraging implementation of government protocols and generally reinforcing proper service delivery. This method could be used in attempting to strengthen problematic system components, like patient and specimen referral systems. Best practices could be communicated to all health professionals within a district, extending also to facilities that are not directly supported by the project. Feedback from the DHIS is another area which needs to be communicated more constantly to all facilities and health professionals Proposed Communication Products and Activities The IPHC team will implement this communications plan in concert with technical work; the communications calendar/work plan will dovetail with the project’s technical activities, and communications deliverables—like those in the rest of the project—will be results-oriented and demonstrate evidence-based decision-making. • Calendars of planned communications events and new initiatives for communities and sub-districts, across districts and provinces, and at the national level will help to keep communications efforts relevant and complementary to the technical work which is our foundation. • Ongoing support from the home office will help to build the capacity of local staff to conduct effective and strategic communications activities on this project. • Case Studies and Success Stories will be regularly produced and widely distributed at national, provincial, district, and sub-district levels; to USAID and PEPFAR (as relevant); and, whenever possible, to the media and via relevant websites (USAID, MSH, NDOH, etc.). These documents will present compelling stories demonstrating the overall impact of MSH Extension Proposal, TO 800 under IQC GHS-I-00-03-00030-00 Page 42 the project through the experience of an individual or small group of individuals; they also will be included in semiannual and annual technical reports. • Voices of Children: The OVC Program has outsourced the creation and development of the “Voices of Children Report.” • Broaden annual report distribution to include sub-districts and facilities as well as sub￾contractors. This will enhance outreach to our tertiary audiences, help facilitate knowledge exchange among stakeholders and actors, and help build the project’s technical credibility. • Technical Seminars will be sponsored and facilitated by the project. In consultation with the NDOH, we will choose relevant themes and topics for workshops/seminars to provide avenues for the effective exchange of best practices and lessons learned. Furthermore, these sessions can help bolster the perception of the technical leadership of the NDOH, IPHC, and USAID. • End-of-Project Report. A full description of the project’s work and accomplishments will be further enhanced by compelling photography and design, and stories from individuals to highlight the important impact of this partnership. • Events (seminars, consultations, and planning meetings) with the health departments at national, provincial, and district levels. Use or disclosure of data contained on this sheet is subject to the restrictions on the title page of this proposal For more information, please visit: http://ghtechproject.com Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com