SUDAN HEALTH TRANSFORMATION PROJECT ASSESSMENT REPORT March 2008 This publication was produced by IT Shows, Inc. for review by the United States Agency for International Development SUDAN HEALTH TRANSFORMATION PROJECT (SHTP) ASSESSMENT REPORT March 2008 Margaret Neuse, M.A., M.P.H. Team Leader Connie Davis, M.D., M.P.H. Infectious Disease Specialist Victor Masbayi, M.P.H. Maternal Health Specialist Mary Harvey, M.P.H. Child Health Specialist Yogesh Rajkotia Health Economist DISCLAIMER The authors' views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID/Sudan: Sudan Health Transformation Project Assessment Report i ACKNOWLEDGEMENTS The Assessment team thanks Khadijat Mojidi, the USAID PHN Officer for Southern Sudan for her outstanding support during the entire Assessment process and for ensuring a positive outcome. The team greatly appreciated her clear vision and guidance, which helped hone the recommendations and ensure their usefulness in shaping the future of Sudan Health Transformation Project (SHTP) and the follow-on. The USAID SHTP team and Mission staff were also critical resources during the field trip and in Juba; Martin Swaka provided invaluable support to the team in Juba. The team also thanks Allan Reed, USAID Director for Southern Sudan Program, for his enthusiastic support for our endeavors and for accompanying us on one leg of the fieldwork and sharing his stories of travel in Southern Sudan. We also thank Mary Hobbs for her company in Mundri and for providing historical perspective on SHTP. The Assessment team expresses our profound appreciation to the leadership of the Ministry of Health for their warm welcome, their willingness to share frankly their views of the SHTP, the positive aspects, the challenges, the history and their concerns, and their understanding of the need at this juncture to look to the future. Dr. Monywirr Arop, Under Secretary, Dr. Nathan Atem, the Director General of Primary Health Care, Dr. Mawien Atem, Director, Quality Assurance and Control, and Ms. Janet Michael, Director of Nursing/Midwifery has our appreciation for their attendance at the de-briefing by the Assessment team. We were especially pleased by their designation of Dr. Baba as the MOH representative for the field trip. He provided invaluable insights into the project background, issues regarding the partners and the sites, and information about the ever￾evolving context of health programming in Southern Sudan. The usefulness of the field trip relied on the untiring logistical and programmatic support and hospitality of the INGOs and their local partners: AAHI and MRDA in Mundri; IMC in Tambura; IRC in Panyijar; and CARE in Twic East. We greatly appreciated the partners’ work in organizing the visits to the PHCCs and PHCUs, as well as the meetings with the various community groups, Village Health Committees, County Health Departments, women’s organizations, TBAs, youth groups, and others. We appreciated their thoughts about the health needs of their communities, problems they encounter, and ideas on how the project can help them improve their health and welfare. We thank the INGOs and their partners for sharing with us their successes, their concerns, and their ideas about future programming. We were honored by the warm welcome extended to us by local officials at all the sites and want to thank them as well for sharing their views of the SHTP and local health issues with us. We also want to thank SAVE and their partner SIDF for their visit to Juba to share experiences with the SHTP, and regret that we were not able to visit them in Mvolo. JSI/Juba provided a phenomenal amount of briefing documentation and staff time and attention to the Assessment team and we recognize that this required extraordinary efforts on the part of many staff. The materials were useful in gaining understanding of the strengths, challenges and issues with the project. We also thank the JSI team for their logistical management of the field visits which took a great deal of time and effort; it paid off in an exceptionally smooth field trip!! We also want to thank the donor agencies and other stakeholders with whom we met in Juba for sharing their insights and experiences with us. We appreciated the opportunities USAID/Sudan: Sudan Health Transformation Project Assessment Report ii to meet with other USAID-funded Cooperating Agencies as a group and individually in Juba and learn about their work. We had an extraordinary team. The team members had extensive, complementary experiences, shared different perspectives, strong commitment and convictions, developed agreement on the conclusions and recommendations in fairly short order, and each member wrote quickly and very well. We hope that the report adequately reflects all team member efforts. The team recognized that there was no way we could fully understand every aspect of this complex project. We regret any and all errors of fact and omission. We hope that the recommendations, especially for actions in the near-term, will be valuable for strengthening the SHTP during its final 1-2 years and for improving primary health care services in the SHTP counties. In the end, this is what matters, that the people of Southern Sudan get better PHC services and improved health from these investments. We sensed a strong commitment from all stakeholders to do all possible to achieve this outcome. USAID/Sudan: Sudan Health Transformation Project Assessment Report iii ACRONYMS AAHI Action Africa Help-International ACT Artemisinin Combination Therapy AMREF African Medical Research Foundation ANC Antenatal Care ARI Acute Respiratory Infections AS Artesunate AQ Amodiaquine BCC Behavior Change Communications BEmOC Basic Emergency Obstetric Care BPHS Basic Package of Health Services CA Cooperative Agreement CAC Community Action Cycle CARE Cooperative for Assistance and Relief Everywhere CBD Community Based Distributors CBHCO Community Based Health Care Officer CBO Community Based Organization CHD County Health Department CHW Community Health Worker CMO County Medical Officer CO Clinical Officer CPA Comprehensive Peace Agreement CQ Chloroquine CRS Catholic Relief Services DPT Diphtheria, Pertussis, and Tetanus EmOC Emergency Obstetric Care EOP End of Project EPI Expanded Program on Immunizations FBOs Faith Based Organizations FP Family Planning GAVI Global Alliance for Vaccines and Immunizations GFATM Global Fund for AIDS, Tuberculosis, and Malaria GNU Government of National Unity GOS Government of Sudan GOSS Government of Southern Sudan HBLSS Home-Based Life Saving Skills HIV/AIDS Human-Immuno-Deficiency Virus/Acquired Immune Deficiency Syndrome HMIS Health Management Information System HSS Health Systems Strengthening HHPs Home Health Promoters IDPs Internally Displaced Persons IDSR Integrated Disease Surveillance and Response IEC Information Education and Communication IECHC Integrated Essential Child Health Care IMC International Medical Corps INGO International Non-Governmental Organization ISS Immunization Systems Support USAID/Sudan: Sudan Health Transformation Project Assessment Report iv ITNs Insecticide Treated Bednets IPT-p Intermittent Presumptive Treatment in Pregnancy IRC International Rescue Committee JDT Joint Donor Team JSI JSI Research and Training Institute, Inc KAP Knowledge Attitude and Practice LLITNs Long-Lasting Insecticide Treated Nets LMIS Logistics Management Information System LNGOs Local Non-Governmental Organizations MA Medical Assistant MCH Maternal and Child Health MCHW Maternal and Child Health Care Worker MDTF Multi-Donor Trust Fund M&E Monitoring and Evaluation MIP Malaria in Pregnancy MRDA Mundri Relief and Development Association MMR Maternal Mortality Ratio NGOs Non-Governmental Organizations NID National Immunization Day NPA Norwegian Peoples Aid OFDA Office of U.S. Foreign Disaster Assistance OLS Operation Lifeline Sudan OPD Outpatient Department ORS Oral Rehydration Salts ORT Oral Rehydration Therapy PDQ Partnership Defined Quality PEI Polio Eradication Initiative PHC Primary Health Care PHCC Primary Health Care Center PHCU Primary Health Care Unit PHN Population, Health and Nutrition PIO Public International Organization PMP Performance Monitoring Plan PSI Population Services International RFA Request for Application RH Reproductive Health SCF-USA Save the Children Federation, Inc/USA SHTP Sudan Health Transformation Project SIDF Sudan Inland Development Foundation SINGO Sudanese Indigenous Non-governmental Organization SMC Sudan Medical Care SOH Secretariat of Health SORD Sudan Organization for Rehabilitation and Development SP Samaritan’s Purse SPLM/A Sudan People’s Liberation Movement/Army SRRC Sudan Relief and Rehabilitation Commission SS Southern Sudan STIs Sexually Transmitted Infections TA Technical Assistance USAID/Sudan: Sudan Health Transformation Project Assessment Report v TB Tuberculosis TBA Traditional Birth Attendant TT Tetanus Toxoid UN United Nations UNICEF United Nations Children’s Fund USAID United States Agency for International Development VCT Voluntary Counseling and Testing VHC Village Health Committees WATSAN Water and Sanitation WHO World Health Organization WV World Vision USAID/Sudan: Sudan Health Transformation Project Assessment Report vi TABLE OF CONTENTS Page Acknowledgements .............................................................................................i Acronyms ...........................................................................................................iii Executive Summary.........................................................................................viii Section I. Introduction.......................................................................................1 Section II. Background......................................................................................3 A. Southern Sudan .........................................................................................3 B. Health Status in Southern Sudan...............................................................3 C. Primary Health Care Structure in Southern Sudan ....................................4 D. Sudan Health Transformation Project (SHTP) ...........................................5 E. Early SHTP Implementation Issues...........................................................5 Section III. Assessment Methodology..............................................................7 Section IV. Analysis and Findings....................................................................9 A. Access and Use of High Impact Services...................................................9 B. Demand for Services and Behavior Change to Improve Health...............16 C. Availability of Essential drugs at PHC sites and Logistics Management ..18 D. In-Service Training and Service Delivery Capacity Building....................20 E. Infrastructure...........................................................................................22 F. Strengthening Decentralized, County Level Capacity for Management And Delivery of PHC services..................................................................24 G. Strengthening the Capacity of Local NGOs to Manage and Deliver High Impact PHC Services ..............................................................................26 H. National and Level Policy Framework for Maternal and Reproductive Health and M & E ...................................................................................27 I. Collaboration of INGOs, GOSS, JSI and USAID to support SHTP..........28 J. Leadership and Management: USAID, JSI, INGOs, and GOSS..............28 Section V. Achievements of the SHTP, Major Shortfalls, and Recommendations............................................................................................32 A. Achievements..........................................................................................32 B. Major Shortfalls .......................................................................................33 C. Recommendations for Actions in the Near-Term ....................................35 D. Considerations for Future Health Investments........................................38 USAID/Sudan: Sudan Health Transformation Project Assessment Report vii ANNEXES ............................................................................................................1 Annex A: Statement of Work ...............................................................................2 Annex B: Reference Documents for SHTP Assessment Team ...........................9 Annex C: Stakeholders Consulted/Interviewed..................................................12 Annex D: USAID Evaluation Team Program......................................................15 Annex E: Interview Guides.................................................................................19 Annex F: SHTP Tools/Products for Dissemination.............................................33 Annex G: Technical Notes on HIV/AIDS, TB, Malaria and Family Planning ................................................................................34 Annex H: Recommendations for Actions in the Near-Term ...............................39 Annex I: SHTP Performance Assessment PowerPoint Presentation..............43 FIGURES AND TABLES Page Figure 1: Map of Southern Sudan with SHTP Counties ........................................2 Table 1: Annual DPT 1 and DPT 3 Coverage of Children under One Year of Age by County .....................................................................................10 Table 2: Vitamin A Supplementation for Children under Five Years of Age.......12 Table 3: Percentage of Women Receiving Intermittent Presumptive Treatment (IPT) of Malaria During Pregnancy per MOH Protocols .......................13 Table 4: Percent of Deliveries Assisted by a Skilled Attendant..........................14 USAID/Sudan: Sudan Health Transformation Project Assessment Report viii EXECUTIVE SUMMARY PURPOSE AND BACKGROUND In April 2004 USAID/Sudan signed a Cooperative Agreement (CA) with JSI Research and Training Institute, Inc (JSI) to implement the five-year, $29.5 million Sudan Health Transformation Project (SHTP). The CA was increased by $10 million to $39.5 in September 2007. The SHTP focuses on: (1) provision of primary health care services through grants to International Non-Governmental Organizations (INGOs) partnering with Local NGOs (LNGOs) so as to increase their capacity to provide these services on their own; and (2) strengthening of County Health Departments (CHDs) and state and national capacity to manage the health care system. The objectives of the assessment were to: • Assess progress to date in achieving the deliverables of the SHTP; • Assess the models and approaches used in implementing the SHTP; and • Recommend immediate adjustments to improve performance for the remaining project period and follow-on investments for the USAID health program in Southern Sudan. The five-member team covered the major technical areas of the project. Given the paucity of reliable quantitative data to assess progress under the SHTP, the team relied on qualitative information. Sources of information included: project, government and other documents; interviews with key informants and stakeholders in Washington and Juba; and observations, interviews, group meetings, and documents collected during a ten-day field trip to four of the six project-supported counties and visits to 21 of the 99 SHTP PHC facilities. The team members did not have enough time to comprehend the many documents available and to fully understand a very complex situation, rapidly changing environment, and a project history marked by confusion and turmoil. The team learned early on that the participants in this project history had their own perceptions of the causes and consequences of various problems. Since it was not be possible to disentangle the many threads of the stories, the team agreed to focus on the consequences for project deliverables and propose actions to correct the problems identified. The SHTP was designed during the last years of a long-standing civil war between Northern and Southern Sudan, which ended in the signing of the Comprehensive Peace Agreement (CPA) of January 2005. Implementation of primary health care in Southern Sudan has been affected by important environmental, social, and political factors such as: • Poor roads and communication infrastructure and health infrastructure that is rudimentary and affected by war and neglect; health care workers with poor access to training and other learning resources; • Poor health status of the population with infant mortality rates and maternal mortality ratios among the highest in the world and with limited education; • Pockets of insecurity and uncertainty about the future political situation along with large numbers of returnees, i.e., Southern Sudanese who fled to neighboring countries during the war who are now coming home with high expectations; • Nascent government institutions: the Government of Southern Sudan (GOSS) is developing a new decentralized structure of ten states which in turn devolve USAID/Sudan: Sudan Health Transformation Project Assessment Report ix authorities to counties (which have existed for a longer time); these state and county structures have few human and financial resources, and the GOSS/MOH is developing its own health policies, strategies, and implementation approaches, with limited staff and infrastructure. The SHTP was intended to provide seven high impact interventions: immunizations, Vitamin A, antenatal care (ANC), treatment of diarrheal disease with oral rehydration therapy (ORT), Long-Lasting Insecticide Treated Bed Nets (LLITNs), case management of malaria, and case management of acute respiratory infection (ARI). Although the project had been planned to cover 20 counties, to date the project has funded PHC services in 6 counties, with three more counties and three urban areas to be covered shortly. PHC services are delivered through a system of peripheral sites called Primary Health Care Units (PHCUs) that refer clients to Primary Health Care Centers (PHCCs) for comprehensive primary health care. To provide the seven high impact interventions, the project supported these systems components: behavior change communication/information, education and communication (BCC/IEC) and community mobilization (primarily through support for Village Health Committees); provision of and logistics to deliver essential drugs; pre- and in-service training and service provider capacity building; rehabilitation of PHCCs and PHCUs and the drilling of boreholes and construction of pit latrines and waste pits to assure adequate sanitation at the facilities; and, capacity building and systems strengthening for the County Health Departments, Local Non-Governmental Organizations, and the Village Health Committees (VHCs) so that all can effectively play their roles in managing and supporting the delivery of PHC services. SHTP was to review and revise, as needed, national health policies that affect primary health care programs. SHTP implementation relied on the partnership of the INGOs, GOSS/MOH, USAID, and JSI. FINDINGS Access and Use of High Impact Services: The SHTP does not have a solid statistical base and reliable denominators to assess if and by how much access to and use of the seven high impact interventions have increased since its inception. SHTP reports that more service sites meet GOSS standards for staffing; the team’s visits verified that many service sites have trained staff and have been refurbished and equipped, thereby making access more likely. However, other systems failures (see components below), such as erratic vaccine and drug supplies at the county and service points, have compromised service availability. Finally, the Assessment team found no evidence that SHTP used any means (e.g., standardized, systematic supervision) to assess the quality of the services being provided in its facilities. IEC/BCC: The IEC/BCC materials and activities of the SHTP that were observed were spotty and limited in scope and reach. The team found no evidence of a BCC/IEC strategy, no identification of key messages and behaviors to change or reinforce, many missed opportunities to mobilize communities and provide health education. For example, the SHTP did not actively utilize innovative strategies such as using women’s groups or other civil society client groups to pass health messages or conduct health education during, waiting time in the facilities. Nor did SHTP capture or share successes models and experiences among the INGOs. The INGOs themselves have taken most of the initiatives in the area of IEC? BCC. Until the award of a sub-grant to Counterpart International in 2007, JSI appears to have neglected this critical the IEC/BCC component. USAID/Sudan: Sudan Health Transformation Project Assessment Report x Availability of Essential Drugs and Vaccines, and Logistics Management: Although SHTP INGOs have leveraged non-SHTP drug supplies and developed their own innovative drug management systems, they are plagued by frequent stock outs of malaria drugs, some antibiotics, and vaccines. These drug stock outs have been caused by poor oversight and management of several procurements from UNICEF by USAID. In addition, there is confusion regarding the roles and responsibilities of UNICEF and GOSS in the distribution of vaccines. Given the need for coherent, standardized drug management systems at the national, state and county levels and JSI’s world-wide experience in logistics systems, the Assessment team and others were puzzled by JSI’s lack of leadership and technical assistance to the SHTP in this area. Pre- and In-service Training and Service Delivery Capacity Building: From the evidence at one of the five rehabilitated Regional Training Centers, the rehabilitation part of this component was done well. In addition, the AMREF-contracted pre-service training materials for laboratory technicians and nurses are well designed, competency￾based curricula. The effectiveness of pre- and in-service training for facility staff was more difficult to assess, as the team was not able to systematically observe or test providers during their field visits. Most providers in the field sites of three of the four counties reported that they had had some refresher training in the last 2-3 years; however, its relevance and usefulness could not be corroborated. The INGOs, using their own technical know-how and curricula, provided refresher training to strengthen service delivery skills and capacity. The team observed very limited use of job aids and on-the-job training through supportive supervision to reinforce skills. The team found no evidence that SHTP had conducted a systematic needs assessment and developed a strategy for in-service and on￾the-job training of the INGO and LNGO staff. JSI provided no leadership in the development of standardized training modules for the SHTP interventions and sharing of promising practices from the INGOs and other countries with similar health challenges. Infrastructure: The SHTP has accomplished a great deal in the renovation and rehabilitation of PHCCs and PHCUs, assuring a close water source, latrines and waste pits. However, problems remain: waste pits are poorly constructed and maintained, most pit latrines have no covers. Furniture and shelving for PHCCs and PHCUs appear inadequate for the needs of the staff and clients and some providers continue to work in poorly constructed and maintained facilities. Renovation and rehabilitation in Southern Sudan is costly (most materials must be imported from outside the country) and funding limited. The team found the following gaps: no assessment was done to allocate resources for deciding on these capital investments; no standard list from GOSS or SHTP of basic furniture required for PHCCs and PHCUs against which to assess needs; and, no inventory controls on purchases of furniture and transport. Moreover, the team found that JSI had not facilitated the sharing of promising practices among the INGOs (e.g., use of solar), or tracked progress and quality in renovation/rehabilitation. Strengthening Decentralized County Level Capacity: The Assessment team found that performance in this technical area has been poor, with most initiatives coming from the INGOs and their partners. Experiences and practices vary greatly by county and INGO. The lack of staff and salary and budget support from the GOSS for the VHCs and CHDs made this a particularly challenging component for the SHTP, so JSI leadership was critical. In all sites visited, the team found no leadership from JSI in: fostering the linkage of VHCs and CHDs, and CHDs to State governments; importing and customizing tools USAID/Sudan: Sudan Health Transformation Project Assessment Report xi and best-practices to/from SHTP; and, harmonizing processes and tools across INGOs. JSI’s inattention to this important component, health systems strengthening, is due to lack of technical expertise in the organizational structure at JSI/Juba and no expert support by JSI/Boston. Strengthening of Local NGOs: Performance varied by INGO. Two examples of successful collaboration were found among the five counties with which the Assessment team had contact. Other arrangements were poor and resulted in conflict and divorce. In Tambura no LNGO existed when the SHTP started activities there. JSI’s performance in this area was disappointing. JSI did not have the staffing to offer: strategic oversight to help INGOs find alternatives to their partner LNGO and leverage the loosely organized, nascent community-based organizations (CBOs) such as church groups; or tools, technical assistance (TA), mentorship, or r support to the INGOs struggling in an environment in which LNGOs are just forming and experienced ones difficult to find. USAID also was inconsistent in its policy direction in this regard; the requirement that INGOs partner with an LNGO changed with each CA modification. National Policies: In this component, there were two key achievements: the development of the Draft Policy for Maternal and Reproductive Health and assistance in the development of a national framework for Monitoring and Evaluation for the National Health Services program. In both instances, JSI’s support was much appreciated as highly collaborative and useful. Collaboration among the INGOs, USAID, JSI and GOSS: The structures developed for the partnership of USAID, JSI and GOSS and then for the all of the stakeholders in the SHTP were never formalized. The Core Group, consisting of GOSS/MOH, USAID and JSI, met during the first six months. However, relations grew contentious. The GOSS/MOH periodically withdrew from management oversight when they felt that their concerns were not adequately addressed. The quarterly meetings of all the stakeholders have been more regular, but have not met the expectations of the parties involved. Communication breakdowns have been frequent and have affected decision-making and, therefore, progress for the project activities. Leadership and Management: JSI, USAID, INGOS, And GOSS: The Assessment team found leadership and management shortfalls by all SHTP partners. To date, the leadership and technical skills for setting a vision and direction for the SHTP are not found in the JSI project staff and no long-term technical assistance from JSI/Boston has been provided for this purpose. Except for the development of the PHC record keeping system and some instances mentioned in quarterly reports, there is little evidence of JSI leadership in documenting and sharing best practices and lessons learned among the INGOs in SHTP or from other countries. No technical staff demonstrated organizational development skills to work with the relevant government bodies on strengthening the capacity of CHDs and VHCs and linking them to each other, or with the new State Ministries of Health and GOSS. JSI has not spelled out the roles they expect the INGOs to play in this regard. There were several glaring deficiencies evident in JSI management. The lack of fundamental grants management led to a long delay in awarding follow-on grants to the INGOs and no extensions processed to cover the five-month gap period. The result was reduced activities by the sub-grantees and delay in paying service providers. Also, an in depth financial review by the Regional Financial Management Office revealed major USAID/Sudan: Sudan Health Transformation Project Assessment Report xii deficiencies in financial accounting and management in inventory control for furniture, other supplies, and transport - vehicles, motorbikes and bicycles. USAID’s monitoring and follow up of SHTP has been insufficient. The team found no evidence of a monitoring plan for field visits for USAID staff and no reports of monitoring visits by USAID were shared with the team. USAID’s tracking of actions, such as the $3 million provided by USAID to UNICEF for drug procurements, highlighted management lapses and serious gaps in USAID’s monitoring and documentation for this critical component. If there had been adequate monitoring by the SHTP partners, the difficulties and lack of progress identified in this Assessment would have been identified some time ago. There was inadequate, unsystematic monitoring of field sites by JSI, USAID, and the INGOs. Also, the team found that USAID, JSI, the INGOs, and GOSS/MOH made little use of the SHTP system for monitoring and reporting. There was little analysis and review of the quarterly reports and other documentation. Consequently, the reoccurring issues with data quality, inaccurate analysis, and implementations problems were not caught and dealt with in a timely manner. The result has been crisis management and difficulty in tracking SHTP results in a meaningful way. ACHIEVEMENTS The SHTP has made contributions to primary health care services in Southern Sudan. In the four counties the Assessment team visited, the INGOs with their LNGO partners were the sole providers of PHC services. If the SHTP stops, services will stop. Moreover, service delivery has increased in scope and scale. Project data indicate that more clients are being served now than in the early reporting periods and in more functional facilities. Facilities staffed to GOSS standards have increased from 10 to 45 of the 99 SHTP￾supported PHCC/Us. Village Health Committee members and County Officials acknowledged (with varying degrees of concern) that the facilities were making a difference to their communities. Most SHTP health facilities had: • Staff. At PHCUs, a Community Health Worker, Maternal Community Health Worker and/or TBAs; at PHCCs, Village Midwife, Certified Nurse, Clinical Officer, and/or Medical Assistant, Lab Technician and TBAs. • Record-keeping. Evidence of patient registers recently filled. • Clients and Space. Outpatients waiting for services and availability of areas for Reception, Examination, and Dispensing of drugs. • Drugs. Availability of many basic drugs. • Refresher Training. Health Staff in Mundri, Tambura and Panyijar reported having received some training within the last two years. The team noted several other achievements. At the initiative of the INGOs, BCC/IEC activities are occurring, through some local puppet shows and skits s and one-on-one counseling sessions. The INGOS have also implemented some innovative management practices and community development and participation approaches, such as those reported by Save the Children in Mvolo and the stock management system by International Rescue Committee (IRC) in Panyijar. Where renovations have been done, they made an important difference in the functioning of the PHCU or PHCC. Access to USAID/Sudan: Sudan Health Transformation Project Assessment Report xiii water and latrine facilities has contributed to the health image and status of the facility itself, and presents a positive model to the community. The SHTP, through its technical assistance, has played a central role in the development of the Maternal and Reproductive Health Policy for Southern Sudan, an important step in incorporating family planning into PHC services. SHTP-funded consultants have also made contributions to the development of a National M&E Framework for the GOSS/MOH Health program. MAJOR SHORTFALLS Although SHTP has had some achievements, the shortfalls are significant and have seriously compromised the ability of the project to achieve its objectives and have a positive impact. Technical Deficits: The Assessment team found that JSI had made limited efforts to strengthen County Health Departments, Village Health Committees, and had done little in community mobilization and IEC/BCC. Leadership: The Assessment team found that JSI had not provided leadership that: offered a strong vision for SHTP; used a systematic approach to identify gaps in programming in the counties (e.g., in community mobilization); stayed attuned to the rapidly changing policy environment; and looked for opportunities for helping the INGOs to take advantage of new resources such as the Multi-Donor Trust Fund (MDTF); share and harmonize tools and processes, and cross-fertilize innovative practices. Management: The team members found that the lines of authority in both USAID and JSI are confusing to all partners. For the SHTP, the fundamental business operations of JSI are not functional: financial management and grants management are two glaring examples. Problem solving on the part of all partners has been reactive rather than anticipatory and proactive. Monitoring and Evaluation: The on-going problems with the SHTP registers and summary sheets for reporting raised questions about the reliability of the data and suggest a need for data quality verification. The data that are available are not analyzed and used for decision-making, by JSI, USAID, GOSS, and rarely the INGOs. The challenges of delivering primary health care services in Southern Sudan warranted extensive on-site monitoring, as feasible, given the difficult circumstances. However, on-site monitoring and feedback by all partners (INGOs, USAID, JSI and GOSS) was irregular and not standardized. Communication, Collaboration and Coordination: Inadequate information dissemination and communication have plagued all of the partners in the SHTP individually and collectively. A particular gap is in the dissemination of information from the GOSS to the States and then to the Counties. The Core Group meetings have lapsed. The quarterly partners meetings have not provided a venue for sharing of lessons and harmonizing approaches needed by SHTP. RECOMMENDATIONS FOR ACTION IN THE NEAR-TERM The SHTP Assessment team has recommended specific actions with assignments to each partner to address these shortfalls: technical deficits. These include; leadership; management; monitoring and evaluation; and communication, collaboration and USAID/Sudan: Sudan Health Transformation Project Assessment Report xiv coordination. In addition, immediate actions are required to address problems with vaccines, drugs, reporting and several technical issues. Many of the actions require the development of plans for monitoring field sites, strengthening technical competencies, leading and facilitating the sharing of promising practices among the NGOs, and the like. These plans need to be consolidated into an SHTP work plan for the next 12 months. The team strongly recommended that these actions be taken as soon as possible, and completed no later than May 15, 2008, with corresponding milestones scheduled for March 15, 2008 and April 15, 2008. This detailed plan is presented on pages 32-35 and in Annex H of this report. CONSIDERATIONS FOR FUTURE HEALTH INVESTMENTS The team took into consideration the achievements and shortfalls of the present project, the needs for future health systems development and primary health care services in Southern Sudan, as well as USAID’s comparative advantage. The Assessment team proposed the following major components for the follow-on project to the Sudan Health Transformation Project: • Bolster decentralized health systems – Continue and further strengthen County Health Department and Village Health Committee development; information dissemination from GOSS to peripheral levels, e.g., funding available, training schedules, policies re vaccines, etc. – Start support to State level and State-County interaction – Monitor needs and target TA for key functions at the GOSS/MOH. • Continue support for and build on the delivery of high impact services. Although the Assessment identified many problems and gaps, the SHTP has made an important beginning and should be continued. – Further strengthen outreach, community mobilization, and community-based approaches, pre- and in-service training, including for maternal care for TBAs and MCHWs and add family planning; maternal health; nutrition; TB as per Basic Package of Health Services (BPHS). • Support improved Epidemic Preparedness. • Initiate/strengthen linkages with other sectors (e.g. Education, Economic Growth, Democracy and Governance). – Develop mechanisms to reinforce civil society groups: These groups can serve as outreach linkages and advocates for better health services, community norms, and financing. – Link Village Health Committees and other groups with livelihoods, income￾generating opportunities, etc.: Consider opportunities for complementary financing of these groups and members. FOLLOW UP TO THIS REPORT: BRIEFINGS ON THE RECOMMENDATIONS Three members of the Assessment team remained in Juba the week of February 11, 2008, to prepare a briefing on the conclusions and recommendations of the team. The briefing was presented to three different groups: the USAID/Juba Mission team on Thursday a.m., February 14, 2008; GOSS/MOH leadership on Thursday p.m., February 14, 2008; and to the JSI staff on Friday a.m., February 15, 2008. (See Annex I for the PowerPoint presentation used for the briefings and Annex H for the Handout also provided to those USAID/Sudan: Sudan Health Transformation Project Assessment Report xv who attended.) A briefing is scheduled for USAID/Washington and JSI staff in the U.S. for March 7, 2008. USAID/Sudan: Sudan Health Transformation Project Assessment Report 1 SECTION I. INTRODUCTION USAID/Sudan signed a Cooperative Agreement (CA) with JSI Research and Training Institute, Inc (JSI) in April 2004 to implement the five-year, $29.5 million Sudan Health Transformation Project (SHTP). The CA was increased by $10 million to $39.5 in September 2007. The SHTP focuses on: (1) provision of primary health care services through grants to International Non-Governmental Organizations (INGOs) partnering with indigenous Local NGOs (LNGOs) so as to increase their capacity to provide these services on their own; and (2) strengthening of County Health Departments (CHDs) and state and national capacity to manage the health care system. Since the SHTP was designed, the political context of Southern Sudan has changed dramatically as described in Section II below. Although a mid-term assessment of SHTP was planned in 2006, for a number of reasons, its implementation was postponed. With less than two years remaining under this CA, USAID/Juba wanted to take a critical look at the SHTP to assess what worked and what did not; identify models that could be replicated; decide on immediate actions to get the project on track; and propose future directions for USAID’s health investments to support the Government of Southern Sudan (GOSS). The Objectives of the mid-term assessment were to: • Assess progress to date in achieving the deliverables of the SHTP Cooperative Agreement (as well as modifications to it); • Assess the models and approaches proposed and used by SHTP, including: – The seven high impact services that were chosen to start the PHC program and to serve as a platform for expanding services to the GOSS-mandated Basic Package of Health Services (BPHS); – Community mobilization and behavior change communication through use of techniques such as the “Community Action Cycle;” – County-level capacity building to support local government management and oversight of INGO/LNGO service delivery programs; and – The use of an Umbrella Grants Mechanism to provide leadership and channel funding for PHC service delivery to the INGOs and LNGOs. • Make recommendations for: – Short-term adjustments and actions to improve performance during the remaining period of the agreement – Key components of follow-on investments for the USAID health program, so as to build on SHTP investments and strengths, support the Comprehensive Peace Agreement (CPA), provide tangible “health dividends” to the people of Southern Sudan, and support the development of the GOSS/Ministry of Health (MOH) and decentralized structures for delivering quality PHC services. Note: The details of the proposed follow-on investments are presented in a separate memorandum to USAID/Juba. Five team members participated in the Assessment. Margaret Neuse, a former USAID Population, Health, and Nutrition (PHN) Officer, was contracted to serve as the Assessment Team Leader. Connie Davis, an expert in infectious diseases, and Victor Masbayi, an expert in maternal and child health, based in USAID’s regional office for East Africa in Nairobi, focused on the delivery of services for infectious diseases and maternal health respectively. Mary Harvey, a child health expert in USAID’s Africa Bureau in Washington, focused on child health, immunization, and disease surveillance and preparedness. Health systems strengthening and capacity building was the focus of Yogesh Rajkotia, an economist and health systems expert from the Bureau for Global Health at USAID/Washington. The team was supplemented during their field work by the participation of Dr. Samson Baba, the former Director General for Primary Health Care and now the Director General for External Assistance and Coordination of the GOSS/MOH. Khadijat Mojidi, the USAID Population, Health and Nutrition Officer and Health Team Leader, provided both expert guidance on the desired outcomes of the Assessment as well as invaluable logistical support. Figure 1: Map of Southern Sudan with SHTP Counties USAID/Sudan: Sudan Health Transformation Project Assessment Report 2 USAID/Sudan: Sudan Health Transformation Project Assessment Report 3 SECTION II. BACKGROUND A. SOUTHERN SUDAN A brutal civil war between the North and the South has ravaged Sudan for all but eleven of the last 50 years. Some of the main drivers of this conflict have been the historical concentration of wealth and power in the central government in the North, racial and religious discrimination of marginalized southerners, and conflict over land for crops or pasture, and exploration for vast oil reserves. The signing of the Comprehensive Peace Agreement (CPA) on January 9, 2005, by the Sudan People’s Liberation Movement (SPLM) and the Government of Sudan (GOS) brought an end to Africa’s longest running conflict. The agreement, addressed the causes of war and instability in Southern Sudan, and established the Government of National Unity (GNU) and the Government of Southern Sudan (GOSS). The GOSS has had many challenges in its short history: the establishment of a central government and local government structures (states and counties) virtually from scratch; continuing insecurity in some areas; the death of the SPLA/M leader six months after the signing of the CPA; and the relocation of the capital from Rumbek to Juba in 2006. In addition, many Southern Sudanese who fled to neighboring countries during the war are returning; these returnees are coming with high expectations and limited resources. USAID, since the late 1990s, has provided emergency and humanitarian relief through its Office of Foreign Disaster Assistance (OFDA) using the UN Operation Lifeline Sudan (OLS) structure. To show support for Southern Sudan, USAID was the first bilateral agency to engage with the South in the development of a primary health care project that would provide an opportunity for the SPLM to become familiar with working with donors and test out one model for providing health services to populations in Southern Sudan. At the time of the design and launch of the Sudan Health Transformation Project (SHTP) in April 2004, the SPLM was still at war with the North, Rumbek was the de facto capital of the South, and the County was the only local administrative structure, but in name only. Within this constantly changing and complex political environment, it was difficult to plan for health interventions or systems. There were many logistical challenges as well. There were no roads and other communication infrastructure. Food and supplies had to be ferried to isolated communities by air. Communication was by radio or satellite phone. Internet connectivity was available in only a few places. Once the rainy season starts, people are effectively cut off from all communication as the rivers rise, and the Nile and tributaries flood the lowlands. Depending on the area, people are marooned for 4 to 6 months until flights can once again resume. Most of these challenges remain. B. HEALTH STATUS IN SOUTHERN SUDAN At the time of the signing of the CPA, the health status of the people of Southern Sudan was among the worst in the world. Most health services were provided by INGO emergency and humanitarian relief organizations. At best they covered less than 25 percent of the estimated population. The 2006 Sudan Household Survey (Southern Sudan Report) estimates the Infant Mortality Rate (IMR) at 102.4 per 1000 live births and the under five mortality at 135 per 1000 live births. The Maternal Mortality Ratio (MMR) is USAID/Sudan: Sudan Health Transformation Project Assessment Report 4 possibly the highest in the world with estimates ranging from 1000 to over 2000 per 100,000 women of reproductive age. The total fertility rate was estimated at 5.9 live births per woman (UNICEF). Southern Sudan has a wide range of infectious diseases and is prone to epidemics. Infectious Diseases: Both UNICEF and WHO classified malaria as the number one cause of under-five mortality. Several studies suggested that resistance was emerging to both chloroquine and sulphadoxine-pyrimethamine (SP). Use of insecticide treated bed nets and intermittent presumptive treatment for pregnant women was very low. The 2006 Sudan Household Survey found that 30 to 40 percent of children surveyed in Southern Sudan had suffered from diarrhea within two weeks of the survey that 6 to 19 percent of children throughout Southern Sudan reported recent respiratory infections, and that routine immunization coverage is less than 17 percent. Outbreaks of measles are reported annually and neonatal tetanus is endemic in many counties. Maternal and Reproductive Health: About 94 percent of births take place at home (UNICEF). Although about 77 percent of these deliveries are assisted by attendants, most of these are traditional birth attendants with little training. In addition, antenatal care coverage is low and usually lacks tetanus toxoid immunization and other services. There is a near absence of family planning and child spacing information and services. Nutrition: The nutritional status of children and adults is poor. One state of Southern Sudan, for example, suffers from recurrent drought and the wasting rates for children under 5 years are at emergency levels. Some 30 to 40 percent of babies were born with low birth weights. Sub-clinical Vitamin A deficiency affects one of seven children in Sudan and goiter is common in areas such as the Nuba Mountains. Only about 30 percent of the population use water from a protected source and only about 20 percent report having received any hygiene or sanitation information. HIV/AIDS: Southern Sudan is a low prevalence area for HIV/AIDS (available data indicate a HIV prevalence rate below 3 percent), but it is at risk for a rapidly escalating epidemic. Since the CPA, many Sudanese who had fled to high HIV/AIDS prevalence countries are returning. Other possible contributing factors to a growing epidemic are cultural practices and a steep decline in the age of sexual debut for both genders. C. PRIMARY HEALTH CARE STRUCTURE IN SOUTHERN SUDAN Primary health care is delivered at the community level through a system of Primary Health Care Units (PHCUs) serving a population of 3,000 to 5,000 people. Four to six PHCUs refer to a Primary Health Care Center (PHCC) that serves a population of 12,000 to 15,000; the PHCCs have more and better trained staff and higher capacity and generally provide more specialized services than the PHCUs. According to MOH guidelines on staffing, PHCC and PHCU should have the following structure: PHCC Staff PHCU Staff − 1 Clinical Officer − 3 Community Health Workers (CHWs) − 2 Maternal Community Health Workers (MCHWs) − 1 nurse, 1 midwife − 1 dental technician, 1 laboratory assistant − 1 pharmacy technician/assistant, 1 public health technician, and support staff, including cleaners, watchmen, and a statistician/bookkeeper. − 1 Maternal Community Health Worker − 2 Community Health Workers − 3 support staff USAID/Sudan: Sudan Health Transformation Project Assessment Report 5 A CHW is trained for nine months in PHC service delivery; an MCHW receives similar 9-month training, with more emphasis on management of pregnancy, labor, delivery and postnatal care. Referral sites for the PHCC/Us are often difficult to reach. When they are accessed, they may not have the trained personnel, drugs and supplies needed. D. SUDAN HEALTH TRANSFORMATION PROJECT (SHTP) The Cooperative Agreement of the SHTP signed on April 24, 2004 with JSI, was officially launched in May 2004. JSI agreed to: • Provide high impact services at the community level through the PHCCs and PHCUs; • Strengthen the County Health Departments so they could supervise those structures and PHC service delivery; • Develop community mobilization and outreach to create demand to use services and healthy behaviors; • Provide water and sanitation for the PHCCs and PHCUs; and • Strengthen five facilities for training Community Health Workers (CHWs). Through sub-grants, JSI was to select a lead INGO in each SHTP county, which in collaboration with a local NGO/Community Based Organization (CBO), would provide primary health care services at the county level. JSI was to procure drugs for the PHC services and provide technical assistance to the Secretariat of Health (SOH) by arranging consultants to assist in the development of PHC policies and guidelines. Since the design of the SHTP and the signing of the CPA, the administrative structure of Southern Sudan has changed radically with the establishment of a Federal system. The central government is now responsible for setting policy and guidelines; the ten States are responsible for implementing the policies along with more than 120 counties. However, the State and County structures have limited human and financial resources. Worryingly, pockets of insecurity in the South caused by ethnic conflicts have caused INGO staff to be evacuated from some areas during the past three years. The GOSS/Ministry of Health moved from Rumbek to the new capital in Juba (in July 2006) with USAID and JSI following several months later. After the signing of the CPA, the World Bank, numerous bilateral governments, and multinational agencies have descended in force to support the South. JSI, USAID, and GOSS planned to manage the SHTP by means of a tripartite Core Group, which was to meet quarterly. The selection of the 20 counties the SHTP was based on criteria such as geographical location, health needs, and the need for equity between regional areas. The initial six counties selected were: Twic East, Panyijar, Tambura, Mvolo, Mundri, and Tonj South (see map on page 2). A competitive process using Requests for Applications (RFAs) was used to select the lead INGO agency for each county. E. EARLY SHTP IMPLEMENTATION ISSUES From the start, because of USAID regulations and other factors, JSI had difficulties procuring the drugs required for PHC services in the six counties. USAID then decided to procure the drugs through UNICEF in two batches. However, USAID and UNICEF USAID/Sudan: Sudan Health Transformation Project Assessment Report 6 bureaucratic processes resulted in delays; the arrival of the first batch was delayed until early 2007, nearly three years after SHTP start-up. As a result of these delays in procuring essential drugs, the GOSS/MOH resisted further expansion beyond the first six counties. In June 2006, the United States Government announced a unilateral change in its platform and introduced the “Fragile State Strategy” for Southern Sudan without consultation or negotiation with the GOSS. This change was to have serious and far￾reaching ramifications. While the strategy’s goal was to achieve a just and lasting peace through the successful implementation of the CPA and emphasized investment in essential services to encourage and sustain the return of internally displaced people (IDPs) and refugees. The new strategy shifted support from the planned 20 counties to target three border Areas (South Kordofan, Southern Blue Nile, and Abeyei) and the three garrison towns of Juba, Malakal and Wau. Informal commitments had been made to the next 14 counties to be covered by the SHTP, and other donors were told not to focus their efforts elsewhere. Because of the delays in drug procurements, the imposition of the Fragile State Strategy, and continuing conflict between JSI and GOSS over the management of the initial six counties, the MOH disengaged from active participation in the project. It has been difficult to regain the trust and confidence of the MOH following these events. At the time of this assessment in early 2008, the SHTP has continued implementation in the six original counties, and is adding three new counties, Aweil South, Terekeka and another still under discussion (not those areas proposed in the Fragile State Strategy), and the three towns, Juba, Malakal, and Wau. USAID/Sudan: Sudan Health Transformation Project Assessment Report 7 SECTION III. ASSESSMENT METHODOLOGY The Assessment team used the Cooperative Agreement Program Description and Modifications as the standards against which to assess the status of deliverables and progress. Because quantitative data were of questionable quality, the team relied primarily on qualitative information from the sources described briefly below. Quantitative Data: Very little, reliable quantitative data are available, particularly population-based data specific to the counties targeted by the SHTP. Although three of the INGOs conducted baseline assessments, they were completed only in 2007. The SHTP also regularly submits quarterly and annual reports that include its service statistics for the reporting period. After their field visits, the Assessment team questioned both the accuracy of the data and the population statistics that were used to compute coverage for the counties. In one county, for example, the team was given three different estimates of the population. Service providers at times did not have the proper registers for recording their client contacts and services provided. Although some INGOs provided assistance to their facilities in mapping and assessing the size of the catchment areas for their services, this was not a generalized practice across the six counties. Qualitative Data: The Assessment team relied on the following qualitative data sources. The team reviewed a large number of important documents (see Annex B for the full list of reference documents consulted), including: • Project documents, such as program descriptions from the Cooperative Agreement and modifications; quarterly progress report; , annual reports; reports of technical assistance and monitoring visits from JSI/Boston; presentation documents and other specially prepared briefing materials; training materials; monitoring and evaluation reports, and materials produced by partner INGOs. • Government documents such as the draft policy document for Maternal and Reproductive Health, and the Basic Package of Health Services. • Documents from other donors, for example, reports from the Multi-Donor Trust Fund. The team also had briefings with key staff at USAID/Washington, and USAID/Juba (See Annex C for complete list), and held individual and group interviews with: • Project managers (from JSI, , INGOs, and LNGOs) • Service providers at sites visited, including Traditional Birth Attendants (TBAs) • Beneficiaries and community groups (including women’s groups, youth, and Village Health Committees) • Government officials (GOSS and County) • Other donors, UN Agencies, the World Bank, and stakeholders Most important, the Assessment team participated in a 10-day field trip to four of the six counties where the SHTP is being implemented. The partners include: Action Africa Help, International (AAH-I) in Mundri; International Medical Corps (IMC) in Tambura; International Rescue Committee (IRC) in Paniyar; and Care International in Twic East. The team visited 21 of the 99 facilities supported by the project, and observed service delivery, facility renovations, as well as boreholes, latrines, and waste management with the use of waste pits. The team also observed project operations at the INGOs’ USAID/Sudan: Sudan Health Transformation Project Assessment Report 8 compounds and met with County Health Departments and County officials in the four counties. (See Annex D for the itinerary) Upon return to Juba, the team interviewed representatives of Save The Children and the Sudan Inland Development Foundation (SIDF) from Mvolo County. Cautionary Notes: Given the circumstances and limited time for the field visits, it was not possible to select a random sample of facilities to visit or people to interview. Moreover, some interviews were challenging given the need at times to translate questions and responses through several languages. For similar reasons, the team members did not use formal questionnaires, but instead followed interview guides to ensure that a common set of questions was asked and similar information collected. The team was not able to visit any State MOHs to discuss their needs and circumstances. Even with five team members, there was not enough time to read and analyze and fully comprehend the many documents available and to fully understand a very complex situation, rapidly changing environment, and a project history marked by confusion and turmoil. The team learned early on that the participants had their own perceptions of the causes and consequences of various problems and that it would not be possible to disentangle the many threads of the stories. Rather the team agreed to focus on the consequences to project deliverables and to propose actions to correct the problems identified. Given these limitations, the team was careful in reviewing its findings, reaching conclusions, and making recommendations for corrective action. However, given the multiple sources used, in particular the visits to the field sites, the team is confident in its findings and conclusions. USAID/Sudan: Sudan Health Transformation Project Assessment Report 9 SECTION IV. ANALYSIS AND FINDINGS A. ACCESS AND USE OF HIGH IMPACT SERVICES What was expected? The SHTP intended to focus on health interventions that address the basic health needs and highest impact on the morbidity and mortality of women of reproductive age and children under five years of age. Although the Cooperative Agreement did not identify them explicitly, by Modification 10 in 2007, they had been selected. The results anticipated are increased use of the seven services and improved quality of care. The seven high-impact interventions, a subset of the GOSS’s Basic Package of Primary Health Care Services are: 1. Immunizations 4. Long-Lasting Insecticide Treated bed Nets 2. Vitamin A 5. Case management of malaria 3. Antenatal care 6. Case Management of Acute Respiratory Illness 7. Treatment of Diarrhea with Oral Rehydration Findings General Service Delivery While the initial SHTP work plan included the collection of baseline data in each of the six counties, only three of the six NGOs collected baseline data in 2007. Since their baseline reports were not readily available to the team, it was difficult to determine the amount of change in use of health services since the beginning of the project in 2004. However, through a review of the quarterly reports and, at the project sites, quick record reviews of the monthly reports from the project designed health records, the Assessment team concluded that these services are reaching more people than before the project began. All increases are also directly attributable to the project NGOs. They are the only source of PHC services in these six counties, as all relief NGOs have departed since the signing of the CPA. The Assessment team also recognized that greater change would have resulted if drugs and vaccines had been consistently available. To effectively reach the people and provide services in the catchment areas, the SHTP increased the numbers of facilities that were adequately staffed, refurbished and equipped. At the start of the SHTP, only ten of the 99 SHTP facilities were staffed according to MOH guidelines. By the end of December 2006, 45 of the 99 PHCCs and PHCUs were fully staffed, an important accomplishment given the paucity of skilled staff from which to recruit and the challenges of retaining staff in the rural areas. (Reporting on this indicator is annual; the data for 2007 were not available at the time of this report.) The data available do not indicate the number of facilities that were refurbished and equipped. However, most of the facilities observed in Mundri, Tambura, and Panyijar had been renovated and/or rebuilt. Twic East had few, if any, upgraded facilities. The indicators reported in these findings as well as the baseline and targets were selected by SHTP and are part of the SHTP Monitoring and Evaluation System. The total population covered in the six SHTP counties is approximately 926,400 based on population estimates provided by the 2004 grant applications submitted by the SHTP INGOs. Using a standard estimate of 20 percent for women of childbearing age and children under five years of age, the total target population for high-impact services was approximately 370,000. However, the team found that population estimates vary USAID/Sudan: Sudan Health Transformation Project Assessment Report 10 significantly depending on the source. The census now scheduled for April 2008 should provide more accurate population data. Immunizations (including EPI) Southern Sudan relies heavily on national polio and measles campaigns for the delivery of Vitamin A and immunizations. Preference for campaigns are keeping with international priorities, lack of staff and equipment for routine delivery; poor infrastructure; difficult climate conditions; and the need for quick wins/results to show that something is being done. Routine EPI is in a nascent stage. With the December 2007 arrival of GAVI Immunization Systems Support (ISS) funds, the GOSS will hold its first meeting with the State EPI Managers and some NGOs from February 18-21, 2008 to conduct micro-planning for Routine EPI. States will then receive some ISS funds to support implementation of their EPI plans. Although the GOSS has no separate, official policy for EPI, the GOSS Basic Package of Health Services for Southern Sudan provides some guidance. EPI is to be conducted daily at the Primary Health Care Centers, on fixed days from the PHCUs, and outreach sessions conducted regularly from both sites. While the EPI focus is on vaccinating children under one year of age and women of childbearing age, the policy includes vaccinating children up to five years of age who are unvaccinated. The County Health Department, in close collaboration with UNICEF and GOSS, is responsible for identifying and registering target populations; developing county EPI micro plans; maintaining the cold chain equipment; requesting, storing and distributing vaccines and other materials; and training r staff at PHCC and PHCU levels. Target DPT3: Stated Baseline: <7.5%. End of Project Target: >21.5% (Design team target was 40 %.) FY 07 Target: 13.5 %. Achievement: 16.4 % overall, with a range from 7% in Tonj South to a high of 28.6% in Mundri. Table 1: Annual DPT 1 and DPT 3 Coverage of Children under One Year of Age by County County Estimated number of Children <1 year of age DPT 1 Oct. 05- Sept. 06 DPT 3 Oct. 05- Sept. 06 DPT 1 Oct. 06- Sept. 07 DPT3 Oct. 06- Sept. 07 Twic East 4,819 76.4% 38.4% 28.4% 23.3% Tonj South 15,000 26.6% 6.0% 10.0% 7.0% Tambura 9,850 23.6% 5.8% 18.5.% 13.2% Mundri 8,006 38.9% 16.2% 32.1% 28.6% Mvolo 7,616 18.1% 7.1% 27.2% 25.6% Panyijar 6,500 59.6% 42.0% 18.8% 12.0% Total 51,791 35.5% 15.3% 20.4% 16.4% SHTP established the DPT 1 and 3 baselines and targets at the beginning of the Project. How these were estimated remains unclear as no baseline surveys were conducted in 2004/5. Given the lack of good census data for determining the target population and that the information on actual immunizations is collected through routine reporting, the team can not say if there has been a statistical change in immunization coverage. USAID/Sudan: Sudan Health Transformation Project Assessment Report 11 Vaccination with the first dose of Diphtheria, Pertussis, and Tetanus (DPT 1) is widely used as a proxy for measuring access to health services. Access to DPT 1, as reported by the six counties on the chart above, shows it declining from the 2005/2006 timeframe to 2006/2007 in all but Mvolo County. This is most likely due to shortages of DPT vaccine. The high coverage for Twic East and Panyijar is most likely due to the campaigns conducted in these counties in 2006. Routine vaccination in all of the counties the team visited effectively stopped in October 2007 because of the scheduling of the Polio Campaigns and lack of vaccine, in particular DPT. DPT 3 coverage (often used as a proxy for the quantity and quality of service delivery) increased in four of the six counties over these time periods but declined in two counties (Twic East and Panyijar). Poor EPI coverage has resulted in a number of reported outbreaks. During the Assessment team’s visit, Twic East reported no DPT vaccines since September; this was particularly troubling as there was an outbreak of whooping cough (pertussis) that began in November 2007. The county is still waiting for vaccines to arrive, now expected by February 15, 2008, to arrest the spread of this disease. The team also learned of measles outbreaks in two counties in early 2007 due to delayed campaigns and low coverage. Of the counties the team visited, only Mundri and Tambura had vaccines at the county level. The measles vaccines available had an expiration date of February 2008, and it was unclear when the next shipment would arrive. In Tambura County trivalent polio vaccine was mixed with monovalent Polio 1 vaccine. The EPI team did not know that they should not provide monovalent polio for routine administration. This vaccine is used only in polio campaigns directed at this one type of polio virus. In discussions with UNICEF it became clear that, at the national level, there are no vaccine stock outs and that there is a tracking system in place to identify quantities of vaccine distributed and used. However, it also is evident that in spite of the fact that UNICEF has conveyed information to most NGOs through the NGO forums and letters to State and County Health Departments, the process for requesting and obtaining and reporting on vaccine use on a monthly basis is not known or understood by most of the SHTP INGOs. Also, in two States, Jonglei and Warrup, the State cold chains have not been established according to the National EPI Manager. There are multiple causes for the stock outs of vaccines and low coverage affecting EPI and shared responsibility among the partners: • Unclear roles. County EPI personnel are unclear who is in charge and the roles of the various organizations: UNICEF, GOSS (central, state and county), and INGOs. • Staff skills. The most recent training programs were done in 2005 by UNICEF. It is not clear who is responsible for refresher training. • Management. Coverage targets for routine EPI are low. Systems for routine immunization and outreach are poor. Focus is not on children under one year. Vaccine management is unsatisfactory, both in assuring good vaccine at right time to the right people and in understanding amounts to order and when. • Poor monitoring. Monitoring of the cold chain (vaccine temperature) in Mundri is not well done, and the temperatures for the vaccines supplied by WHO are not recorded. • Poor record keeping. Recording of vaccinations is done on a tally sheet and these are submitted monthly to the County Health Department and the SHTP INGO. Vaccination Cards and handmade registers exist but, for the most part, the registers USAID/Sudan: Sudan Health Transformation Project Assessment Report 12 and cards do not link. As a result, it is not possible to review either record and see which child, if any, is fully vaccinated and to follow up children who have not returned for their next dose. Vitamin A The distribution of Vitamin A to children 6 to 59 months is conducted primarily through polio and measles campaigns. At baseline, coverage was high because of these campaigns. The WHO recommendation is to provide one dose of 100,000 IU to infants 6 to 11 months of age at any opportunity, to provide 200,000 IU in a single dose every four to six months to children 12 months and older, and to include it in routine services. End of Project (EOP) Target: 90%; Baseline: 64.3 % based on national and sub￾national EPI campaign reports from UNICEF (per SHTP reports); FY07 Results: 11.6% Table 2: Vitamin A Supplementation for Children under Five Years of Age County Estimated number of Children <5 year of age Oct. 05-Sept. 06 Oct. 06-Sept. 07 Twic East 17,830 2.4% 3.3% Tonj South 55,500 1.3% 3.1% Tambura 36,445 0.1% 13.5% Mundri 29,620 3.0% 18.8% Mvolo 28,179 3.7% 20.0% Panyijar 24,050 5.7% 16.0% Total 191,624 2.3% 11.6% The data used for this indictor are questionable. The child health card does not record the provision of Vitamin A which makes it is impossible to tell if a child received the recommended two doses, spaced four to six months apart. During polio campaigns, Vitamin A is provided but is not recorded on the cards. Fearing overdosing, most providers do not provide Vitamin A on a routine basis. Vitamin A and albendazole were provided during the Polio National Immunization Days (NIDs) conducted in November 2007 and plans are to provide it during the next NIDs planned for April 2008. Thus it is likely that coverage will increase. However, to achieve and maintain high coverage, Vitamin A needs to be included in routine services. GOSS and SHTP should give guidelines for service providers on the routine provision of Vitamin A and adjust the record keeping system to document the timing and dosage provided to children. Antenatal Care (ANC) In addition to antenatal care (ANC) as one of the seven high impact services, JSI intended to include implementation of training in Home-Based Life Saving Skills (HBLSS) to improve home deliveries. No indicator or data are available on HBLSS. The indicators SHTP has used to measure ANC care are Intermittent Presumptive Treatment of Malaria (IPTM) and the percent of deliveries assisted by a skilled attendant. Neither of these is a direct measure of ANC care. In SHTP’s most recent Quarterly Report (October 1- December 31, 2007), ANC attendance (one visit and four visits) is reported as an indicator for the first time. The Project indicator for measuring treatment of malaria in pregnancy and as a “proxy” for ANC attendance is the percentage of women receiving intermittent presumptive USAID/Sudan: Sudan Health Transformation Project Assessment Report 13 treatment (IPT) of malaria during pregnancy per MOH protocols. The percent of women receiving IPT2 (19.6%) is close to the percent reported attending ANC for a first visit (21%). End of Project Target: > 20%; FY07 Target: 18.2%; FY 07 Achievement: 19.6%. Table 3: Percentage of women receiving intermittent presumptive treatment (IPT) of malaria during pregnancy per MOH protocols County Estimated number of pregnant women IPT2 coverage Oct. 05-Sept. 06 IPT2 coverage Oct. 06-Sept. 07 Twic East 4,819 17.9% 21.9% Tonj South 15,000 16.7% 15.9% Tambura 9,850 13.6% 20.8% Mundri 8,006 12.6% 20.7% Mvolo 7,616 5.1% 29.0% Panyijar 6,500 9.5% 11.9% Total 51,791 13.0% 19.6% The increases shown by these data may be an indication of demand for these services and/or the availability of Sulfadoxine Pyrimenthamine (SP), the MOH prescribed drug of choice. The increases must also be questioned given doubts about the quality of data and the denominator. The project uses the number of children under one year of age to estimate the number of pregnant women. This is not accurate given the infant mortality rate of 102.4 per 1000 live births (UNICEF, Sudan Household Survey 2006, Southern Sudan Report). To better track ANC attendance, SHTP needs to set targets, monitor ANC attendance (1 visit, 4 visits), and continue to include these data in its quarterly reports. The SHTP also tracks the percent of deliveries assisted by a skilled attendant, a useful indicator for maternal care, but not ANC. Given that change in attendance at birth is likely to be slow in Southern Sudan, annual reporting may be sufficient. Skilled providers are defined as trained midwives and nurses, a cadre of staff sorely lacking in Southern Sudan. Most deliveries are attended by Traditional Birth Attendants (TBAs) and Maternal and Child Health Workers (MCHWs). TBAs also carry out mobilization for ANC at the community level, deliver women at home and report deliveries to the health facilities. MCHWs conduct the ANC visits at facility level. At the PHCC level, MCHWs assist the village midwife, if one is available, with deliveries. Given the Assessment’s team’s review of how this information is collected in the field; team members doubt the accuracy of these data. USAID/Sudan: Sudan Health Transformation Project Assessment Report 14 Table 4: Percent of deliveries assisted by a skilled attendant County Estimated number of pregnant women Oct. 05-Sept. 06 Oct. 06-Sept. 07 Twic East 4,819 0% 1.4% Tonj South 15,000 3.0% 2.5% Tambura 9,850 0.5% 2.3% Mundri 8,006 0.3% 3.4% Mvolo 7,616 3.7% 3.7% Panyijar 6,500 3.5% 3.0% Total 51,791 1.4% 2.5% It was evident from the field visits that TBAs are the key promoters of information on the importance of ANC attendance and safe deliveries to pregnant women in the community and that they are often the only link between health facilities and pregnant women. They also carried out all of the recorded home deliveries and most TBAs reported assisting in several home deliveries (most above seven) over the past year. TBAs mentioned delivery problems such as excessive bleeding, breach presentation and prolonged labor. During the Assessment team’s interviews, TBAs reported stillbirths, but no maternal deaths. TBAs in Mundri East and West reported having delivery kits that included towels, iodine, plastic sheets, scissors, weighing scale, and cord-clamping materials. These kits are often replenished by the PHCCs and PHCUs. In the other counties visited, most TBAs did not have a complete set of delivery supplies or kits and had not been issued any materials in over two years. In Twic East no TBA interviewed had ever received a delivery kit. Treatment of Diarrhea with Oral Rehydration Therapy Diarrheal diseases, along with malaria and acute respiratory infections, were the most frequent diseases found in the morbidity outpatient logbooks. Although this is one of the seven high-impact interventions, the SHTP does not report on treatment rates with Oral Rehydration Salts (ORS) or Oral Rehydration Therapy (ORT) in the SHTP Quarterly Reports. While ORS appeared to be available in all sites and the providers from pharmacists to CHWs were aware of the correct amount of ORS and water to use and the messages to give mothers and caregivers, there were no ORS treatment corners and few posters or other educational materials. In one in-patient ward in Twic East, all cases of diarrhea were being treated with Flagyll and intravenous solutions. In short, the team was unable to assess progress on this intervention but had reasons to be concerned. Only IMC in Tambura, according to the team’s observations, has instituted any related reporting, specifically weekly reporting of selected epidemic-prone diseases, including cholera. Their Integrated Disease Surveillance and Response (IDSR) system alerted the Ministry of Health when there was a sudden increase in cases of acute watery diarrhea. A team from the MOH and the World Health Organization (WHO) then visited the sites, conducted case investigations and collected stool samples for testing. The causative agent was not found to be cholera. USAID/Sudan: Sudan Health Transformation Project Assessment Report 15 Provision of Long Lasting Insecticide Treated Nets (LLITNs) JSI did not procure LLITNs until 2006 due to a worldwide shortage of long lasting nets and then, was able to procure only 60,000. A second order for an additional 100,000 nets is expected to arrive by Malaria Day, April 24, 2008. (By contrast, Population Services International/Sudan has procured over one million LLITNs with resources from the Global Fund.) All the LLITNs brought in to date were distributed to the counties. No nets were seen in any of the service sites except one, Twic East where they were kept in storage room at the PHCC and not placed over the in-patient beds. According to GOSS policy, LLITNs are targeted for pregnant women and children under five who are most at risk. The project did not set separate targets for the two groups. Most of the nets were distributed through the ANC clinics at the PHCC/Us. It was not possible to ascertain how many were provided to children under five. The INGOs used several strategies to distribute nets. Some used them as an incentive to come to ANC, or for children to return for DPT3. Both of these strategies may seriously limit the impact of protection that the LLITNs can provide. Relatively few women access ANC, and, when they do come, they arrive late in the pregnancy. Also given the erratic supplies of EPI vaccine, tying the distribution of nets to children who obtain routine immunizations will limit their access. The malaria burden is high, is endemic in most parts of Southern Sudan, and the rainy season will only increase the problem. To have high impact, the nets need to be distributed and their correct use ensured through practical demonstrations (such as those done in Mundri). Malaria Case Management After studies in 2001-2003, malaria first-line treatment changed to Artesunate AS+ Amodiaquine AQ. Second line treatment is Artemether/Lumefantrine and quinine is reserved for severe malaria. To harmonize treatment regimens throughout Southern Sudan, the National Malaria Control policy was issued in 2005. The policy also recommended the use of rapid drug tests in PHCUs and smear microscopy in PHCCs for confirmation of the diagnosis. The main interventions were malaria in pregnancy (MIP), intermittent preventive treatment in pregnancy (IPTp), LLITNs, effective treatment, and early treatment-seeking behavior. At all the counties and sites visited, the CHWs correctly responded to questions about appropriate treatment and appeared to be following the national program guidance for the treatment of malaria. Malaria posters and the treatment regimen algorithm were seen in Tambura and Mundri. Since malaria is the number one cause of outpatient visits, artemesinin combination therapy (ACT) was in use but many PHCC/Us reported stock outs within the past 3 to 6 months. In addition the malaria rapid test kits (Para-check) were also not in stock at some sites and were expired at others. Many of the PHCCs did not have functioning laboratories so the availability of rapid test kits are critical at this level. Case Management of Acute Respiratory Infection (ARI) The Project did not report on case management of acute respiratory infection (ARI); therefore, the team could not assess its coverage. This should be corrected as soon as possible. The Assessment team did observe that in Panyijar, IRC, with funding from the Canadian Development Agency (CIDA), began a program of Community Based Distributors (CBD). The CBD agents were selected from the community where they live USAID/Sudan: Sudan Health Transformation Project Assessment Report 16 and trained to provide case management for ARI and diarrhea. They were also provided with drugs for treatment. The CBDs’ work being done with IRC should be monitored and captured as a promising practice and introduced to the other SHTP NGOs. Summary The SHTP does not have a solid statistical base nor does it have reliable denominators to assess how much access to and use of the high-impact interventions promoted by the SHTP have increased since its inception. The project uses the same denominator for children under one year of age and pregnant women, thereby not accounting for infant mortality. For each high-impact service, there are problems with data availability and quality. Although SHTP has made important and useful investments to improve and harmonize record keeping and reporting, more investments are needed to refine and maintain a reliable system. SHTP does not routinely report on three of the seven interventions, ANC visits, ARI and ORT, even though the data needed are available from the facility service statistics. To better estimate coverage, Lot Quality Assurance surveys and, for immunizations, vaccine coverage surveys should be considered in the SHTP counties. Although more service sites are adequately staffed and have been refurbished and equipped, thereby making access more likely, other problems and systems failures, such as drug and vaccine stock outs, have compromised service availability. (These systems failures are discussed in more detail in later sections.) Even with the substantial investments by SHTP it has only start in developing the PHC systems (training, IEC/BCC, supervision, monitoring and management) and ensuring the delivery of these seven services. The challenges in delivering the larger Basic Package of Health Services will be greater. Finally, the Assessment team found no evidence that SHTP used any means to assess the quality of the services being provided in its facilities. The team found no standardized reporting on supervision visits by the INGOs or JSI to monitor quality of care. In the limited time and circumstances of the team’s field visits, the team could not observe the health providers to assess their performance according established guidelines and standards. B. DEMAND FOR SERVICES AND BEHAVIOR CHANGE TO IMPROVE HEALTH What was expected? In the Project Description for the Cooperative Agreement, JSI proposed a good, comprehensive design for this component. The elements included: mobilizing communities in improving their own health by using village health committees and applying an approach called the Community Action Cycle (CAC); working with their sub-grantees to develop and implement appropriate behavior change strategies and materials; and developing/ harmonizing and then using critical health messages in the schools. Under Modification 10 of September 2007, another sub-grant is identified to “develop and implement appropriate behavior change communication strategies” aimed at promoting use of the seven high impact services offered at SHTP facilities. JSI funded Counterpart International to work with the INGOs in the six counties on formative research and develop these strategies. The anticipated results of this component were an “Increased proportion of the population who adopt appropriate health seeking behavior,” and “increased utilization of PHC facilities in USAID supported areas.” USAID/Sudan: Sudan Health Transformation Project Assessment Report 17 Findings The effectiveness of SHTP IEC and BCC efforts is difficult to assess because of the lack of baseline data, definition of the behaviors the project expected to change, and current data on relevant behaviors and use of services. Therefore, the team used two proxy measures: 1) understanding or knowledge derived from group interviews that community members knew of the services at the facilities and how to access them and 2) IEC/BCC “efforts,” namely, evidence of IEC/BCC materials, communication and dissemination activities, and one-on-one communications and counseling. In all of the team’s discussions with VHCs, community groups, and others, the communities around the PHCCs and PHCUs appeared to be familiar with the services offered and how to access them. They also knew of the constraints and problems of the services and expressed concerns about lack of a consistent supply of drugs and vaccines, limited staff, and other difficulties. Some TBAs indicated that they encouraged the pregnant women they were serving to attend ANC at the local facility, while others did not. Overall, IEC/BCC efforts appeared to be spotty and limited in scope and reach. During their field visits, team members observed: some posters (mainly on HIV/AIDS, malaria, and acute watery diarrhea); T-shirts that promoted immunization campaigns, prevention of malaria, and HIV/AIDS; and one flip chart on EPI, one on malaria, and one on HIV/AIDS. The team was informed about the dissemination of messages in churches about the time and place for immunizations. The SHTP provider staff, including CHWs, MCHWs, and TBAs (both in-facility and community-based), described the advice they give to mothers during pregnancy, after pregnancy and regarding, for example, breast￾feeding, use of LLITNs, and hygiene and sanitation. However, team members did not observe one-on-one counseling and cannot attest to the accuracy of the information or its effectiveness. In a few facilities, mainly PHCCs, the team noted areas where health education was offered to waiting clients. With its multiple approaches to dissemination of prevention messages, use of materials and folk media, such as puppet shows, and peer educators, the HIV/AIDS prevention IEC/BCC program was by far the strongest under the SHTP and most likely to get results. In Mundri County the team observed a puppet show on HIV/AIDS prevention that was also enjoyed by many members of the local community. Village health committees were used for community mobilization primarily to generate community participation in the construction and refurbishment of the local health facility and latrine and to disseminate information about EPI. In Mvolo, SAVE has used the Community Action Cycle (CAC) to identify health problems and solutions, including behavior change, but the team was not able to visit Mvolo County to verify the outcome of this activity. Given the evidence from the field visits and other sources, the IEC/BCC activities of the SHTP were disappointing. Weaknesses in this component include: • No identification of key messages and behaviors to change or reinforce • Lack of any IEC/BCC strategy, even in 2008, for concerted efforts in this area • Meager materials in scope (few topics and aspects of the seven high impact services covered) and quantity USAID/Sudan: Sudan Health Transformation Project Assessment Report 18 • No adaptation of materials available from other organizations, like UNICEF, WHO • Reliance on the NGOs themselves to do what they could to provide health education materials and support (e.g., an IMC hygiene guide) • Many missed opportunities to mobilize communities and provide health education, for example, targeting civil society groups, e.g. churches and women’s group, mobilizing the Village Health Committees, and systematic use of client waiting time for reinforcing key health messages The lack of a coherent and aggressive strategy and support for this critical component stems from overall neglect by JSI. The team could find no evidence (until the sub-grant award to Counterpart International in 2007) of focus on IEC/BCC, of leadership in developing and implementing a strategy, of supervision and monitoring of IEC/BCC activities, or the sharing of experience or collaboration among the INGOs and other partners and stakeholders. Given the lack of a coherent approach to community mobilization and BCC, the team cannot comment on whether SHTP can serve as a model for scale up in Southern Sudan, although JSI implements successful BCC programs elsewhere in the region. C. AVAILABILITY OF ESSENTIAL DRUGS AT PHC SITES AND LOGISTICS MANAGEMENT What was expected? Drug and Vaccine Supplies Under the Cooperative Agreement, JSI was initially assigned the task of procuring all essential drugs and medical supplies for the counties under SHTP. With USAID development funds, grantees are required to buy drugs from the US unless a waiver is obtained. U.S. sourced drugs are exorbitant, and transport of the drugs can add an additional 30-40% charge. USAID/Juba researched other options and decided to purchase essential drugs from UNICEF. By using the PIO mechanism of the Global Health Bureau in Washington and ordering from Copenhagen, no waiver from Washington was required. In FY 2005 USAID/Juba placed an order for $400,000 of drugs and in 2006 another order of $2,600,000 in essential drugs and medical supplies for the six counties. UNICEF provides vaccines for the EPI in Southern Sudan. Logistics Management Under the Cooperative Agreement, JSI was also to develop, train staff, and implement a Logistics Management Information System (LMIS) that would establish procedures for procuring and distributing from the national warehouse to the concerned counties. The system was also to specify how drugs would flow at the county level (i.e. from the CHD to the PHCC to the PHCU). Findings Drug and Vaccine Supplies It is unclear that which USAID/Juba carefully monitored the procurement process with UNICEF. The Assessment team was informed that the first consignment of UNICEF drugs did not arrive in country until November 2006. In March 2007, USAID requested JSI to clear the first tranche of drugs and place them in a GOSS warehouse. While examining the drugs, JSI noticed that many drugs in the consignment would expire in six USAID/Sudan: Sudan Health Transformation Project Assessment Report 19 months, and refused to distribute them. The INGOS were notified that the drugs were available and that they could collect them from Juba. With this delay in getting drugs in country, all counties under SHTP have experienced difficulty in stocking drugs for their health facilities. Individual INGOs made creative, ad hoc local arrangements to obtain drugs from other sources, such as GOSS and UNICEF, or used their own private funds to purchase drugs. Drugs received from GOSS using the kit system do not contain adequate amounts supplies of the drugs used for the most common illnesses in Southern Sudan such as malaria. Drug kits are kept at the county level and facilities must request the drugs they need. However, because of the erratic supply, most PHCC/Us visited reported shortages and stock outs of selected drugs since September 2007. For example, Lakamadi PHCC in Mundri East reported stock outs in penicillin, amoxicillin, ACT, and SP for malaria in pregnancy. They had some supplies of iron and folate tablets. Supplies of malaria rapid tests ran out in December 2007. This PHCC also reported a stock out of EPI vaccines. Mvolo reported that the most frustrating thing was the lack of communication from JSI on the status of drugs. On Thursday, February 14, 2008, USAID and UNICEF met to review the status of USAID’s drug orders. USAID learned that 90% of the second consignment of drugs had arrived in Southern Sudan in September 2007. However, USAID had not contacted UNICEF about the whereabouts of the drugs, nor had UNICEF notified USAID about their arrival until this meeting. USAID did not keep proper documentation of its orders with UNICEF (the team heard various versions of the number and size of the orders) and in the last 6 to 8 months did not follow up with UNICEF in a concerted way. At the moment, the drugs are being stored in containers that are likely to reduce the shelf life and quality of the drugs. The very large size and quantities of the last order would indicate poor forecasting of the needs for the USAID-supported counties and the prospect that drugs could expire before they are used. The team found a range of issues affecting the supply of EPI vaccines. All four counties visited reported stock outs of some vaccines, or vaccines expiring in February 2008, significantly disrupting the routine delivery of EPI. No vaccines were found in Twic East County where there was an outbreak of whooping cough. There was general confusion in the field over who was responsible for ensuring the supply of vaccines and for the functioning of the cold chain, UNICEF, GOSS, or the INGO. Both GOSS and UNICEF need to clarify their roles and responsibilities, at the national, state, county and peripheral levels. Given the limited staffing and capacity at the more peripheral levels and the challenges of transportation and communication, annual EPI reviews would be helpful to pinpoint problem areas and target systematic follow-up. Logistics Management Although logistics management is an important part of JSI’s Cooperative Agreement, the team found little evidence of the development of a logistics management system for drugs and supplies either at the national level or at the SHTP project sites. The team was informed that JSI played a significant role in the recent development of a Trainer’s Manual for Supply Chain Managers in Primary Health Care for use at the national level. The Manual was developed through active participation of GOSS and other key partners to build consensus and ownership. A workshop to review the second draft of the LMIS is scheduled for February 22, 2008. However, in reviewing the manual, team members noted that the contents appeared to be generic and compiled from many other sources. It was not adapted to the specific conditions and situations foundin the PHCCs and PHCUs USAID/Sudan: Sudan Health Transformation Project Assessment Report 20 in Southern Sudan and not of the caliber expected given JSI’s world-wide experience and reputation in this area. Given the delays in developing an LMIS or other tools to manage supplies, some INGOs such as IMC set up a “pull” system whereby PHCUs used their records to order the drugs and supplies they needed before they ran out. In other counties such as Panyijar, IRC has trained staff to maintain stock cards to keep track of drug supplies and to know when to order. Summary Although SHTP INGOs have managed to leverage non-SHTP drug supplies and develop their own innovative management systems, they are plagued by frequent stock outs of malaria drugs, some antibiotics, and vaccines. These drug stock outs have been caused in large part by poor oversight and management of the procurements from UNICEF by USAID. The vaccine stock outs are caused in large part by confusion regarding the roles and responsibilities of UNICEF and GOSS (including states and counties) in the distribution of vaccines and management of the cold chain, but also by lack of training and follow up, and by logistics constraints at the local level. The Assessment team noted that many factors are involved in drug procurement at the national level, GOSS, MDTF, UNICEF, and other INGOs. Overall there is a strong need for a coherent drug management system. To establish this, the GOSS needs assistance with policies, guidelines, the rational use of drugs, standardization, and harmonization of drug and medical supplies. Coordination and management of the supplies already in place are also urgently needed. Some INGOs thought that JSI’s role was to assist in setting up standardized systems to be used by the six counties rather than each individual agency developing its own system. With JSI’s experience in logistics systems worldwide, it should have taken a leadership role in this critical component of the project and provided timely technical assistance for the systems needed for its counties and nationally in Southern Sudan. D. IN-SERVICE TRAINING AND SERVICE DELIVERY CAPACITY BUILDING What was expected? In the Cooperative Agreement, JSI was to rehabilitate the facilities of up to six training schools; review and revise curricula and develop standardized curricula for community￾based workers; strengthen existing schools and new training facilities for training nurses, midwives, clinical officers and doctors; and, emphasize the recruitment and training of women. Under the 2007 Modification, JSI was to: rehabilitate 5 Regional Training Centers (RTCs); develop or review the curricula for nurses, laboratory technicians, midwives and clinical officers (the 2006-2008 Revised Project Description states that JSI will work with the Ministry of Health on curricula for the nursing and laboratory cadres; in 2005, JSI made two sub-grants to AMREF for this work); support sub-grantees to select and provide training for nurses, laboratory technicians, midwives and clinical officers from the six project counties; provide on-the-job training for PHCC and PHCU staff. To mentor and transfer skills, JSI was to collaborate with the USAID “Skills Transfer” Diaspora Project and help place Sudanese Diaspora in the PHCCs and PHCUs. USAID/Sudan: Sudan Health Transformation Project Assessment Report 21 Findings Rehabilitation of Regional Training Centers The Assessment team was able to visit only one of the RTCs that had been rehabilitated with SHTP funds, the RTC in Ganyiel. The renovation had expanded and significantly upgraded the class and practical training rooms and library as well as the office space for the RTC staff. The Assessment team noted that the rehabilitation appeared to have been done well. However, the RTC had other l infrastructure and equipment problems. The students live in poor housing, and the borehole is not close to the compound and is shared with the community, creating inconvenience for students who have a tight schedule. The RTC has not yet received a promised, updated curriculum and new materials for training CHWs from GOSS. The RTC lacks computers for access by the students to the important resources available on the internet. The RTC is also struggling with effectively analyzing its costs and budgeting so as to secure adequate funding for its program and students. Curricula for Laboratory Technicians and Nurses from AMREF The training materials for these cadres include: manuals containing the curricula, which present the course outlines, time periods, prerequisites, etc. for the course; trainers manuals; procedures manuals (as appropriate); books for recording clinical experience; and assessment checklists. Although the team could not review the materials in depth, they appeared to meet requirements for competency-based training materials. The team was not able to verify whether the new curricula have been distributed to the relevant training centers and are being used. Service Provider Competence To increase the number of trained staff for the PHCCs and PHCUs, the project proposed to train CHWs and MCHWs in the Regional Training Centers, as well as nurses, midwives, laboratory technicians, doctors and clinical officers. Although the quarterly reports present the numbers sent each quarter by the INGOs for pre-service training, the Assessment team was not able to find a target number to be trained during the SHTP or get a cumulative number of those who had received pre-service training to date. Given the limited education and skills of the CHWs and MCHWs, refresher training and reinforcement of their skills through job aids, supportive supervision, and on-the-job training are critical. Training effectiveness is measured in the competence of the service providers. Unfortunately, the Assessment team was not able to directly observe service providers or to test their skills. Team members, therefore, used proxy measures of competency: 1) the responses of service providers at the sites visited to questions on key topics and 2) the amount and timing, frequency, and relevance of their refresher training. In general, questions on key topics, for example, management of EPI vaccines, the vaccination schedule, malaria treatment guidelines, and management of antenatal women and delivery were answered correctly by service providers during the Assessment team field visits. The project reports that 1,567 staff had received some structured training of at least 1 to 2 days duration during the project period. This result is good progress against a target of 2,000. Most service providers in the field sites visited reported to the Assessment team that they had received some refresher training in the past 3 years. In three counties, some staff reported that they had participated in up to 5 refresher workshops on such topics as HIV/AIDS prevention, delivery skills, and the like. Twic East was the one exception: service providers interviewed did not report receiving any USAID/Sudan: Sudan Health Transformation Project Assessment Report 22 refresher training. The relevance and usefulness of the refresher training could not be corroborated. Most providers rarely mentioned family planning, child spacing or breastfeeding as one of the key components of information or services provided to clients All TBAs interviewed reported having received some form of training in 2006 or before. In Twic East the TBAs reported receiving training before SHTP was launched. None of them had received refresher training after the start of SHTP. From the discussions with TBAs, it was evident that some were more familiar with key messages than others. In Mundri and Tambura most TBAs listed such topics as TT immunizations, IPT, anemia and safe delivery as messages to encourage mothers to attend ANC. In other places, the team had to prompt for TT and IPT. The team found no evidence that SHTP had conducted a systematic needs assessment and had developed of a strategy and plans for in-service and on-the-job training of the INGO and LNGO staff. The INGOs did their best to provide training using their own technical know-how, experience, approaches and plans. The opportunity for creating a common curriculum for refresher training for the CHWs among the sub-grantees has not been seized. Given the need to rely on these lower level cadres for health services at the periphery for some time to come, the lack of a more focused, systematic approach to refresher training may seriously compromise the quality of care being offered. Moreover, some INGOs complained that in recent revisions of their budgets by JSI, the training line items had been cut, potentially compromising their own training efforts. The team also noted that the INGOs have relied primarily on training to improve performance, with limited follow-up and reinforcement with job aids and on-the-job training through supportive supervision. JSI has provided no leadership in any of these areas and has not facilitated the sharing of INGO promising practices and experiences from similar countries. Staff performance can also be affected by low morale. During the Assessment team interviews, the team learned that morale had been seriously affected by a number of factors. They include lack of a uniform pay standard; lack of pay for many government staff and for TBAs; and delay in pay for many INGO staff. VHCs, women’s groups and others informed the team that many health workers are not regularly at work and when at post their performance is poor due to nonpayment of salaries or incentives. E. INFRASTRUCTURE What was expected? SHTP Project Descriptions address infrastructure and functionality as part of making high-impact services available. Results include minimal renovation of PHC facilities on a case-by-case basis so that the infrastructure is able to meet Ministry of Health PHC specifications; equipping the PHC facilities with basic furniture and non-medical supplies; and, recruitment of staff to ensure fully staffed PHC facility. Notably, the SHTP called for construction or rehabilitation of boreholes so that each facility has access to water; establishment and strengthening of water management committees for each borehole; and facilitation of construction of VIP latrines in the communities. Some of the expected outcomes are reliable water points for all health facilities, functional water management committees; increased number of facilities with latrines; and an increased number of community members who use latrines. USAID/Sudan: Sudan Health Transformation Project Assessment Report 23 Findings Renovation/Construction Most facilities visited in Mundri, Tambura and Panyijar had been either renovated or new ones built, or a combination. Renovation and construction work was done by either SHTP or other NGOs, such as CRS and OXFAM. Some INGOS have been innovative in their rehabilitation efforts. Save the Children reported an approach to construction of PHCUs that was low-cost and yet used permanent materials that are not subject to termite damage. Others have made effective use of solar power or of improved water catchment and storage during the rainy season. Many leveraged additional funds to support more renovation work. Nonetheless health providers in some areas, especially Twic East, continue to work in poorly constructed, substandard buildings that are poorly designed and maintained. The team had no access to a standardized list of basic furniture (from either GOSS or SHTP) for a PHCU or for a PHCC against which the team could assess whether the facilities were well equipped. Some facilities had received chairs, tables, and maternity beds. However, many facilities had very limited furniture, with few tables and no shelving for the storeroom and consultation rooms. As a result, service providers had to find space for their large registers, drugs for dispensing, and other items on very crowded tables and spaces. In some cases, drugs were dispensed from boxes on the floor. In some instances in the field, the team was struck by what appeared to be poor allocation of the scarce renovation/reconstruction resources. In Karika the PHCU had requested and received a maternity ward equipped with about 10 beds (but no delivery bed), even though by GOSS standards a PHCU is not designated for maternity service. The County Health Department has not found midwifery staff to run the facility; hence, it is not functional. In contrast, the Mopoi PHCC in Tambura County, which was designed as a PHCU, operates as a PHCC. It has a maternity ward with only two beds and no general ward. As a result, the maternity ward doubles as a general ward. In Twic East, there was no evidence of any renovations of the facilities that are in great need of upgrading. Latrines JSI reports that 83 percent of the facilities supported by the project have pit latrines. Nearly all the facilities visited by the Assessment team had a latrine with one or more doors. These facilities were appropriately located away from the clients’ area and most were reasonably well maintained. Some facilities with newly built latrines, such as Lakamadi, had not started using them as they were still waiting for an “official’ opening. It is important to note that the revised project description called for VIP latrines, which, in addition to vent pipe ventilation, must have a cover over the pit hole to keep out flies. Most of the SHTP latrines did not have this cover. There was no evidence of construction or availability of latrines in Twic East. Refuse Pits for Medical Waste Most facilities had refuse pits and some had, in addition, “incinerators” (these were steel drums in which they burn refuse). The refuse pits were abysmally poor. They were all too shallow (deep pits collapse during the rains), were uncovered, and, in several situations, had waste lying beside the open pit. Most pits were appropriately located away from where clients were provided medical services and residential areas; however, in some cases such as Lakamadi, the pit was close to residential areas. These pits pose a health USAID/Sudan: Sudan Health Transformation Project Assessment Report 24 hazard to surrounding communities and a danger to children who often wander in these areas. Water JSI reports that 93 percent of the SHTP facilities now have access to boreholes. Most facilities are within a 5- or 10-minute walk from a water source, usually a borehole. In some cases the boreholes existed prior to the SHTP. It is important to note that even though SHTP signed a 9-month contract with CRS for the drilling of boreholes, there was a long delay in starting the work. By then, CRS had spent all funds on contractors who had not completed the assignment. SHTP refused to give CRS a cost extension and asked CRS to complete the work at their own cost. CRS agreed to do this for Mundri and Tambura. Summary The team recognized that renovation and rehabilitation in Southern Sudan are costly because most materials must be imported into the country and funding water and sanitation was likely too little to meet the needs. Although much has been accomplished in the renovation and rehabilitation of PHCCs and PHCUs and in assuring a close water source and sanitation with latrines and waste pits, problems remain. The team found the following important gaps: • No strategic, assessment-based approach to the allocation of the resources for these capital investments; • No standards or guidelines on the basic furniture for PHCUs and PHCCs from either GOSS or SHTP; • No inventory or other controls to guide decisions on the provision of basic furniture and shelving (and transport). • Environment Health Engineered only hired in mid 2007 to provide TA to GOSS and monitor the water and sanitation components for SHTP. JSI did not have or implement a monitoring plan for tracking the progress of the renovation activities and checking on the quality of the work. Nor did JSI monitor the development and activities of the water management sub-committees who were to oversee the use and repair of the boreholes and the periodic testing of the adequacy and safety of the water. The water management committees and the boreholes also represent a missed opportunity for intensive hygiene and sanitation education. The INGOs developed many promising practices in conducting the renovations in their counties and would have benefited from sharing among themselves. F. STRENGTHENING DECENTRALIZED, COUNTY LEVEL CAPACITY FOR MANAGEMENT AND DELIVERY OF PHC SERVICES What was expected? During the SHTP design, even before the CPA, a decentralized system was envisioned with heavy reliance on the counties, which already existed to provide oversight and management support for the delivery of health services. The GOSS National Health Policy specifies a decentralized health structure, with States and Counties largely responsible for the management and implementation of health care. One of the main pillars of the SHTP was to support this policy by strengthening Village Health USAID/Sudan: Sudan Health Transformation Project Assessment Report 25 Committees and County Health Departments. As a direct provider of health services, SHTP was well positioned to transfer key management skills to the nascent CHDs and VHCs. Structurally, JSI was to work with its service delivery partners to develop a health system strengthening strategy, common tools and processes, and a system for sharing best-practices across counties. In JSI’s original Cooperative Agreement Project Description and in Modification 10, JSI agreed that: • “SHTP will work closely with CHD to strengthen their capacities to effectively manage and monitor primary care services” • “The project will continuously implement various activities to transfer basic management and administrative skills to the CHD and State MOH” • “JSI will design study tours for CHD staff and develop systems to maintain their links with other CHDs” • “Anticipated Results: Pre-existing decentralized government health institutions strengthened through training, sharing of materials, and collaboration with NGOs through regular meetings.” Findings The Assessment team found that performance in this technical area has been poor. Initiatives have come from the INGOs and their partners and not from SHTP. Experiences and practices greatly vary by county and INGO. In Twic East, for instance, Care International has made little effort to strengthen the County Health Department. The members of one VHC in Twic East remarked that this assessment was the first time that anyone had ever asked for their input. Conversely, in Mundri, despite the lack of government salary support for CHDs, AAHI and MRDA found innovative ways to create an environment for the CHD to grow and function. The CHD was well staffed and the CMO made a cogent presentation on the County’s health situation to the Assessment team. In Tambura, IMC developed its own supervisory checklist for the CHD and made joint supervision with the CHD a priority. In Panyijar, IRC developed its own supervisory checklists, holds monthly meetings with the CHD, and gives advance notice to the CHD before any supervisory visits. Although the Assessment team did not visit Mvolo, interviews with the Save the Children and SIDF management team revealed that the INGO brought in many of their own tools, including Partnership-Defined Quality (PDQ) and supervisory checklists, to facilitate the activities of the VHCs and CHD respectively. In all cases, the team found that VHCs were rarely clear on their roles and responsibilities, and were not linked to the CHD. Further, JSI has failed to foster the linkage of CHDs to State governments and has not begun any work to strengthen State Ministries of Health, as indicated in Modification 10 of the Cooperative Agreement. The Assessment team found that VHC and CHD development has failed largely because of JSI’s lack of technical leadership rather than because of any evidence that this model may not be appropriate for Southern Sudan. The team’s visits to SHTP sites found no evidence of technical assistance provided by JSI for this purpose; the sub-grantees were left to their own devices for this complex area of work. Despite the fact that JSI has been successful in this technical area elsewhere around the world, the organization failed to customize and import its tools and best practices to SHTP. No effort was even made to identify local innovation, to harmonize processes and tools, or to cross-fertilize best practices across INGOs, even at the quarterly partner meetings. JSI had never organized a meeting or other forum for existing VHC or CHD officials to meet and share best practices. The team interviews revealed that no directives, or even suggestions, were USAID/Sudan: Sudan Health Transformation Project Assessment Report 26 given by JSI to its sub-grantees on joint supervision, joint planning, or joint budgeting with CHDs. This was not due to the lack of initiative by INGOs; their staff told the Assessment team that despite repeated calls for JSI’s technical support, no TA was provided. The Assessment team found that JSI’s failure in this area of work is due to the limited attention given to health systems strengthening, the lack of technical expertise at JSI/Juba, poor monitoring and supervision, and their inability to provide strong and strategic leadership. This is demonstrated by the lack of any health systems strengthening expertise in their organizational structure. Despite the lack of technical staff, JSI/Juba did not ask JSI/Boston for technical assistance for systems strengthening, nor did JSI/Boston proactively identify this as a major technical gap. Further, team interviews in the field suggest that JSI/Juba may have actively minimized the importance of this area; although this could not be verified. However, several sub-grantees reported that their budget proposals for CHD and VHC strengthening were cut by JSI in the latest grant proposals, despite the sub-grantees protests that CHD and VHC development is critical. Finally, interviews with JSI staff revealed that they were not aware of the strategies, innovations, successes, and failures by the INGOs. G. STRENGTHENING THE CAPACITY OF LOCAL NGOS TO MANAGE AND DELIVER HIGH IMPACT PHC SERVICES What was expected? Under the Cooperative Agreement, each INGO was to develop a partnership with a Local NGO (LNGO) that would develop the capacity to be the primary implementing partner and provide PHC services at the county level. While not a specific component of the project in the Project Description, under the section on Program Management Structure, JSI proposed “partnering one INGO with an SNGO grantee to ensure clarity of capacity building requirements and specific responsibilities.” JSI planned to “consider giving INGO/SNGO partner grantees sole regional level implementation responsibility and authority, with JSI providing policy guidance, technical assistance and management oversight.” Findings Performance varied by INGO. In Mundri and Mvolo, the team saw evidence of successful collaboration resulting in stronger LNGO institutions. In Mundri, AAHI and MRDA operate as a consortium, with joint responsibility for service delivery. However, Save the Children reported that they had to use some financing from other sources for the training and TA for their partner LNGO because JSI did not provide an adequate funding for this area of work. In Twic East and Panyijar, collaboration between the INGOs and LNGOs was poor and mired with conflict, partly because of inappropriate practices on the part of the LNGO. For the most part, JSI did not proactively monitor each situation, but rather reacted when conflict became severe. In Tambura, an appropriate LNGO did not exist when the SHTP started activities there. The Assessment team found JSI’s performance in this area very disappointing. For example, JSI offered no strategic oversight to help IRC find alternatives to their partner LNGO. It did not create an enabling environment and leverage the loosely organized, nascent CBOs such as church groups. In no case did JSI offer strategy, tools, TA, mentorship, or any other support to any of the INGOs that were struggling with this issue USAID/Sudan: Sudan Health Transformation Project Assessment Report 27 in an environment in which LNGOs are just getting started and experienced ones were difficult to find. Poor performance in this area was largely due to JSI’s lack of strategic leadership and/or lack of dedicated technical staff in capacity building. The Assessment team’s interviews with JSI staff suggest that limited thought and effort had been given to this area. USAID also was inconsistent in its policy direction in this regard. Over the period of SHTP implementation, the requirement that INGOs partner with an LNGO partner changed from modification to modification. In addition, without clearly understanding the need for JSI leadership in this area, USAID approved the realignment of JSI staffing which effectively eliminated the only position focused on capacity building. This could have been an important position to provide TA to strengthen the organizational capacity of the INGOs, GOSS and county health departments in trying to implement decentralization in the health sector. H. NATIONAL LEVEL POLICY FRAMEWORK: FOR MATERNAL AND REPRODUCTIVE HEALTH AND M&E What was expected? Under the SHTP Cooperative Agreement, JSI planned to: • Review and revise (or develop) health policies that affect primary health care programs that are related to the Basic Package of Health Services • Orient and disseminate a plan for awareness and utilization of existing policies, and foster community-based support for monitoring budgets and implementation of policies. Over the project period and as reflected in Modification 10, the scope of SHTP’s policy work to be done in collaboration with the MOH and other stakeholders narrowed to selected technical areas: Maternal and Reproductive Health; the M&E framework and policy including development of a database system that can be adopted by GOSS/MOH; and malaria. Findings A major achievement in the area of Maternal and Reproductive Health was the development and review of the Final Draft Maternal and Reproductive Health Policy for Southern Sudan. This policy, once adopted officially by the GOSS, will set the stage for development of a strategic plan for the eventual roll out of family planning and other maternal health services. SHTP provided two consultants to assist the MOH in the process. The consultants worked closely with UNFPA, the GOSS and other stakeholders, and SHTP supported meetings to discuss and agree on the contents. Feedback to the Assessment team was very positive because of the inclusive and collaborative nature of the process and the outcome. SHTP contracted consultants to assist the GOSS/MOH in the development of an M&E Framework for the Ministry’s health program overall, including the identification of indicators for all of the key health components and consideration of data collection tools. This work was done in collaboration with the MDTF and another consultant provided through another USAID project and was well received by stakeholders, including the GOSS/MOH, other donors and the States. JSI has also participated in the consultative meeting of key partners to review and finalize the Malaria Control Strategic Plan (July 2006-July 2011). USAID/Sudan: Sudan Health Transformation Project Assessment Report 28 I. COLLABORATION OF INGOS, GOSS, JSI AND USAID TO SUPPORT SHTP What was expected? The SHTP was designed to be a highly collaborative project between JSI, GOSS, and USAID. JSI devoted a crosscutting section of the Cooperative Agreement Project Description to the need for coordination and collaboration. When the project was launched, the partners developed an arrangement for this collaboration via a Core Group, in which representatives from each institution, JSI, USAID, and the GOSS/MOH, were to meet regularly to provide broad direction and oversight to SHTP activities. JSI was to coordinate its activities with its sub-grantees by holding regular quarterly meetings for all partners including USAID and GOSS/MOH. Findings Because of the lack of documentation on this process, the Assessment team could rely only on interviews to assess the intentions of the members of the Core Group in its formation and its operations over time. The team learned that the Core Group only met regularly for the first 6 months of SHTP. Lack of clarity on roles and responsibilities by all parties, the unilateral strategy shift by USAID without consultation with the GOSS, which would have affected the geographic coverage of SHTP, and inappropriate behavior by JSI staff towards GOSS officials and JSI’s lack of consultation with GOSS on key actions were the main contributors to the breakdown of this group. The results were far￾reaching in delays in decision-making, non-involvement by GOSS in project support, oversight and review, and on-going friction. The GOSS reported that the last Core Group meeting was held in February 2006. During interviews with the Assessment team, some of the SHTP partners discussed the quarterly meetings of the partners and expressed frustration with the confrontational tone of the meetings. They also noted that these meetings did not foster the sharing of lessons, harmonization of tools and approaches, and collaborative problem solving that they had expected. JSI did not use these meetings as an opportunity to identify local innovations. Finally, these meetings have also not been regular and the last partners meeting was held in August 2007. J. LEADERSHIP AND MANAGEMENT: USAID, JSI, INGOS, AND GOSS What was expected? From the design stage of this project to its implementation, strong leadership and management by JSI, USAID, the INGOs, and the Ministry of Health were stressed. JSI’s overall approach as stated in the Cooperative Agreement was to “provide technical assistance, training and commodity support toward achievement of USAID/Sudan’s Strategic Objective for Health: Increased Use of Health, Water, and Sanitation Services and Practices.” Section III of this document lays out their management plan and structure. Emphasis is placed on the management oversight of the program by: • Assigning a permanent corporate-level Senior Technical Advisor to the project • Using an internal quality control system geared particularly toward ensuring client satisfaction • Applying internal organizational learning and dissemination guidelines USAID/Sudan: Sudan Health Transformation Project Assessment Report 29 • Conducting quarterly corporate and field level internal management reviews of all bilateral projects. As prime recipient, JSI agreed to establish and maintain quality standards, both managerial and technical, and to serve as the central point for accountability and for Cooperative Agreement deliverables. JSI was to manage relationships with USAID and with the SOH/MOH, and ensure maximum communication and collaboration with grantees. JSI was also responsible for sub-grant management, the efficient management of program funds, and for maintaining adequate staffing, facilities, and other support for the program. JSI had a central leadership role in setting direction for the capacity building of an evolving, nascent primary health care delivery system and shaping the GOSS management structures and systems for the county level health system. JSI was to advise on the development of the PHC system at all levels, from the national to peripheral service delivery points. In implementing this scope in an efficient and effective way, JSI agreed that communications would be channeled through the Chief of Party, who would speak for the project team and the partners in all technical and non-technical program issues, whether with USAID, other program collaborators, or any other stakeholder. The JSI Senior Advisor role was for providing strategic management, program oversight, and technical assistance as needed to the COP and program field team, as well as serving as the key contact between the JSI home office and USAID for monitoring of client satisfaction. USAID’s role was governed by the standard substantial involvement clauses. While the Agreement was not signed with the MOH directly, the Core Group of JSI, USAID and GOSS/MOH anticipated joint decision making on key project issues. Findings Over the course of the Assessment, the team found that there were major shortfalls in the leadership and management of SHTP by all parties, in particular by JSI/ Boston and the SHTP field staff. These shortfalls have had a major impact on project performance. JSI Leadership The Assessment team defined leadership as the brain of the SHTP. The brain or leader sets the vision and course and puts a strong management team in place to translate the vision into project activities. Leadership works closely with its management team to identify the goals and objectives within the country context and plans the way forward to achieve those objectives. The leader and leadership team anticipates problems and identifies and resolves bottlenecks. For SHTP and under the terms of the Cooperative Agreement and Modifications, JSI was to be the leader. One of JSI’s child health consultants spelled this out: “JSI had an important leadership role to play to assist the MOH to harmonize international and local NGO approaches including standards for child health programs, training modules and supervisory systems into one national approach for child health.” The leadership and technical skills to serve this function are not found on the project staff nor has long-term technical assistance been provided from JSI/Boston. Except for the development of the PHC record keeping system and some other instances mentioned in quarterly reports, there is little evidence of JSI leadership documenting and sharing best practices and lessons learned by the INGOs or by other countries. Yet, the INGOs the team visited saw this as a great need. The team found that limited attention was paid to the changing policy environment of Southern Sudan and the growing health resources USAID/Sudan: Sudan Health Transformation Project Assessment Report 30 (such as from the MDTF) being made available to the States. No technical staff is in place with the right skills to work with the relevant government bodies on developing the capacity of County Health Departments and Village Health Committees (and Water Subcommittees) and linking them to each other, not to mention links with the new State Ministries of Health and GOSS. JSI Management The most glaring management deficiency at the time of this visit was that the follow-on grants to five of the six implementing INGOs had not been finalized. The previous grants ended in September 2007. The delay in getting these follow-on grants signed was due to the lack of fundamental good business practices. The proposals presented to USAID were late and fraught with errors, indicating poor quality control. JSI/Boston showed a consistent unwillingness to listen and follow USAID requests and requirements. While most of the NGOs managed to continue funding salaries through funds from their corporate offices, the ability to continue this past January 2008 was viewed as not feasible. Some NGOs reduced their activities in September, yielding lower results from service sites. IRC headquarters requested that their team in Panyijar give their required one-month notice to all employees that their contracts would be terminated. Many emergency calls by USAID/Juba and the USAID Agreements Officer to JSI/Boston were required to resolve the issue. It was agreed that JSI corporate funds would fill the gap from October 2007 to February 2008 due to delays in submitting sub-grant award packages that were accurate and complete. This funding will cover salaries and other costs for this period. The follow-on grants were signed on February 15, 2008. At the request of the new PHN Officer, the Regional Financial Management Office conducted a financial review of SHTP in early January 2008. The review identified some glaring deficiencies in financial accounting and management. During the Assessment team’s field visits, team members grew concerned whether JSI had exercised due diligence in the use of USAID funds. Many facilities had little furniture, which raised questions about baseline inventory and project inventory control, and guidelines for distribution of bicycles and other transport were unclear. In Twic East, for example, it was not apparent how the project funds had been used, as there was no evidence of renovation, latrine construction, project-funded furniture, or training. Currently it is unclear who from JSI is in charge. From the team’s observations during their time in Southern Sudan, the COP for SHTP no longer serves the communication and leadership function defined for the position in the CA. USAID Management USAID’s monitoring and follow up of SHTP has been insufficient. The team found no evidence that a field visit monitoring plan by USAID staff had been developed or followed. No reports of monitoring visits by USAID were shared with the team. USAID’s tracking of actions, such as the UNICEF drug procurement, highlighted management lapses and serious gaps in USAID’s oversight and documentation for this critical component. The USAID-approved 2006-2007 Project Work plan called for a Mid-term Assessment. The Assessment now completed is not mid-term, but rather four years after the beginning of this five-year agreement. If USAID (and its partners) had been critically reviewing the quarterly reports and conducting more frequent visits to the project sites, the urgency of this assessment would have been clear long before the new PHN Officer took her position in November 2007. USAID/Sudan: Sudan Health Transformation Project Assessment Report 31 Joint Management Although a system for monitoring and reporting was established by SHTP and the INGOs have been trained in its use, the reports from the system have not been reviewed or used by the partners, INGOs, JSI, USAID and GOSS. The INGOs have been dutifully sending in their quarterly reports and SHTP has sent these reports as required to the partners. However, it does not appear that the reports were being critically reviewed by anyone. If careful and regular reviews had occurred, the partners would note, for example, that the service statistics and population targets are confusing. As noted earlier, the reports use the same target population figures for children under one year of age and women of reproductive age. The reports also highlight implementation problems. For example, issues such as drug and vaccine stock outs were often raised. Yet, it does not appear that TA has occurred and demonstrates problems in SHTP management, quality of reporting and follows up. While the original Cooperative Agreement recognized that careful planning, good communication, frequent on-site monitoring, and effective logistical support systems would be needed, the Assessment team found little evidence of these functions by all partners, JSI, USAID, GOSS and the INGOs. No written reports were provided to the Assessment team on any monitoring visits. While written feedback was promised to several INGOs by JSI as follow up to technical visits conducted in December 2007, the INGOs told the team that they are still waiting for the reports. One result of insufficient field monitoring and review of the SHTP reports is that problem solving by all parties is reactive. The grants management debacle is one example; a flurry of activity occurred only when faced with the crisis that staff in the field providing services were not being paid. Similarly, stock outs of drugs and vaccines have plagued SHTP, yet JSI has not facilitated resolution of these problems with the key stakeholders, such as UNICEF and USAID. USAID/Sudan: Sudan Health Transformation Project Assessment Report 32 SECTION V. ACHIEVEMENTS OF THE SHTP, MAJOR SHORTFALLS, AND RECOMMENDATIONS The Assessment team analysis and findings identified some important achievements of the SHTP and its INGO and LNGO partners. However, the team also identified serious, major shortfalls and these are summarized briefly below. The recommendations for immediate, near-term actions are intended to address these shortfalls. A. ACHIEVEMENTS Despite the many political, logistical, and contextual problems of the environment in Southern Sudan, the SHTP has made contributions to primary health care services delivery. As the County Administrator stated in Twic East, “The SHTP has saved lives.” Village Health Committee members and County Officials acknowledged (with varying degrees of concern) that the facilities were making a difference to the communities. In the four counties that the Assessment team visited, the INGOs with their LNGO partners were the sole providers of PHC services. If the SHTP stops, services will stop. Moreover, service delivery has increased in scope and scale. Project data indicate that more clients are being served now than in the early reporting periods and in more functional facilities. INGOs have increased the number of trained staff and provided refresher training for these staff. More than 45 percent of SHTP health facilities met GOSS’s high standards for staffing; most SHTP facilities visited by the team had at minimum: • Staff: At the PHCU, a Community Health Worker, Maternal Community Health Worker and/or TBAs; at the PHCC, Village Midwife, Certified Nurse, Clinical Officer, and/or Medical Assistant, Lab Technician and TBAs. • Record-keeping: Evidence of patient registers recently filled (including day of the visit), including the Outpatient Register, ANC Register, Delivery Register, and Drug Dispensing Register • Clients and Space: Outpatients waiting for services and availability of areas for Reception, Examination, and Dispensing of drugs • Drugs: Indication of availability of many basic drugs • Refresher Training: Health Staff in Mundri, Tambura and Panyijar reported having received some training within the last two years. • The team noted several other achievements. At the initiative of the INGOs, BCC/IEC activities are occurring, through some local puppet shows and skits s and one-on-one counseling sessions. The INGOS have also implemented some innovative management practices and community development and participation approaches, such as those reported by Save the Children in Mvolo and the stock management system by International Rescue Committee (IRC) in Panyijar. Where renovations have been done, they made an important difference in the functioning of the PHCU or PHCC. Access to water and latrine facilities has contributed to the health image and status of the facility itself, and presents a positive model to the community. . USAID/Sudan: Sudan Health Transformation Project Assessment Report 33 The SHTP, through the provision of technical assistance, has played an important role in the development of the Maternal and Reproductive Health Policy for Southern Sudan, an important step in incorporating family planning into the PHC services. SHTP-funded consultants have also made contributions to the development of a National M&E Framework for the GOSS/MOH Health program. Although some tensions remain among the key SHTP partners, JSI, USAID, and GOSS/MOH, the relationships are under repair and improving. B. MAJOR SHORTFALLS Although SHTP has had some important achievements, the shortfalls identified by the Assessment team throughout this report are significant and have seriously compromised the ability of the project to achieve its objectives and to have a positive impact in the six counties. The shortfalls can be categorized in these areas: technical deficits; leadership; management; monitoring and evaluation; and collaboration, communication and coordination. Technical Deficits The Assessment team found that JSI had made limited efforts to strengthen County Health Departments and take advantage of emerging civil society groups such as, Village Health Committees in operationalizing the GOSS decentralization policy. In particular, JSI had done little in community mobilization, and IEC/BCC activities, and refresher training of health providers, although reflected in the Cooperative Agreement and Modifications. The team did observe some valiant efforts of the INGOs themselves in training and mentoring CHDs and VHCs and some had prepared in-service and IEC/BCC materials. Leadership Despite the requirements in the signed in the Cooperative Agreement, the Assessment team found that JSI had not provided any leadership that offered a strategic vision for SHTP. JSI did not use a systematic, assessment-based approach to identify gaps in programming in the counties to ensure delivery of basic PHC services on a consistent bases. The leadership was often not attuned to the rapidly changing policy environment in Southern Sudan and missed opportunities for helping the INGOs take advantage of resources from such sources as the MDTF. For example, the $1 million infrastructure grant now available through the states for emergency repairs could be accessed by some of the INGOs for upgrading health facilities. From the evidence available, JSI did not effectively facilitate the sharing and harmonization of tools and processes across INGOs, and did not foster the sharing of the innovative practices that some INGOs were using, such as community mapping and use of CBDs. Management The team found that the lines of authority in both USAID and JSI are confusing, to them and to partner institutions. Fundamental business operations of JSI are not functional. The USAID an analysis of its financial management has revealed significant problems that need immediate attention and call into question its stewardship of USAID funds. Grants management, a core part of the business for this project, is not working and PHC services are being adversely affected. Problem solving on the part of JSI (and others) is reactive rather than proactive. Effective problem solving is vital for management in an environment as challenging as Southern Sudan. Monitoring and Evaluation USAID/Sudan: Sudan Health Transformation Project Assessment Report 34 Although M&E are usually considered part of management, in this case, the team considered it so important that it required separate attention. The complexity of supporting the delivery of primary health care services by six INGOS in six diverse counties of Southern Sudan required as much on-site monitoring as was feasible under the circumstances of on-going conflict, poor infrastructure, and weather conditions. The team found that on-site monitoring and supervision was irregular and not standardized. The INGOs complained that they received little feedback from the on-site visits that were conducted. Although furniture, transport and other supplies were purchased for the PHCCs and PHCUs, JSI has had no standardized list of furniture and other supplies needed by the different facilities, no baseline inventory for the facilities, and no updating and monitoring process to assess the appropriate use of the furniture and other supplies. While the SHTP developed and used standardized registers and summary sheets for reporting service and other statistics, on-going MIS problems and issues accurate recording and reporting indicated a need for data quality verification. The data that are available are not analyzed and used for decision-making, by JSI, USAID, GOSS, and only rarely by the INGOs. Since no baselines were conducted before 2007, it difficult to assess the impact or progress of SHTP activities and interventions. Communication, Collaboration and Coordination Inadequate information dissemination and communication have plagued all of the partners in the SHTP individually and collectively. There is a gap in e dissemination of information from the GOSS to the States and then to the Counties. Only one county reported to the team that it knew of the funds available from the States for emergency health infrastructure repairs. Although the quarterly partners meetings have been more regular than the Core Group meeting, they have not provided the forum needed for regular communication, highlighting new developments and resources in the sector, sharing of experiences, and harmonizing of approaches needed for the SHTP. In addition, the INGOs complained of poor responsiveness from JSI/Juba for their requests for help and information. Other Technical Concerns In the course of the field visits and review of documents, interviews, and other sources of information, the Assessment team members identified a number of other concerns. These are noted briefly below. Maternal Health GOSS is discontinuing the training of Maternal Community Health Workers (MCHWs) and has adopted the WHO recommendation to discontinue the TBAs). These two cadres now form the backbone of the maternal health services at PHCU and in the communities. The implications of discontinuing these should be analyzed in light of the likely consequences, that is, no attendants at births. If the MOH reconsiders this decision, as the team suggests, the training, updating, and motivation of these staff should also be reconsidered. Another challenge is how to improve access to EmOC. It may be possible in some cases to introduce basic EmOC in selected PHCCs. However, the services will be difficult to develop and sustain, especially given the difficulties in retaining midwifery staff community level where births are occurring. With maternal high mortality, this policy shift on MCHWs and TBAs is a worrisome development in the short term. USAID/Sudan: Sudan Health Transformation Project Assessment Report 35 EPI Routine EPI has been particularly challenging to implement because of the cold chain requirements and other logistical issues at the county level. Access to roads and communication links remain tenuous. However, many other problems with EPI delivery appear to be managerial and communication mix-ups, which should be resolved as quickly as possible with GOSS and UNICEF leadership. C. RECOMMENDATIONS FOR ACTIONS IN THE NEAR-TERM The Assessment team strongly recommends that the following actions be taken as soon as possible, and completed no later than May 15, 2008, with corresponding milestones scheduled for March 15, 2008 and April 15, 2008. All partners should monitor progress and take appropriate action to stay on track. Many of the actions require the development of plans for monitoring field sites, strengthening technical areas, and leading and facilitating the sharing of promising practices among the NGOs. These plans need to be consolidated into an SHTP work plan for the next 12 months. Technical Deficits JSI Develop plans with specific activities to strengthen the following technical areas: • Development of County Health Department capacity in core competencies including planning, monitoring, supervision, drug and vaccine management, logistics, and HIS analysis • Development of linkages between County Health Departments and States to ensure coordination on core health systems functions such as drug and vaccine management • Development, empowerment, encouragement, training and mentoring of Village Health Committees (and water sub-committees). Ensuring these committees understand their roles and responsibilities, can solve local problems, understand what they can and cannot solve, and are linked to County Health Departments • Ensuring the use of PHCU and PHCC staff and TBAs in community mobilization and IEC activities through outreach --Incorporate IEC plans of Counterpart International into SHTP work plan Milestone: By April 15: Technical Plans submitted to and approved by USAID. Leadership • Restructure and empower strong leadership in JSI/Juba. • Assess leadership gaps and prepare plan for addressing these. • Provide effective technical backstopping from JSI/Boston. • Realign staffing pattern of JSI/Juba to work priorities, ensuring that all technical expertise exists (for example, health systems and decentralization). • Develop strategy to identify and share promising and best practices among SHTP partners. USAID/Sudan: Sudan Health Transformation Project Assessment Report 36 Milestones: By March 15: • Process for identifying promising and best practices from NGOs developed. • Process for regular sharing of lessons learned from NGOs developed. By April 15: • Re-structured and empowered SHTP leadership team approved by USAID. Management JSI • Re-align organization chart of JSI/Juba to support key technical areas. • Ensure strong management and technical staff is in place in both Boston and Juba. • Implement findings of USAID financial review. USAID • Clarify USAID staff roles and authorities; institute processes and procedures to handle staff absences and transitions. • Consider technical advisors/consultants to GOSS and county health departments in decentralization during transition Milestones: By March 15: • Procedure for staff absence and transitions developed by USAID. By April 15: • USAID produces and disseminates document that outlines roles and responsibilities of SHTP CTO team. • JSI-Updated organization chart with highly skilled and relevant staff approved by USAID.JSI-All financial review recommendations implemented by dates specified in report. Monitoring and Evaluation JSI • Develop field visit monitoring plan, with clear monitoring guidelines. • Establish regular feedback mechanisms to INGOs and other sub-grantees. • Conduct facility and transport inventory for all 6 SHTP counties (consistent with methodology suitable for national inventory). USAID • Arrange for/conduct external data quality check of SHTP service statistics. • Arrange for/conduct financial audit and verify use of project funds, starting with Twic East. Milestones: By March 15: • JSI-Present monitoring plan and guidelines for USAID approval. • USAID-Arrange for conduct of financial audit. By April 15: • USAID-Arrange for conduct of external data quality check. USAID/Sudan: Sudan Health Transformation Project Assessment Report 37 • JSI-Conduct facility and transport inventory. Communication, Collaboration and Coordination GOSS • Develop plans and mechanisms for dissemination of information (such as newsletters, monthly meetings, etc) to and across States, Counties, INGOs, LNGOs, and VHCs. USAID, JSI and GOSS • Reinstitute regular Core Group meetings. • Advocate and support resolution of EPI bottlenecks; support an EPI review with all partners. • Assure that all SHTP counties have a vaccine supply and schedule for vaccinating before and during the rainy season. (The vaccination schedule should be developed with County Medical Officers, where they exist.) • Consider individual agency and joint retreat(s) to clarify relationships and work plans. JSI • Establish focal person to serve as main point of contact for each INGO partner. Ensure focal person proactively communicates with INGO. • Assure that all messages developed by GOSS and UNICEF pertaining to the supply and distribution of EPI vaccines, EPI policy, etc. are clearly understood by the States, Counties, and INGOs where SHTP is working. • Ensure that GOSS, WHO, and UNICEF are familiar with the INGOs and the counties where SHTP is working, in particular, CARE in Twic East. Follow up with the INGOs that they have made needed contacts with GOSS, States, UNICEF and WHO. Milestones: By March 15: • JSI-Each NGO partner is assigned one point of contact. • USAID and Partners: Establish monthly meetings for GOSS, JSI and USAID; reinstitute quarterly Core group meetings; reinvigorate Quarterly Partner meetings. By April 15: • GOSS-State and County health officials informed of funding and programs available for their use. • USAID, GOSS and JSI-Develop plan for quarterly joint monitoring visits for the year. • GOSS lead with UNICEF to plans for National review of the EPI program. TA available on request from USAID through JSI • Plan for individual agency and/or joint retreat(s). Other Immediate Steps The Assessment team strongly recommends the following actions by May 15, 2008 to solve the high priority problems of drugs, NGO budget allocations, EPI and other technical areas. USAID/Sudan: Sudan Health Transformation Project Assessment Report 38 USAID • Follow up on the distribution of SHTP drugs ordered through UNICEF: find other location for storage and means of delivering them to the six counties. • Re-assess needs for any additional drugs and LLITNs given the quantities of drugs now available from the UNICEF order and other sources. • Consider assistance at national level to address needs for a comprehensive national logistics system. (With MSH and coordinated with MDTF and other sources) JSI • Re-align/adjust follow-on grant budgets for the six county NGOs to support priority actions in the SHTP work plan, with attention to sufficient funds for: – Provider Training – CHD and VHC mentoring and support – Outreach and IEC – Sharing among the NGOs, and – Leveraging funds from other sources. • Ensure that SHTP NGOs are collaborating with county and state EPI personnel in the implementation of the EPI Plans developed at the February 2008 meeting. • Consider providing long-term TA at the National level for EPI Policy development and technical support to the GOSS EPI manager in collaboration with UNICEF and WHO. For other technical areas • SHTP should work closely with the Department of Nutrition and EPI to consider how to track the provision of Vitamin A in routine services. For now, SHTP should assure that Vitamin A is available at its sites during the National Immunization Days and report on Vitamin A coverage on a semi-annual or annual basis SHTP should start reporting quarterly on ORT and ARI treatment for under 5s, using the service statistics already available to develop the indicators. For maternal health • SHTP should work in collaboration with other partners (MOH, UNFPA WHO) to reproduce and dissemination MRH policy and guidelines to roll out at county level. • SHTP should follow up with the MOH to determine the pros and cons of discontinuing MCH workers and TBA training given the circumstances of Southern Sudan and the potential time lag in replacing these auxiliary workers with more skilled cadres. • Use the next one year to review the PHCCs within the project areas and determine what it would take to upgrade these facilities to offer basic emergency obstetric care. • If possible, it is not too late influence the Capacity project staff analysis currently under way to include an analysis of “certification and training” received by the current cadres who are delivering ANC at facility level. This will provide a basis for determining upgrading or pre-service training of new health workers. D. CONSIDERATIONS FOR FUTURE HEALTH INVESTMENTS In order to provide guidance on future investments in USAID health program in Southern Sudan, the team reviewed the achievements and shortfalls of the present project, the USAID/Sudan: Sudan Health Transformation Project Assessment Report 39 needs for future health systems development, critical PHC needed to respond to growing numbers of returnees, a rapidly evolving donor environment at the national, State, County and community levels, and USAID’s comparative advantage. USAID should continue to build the foundation for future investments in the health sector from emergency/reconstruction programming to orchestrating a well-coordinated plan with other donors to support GOSS decentralization efforts in the health sector. The Assessment team proposes the following major components in the follow-on project to the Sudan Health Transformation Project: • Bolster decentralized health systems in supporting CPA implementation – Continue and further strengthen interventions for empowering County Health Department and engaging civil society groups including Village Health Committee (including water sub-committees) development; information dissemination from GOSS to the counties to leverage GOSS and donor funding, institutionalizing training , harmonizing policies re vaccin – Support State level and State-County interaction. – Monitor needs and target technical assistance for key functions at the GOSS/MOH, State and County levels • Continue to build on the delivery of high impact services. Although the Assessment identified many problems and gaps, the SHTP has made an important beginning and should be continued. – Further strengthen outreach, community mobilization, and community￾based approaches; pre- and in services -service training; job aids and supervision; and performance support for maternal health, TBAs and MCHWs; add in family planning/child spacing and breastfeeding; expanded maternal health services; nutrition; and TB as per BPHS. • Support improved epidemic preparedness and expanded attention to water and sanitation • Initiate/strengthen linkages with other sectors including HIV/AIDS targeting men and youth; and integration of health with education, economic growth, democracy and governance – Develop mechanisms to reinforce civil society groups. These groups can serve as outreach linkages and advocates for better health services, supporting health governance, community norms for healthy behaviors, and community financing of health interventions, where possible. – Link Village Health Committees and other groups with livelihoods, income-generating opportunities, etc and consider opportunities for complementary financing of these groups and members. – Create fast-track, rapid response initiatives to speed up delivery of basic health services responding to needs of marginalized groups and returnees. USAID/Sudan: Sudan Health Transformation Project Assessment Report 1 ANNEXES ANNEX A: STATEMENT OF WORK SUDAN HEALTH TRANSFORMATION PROJECT (SHTP) ASSESSMENT ANNEX B: REFERENCE DOCUMENTS FOR SHTP ASSESSMENT TEAM ANNEX C: STAKEHOLDERS CONSULTED/INTERVIEWED ANNEX D: USAID EVALUATION TEAM PROGRAM ANNEX E: INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH COMMUNITY GROUPS AND BENEFICIARIES ANNEX F: SHTP TOOLS/PRODUCTS FOR DISSEMINATION ANNEX G: TECHNICAL NOTES ON HIV/AIDS, TB, MALARIA AND FAMILY PLANNING ANNEX H: RECOMMENDATIONS FOR ACTIONS IN THE NEAR-TERM ANNEX I: SHTP PERFORMANCE ASSESSMENT POWERPOINT PRESENTATION USAID/Sudan: Sudan Health Transformation Project Assessment Report 2 ANNEX A: STATEMENT OF WORK SUDAN HEALTH TRANSFORMATION PROJECT (SHTP) ASSESSMENT JOHN SNOW INC. (JSI) PROJECT PERFORMANCE ASSESSMENT & LESSONS LEARNED January 19 -February 15, 2008 INTRODUCTION Sudan is the largest country in Africa, borders 9 countries and has a population estimated at 40 million. Although the first census in decades is planned, the current Southern Sudan (SSudan) population is estimated at 10 - 12 million people — with an estimated 4 million displaced to Northern Sudan and as refugees outside the country. Since independence in 1956, SSudan has suffered from civil war with only a decade of troubled peace from 1972 to 1983. The civil war period, characterized by devastation of country’s economic, political and social structures, left the health status of the Sudanese people among the poorest globally. Since the mid-90s, non-governmental organizations (NGOs), faith￾based organizations (FBOs), and multilateral/bilateral agencies offering humanitarian relief became the prime providers of health services. NGOs and FBOs continue to play the lead role in health service delivery; of the 30% of the population covered by health services 68% are provided by NGOs or FBOs. With the signing of the comprehensive peace agreement (CPA) on January 9, 2005, the political climate in South Sudan is changing to enable the transformation of health services delivery from humanitarian relief to a more developmental approach — focusing on developing systems, infrastructure, strategies and policies focused on sustainability while at the same time ensuring that basic health services are initiated and provided to the people of SSudan. The onset of peace has created expectations for a return to normality, including the provision of health services. The Government of SSudan (GOSS) is under significant pressure to improve health services and status and thereby make rapid and visible progress toward establishing legitimacy with the people of SSudan. Since physically reestablishing itself in Juba (about 1 year ago), the GOSS has been extremely committed to articulating a strong legal/policy framework for implementing its basic package of primary health care services. The GOSS/MOH has demonstrated impressive progress in generating a range of policies for setting the health policy framework for primary health care, including maternal and child health services for SSudan. The framework, which comprises 9 key policy inputs is progressive and represents the state of the art thinking in programming in post conflict settings. The framework captures key policies which have been developed over the past several years. They include: • The Comprehensive Peace Agreement (2005) to accelerate development to bridge the gaps across the regions; • The 2005 Joint World Bank UN Assessment Mission framework for Sustained Peace, Development and Poverty Eradication (2005); • The Interim Constitution of SSudan (2005); • The Health Policy of the New Sudan (1998); • Health Sector Recovery Strategy (2004); USAID/Sudan: Sudan Health Transformation Project Assessment Report 3 • The Basic Package of Health Services (2005) which spells out structure and function of services at the five levels; • The Interim Health Policy (2006-2011); and • The GOSS/MDTF South Sudan Umbrella Program for Health System Development. These GOSS policies flow from and complement the ICPD Global Program of Action (1994) and Millennium Development Goals. The overall Health Policy is well written and comprehensive, outlining the basic mission statement, values, and principles for implementing primary health care in SSudan as transition from war to peace in a post-conflict environment. This policy was developed through a consultative process with key stakeholders, international agencies, local partners and the Diaspora. For many years, until the 2005 CPA was signed, USAID was the only donor in the health sector, working closely with the Southern Sudan Secretariat of Health (pre-cursor to the MOH). In support of the GOSS health plan, in 2004, USAID signed a cooperative agreement with John Snow International (JSI) to implement the Sudan Health Transformation Project (SHTP) project. SHTP is a $27 million, five-year cooperative agreement that was designed, to provide basic primary health care services through small grants to local and international NGOs. The seven high impact services include: immunizations, child survival, antenatal care, malaria, water and sanitation, maternal and reproductive health and HIV/AIDS. As of September 2007, approximately $25m has been obligated out of the total amount cooperative agreement budget of $29m. At present, the SHTP is working in 6 counties (Tambura, Mvolo, Panyijar, Twic East, Mundri, and Tonj South. See the map on page xiv.) The JSI project was designed and awarded under the Mission’s Interim Strategy with the goal of “Increased Use of Health, Water & Sanitation Services and Practices”. The key results included: • IR 7.1: Increased access to high impact services • IR7.2: Increased Sudanese capacity, particularly women’s, to deliver and manage health services • IR7.3: Increased demand for health services • IR7.4: Increased access to safe water and sanitation • IR7.5: Increased access to HIV/AIDS prevention, care and support services In 2005, the Sudan Mission developed and approved a new strategy. The Sudan Mission’s primary goal under the Fragile States Strategy is to nurture the achievement of a just and lasting peace in South Sudan through the successful implementation of the Comprehensive Peace Agreement (CPA). The Strategy has two strategic objectives (SO). SO9 is to avert and resolve conflict and SO10 is to promote stability, recovery and democratic reform in South Sudan. The Health Portfolio contributes to both SO9 and SO10 through program activities which bolster confidence in the CPA and the new Government of Southern Sudan among its constituents. The activities serve to support the foundations for a fledgling health system in Southern Sudan to stand up the Ministry of Health and by improving health service delivery through community-based facilities. These efforts strengthen the government’s health institutions through leadership training of mid- and senior level program managers and policy makers at the Ministry; strengthen the financial and data management systems USAID/Sudan: Sudan Health Transformation Project Assessment Report 4 of the MOH at the National and County levels; and provide technical assistance to develop policies and laws which allow effective implementation of health services. The Health Portfolio also works directly to improve service delivery at the community level in six counties (and eventually to six more counties over the next year) by provision of an integrated package of seven life-saving interventions; essential medicines; in-service capacity building of community and facility-based health care providers, and building/rehabilitating Primary Health Care Centers and Units. The change in the strategy affected SHTP programming. Initially, the project was slated to work in 20 counties, but due to the shift in strategy the target counties were reduced to 9 counties and 3 urban areas to increase support of the CPA. In addition to the six current counties the program will expand to: Aweil South, Terekeka, and Kordofan counties, and three urban areas: Malakal, Wau and Juba. Transition from Relief to Development: As USAID/Sudan moves further to a long-term development mode, OFDA will phase out of health care activity to focus attention on critical emergency and relief operations. Therefore, it is vital that the investments in relief activities are maintained in transitioning over to a long-term development approach. From some current OFDA areas, this means handing over to the government, other donors or possibly picked up by USAID. Over many years, the Mission has discussed this transition and currently concrete plans are emerging for OFDA and the USAID/Sudan Health Team to work closely with the GOSS on this transition effort. Therefore, USAID/Sudan’s future health investments will reflect this transition and tackle the challenge of working with GOSS toward a sustainable health system. PURPOSE Given SSudan importance for the USG, it is critical to highlight and document key results in the health sector for all stakeholders, including host country counterparts, USAID/Washington, US Congress, tax payers and outline the impact of USG assistance in the lives of Sudanese. Many years have passed since the Sudan Health Transformation Project was designed, procured and implemented. The SSudan landscape has changed significantly: the CPA was signed; the Southern Sudan capital moved from Rumbek to Juba, multi-lateral and bilateral donors (since the signing of the CPA) have come into the SSudan in a significant way, and the development of the USAID/Sudan Fragile States Strategy has stimulated significant change in Mission program direction. Although a mid-term assessment of SHTP was planned in 2006, due to the constantly changing environment, it was postponed. With less than 2 years remaining in the project, it is time for a critical look at SHTP to determine what worked and what did not; outline models that could be replicated; and establish future directions of USAID’s health investments to support the GoSS. Objectives of the Mid-Term Assessment 1. To assess SHTP progress to date in meeting the deliverables of the SHTP Cooperative Agreement (including amendments to the original agreement) and lessons learned for future USAID/Sudan investments. 2. To assess SHTP approach in the context of high impact. Is this the best model the MOH existing sector priorities, policies and strategies. In the context of USAID/Sudan: Sudan Health Transformation Project Assessment Report 5 coordination efforts with USAID partners, other stakeholders, including international NGOs and CBOs. 3. Make recommendations for: a) Short-term adjustments in the JSI agreement that would improve performance in the remaining period of the agreement. Identify project components that could be scaled up or phased out for the greatest impact in the time remaining in the agreement. b) Long-term recommendations for a dynamic follow-on USAID health program which responds to priorities to provide tangible “health dividends”, supports the CPA, works with other sectors such as Economic Growth, OFDA, and Governance, and selective interventions for the Three Areas (S.Kordofan, Blue Nile and Abeyi). What aspects of current project activities should be continued, scaled up, omitted or added to substantially increase the impact of USAID’s health program in SSudan. Issues to be investigated The following questions should assist to investigate specific issues to fulfill the assessment objectives: Program Questions Is the SHTP approach, specific interventions and geographic coverage adequate for a significant health impact on child and maternal health? 1. According the Cooperative Agreement, is the program on the right track? What were the challenges and successes? 2. Is the SHTP approach, specific interventions and geographic coverage adequate for a significant health impact on child and maternal health? 3. Does the current project respond to the MOH’s desired directions for SSudan? 4. What is the nature and quality of the relationships between the SHTP and its local partners, communities, MOH (central, state, county), other USAID cooperating agencies, other NGOs and donor partners? 5. Are the technical areas and current approach appropriate for USAID/Sudan’s follow-on investment? How should USAID/Sudan focus its future health investments? Question 1: According to the Cooperative Agreement, is the program meeting its objectives and achieving results? a. Did the project achieve the activities agreed to under each IR? IR 7.1: Increased access to high impact services. • PHCU/C equipped and coverage improved • Community construction of PHC Units • Support for staffing, supervision, start-up drugs, and outreach services to underserved areas • Essential drugs available • High impact services increased: immunizations, antenatal care, vitamin A supplementation, anti-malarials and ITNs, • Additional components such as HIV/AIDS prevention activities and VCT Centers in selected areas; and USAID/Sudan: Sudan Health Transformation Project Assessment Report 6 • IEC programs developed and disseminated. IR7.2: Increased Sudanese capacity, particularly women’s, to deliver and manage health services. • Five CHW Training Institutes rehabilitated/developed • Develop and implement strategies and programs to improve women’s access, retention and learning in MCHW institutes • Updated, competency-based CHW/MCHW curriculum • CHW handbook of standard procedures developed and disseminated • Strengthen County Health Department in epidemiology, disease surveillance & response; and • Key national health policies developed for Tuberculosis control, Malaria Prevention & Control, and Reproductive Health policy. • Discuss SHTP efforts at capacity building among its grantees, central government, state government and local county health departments. Is the project strengthening county capacity to deliver health services? What are the major obstacles? How are they addressed at the various levels? What were the major breakthroughs and accomplishments? Give evidence and site examples. IR7.3: Increased demand for health services. • Immunization coverage increased; • Use of ITNs by pregnant women and children increased; • IEC messages in schools and in the communities increased • Vitamin A usage increased; • Appropriate treatment for malaria increased; and • Health educational messages broadcast by the radio increased; and • Use of PHCU services increased. IR7.4: Increased access to safe water and sanitation. • Promotion and monitoring of water management committees; • IEC messages on safe handling and storage of water; • Number of individual houses with latrines; and • Number of functioning water points six months after installation. b. Discuss how the basic package of “High Impact Interventions” integrated health program strategy is implemented. What were the challenges in meeting the IR results and the successes for each intervention? Discuss missed opportunities for linkages with HIV/AIDS PEPFAR funded activities. c. Discuss baseline and current service statistics in each of the technical domains. What were trends? Results achieved? Successes? Generate a two page fact sheet summarizing findings and lessons learned in each of the technical domains. Secure photos, if possible, for future desktop publishing and website use. Question 2: Is the SHTP approach, specific interventions and geographic coverage adequate for a significant health impact on child and maternal health? • Is the grant-making model the best approach in Sudan for major impact on maternal and child health? What were the limitations/advantages of this model? USAID/Sudan: Sudan Health Transformation Project Assessment Report 7 • Were the interventions selected the appropriate ones for high impact? • What other interventions make sense in a fragile state like Sudan for high impact? • What could be done to strengthen this impact? • Are the current project areas rational? Do they make sense considering the CPA and maternal and child health impact? • If new areas are selected in the future, what geographic coverage would make sense, considering the Mission’s fragile state focus, OFDA programs and transition, and the MOH’s plan for strengthening the health sector? Question 3: Does the current project respond to the MOH’s desired directions for SSudan? • Does the current project respond to the MOH’s desired directions for SSudan? • How does SHTP work with the MOH? • How can this relationship be further strengthened? • Is the program working towards sustainability? Building capacity? What else could be done? • How does the program complement other MCH and FP services in the country? Question 4: What is the nature and quality of the relationships between the SHTP and its local partners, communities, MOH (central, state, and county), other USAID cooperating agencies, other NGOs and donor partners? • How does SHTP work with its grantees? Is there a better way to strengthen SHTP’s support to its grantees? • How does SHTP work with other USAID CAs, NGOs and donors? How can this relationship be further strengthened? Question 5: Are the technical areas and current approach appropriate for USAID/Sudan’s follow-on investment? How should any future USAID investments be implemented/focused? • What approaches/models should be expanded in the follow-on project? What are the strengths and innovative activities being undertaken that should be continued, scaled-up and emphasized? • How can activities focus to support the current strategy? Strengthen the CPA? • What is the best approach for working in this transitional period? • How can USAID’s health development investments work better with OFDA investments? How can future programming work synergistically to transition from relief to development? • Should USAID work more in capacity building and systems strengthening at the county level? State level? Central MOH level? Community Level? What activities would have the greatest impact? Could USAID strengthen state and county planning systems and oversight role in future? • What should be the balance between service and health capacity/systems work? • If working with NGOs is the desired future direction, what are future strategic directions in strengthening NGO? • With limited resources, what are feasible and practical models to support access and quality of maternal and child health services and health systems on a large scale (e.g. nationwide)? • Are the high impact interventions adequate for improving access to quality services? USAID/Sudan: Sudan Health Transformation Project Assessment Report 8 • Is there a role for utilizing the private sector implementation and service delivery? Is there a role for the Diaspora? Are there more innovative/appropriate approaches for Sudan? If new areas are selected in the future, what geographic coverage would make sense, considering the Mission’s fragile state focus, OFDA programs and transition, and the MOH’s plan for strengthening the health sector? USAID/Sudan: Sudan Health Transformation Project Assessment Report 9 ANNEX B: REFERENCE DOCUMENTS FOR SHTP ASSESSMENT TEAM GOVERNMENT OF SOUTHERN SUDAN 1. Basic Package of Health Services for Southern Sudan, 2006. 2. EPI Comprehensive Multi-Year Plan, 2007-2011. 3. Laboratory and Nurses Training Curriculum. GOSS, USAID, JSI, AMREF, 2007. 4. Logistics Management in Primary Health Care. A Trainer’s Manual for Supply Chain Managers in Primary Health Care. GoSS, JSI, USAID; Compiled by Dr. W.B. Odinga Oduol and Dr. Richard Igu Keri, December 2007. 5. Final Draft Maternal and Reproductive Health Policy for Southern Sudan. 2007 (January).GoSS, USAID, SHTP, JSI 6. Trainers Manual on Sexually Transmitted Infections. GOSS, JSI, USAID. 7. First Steps Towards the Recovery of Southern Sudan Health Sector Report. 8. Sudan Household Survey (Southern Sudan Report). 2006. GOSS, UNICEF, UNFPA, WHO, USAID. 9. South Sudan Umbrella Program for Health Systems Development. April 30-May 14, 2007.GOSS-MDTF 10. South Sudan Umbrella Program for Health Systems Development; Aide Memoir. April 30- May 14, 2007. GOSS-MDTF. 11. Proceedings of a Stakeholders Workshop on Human Resources for Health. August 5-11, 2006. MOH, Directorate of Human Resource Development and Planning, Southern Sudan, and the Capacity Project. 12. Health Policy for the Government of Southern Sudan 2006-2011. 2007 13. Report on Situation Analysis and Recommendations on Human Resource Assessment for health in Southern Sudan. Assistance from WHO and AMREF. July 2006. USAID 1. JSI Cooperative Agreement and 10 Modifications 2. Michael, Janet; Andreini, Michael; Mojidi, Khadijat; Pressman, Willa; Rajikotia, Yogesh; Stanton, Mary Ellen; 2007 (September), GOSS and USAID Southern Sudan Maternal and Reproductive Health Rapid Assessment. 3. Performance Management Plan, Health. October, 2007. 4. USAID/Sudan Health, 2007. 5. USAID Sudan Strategy, 2006-2008. 6. Gender Assessment Sudan ISP: 2003-2005 7. Greene, Richard, Aleer, Dau, M’bior, Michael, Buckingham, Warren, Davis, Connie, and Morris, Peter. Health Assessment for Southern Sudan, Executive Summary. June 2003. USAID/Sudan: Sudan Health Transformation Project Assessment Report 10 8. Boulanger S, Rajkota Y and Pressman W. USAID Health 20/20. June 2007. Building an equitable Health System for Southern Sudan: Options for GAVI HSS Funding. 9. Health Programming for Rebuilding States. USAID/BASICS. SHTP REPORTS AND DOCUMENTS 1. Mundri County RFA: Action Africa Health- International, December 3, 2004. 2. Mvolo County RFA: Save the Children Federation, U.S.A., September 24, 2004. 3. Panyijar Country RFA: International Rescue Committee, September 24, 2004. 4. Tambura County RFA: International Medical Corps, September, 2004. 5. Twic East County RFA: CARE, January 31, 2006. 6. Tonj South County RFA: World Vision, December 30, 2004. 7. Shuruma, Andrew, SHTP Monitoring and Evaluation Plan, October 1, 2004- September 30,2009: M& E Specialist , September 30, 2005. 8. SHTP Quarterly technical and Financial reports April 2004– September, 2007 9. SHTP Project Description, April 23, 2004 10. SHTP Revised Project Description, 2006-2008 11. SHTP Overview 12. SHTP Country Briefs 13. SHTP Indicator Matrix 14. SHTP Annual Report, FY 2006 15. SHTP Work Plans, FY 04-08 JSI CONSULTANT REPORTS 1. Conroy, Colleen, April 19, 2006. SHTP Maternal and Child Health Programming Guidance. 2. Waldman, Ron. June 19, 2006. SHTP Trip Report 3. Keeney, Gwen Brumbaugh, 8-29 July, 2006. Home Based Life Saving Skills, Southern Sudan. 4. Cogswell, Lynn and Eshuchi. July 28, 2006. Rapid Assessment of Behavior Change Communications in Southern Sudan. 5. Sthreshley, Larry. June 18-21, 2007. Government of South Sudan Health Assembly. 6. Olivola, Ken. July 22-25, 2007. Trip Report 7. Rae, George and Castrillo, Marcelo. August 1-15, 2007. Support to the Ministry of Health to Develop and Implement an M&E Framework. 8. JSI Financial Review Scope of Work. USAID/Sudan: Sudan Health Transformation Project Assessment Report 11 9. Roberts, Susan and Narwil, Noah. October 29-November 16, 2007. Financial Review of SHTP. 10. Carnell, Mary and Tain, Francis. November 6-17, 2007. SHTP Trip Report on Child Health. OTHER REPORTS 1. Situational Analysis Reproductive Health-Jonglei. UNFPA 2. Situational Analysis Reproductive Health-NGB. UNFPA 3. Situational Analysis RH - S Sudan. UNFPA 4. Situational Analysis RH – Warrap. UNFPA 5. Situational Analysis RH – WBG. UNFPA 6. Situational Analysis - Institutional Framework for RH. UNFPA 7. Health Programming for Rebuilding States. USAID/BASICS 8. Waldman, R. Health Programming Guidance. 2006 USAID/Sudan: Sudan Health Transformation Project Assessment Report 12 ANNEX C: STAKEHOLDERS CONSULTED/INTERVIEWED JUBA US GOVERNMENT Allan Reed Director for Southern Sudan Program Chris Datta Consul General, State Department, Sudan Khadijat Mojidi Population, Health and Nutrition Officer, USAID/Juba Martin Swaka Health Specialist, USAID/Juba Mary Hobbs Supervisory GDO, USAID/Khartoum Inez Andrews Education TL, USAID/Juba John Kimbrough OFDA, USAID/Juba Rosemary Onduru Administrative Assistant, USAID/Juba Ruth Buckley USAID/AFR/DP/W Yves Kore Agreement Officer, USAID/Sudan MINISTRY OF HEALTH Monywirr Arop Under Secretary, Ministry of Health Samson Baba Director General of External Assistance & Coordination Nathan Atem Director General of Primary Health Care Manyang Agoth Thon Director General of Pharmaceutical Services Mawien Atem Director, Quality Assurance and Control Janet Michael Director of Nursing/Midwifery Makur Kariom Director for Reproductive Health Anthony Lako EPI Manager OTHER COLLABORATING AGENCIES Margaret Itto Sudan Coordinator, AMREF Tomoko Horii MDTF-Health Program Coordinator Adulahi M. Ahmed Head of Office, WHO Southern Sudan Asseffa. Afework Polio Coordinator, WHO Southern Sudan Peter Crowley Director, UNICEF, Southern Sudan David Deputy Director, UNICEF, Southern Sudan Romanus Mkerenga Chief Health and Nutrition, UNICEF, Southern Sudan Okiror Director EPI, UNICEF, Southern Sudan Dragudi Buwa Head of Office, UNFPA Bengt Herring Basic Health Services & HIV/AIDS Advisor, Joint Donor Team (JDT) John Mel Policy Officer, Basic Services, Health, Education, JDT Tesmerelna Atsbeha GF/UNDP Mariam Traore TB&TB/HIV Program Manager, UNDP Tessa Mattholie Technical Officer Malaria Consortium Sapana A. Abuyi Executive Director, Sudan Inland Development Foundation Abebe Gabremariam Regional Manager, Save the Children Marcie Cook PSI Country Representative USAID/Sudan: Sudan Health Transformation Project Assessment Report 13 SUDAN HEALTH TRANSFORMATION PROJECT Darshana Vyas Chief of Party, SHTP Omari Mohamed HIV/AIDS, SHTP Ephantus Wahome Water and Sanitation, SHTP Anthony Laki Monitoring and Evaluation Assistant, SHTP Jennifer Omino Grants Manager, SHTP Francis Khamis Finance Manager, SHTP Kirogo Mwangi Monitoring and Evaluation Manager, SHTP Felix Lon Lado MCH Advisor, SHTP Isaac Ondoga Grants Finance Officer, SHTP Colleen Conroy JSI/MNH/ Consultant MUNDRI / ACTION AFRICA HELP-INTERNATIONAL Redento Tombe Field Coordinator, AAH-Sudan Omer Mohamed PHC Coordinator, AAH-I/SHTP Lextion May Kenneth Health Coordinator, MRDA/SHTP H.E. Bullen Abiatara Mundri West Commissioner H.E. Wilson Abi Mundri East Commissioner Kenneth Korayi Coordinator, MRDA/SHTP James Smith SSRC Secretary, Mundri West Hellen Turkia SSRC Secretary, Mundri East Mary Lucy Lyabo PHC Supervisor, Mundri East Wilson Sakit PHC Supervisor, Mundri West Penemona Amoya C.MOH, Mundri East Lawrence B. Brock Ag/CMOH, CHD, Mundri West Henry Isaac Kayanga Store Keeper Martin M. Monea HMI Mundri CHD EPI Personnel Mundri PHCU/CC Personnel Mundri East and West Community Organizations Mundri East and West Village Health Committees Mundri East and West TBAs Mundri East and West TAMBURA / INTERNATIONAL MEDICAL CORPS Nicholas Wampoi Health Program Manager, IMC/SHTP Clement Babitimo Logistics Officer, IMC Debero Wolde HIV/AIDs Program Manager, IMC Bimpa IMC, Regional Coordinator Michael Yacob Acting Deputy County Director William Bakata County Medical Officer Samuel Gume Local Partner, Yubu PHCU/CC Staff Community Groups Village Health Committees EPI technical and Cold Chain Personnel New Sudan Women’s Federation Young Women’s Christian Association of Tambura County USAID/Sudan: Sudan Health Transformation Project Assessment Report 14 New Sudan Women Association Anisa Women’s Resource Center PANYIJAR / INTERNATIONAL RESCUE COMMITTEE George Kijana Health Coordinator, IRC/SHTP Goeffrey Olunga Regional Coordinator/Field Manager, IRC James Gatkoi Commissioner, Paynijar Peter Kai Chuol SSRRC Ganyliel Field monitor Bruno Bol Duok SSRC Deputy County Secretary John Tap Puol Security Advisor Kwong Mutik Secretary Panyjiar County Joseph Huet CHD Coordinator Lam Khillchiu Culture and Information Director Henrietta Khamat IRC PHCU/PHCC staff Community Groups Village Health Committees Community Based Distribution Agents TWIC EAST/CARE INTERNATIONAL Harron Angatia Mulongo Program Manager, CARE/SHTP Mary Rose Juwa MCHO/CARE Joseph Diing Community Based Health Care Officer, CARE/SHTP H.E. Ding Akol Ding County Commissioner Abdalla Hussen A/Commissioner Bior Jacob Buf County Office Manager Panyagor PHCC Staff Wangulei PHCU Staff Mabior PHCC Staff Other PHCC/U staff Village Health Committee U.S.A. Willa Pressman USAID/BGH/Sudan Country Team Leader Hope Sukin USAID/AFR/SD/ Health Team Leader Janet Paz-Castillo USAID/OHAA Brad Wallach USAID/EA/Sudan Director Doug Balko USAID/EA/Sudan Andrea Freeman USAID/EA/Sudan Shawn Phillips USAID/EA/Sudan Jamie Fisher USAID/FFP/Sudan Jen Mayer OFDA/Sudan Desk Tiffany Denman USAID/BGH/Sudan Program Assistant Dana Ott Former USAID Desk Officer Khama Rogo World Bank, Washington, D.C. Ken Olivola JSI/ Director International Programs/Boston Mary Carnell JSI Consultant USAID/Sudan: Sudan Health Transformation Project Assessment Report 15 ANNEX D: USAID EVALUATION TEAM PROGRAM County: Mundri East and West. Lead Agency: Africa Aktion Health International (AAH_I) Local Partner: Mundri Relief and Development Agency (MRDA) Date Activity Contact person Time Remarks Tues 29th • Depart Juba 8 a.m. • Arrive Mundri 8:50 a.m. Tuesday, January 29 • Team arrives in Mundri East County • Register with County Authorities/ courtesy call to the County Commissioner • Briefing with AAH_I and MRDA • Meeting with CMOH Dr Omer Mohamed PHC Coordinator￾AAH-I +8821651073699 Morning • Visit 1 PHCU Karika • Meet with Village Health Committee and community representatives; TBAs and women; youth. Afternoon Organize groups for Focus Group Discussion: with -Women and TBAs; Youth group; Village Health Committee and community representatives Wednesday, January 30, 2008 • Interview with CHD staff • Visit to Mundri PHCC • Visit to PHCU Mandi • Visit to PHCC Lakamadi Morning Afternoon Thursday, January 31, 2008 • Visit to PHCU Amadi • Visit to PHCU Lanyi • Visit to PHCU Buagyi • Visit to Oxfam and vaccine store Friday, February 1, 2008 • Depart Mundri 8:50 a.m. • Arrive Tambura 10:15 USAID/Sudan: Sudan Health Transformation Project Assessment Report 16 USAID EVALUTION TEAM PROGRAM County: Tambura County. Lead Agency: International Medical Corps (IMC) Local Partner: Yubu Development Agency Date Activity Contact person Time Remarks Friday, February 1, 2008 • Team arrives in Tambura County • Register with County Authorities/ courtesy call to the County Commissioner • Briefing with IMC • Courtesy call with CHD • Visit to PHCU Matoto • Visit to PHCU Mopoi Dr. Nicholas Wampoi SHTP Health Program Manager Thuraya: +8821643339034 Email: nwampoi@ imcworldwide.org Morning Afternoon Saturday, February 2, 2008 • Community visits/discussions: • TBAs from Tambura PHCC • Village Health Committee • Meeting with Yubu Development Agency • Meeting with IMC managers • Meeting with Tambura PHCC staff • Assessment Team meeting Morning Afternoon Sunday, February 3, 2008 Open Meeting with community groups: women’s groups; church groups Morning Afternoon Monday, February 4, 2008 • Visit to Source Yubu PHCC • Visit to Bambu PHCU • Visit to Mangburu PHCU • Visit to Duma PHCU • Meeting with County Health Department Morning Afternoon Tuesday 5, 2008 • Visit to cold chain and Tambura PHCC • Depart Tambura 1:00 p.m. • Arrive Ganyliel, Panyijar at about 3:00 p.m. USAID/Sudan: Sudan Health Transformation Project Assessment Report 17 USAID EVALUATION TEAM PROGRAM County: Panyijar County. Lead Agency: International Rescue Committee (IRC) Local Partner: 2 local CBOs identified Date Activity Contact person Time Remarks Tuesday February 5, 2008 • Team arrives in Ganyiel • Register with County Authorities/ courtesy call to the County Commissioner • Quick tour of PHCC and Regional Training Center • Briefing with IRC Afternoon Wednesday, Feb 6, 2008 • Visit to PHCU Leidit • Visit to PHCU Tiap • Visit to PHCU Dekom • Visit to PHCC and cold chain Ganyiel • Meeting at RTC with RTC staff • Meeting with County Health Department (CMO) Morning Afternoon Visits to PHCUs included discussions with TBAs, Village Health Committee members, and CBD agents working on a CIDA-funded child survival program. Thursday, Feb. 7, 2008 • Depart Ganyiel 9:30 a.m. • Arrive Mabior/Panyagor, Twic East County 10:30 a.m. USAID/Sudan: Sudan Health Transformation Project Assessment Report 18 USAID EVALUATION TEAM PROGRAM County: Twic East County. Lead Agency: CARE International Local Partner: None Date Activity Contact person Time Remarks Thursday February 7, 2008 • Team arrives in Mabior/ Panyagor • Register with County Authorities/ courtesy call to the County Commissioner • Briefing with CARE • Visit to Panyagor PHCC • Visit to Wangulei PHCU Harron Angatia Mulongo SHTP Program Manager +8821643334137 Email: harrymully@ yahoo.com Morning Afternoon Visit to PHCU included visit with local leaders on the Village Health Committee Friday, Feb 8, 2008 • Visit to Paliau PHCC • Visit to Maar PHCU • Meeting with CARE staff and Assessment team • Informal other meetings Morning Afternoon Saturday, Feb 9, 2008 • Depart for Juba 10:00 a.m. USAID/Sudan: Sudan Health Transformation Project Assessment Report 19 ANNEX E: INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH COMMUNITY GROUPS AND BENEFICIARIES INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Number and general description. Whether they have moved here recently. 1. What are health problems in the community? Which ones are most important? What do children get sick from? Adults? Women? Babies? 2. Where do they go for health services when they need them? How do they get to the services? 3. What do they know about the PHCU or PHCC? What services are provided there? Have they been there? If not, why not? If yes, what did they think? Of quality? Other aspects (waiting time, cost, availability of drugs)? Are they familiar with: [Specific services—like family planning, well baby, antenatal, care for malaria, diarrhea, etc. depending on group and team members] Outreach to the community—how, how often? Community education efforts Status and functioning of the Health Management Committees Other sources for health care. What are the problems in the health facilities they have attended over the last 3 months? If they could make changes in how the PHCU or PHCC functions, what would they do? How would they improve services? How can they improve their own health? How would they like to learn about how to improve their own health? If USAID were to offer to help, what would you want from USAID? Other thoughts USAID/Sudan: Sudan Health Transformation Project Assessment Report 20 INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH COUNTY HEALTH OFFICIALS INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. Specific information about them, as feasible Experience in Southern Sudan and with GOSS, MOH; prior experience; areas of expertise. 1. From what you know about SHTP, how has SHTP contributed to Health Sector development in Southern Sudan? What has the project provided? How? At County level? In delivery of primary health care services? Training? Water and Sanitation? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? Other contributions? 2. From your experience, what have been the strong elements of SHTP’s work? Are these elements that you think should continue? Why? 3. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? If so, how would you change what SHTP has done? 4. How well has SHTP coordinated its work with other stakeholders working in your County in Health Sector? What specifically could it do to improve its coordination and support? 5. Are there other activities or areas that you think SHTP should be engaged in from now and the end of the project in 2009? 6. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? At the County level? At the State level? USAID/Sudan: Sudan Health Transformation Project Assessment Report 21 Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. Capacity building—at what levels; for what aspects. National level policy formulation and implementation Infrastructure Support for models of service delivery and management, e.g., development of MOH unit for contracting out to NGOs Systems development: logistics; other. Other components 7. Do you have any other suggestions about how USAID might support the Health Sector of Southern Sudan? 8. Other thoughts. USAID/Sudan: Sudan Health Transformation Project Assessment Report 22 INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH HEALTH SERVICE PROVIDERS INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. Specific information about them, as feasible Experience in health service delivery—where and how long; technical and management training; areas of expertise. 1. Background on your work and service site. Catchment Area Clients/day, month Services provided (look at and get charts with statistics as available) Outreach to the community—how, how often? Availability of staff, drugs, other supplies Availability of water, sanitation facilities Status and functioning of the Health Management Committee Problems/issues 2. How has SHTP supported you and/or your service site? What has the project provided? How? How/in what way has SHTP supported delivery of primary health care services? Essential drugs? Training? Water and Sanitation? Community mobilization and communication? Are you aware of any other activities of SHTP? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? Other contributions? 3. From your experience, what have been the strong elements of SHTP’s work? Are these elements that you think should continue? Why? 4. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? USAID/Sudan: Sudan Health Transformation Project Assessment Report 23 If so, how would you change what SHTP has done? 5. How well has SHTP supported the INGO and Local NGO with which you work here? What specifically could it do to improve its support for and capacity building of your INGO and LNGO? 6. Are there other activities or areas that you think SHTP should be engaged in from now and the end of the project in 2009? 7. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. Infrastructure Systems development: logistics; other. Capacity building—at what levels; for what aspects. Support for INGOs, LNGOs? At the County level? At the State level? National level policy formulation and implementation Other components 8. Do you have any other suggestions about how USAID might support the Health Sector of Southern Sudan? 9. Other thoughts. USAID/Sudan: Sudan Health Transformation Project Assessment Report 24 INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH INGO AND LNGO STAFF INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. Specific information about them, as feasible Experience in health service delivery,--where and how long; technical and management training; areas of expertise; experience with the LNGO and INGO.. 1. Background on your work and service sites in this county. Catchment Area(s) For service sites managed: Clients/day, month for INGO or LNGO Services provided (look at and get charts with statistics as available) Outreach to the community—how, how often? Availability of staff, drugs, other supplies Availability of water, sanitation facilities Status and functioning of the Health Management Committees Health Problems/issues Management Problems/Issues 2. How has SHTP supported your INGO/LNGO and/or your service sites? What has the project provided? How? How/in what way has SHTP supported your management and delivery of primary health care services? Essential drugs? Training? Water and Sanitation? Community mobilization and communication? Are you aware of any other activities of SHTP? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? Other contributions? 3. From your experience, what have been the strong elements of SHTP’s work? USAID/Sudan: Sudan Health Transformation Project Assessment Report 25 Are these elements that you think should continue? Why? 4. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? If so, how would you change what SHTP has done? 5. How well has SHTP supported the INGO and Local NGO with which you work here? What specifically could it do to improve its support for and capacity building of your INGO and LNGO? 6. Are there other activities or areas that you think SHTP should be engaged in from now and the end of the project in 2009? 7. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. Infrastructure Systems development: logistics; other. Capacity building—at what levels; for what aspects. Support for INGOs, LNGOs? At the County level? At the State level? National level policy formulation and implementation Coordination among the INGOs and LNGOs Other components 8. Do you have any other suggestions about how USAID might support the Health Sector of Southern Sudan? 9. Other thoughts. USAID/Sudan: Sudan Health Transformation Project Assessment Report 26 INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH MOH LEADERS INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. Specific information about them, as feasible Experience in GOSS, MOH; prior experience; areas of expertise. 1. How has SHTP contributed to Health Sector development in Southern Sudan? What has the project provided? How? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? At County level? In delivery of primary health care services, esp. high impact services? Training? Water and Sanitation? Other contributions? 2. What have been the strong elements of SHTP’s work? Are these elements that you would like to continue? Why? 3. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? If so, how would you change what SHTP has done? 4. How well has SHTP coordinated its work with other stakeholders working in the Health Sector in Southern Sudan? What specifically could it do to improve its coordination and support? In what ways is SHTP coordination linked with government mechanisms for donor coordination? 5. Are there other activities or areas that you think SHTP should be engaged in between now and the end of the project in 2009? 6. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? USAID/Sudan: Sudan Health Transformation Project Assessment Report 27 Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. Capacity building—at what levels; for what aspects. National level policy formulation and implementation Infrastructure Support for models of service delivery and management, e.g., development of MOH unit for contracting out to NGOs Systems development: logistics; other. Other components 7. Do you have any other suggestions about how USAID might support you in achieving your goals in the Health Sector of Southern Sudan? USAID/Sudan: Sudan Health Transformation Project Assessment Report 28 INTERVIEW GUIDE FOR SHTP ASSESSMENT MEETINGS WITH OTHER DONORS/STAKEHOLDERS INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. Specific information about them, as feasible Experience in Southern Sudan and with GOSS, MOH; prior experience; areas of expertise. 1. From what you know about SHTP, how has SHTP contributed to Health Sector development in Southern Sudan? What has the project provided? How? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? At County level? In delivery of primary health care services? Training? Water and Sanitation? Other contributions? 2. From your experience, what have been the strong elements of SHTP’s work? Are these elements that you think should continue? Why? 3. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? If so, how would you change what SHTP has done? 4. How well has SHTP coordinated its work with other stakeholders working in the Health Sector in Southern Sudan? What specifically could it do to improve its coordination and support? How does SHTP coordination relate to GOSS coordination of donor programming? 5. Are there other activities or areas that you think SHTP should be engaged in from now and the end of the project in 2009? 6. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. USAID/Sudan: Sudan Health Transformation Project Assessment Report 29 Capacity building—at what levels; for what aspects. National level policy formulation and implementation Infrastructure Support for models of service delivery and management, e.g., development of MOH unit for contracting out to NGOs Systems development: logistics; other. Other components Other suggestions about how USAID might support the Health Sector of Southern Sudan 7. Does your organization intend to support health activities in the Southern Sudan over the next 3-5 years? If so, what activities are you planning to support? In what ways will these activities complement those of USAID? 8. How could USAID strengthen the collaboration among the various partners engaged in the Health Sector to ensure maximum impact? 9. Other thoughts. USAID/Sudan: Sudan Health Transformation Project Assessment Report 30 OBSERVATION GUIDE FOR SHTP ASSESSMENT VISITS TO HEALTH SERVICE SITES INTRODUCTION OF TEAM AND PURPOSE OF VISIT Assess implementation of the Sudan Health Transformation Project Identify its strengths and areas where changes may be needed Consider future areas of USAID investment in the health sector in Southern Sudan If needed, give overview of SHTP. ATTENDEES Name, positions. 1. Outside area, around the site Cleanliness Evidence of appropriate garbage, medical waste disposal Accessibility of site to community 2. Inside the service site Waiting area: Suitable? Clean? People able to sit? Educational materials? Crowded? Exam room(s): Suitable? Clean? Private? Other spaces for records, drugs, etc.: Suitable? Clean? Organized appropriately? Preserve privacy and restrict access, as needed? 3. Availability of staff and commodities Staffing: are there sufficient numbers for the client load? Do they have the requisite training and skills? Have they been paid regularly? Do they have a positive approach to their work and client care? Do they understand the need to work in the community? Drugs and other commodities: Are there adequate supplies of essential drugs and other supplies given the client load? Are they accessible? Are the supplies well￾organized to support sound logistics management? 4. Organization of services Does the client flow facilitate access and reduce waiting time? Do all staff members have clear roles and responsibilities? Do they make the clients feel welcome and cared for? Is there a system for identifying those clients in need of emergency or time-sensitive services? 5. Quality of services If possible, observe service provision. Does provider apply most up-to-date international guidelines in providing that specific service? Does the record-keeping system support high quality services? Are the records being kept? Does the client get adequate information about what the provider is doing or will do? Does the client get adequate information about what s/he needs to do for follow up? USAID/Sudan: Sudan Health Transformation Project Assessment Report 31 Does the provide counsel client to identify other services that may be needed? And then provide them, or refer to someone else? Catchment Area Clients/day, month Services provided (look at and get charts with statistics as available) Outreach to the community—how, how often? Availability of staff, drugs, other supplies Availability of water, sanitation facilities Status and functioning of the Health Management Committee Problems/issues 6. How has SHTP supported you and/or your service site? What has the project provided? How? How/in what way has SHTP supported delivery of primary health care services? Essential drugs? Training? Water and Sanitation? Community mobilization and communication? Are you aware of any other activities of SHTP? At central MOH? [May want or need to prompt re policies and guidelines development; development of management capacity, etc.] At State level? Other contributions? 7. From your experience, what have been the strong elements of SHTP’s work? Are these elements that you think should continue? Why? 8. What are areas where you think that SHTP has not done very well? Are they areas where you think assistance is still needed? If so, how would you change what SHTP has done? 9. How well has SHTP supported the INGO and Local NGO with which you work here? What specifically could it do to improve its support for and capacity building of your INGO and LNGO? 10. Are there other activities or areas that you think SHTP should be engaged in from now and the end of the project in 2009? 11. It is anticipated that USAID will continue to support the Health Sector in Southern Sudan after the end of the SHTP. What are your suggestions for the areas that USAID should support in the future? Service delivery—in what aspects; geographical focus; service focus and high impact PHC programming; community mobilization and involvement; BCC programming. Infrastructure Systems development: logistics; other. USAID/Sudan: Sudan Health Transformation Project Assessment Report 32 Capacity building—at what levels; for what aspects. Support for INGOs, LNGOs? At the County level? At the State level? National level policy formulation and implementation Other components 12. Do you have any other suggestions about how USAID might support the Health Sector of Southern Sudan? 13. Other thoughts. USAID/Sudan: Sudan Health Transformation Project Assessment Report 33 ANNEX F: SHTP TOOLS/PRODUCTS FOR DISSEMINATION • The Assessment team considered the following products of the SHTP as appropriate for dissemination at the national level for use throughout Southern Sudan. • Laboratory and Nurses Training Curriculum. GOSS, USAID, JSI, AMREF, 2007 (all volumes) • Clinic-based Record-keeping System, including: Antenatal Register; Delivery Register; Outpatient Register; Drug Dispensing Register; Summary/Tally Forms. USAID/Sudan: Sudan Health Transformation Project Assessment Report 34 ANNEX G: TECHNICAL NOTES ON HIV/AIDS, TB, MALARIA AND FAMILY PLANNING Members of the SHTP Assessment team were asked to focus on several other technical areas t during the visit, specifically TB and family planning. Although malaria is included as part of the seven high impact services, the malaria expert had some additional recommendations for programming that did not fit into the main report. In addition, the team did not specifically review the HIV/AIDS prevention activities as they were added on after implementation of the SHTP started and did not become one of the high impact services of the project. HIV/AIDS PREVENTION ACTIVITIES HIV/AIDS preventive activities started later than the other high impact interventions under SHTP. It was strategically decided to focus on the “corridor” counties that line the road from Uganda to Sudan (i.e., Mundri, Mvolo, and Tambura). These three counties were provided an additional $80,000 to develop prevention interventions. The initial focus was on creating awareness. In the beginning JSI lacked technical staff to assist counties to develop a program. Mundri first trained all their PHCC/PHCU staff in HIV/AIDS. They then identified a CBO, the Mundri Youth group and worked with them on developing a theatre and puppet group that could visit church, schools and villages to get the word out on HIV. They brought in a Kenyan group that showed the youth how to construct puppets and used SHTP funds to construct VCT centers and train counselors. There are now three functioning VCT sites in health facilities. The focus has been on advocacy, awareness, condom promotion, and VCT. The assessment team was entertained by the Maya Youth group which put on a puppet show and skit. It was quite entertaining even to the team that did not know the local language. VCT rapid kits were initially supplied by UNICEF, and some of the remaining kits will soon expire. It is not clear how they will be recovered and additional kits resupplied. In Karika in Mundri West, the Assessment team met with the Village Health Committee, the youth group, and a women’s group. All three mentioned HIV/AIDS as a concern. There were several peer educators in the youth group who spoke knowledgeably about HIV. They mentioned that to date there was no HIV care program to provide ARVs for those identified as being positive. Many HIV/AIDS posters (four different ones) and brochures were seen in the health facilities and also in the town. Tambura County has the highest HIV prevalence rate of south Sudan (25%). Limited data from other areas show prevalence rates of 8% in Yei, 6% in Kaya and 20% clients in the Juba Teaching Hospital. Currently IMC is the only INGO supporting PHC interventions in the county. In August 2005, the INGO started HIV awareness activities, and condom promotion. Later, several other HIV/AIDS organizations came to town: PSI supports peer educators, and CDC/PEPFAR arrived in September 2007 looking for a site to start VCT and a PMTCT sentinel site. CDC has placed a physician in the Tambura hospital/PHCC to manage the sentinel site. The Village Health Committee attached to the Tambura PHCC was quite vocal in its complaint that no HIV care and treatment services are available (no ARVs) and patients are referred to Anzarra (about 160 miles away) for further workup, or they go to Uganda. The community has an incomplete understanding USAID/Sudan: Sudan Health Transformation Project Assessment Report 35 of when ARVs should be accessed and do not understand that even HIV positive people must meet certain criteria before they are placed on drugs. They maintained that now the community was angry and few people now see the need to “get tested” since “nothing is done for you.” At least 20 minutes were spent in explaining the requirements for getting access to drugs. HIV/AIDS training has been provided to PHCC/U staff, village heath committees, and youth and women’s groups. HIV posters were seen in many PHCCs and Units in Tambura County. JSI hired a full time HIV advisor in October 2007. Previously Omari worked for CDC but spent 50% of his time in support of SHTP HIV activities. In 2005 the mission also put funds into the USAID regional HIV program- the FHI ROADS project. Initially the ROADS project was slow to start up, but it was reported that project staff spent considerable time working on collaborating and sharing information with other HIV partners including SHTP. JSI has now turned over a number of their HIV sites to FHI in Rumbek and Aya and in Kapoeta and Loki. Coordination and collaboration with CDC, on the other hand, has not been easy. JSI has worked jointly with WHO to develop guidelines on STIs, training manuals, and participated in a consensus building workshop on syndromic treatment of ulcers and discharge. The USG PEPFAR program is primarily focused on prevention activities in South Sudan and interventions compose of: • Counseling and testing • Communication strategies, education • Prevention from Mother to Child • ABC interventions • STI prevention Cross cutting interventions with GOSS include the development of strategies, policies and guidelines and building capacity of local HIV/AIDS organizations. To date little work is being done in care and treatment. The Assessment team thinks that HIV care and treatment is beyond the scope of the SHTP. However to address community concerns in SHTP-supported areas some interventions that are not too costly could be implemented while waiting for other organizations to provide care: • Introduce provider-initiated C&T at selected PHCCs • VCT • Cotri-moxazole preventive therapy (CPT) • Prompt malaria treatment, LLITNs, and water treatment at the household level. The above will provide specific interventions that will improve the quality of life for HIV positive persons and will prolong life while waiting for care and treatment services to arrive at selected sites such as Tambura. TUBERCULOSIS CARE AND TREATMENT At the time of the design and launch of the SHTP, TB care and treatment was only being provided at about five sites in Southern Sudan. Because SHTP was located at the community level and lab capability would be weak even if present, TB was not included as one of the high impact services. Currently there are 32 TB sputum microscopy centers in 7 states. Three states have no health facilities capable of smear microscopy. There are limited human resources with the expertise in TB diagnosis. Only one private lab training program exists that provides a 2 year certificate course (Malteser, Yei). There is high USAID/Sudan: Sudan Health Transformation Project Assessment Report 36 turnover in lab technicians and assistants due to the low salaries and no career path. The National Reference Lab in Juba is under renovation and is not functional. In 2002 Southern Sudan won a Global Fund Round (GF) 2 award aimed at providing care and treatment initially for 12 centers. The country also won a GF Round 5 award of $15 million to strengthen TB/HIV interventions and recently won a Round 7 award to focus on DOTS expansion, training, provision of anti TB drugs, and strengthening the national lab. This latest funding will also cover staff salaries at GOSS central TB unit at GOSS and provide for 10 TB coordinators to function at the state level. Now that the TB national policy and guidelines have been established and published, and a continuous drug supply has been assured for the next five years, plans can start for the roll out of TB in a systematic manner. The strategic plan calls for TB treatment to be provided by community based DOTS to insure high treatment rates. TB care and treatment is included in the basic minimum package (2006) of services to be provided at the PHCC which in theory should have a laboratory. Of the PHCCs visited during the assessment, few were actually functioning due to the lack of qualified staff. The lab was closed at Lakamadi, Mundri East (no staff), likewise at Pilau PHCC, Twic East (lab was just a bare dirt floor with no water, equipment, or personnel). However at Tambura PHCC/Hospital there is a lab assistant who has been working for the last 11 years. He had an initial training of 6 months followed by 2 years training by NPA in Yei. The lab is able to examine stools, urine, malaria smears, bacteriology, VDRL and skin nips. They also do sputum smears. If the patient has positive sputum he is referred to the Leprosy program which will complete all the documentation and refer the patient to Anzarra. The client must find their own transport to Anzarra which will repeat all the tests but then can hospitalize the patient and start treatment. Most patients are unable to get to Anzarra. No TB treatment is provided at Tambura. The Assessment team recommends that, given the low level of laboratory capability of the PHCCs, the limited qualified staff, and the degree of management and supervision required ensuring low defaulter rates, SHTP should not take on TB prevention and control during the last 18 months of the project. TB funds to the mission should continue to be channeled to TB CAP. However, in whatever form the next project takes, it will be important to insist that TB is included among the high impact services. This will mean that the implementing agency will have to have TB expertise in its managerial staff, and will need to be prepared to establish TB diagnostic and treatment centers. In neighboring countries surrounding Sudan, the laboratory component of TB treatment is the weakest link. The laboratory in Sudan will need considerable strengthening before it can play its important role in the diagnosis and treatment of TB. MALARIA There are many missed opportunities in Southern Sudan to prevent malaria. One has been the limited supply of nets. It is advised that SHTP nets through PSI funded by the Global Fund. SHTP needs to move forward with securing the 2nd batch of nets due to arrive in April and move quickly to finalize procurement of the 3rd batch of nets to ensure continuous supplies to existing and new INGO partners. In addition, SHTP needs to: • Distribute a net to all children who come to consultation clinic for treatment of malaria. Ask if they have/use a net and then provide one. • Use village health committee for social mobilization activities, distribute via community based distributors (CBDs of IRC) (Ganyiel), use community promoters, USAID/Sudan: Sudan Health Transformation Project Assessment Report 37 PDQs (partnership defined quality) to mobilize community to access nets (Mvolo), and health peer educators in Mundri. • Continue ANC net promotion and provide to TBAs to give to pregnant women in their community • Each county should have a target for number of pregnant women sleeping under a net, and the number of children under five sleeping under a net. The national malaria control targets are 60% of pregnant women sleep under a net and 60% of children < five sleep under a net. These should be the targets for the counties. The model for providing technical assistance for malaria, with the advisors sitting with counterparts in the Ministry of Health, has been much appreciated as exemplified by the experience with the support provided by Rational Pharmaceutical Management (RPM Plus). Dr. Robert Azairwe sits at the MOH to strengthen the Malaria Control Program and Pharmaceutical management services. The advisor assists with policy and strategy development at the national level, quantification of program requirements, establish coordination mechanisms, including a malaria newsletter and assists in training of health staff. RPM Plus also supports the MOH to strengthen the malaria M&E systems and works with partners to build capacity for drug supply management. FAMILY PLANNING/CHILD SPACING The development of capacity and programming for family planning (FP) and child spacing service delivery is a growing priority in Southern Sudan. FP is a critical component of MCH care, as the birth-spacing and avoidance of high risk pregnancies are important to the improvement of both child health and maternal health. As the recent Rapid Assessment on Maternal Health pointed out, a critical step in improving maternal health will be to help women avoid unintended pregnancies and help them space their births as they would prefer. Programming for family planning in Southern Sudan is a politically sensitive issue as it has been associated with limiting population growth, a need for which most political leaders do not agree given the large territory of Southern Sudan and the long conflict that took many Southern Sudanese lives. However, most Southern Sudanese with whom the Assessment team met understood the need for spacing births at least 2-3 years apart. In one case, the paramount chief noted that they too should be more like the visitors and have only 2-3 children. Most women, TBAs, and health care providers were not familiar with the full range of family planning methods. Some knew of condoms, pills and in a few cases, injectables. Others were familiar with the calendar method and they were remarkably accurate in identifying the safe period. Thus, while FP may still be politically sensitive at the higher levels of Government, the general population understands the benefits of birth spacing. They also understand that the former, traditional methods of separating from the husband for two years after a birth are not practical or no longer used. The RH/FP advisor on the Assessment team recommends that the Mission use the final time period of the SHTP to make some targeted investments to lay the groundwork for a more concerted, integrated effort for larger scale family planning service delivery to be launched with the follow-on project. Specifically, these actions are recommended over the next 12-18 months: • Direct engagement by USAID and JSI in the Reproductive Health Working Group, which meets once a month to coordinate efforts in RH/FP (UNFPA). USAID/Sudan: Sudan Health Transformation Project Assessment Report 38 • Provide support (TA or some financing of meetings, etc.) to provide assistance along with UNFPA in the development of a strategy and program implementation plans to operationalize the Maternal and Reproductive Health Policy. UNFPA indicated that they are leading this effort and providing TA for it. However, USAID needs to ensure that FP is adequately addressed. Note that while the MRH Policy is close to approval and Technical Guidelines for FP have been developed, there is no planning to link the two in programmatic action. • Review (with UNFPA) the status of FP commodities, specifically condoms, pills and injectables, Cycle beads for the Standard Days Method, and possibly IUDs and Implants. While uptake may be very slow now and will continue for the next few years, programmatic progress will be stalled if an adequate supply of contraceptives is not available. • Develop as part of the strengthened community mobilization and IEC efforts under SHTP (see Actions for the Near Term) messages and materials regarding family planning and child spacing. Strong community sensitization work is needed to promote community understanding of the importance of FP for spacing and good maternal health. • In selected SHTP sites, Mundri County and possibly Tambura, add to the community mobilization efforts the provision of advice and commodities on: condoms, SDM, and LAM. • In selected SHTP communities, use the TBA network for messages and commodities re child spacing: condoms, LAM, SDM. For the follow-on, family planning should be integrated into the PHC services with the following key elements: • Training of CHWs and MCHWs in the provision of condoms, pills, injectables, LAM and SDM, and including the use of the Pregnancy Checklist. • Record-keeping. Revise registers now used to include family planning services and include the means to track clients so that defaulters can be identified. (This is a need for EPI and ANC as well.) • Develop messages and materials re why spacing (or stopping, e.g., at 9), methods available, how to use, side effects • Ensure full supply of commodities. (No product, no program.) • Work with PSI to add pills to their social marketing efforts in the towns. • Consider MRH advisors at state level to follow up and ensure quality services. • Inclusion of FP into supervisory checklist. • Build on experience for last months of SHTP to extend family planning into communities, through sensitization and outreach and CBD. • Ensure strategies that link health, FP and HIV with USG funded activities • Ensure FP/RH initiates interventions which address adolescent health given the decrease in sexual debut by youth in Southern Sudan USAID/Sudan: Sudan Health Transformation Project Assessment Report 39 ANNEX H: RECOMMENDATIONS FOR ACTIONS IN THE NEAR-TERM The Assessment team strongly recommends that the following actions be taken as soon as possible, and completed no later than May 15, 2008, with corresponding milestones scheduled for March 15, 2008 and April 15, 2008. All partners should monitor progress and take appropriate action to keep on track. Many of the actions require the development of plans, for monitoring field sites, strengthening technical areas, leading and facilitating the sharing of promising practices among the NGOs, and the like. These plans need to be consolidated into an SHTP work plan for the next 12 months. TECHNICAL DEFICITS JSI • Develop plans with specific activities to strengthen the following technical areas: – Development of County Health Department capacity in core competencies including planning, monitoring, supervision, drug and vaccine management, logistics, and HIS analysis – Development of linkages between County Health Departments and States to ensure coordination on core health systems functions such as drug and vaccine management – Development, empowerment, encouragement, training and mentoring of Village Health Committees (and water sub-committees). Ensuring these committees understand their roles and responsibilities, can solve local problems, understand what they can and cannot solve, and are linked to County Health Departments – Ensuring the use of PHCU and PHCC staff and TBAs in community mobilization and IEC activities through outreach – Incorporate IEC plans of Counterpart International into SHTP work plan Milestone: By April 15: Technical Plans submitted to and approved by USAID. LEADERSHIP JSI • Restructure and empower strong leadership in JSI/Juba. • Assess leadership gaps and prepare plan for addressing these. • Provide effective technical backstopping from JSI/Boston. • Realign staffing pattern of JSI/Juba to work priorities, ensuring that all technical expertise exists (for example, health systems). • Develop strategy to identify and share promising and best practices among SHTP partners. Milestones: By March 15: • Process for identifying promising and best practices from NGOs developed. • Process for regular sharing of lessons learned from NGOs developed. By April 15: Re-structured and empowered SHTP leadership team approved by USAID. USAID/Sudan: Sudan Health Transformation Project Assessment Report 40 MANAGEMENT JSI • Re-align organization chart of JSI/Juba to support key technical areas. • Ensure strong management and technical staff is in place in both Boston and Juba. • Implement findings of USAID financial review. USAID • Clarify USAID staff roles and authorities; institute processes and procedures to handle staff absences and transitions. Milestones: By March 15: • Procedure for staff absence and transitions developed by USAID. By April 15: • USAID produces and disseminates document that outlines roles and responsibilities of SHTP CTO team. • JSI-Updated organization chart with highly skilled and relevant staff approved by USAID. • JSI-All financial review recommendations implemented by dates specified in report. MONITORING AND EVALUATION JSI • Develop field visit monitoring plan, with clear monitoring guidelines. • Establish regular feedback mechanisms to INGOs and other sub-grantees. • Conduct facility and transport inventory for all 6 SHTP counties (consistent with methodology suitable for national inventory). USAID • Arrange for/conduct external data quality check of SHTP service statistics. • Arrange for/conduct financial audit and verify use of project funds, starting with Twic East. Milestones: By March 15: • JSI-Present monitoring plan and guidelines for USAID approval. • USAID-Arrange for conduct of financial audit. By April 15: • USAID-Arrange for conduct of external data quality check. • JSI-Conduct facility and transport inventory. COMMUNICATION, COLLABORATION AND COORDINATION GOSS • Develop plans and mechanisms for dissemination of information (such as newsletters, monthly meetings, etc) to and across States, Counties, INGOs, LNGOs, and VHCs. USAID/Sudan: Sudan Health Transformation Project Assessment Report 41 USAID, JSI and GOSS • Reinstitute regular tripartite quarterly meetings. • Advocate and support resolution of EPI bottlenecks; support an EPI review with all partners. • Assure that all SHTP counties have a vaccine supply and schedule for vaccinating before and during the rainy season. (The vaccination schedule should be developed with County Medical Officers, where they exist.) • Consider individual agency and joint retreat(s) to clarify relationships and work plans. JSI • Establish focal person to serve as main point of contact for each INGO partner. Ensure focal person proactively communicates with INGO. • Assure that all messages developed by GOSS and UNICEF pertaining to the supply and distribution of EPI vaccines, EPI policy, etc. are clearly understood by the States, Counties, and INGOs where SHTP is working. • Ensure that GOSS, WHO, and UNICEF are familiar with the INGOs and the counties where SHTP is working, in particular, CARE in Twic East. Follow up with the INGOs that they have made needed contacts with GOSS, States, UNICEF and WHO. Milestones: By March 15: • JSI-Each NGO partner is assigned one point of contact. • USAID and Partners: Establish monthly meetings for GOSS, JSI and USAID; reinstitute quarterly Core group meetings; reinvigorate Quarterly Partner meetings. By April 15: • GOSS-State and County health officials informed of funding and programs available for their use. • USAID, GOSS and JSI-Develop plan for quarterly joint monitoring visits for the year. • GOSS and UNICEF-Develop plans for National EPI review TA support available on request from USAID through JSI. • Plan for individual agency and/or joint retreat(s). OTHER IMMEDIATE STEPS The Assessment team strongly recommends the following actions by May 15, 2008 to solve the high priority problems of drugs, NGO budget allocations, EPI and other technical areas. USAID • Follow up on the distribution drugs ordered through UNICEF to SHTP sites and find other location for storage of remaining drugs. • Re-assess needs for any additional drugs and LLITNs given the quantities of drugs now available from the UNICEF order and other sources. • Consider assistance at national level to address needs for a comprehensive national logistics system. (With MSH and coordinated with MDTF and other sources) USAID/Sudan: Sudan Health Transformation Project Assessment Report 42 JSI • Re-align/adjust follow-on grant budgets for the six county NGOs to support priority actions in the SHTP work plan, with attention to sufficient funds for: – Provider Training – CHD and VHC mentoring and support – Outreach and IEC – Sharing among the NGOs, and – Leveraging funds from other sources. • Ensure that SHTP NGOs are collaborating with county and state EPI personnel in the implementation of the EPI Plans developed at the February 2008 meeting. • Consider providing long-term TA at the National level for EPI Policy development and technical support to the GOSS EPI manager in collaboration with UNICEF and WHO. OTHER TECHNICAL AREAS • SHTP should work closely with the Department of Nutrition and EPI to consider how to track the provision of Vitamin A in routine services. For now, SHTP should assure that Vitamin A is available at its facilities during the National Immunization Days and report on Vitamin A coverage on a semi-annual or annual basis. • SHTP should start reporting quarterly on ORT and ARI treatment coverage for fewer than 5s, using the service statistics already available to develop the indicators/reporting. FOR MATERNAL HEALTH • SHTP should work in collaboration with other partners (MOH, UNFPA WHO) to reproduce and disseminate guidelines for implementation of the maternal health component of the national Maternal and RH policy. • SHTP should follow up with the MOH to determine the pros and cons of discontinuing MCH workers and TBA training given the circumstances of Southern Sudan and the potential time lag in replacing these auxiliary workers with more skilled cadres. • Use the next one year to review the PHCCs within the project areas and determine what it would take to upgrade these facilities to offer basic emergency obstetric care. • If possible, encourage the Capacity Project staff analysis currently under way to include an analysis of “certification and training” received by the current cadres who are delivering ANC at facility level. This will provide a basis for determining upgrading or pre-service training of new health workers. ANNEX I: SUDAN HEALTH TRANSFORMATION PROJECT (SHTP) PERFORMANCE ASSESSMENT POWERPOINT PRESENTATION USAID/Sudan: Sudan Health Transformation Project Assessment Report 43 USAID/Sudan: Sudan Health Transformation Project Assessment Report 44 USAID/Sudan: Sudan Health Transformation Project Assessment Report 45 USAID/Sudan: Sudan Health Transformation Project Assessment Report 46 USAID/Sudan: Sudan Health Transformation Project Assessment Report 47 USAID/Sudan: Sudan Health Transformation Project Assessment Report 48 USAID/Sudan: Sudan Health Transformation Project Assessment Report 49 USAID/Sudan: Sudan Health Transformation Project Assessment Report 50