USAID/TANZANIA: MOTHERS AND INFANTS SAFE HEALTHY ALIVE (MAISHA) MIDTERM EVALUATION November 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Lauren Mueenuddin, Rita Leavell, Siriel Massawe, and Jolene Mullins through the Global Health Technical Assistance Project. USAID/TANZANIA: MOTHERS AND INFANTS SAFE HEALTHY ALIVE (MAISHA) MID-TERM EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. MAISHA Midterm Evaluation Report i CONTENTS ACRONYM LIST .................................................................................................... ii EXECUTIVE SUMMARY ...................................................................................... 1 I. INTRODUCTION ............................................................................................... 5 II. MIDTERM EVALUATION METHODOLOGY ............................................... 6 III. BACKGROUND ............................................................................................... 7 IV. USAID ASSISTANCE TO DATE ................................................................... 9 V. MAISHA PROJECT OVERVIEW ................................................................... 10 VI. MAISHA'S SUB-GRANTEES ........................................................................ 13 VII. REVIEW OF TECHNICAL COMPONENTS ............................................. 15 VIII. PMTCT PLUS UP ........................................................................................ 30 IX. PROJECT MANAGEMENT .......................................................................... 32 X. MONITORING AND EVALUATION ........................................................... 36 XI. ACHIEVEMENT OF EXPECTED RESULTS .............................................. 39 XII. RECOMMENDATIONS FOR CHANGES IN MAISHA FOR NEXT TWO YEARS ................................................................................................................... 43 XII. RECOMMENDATIONS FOR FUTURE MNCH PROGRAMMING ........ 44 APPENDICES APPENDIX A. SCOPE OF WORK ................................................................... 45 APPENDIX B. CUMULATIVE TRAINING ..................................................... 60 APPENDIX C. SITE VISIT- SUPPLIES/EQUIPMENT………………..……….61 APPENDIX D. PERSONS CONTACTED ....................................................... 63 APPENDIX E. REFERENCES ............................................................................ 67 APPENDIX F. JHPIEGO ORGANIZATIONAL CHART ............................... 70 APPENDIX G. PROJECT MONITORING PLAN INDICATORS AND PROGRESS TO END OF YEAR 3 ...................................................................... 71 APPENDIX H. POTENTIAL INDICATOR CHANGES WITH REGARD TO SBM-R UNDER MAISHA .................................................................................... 78 APPENDIX I. ILLUSTRATIVE LIST OF ACTIVITIES TO BE SUPPORTED USING PMTCT PLUS UP FUNDING ............................................................... 81 APPENDIX J. INTERVIEW INSTRUMENTS ................................................. 84 ii ACRONYMS ACCESS Access to clinical and community maternal, neonatal, and women’s health services AMREF African Medical Research Foundation AMTSL Active Management of the Third Stage of Labor AMO Assistant Medical Officer ANC Ante-Natal care ARV Anti-retroviral BCC Behavior Change Communication BEmONC Basic Emergency Obstetric and Newborn Care CBDs Community-Based Distributors CCHP Council Comprehensive Health Plan CECAP Cervical Cancer Prevention CEmONC Comprehensive Emergency Obstetric and Newborn Care CHMT Council Health Management Teams CHW Community health worker COMMIT Communication and Malaria Initiative in Tanzania COP Chief of Party CQI Continuous Quality Improvement CS Child Survival CSSC Christian Social Services Commission CTC Care and Treatment Center CTS Clinical Training Skills DHMT District Health Management Team DHS Demographic and Health Survey DoD Department of Defense-United States DMO District Medical Officer DOT Direct Observed Therapy EGPAF Elizabeth Glaser Pediatric AIDS Foundation EH EngenderHealth ENC Essential Newborn Care EOP End of Project FANC Focused Antenatal Care FBO Faith-based organization FE Iron supplements fhi Family Health International (also fhi360) FP Family Planning GHI Global Health Initiative GH Tech Global Health Technical Assistance Project HB Hemoglobin HBLSS Home Based Life Saving Skills HMIS Health Management Information System HPN Health Population Nutrition HPO Health and Population Office HPV Human Papilloma Virus I/C In-Charge ICAP International Center for AIDS Care and Treatment IEC Information Education Communication IMA Interchurch Medical Assistance World Health IP Implementing Partner MAISHA Midterm Evaluation Report iii IPC Infection Prevention and Control IPTp Intermittent preventive treatment in pregnancy (for malaria) IRB International Review Board ITNs Insecticide Treated Nets JHU Johns Hopkins University JHUCCP Johns Hopkins University Center for Communication Programs JSI John Snow International KMC Kangaroo Mother Care LAM Lactational Amenorrhea Method LEEP Loop Electrosurgical Excision Procedure LRP Learning Resource Package LSS Life Saving Skills MAISHA Mothers and Infants: Safe, Healthy and Alive program MCH Maternal and Child Health MDGs Millennium Development Goals M&E Monitoring and Evaluation MgSO4 Magnesium Sulfate MH Maternal Health MIP Malaria in Pregnancy MMAM Mpango Wa Maendeleo Wa Afya Ya Msingi (meaning Primary Health Services Development Program) MMR Maternal Mortality Rate MNCH Maternal, Newborn and Child Health MoH Ministry of Health MoHSW Ministry of Health and Social Welfare-Tanzania Mainland MSD Medical Stores Department (of the MoHSW) MSH Management Sciences for Health MUCHS Muhimbili University College of Health Sciences MW Midwife NMCP National Malaria Control Program PEPFAR President’s Emergency Plan for AIDS Relief PHCI Primary Health Care Institute-Iringa PMI President’s Malaria Initiative PMP Project Management Plan PMTCT Prevention of Mother to Child Transmission (of HIV) PNC Post-Natal Care PPFP Post-Partum Family Planning PPH Post-Partum Hemorrhage PPIUCD Post Partum Intrauterine Contraceptive Device PSE Pre-Service Education QI Quality Improvement QoC Quality of Care RCH Reproductive and Child Health RCHS Reproductive and Child Health Section (of the MoHSW) RH Regional Hospital RHMT Regional Health Management Team RMO Regional Medical Officer RPO Regional Project Officer-MAISHA SBM-R Standards-Based Management and Recognition SC Save the Children SCMS Supply Chain Management System SIP Syphilis in Pregnancy iv SNL Saving Newborn Lives SP Sulfadoxine Pyrimethamine SVA Single Visit Approach SWAp Sector-Wide Approach T-MARC Tanzania Marketing and Communications for AIDS, Reproductive Health and Child Survival TA Technical Assistance TCCP Tanzania Capacity and Communication Project TIMS Training Information Management System TOT Training of Trainers UNICEF United Nations Children’s Fund URT United Republic of Tanzania USAID United States Agency for International Development USG United States Government VIA Visual Inspection with Acetic Acid WHO World Health Organization WRA White Ribbon Alliance WRA-TZ White Ribbon Alliance-Tanzania ZHRC Zonal Health Resource Center ZNZ Zanzibar ZRCHS Zanzibar Reproductive and Child Health Section 1 EXECUTIVE SUMMARY The following are the findings from the Mid-Term Evaluation of the MAISHA (Mothers and Infants, Safe, Healthy, and Alive) Project conducted during October and November 2011. PROJECT OVERVIEW USAID/Tanzania developed the MAISHA project in collaboration with the MoHSW to support Tanzania’s and Zanzibar’s efforts to reduce maternal and newborn morbidity and mortality. The primary focus of the project is on the provision of BEmONC, identified as a major need in Tanzania’s Road Map Strategy Plan to Accelerate Reduction of Maternal, Newborn and Child Death. The project is funded under a cooperative agreement (ACCESS program associate award) from October 2008 through September 2013 for $40 million. Funding is primarily from MNCH ($25 million), including Child Survival (CS) and the President’s Malaria Initiative (PMI) funds. However, the President’s Emergency Plan for AIDS Relief (PEPFAR) funds a significant portion ($15 million) of the project as well. The project is implemented by Jhpiego and its collaborating sub-grantees: WRA-TZ/ Futures, IMA World Health, Save the Children, T-MARC Company, which was replaced by the TCCP in PY3. The first three years of MAISHA have been a gradual process of increasing project complexity, from the core components of FANC, BEmONC and QI to the addition of activities primarily through transitions from other projects. During the first year of MAISHA, Jhpiego staff continued to expand the FANC program to Zanzibar and other regions with ACCESS funding. In late FY2009, FANC, BEmONC and QI activities were conducted under MAISHA with PMI and Child Survival (CS) funding. In April 2010, the Infection Prevention, the Pre-service Nursing/Midwifery and the Pre-service Medical Education components transitioned to MAISHA with PEPFAR funding. This was followed in October 2010 by the cervical cancer (CECAP) and the Integrated PMTCT facility/community programs, originally with the global project ACCESS/FP. For FY2011 (year 3 of the project), MAISHA was provided funding and given the tasks of pre￾service training for Assistant Medical Officers (AMOs), assisting with a national Community Health Worker (CHW) program, PMI pre-service training, a placental parasitemia study in Zanzibar and establishing the foundation for a national Post Partum IUCD (PPIUCD) program (converting to family planning preservice education integration in FY 2012). Finally in FY2010, additional PEPFAR resources were made available to PMTCT implementing partners (IP) as well as MAISHA for increased scale-up of BEmONC, FANC, PNC and cervical cancer prevention (CECAP) activities. TECHNICAL COMPONENTS According to the project proposal, the MAISHA conceptual framework is designed to: • Improve the policy environment for FANC, BEmONC, KMC and PMTCT through advocacy • Improve skills of providers for FANC, BEmONC, KMC and PMTCT through in-service and pre-service training, supervision and quality improvement • Improve the availability of equipment and supplies for FANC, BEmONC, KMC and PMTCT • Increase demand for quality services through behavior change communication and community mobilization 2 Overall, MAISHA’s conceptual framework is solid and in line with international recommendations on key interventions to reduce maternal and newborn mortality. The MAISHA Project has aligned its interventions within the National Road Map Strategic Plan to Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 – 2015 and has developed substantial collaboration and buy-in by the MoHSW, which is vital for the overall sustainability of the interventions after the end of the project. The MAISHA Project has been a leader in supporting national, regional and district-based advocacy efforts to place Maternal and Newborn Health as a priority health intervention. The evidence-based training packages developed by MAISHA for FANC, BEmONC, CECAP and PNC, are an important contribution of the MAISHA project for improving the quality of Maternal and newborn care in Tanzania. These technical resources, combined with the SMB-R Quality Improvement methodology are an effective and powerful combination of resources for strengthening the quality of maternal and newborn care services in MAISHA-supported facilities. The gains made in improving provider skills however, need to be sustained through periodic updates for staff in key technical areas, and through a system of on-site mentoring and coaching which will support staff in maintaining their skills. The SBM-R Quality Improvement methodology introduced by the MAISHA project in facilities and within Regional and District Health Management Teams has been used to great effect, allowing staff and managers to focus on key elements of ante-natal care, labor and delivery and newborn care, and to work towards the improvement in the quality through an internal assessment and scoring mechanism. The facility-based assessment tool of SBM-R is an important innovation, which allows staff to improve planning, prioritize areas of need and advocate for changes in policy and budget. The process allows staff to take ownership for the process of quality improvement by diagnosing challenges in service delivery, implementing changes and monitoring their success in meeting higher standards of care. The SBM-R is however very complex and time-consuming and currently requires very heavy support by the MAISHA program team. In order for the SBM-R tool to be scaled-up for use in other facilities throughout the country, the process needs to be stream-lined and simplified and integrated within the current MoHSW quality improvement and supportive supervision framework. Overall MAISHA has been successful in identifying and supplying critical MNCH equipment for hospitals, health centers /dispensaries, midwifery/nursing schools, and training Institutes. However, the overall issue of drug stock-outs and shortages for key ANC and BEmONC services (such as SP and uterotonics) is a major problem in the ensuring the quality of MNCH care in the long-run. This is a national health systems issue, but USAID/HPO should bring all its resources to bear in helping solve this critical problem by leveraging other USAID funded initiatives such as JSI’s DELIVER and MSH’s SCMS. As of the mid-term evaluation, MAISHA has not yet achieved planned results in increasing demand for quality services through behavior change communication and community mobilization due partially to the delayed performance by one of the sub-grantees (T-MARC) and the desire of USAID to house all its IEC/BCC work with the Tanzania Capacity and Communication Program (T-CCP). The communication and community mobilization component is critical for expanding the project reach beyond the interventions in health facilities and will help address the need to scale-up services and information for the many women and newborns who do not avail themselves of health services at the time of labor and delivery, and the post￾natal period. 3 In order to have national impact in reducing maternal and newborn mortality, the network of BEmONC-trained providers needs to be expanded beyond MAISHA’s current project strategy of supporting two facilities per district. This is a huge undertaking and beyond the scope of the current MAISHA project and beyond current MCH funding levels. Ideally, the model would be taken up and funded by district budgets. However, given the recently announced moratorium on training by districts, the MAISHA project and USAID will need to spend more time thinking about and developing alternative mechanisms for rolling out trainings, such as the proposed plan of working with PMTCT implementing partners and building upon existing platforms to roll out the BEmONC training packages and other technical resources developed for CECAP, IPC, and PNC. RECOMMENDATIONS FOR CHANGES IN MAISHA FOR NEXT TWO YEARS The MAISHA project should continue its strong leadership as an advocate for greater investments in MNCH nationwide, and continue to work with the Ministry in finding solutions for health systems problems such as drug supply and health worker resources shortages through other funding mechanisms such as DELIVER and SCMS. Its work on integrating changes to medical curricula and strengthening medical and nursing/midwifery pre-service and in-service programs are important initiatives and should be vigorously pursued in the last two years of the project. In terms of project implementation, MAISHA should work on consolidating the changes and improvements in maternal and newborn heath service delivery at the regional/district management and health facility levels over the next two years of the project. The project has introduced many new initiatives through its training programs and these need close monitoring and support in order for these improvements to be sustained. The introduction of the Integrated Facility/Community Model is a very important initiative as it seeks to strengthen services in the critical post natal period both for newborns and mothers but also for ensuring strong PMTCT linkages in the post natal period. The development of a community health worker component is a critically important intervention, without which the project will not reach the many women who currently do not avail themselves of facility-based health care services for ANC, labor and delivery and for post-natal care services, including post￾partum family planning. The post natal care training package and services need to be amplified and consolidated both in other regional hospitals, facilities and the community. The project needs to find mechanisms to strengthen on-going support and supervision of staff after initial training at the district and health facility-level. The introduction and strengthening of clinical mentoring on site for health care providers would be in important step in this direction. In order to ensure the very important work introduced with the QI methodology as a way of assessing, monitoring and ultimately improving the quality of care of maternal and newborn services, the project needs to work closely with the Ministry to integrate the principles and tools of the SBM-R into existing Ministry QI methodologies. In terms of MAISHA’s plan for use of PMTCT Plus funds to roll out the BEmONC, PNC and CECAP training packages and tools, better planning and clearer guidance for partners is required if this is to be the primary mechanisms for rolling out BEmONC, PNC, CECAP and other services to achieve national scale. Overall, better collaboration with other MNCH partners would create important efficiencies and would facilitate roll-out for services to other regions. 4 FUTURE OF MNH PROGRAMMING USAID/Tanzania, for a variety of reasons including funding resource constraints and a desire to achieve national coverage in all regions, decided to limit MAISHA’s interventions to regional hospitals and to 2-3 health facilities per district. Given the vast number of outlying facilities in any given district that have not benefited from MAISHA interventions, and the large number of women who currently do not access health care facilities for labor and delivery, the current project is unlikely to achieve impact to affect national mortality rates for mothers and newborns. That said, the model developed by MAISHA, ( i.e. training of key staff in BEmONC, PNC, IPC, CECAP and Kangaroo Mother Care, support for key supplies and equipment at facilities, and the introduction of QI methodologies that empower both heath facility staff and regional/district health management teams to improve the quality of care), should be replicated in future programming. The mechanisms to do this need to be closely analyzed. Ideally, the model developed under MAISHA would be taken up by district HMTs, financed by district budgets and rolled out in all health facilities. However, given constraints on resources of districts, a more realistic mechanism may be to build upon the strong PMTCT platforms and services provided by a large number of PMTCT partners working at the regional level. If this is to be effective, then greater time and efforts need to be put into guiding, facilitating and monitoring this process under strong technical leadership. Given that 60% of maternal deaths and 75% of neonatal death take place within the first week post-partum1, future programming needs to place a strong emphasis, not just on emergency care during labor and delivery, but also on the post-natal period by strengthening the PNC services. MAISHA is in the process of piloting the Integrated Facility/Community PNC package; results from this program need to be closely analyzed for replication and scale-up. Future programming also needs to focus on strengthening Post-Partum Family Planning services (PPFP), given that 42% of Tanzanian women report an unmet need for family planning during their first year post-partum2. Most importantly, future programming should address the need for better outreach and services for mothers and newborns at the community level. The MOHSW needs to commit to the idea of developing a national Community Health Worker cadre, and needs to develop a strong program which combines community mobilization activities and community-based behavior change communication strategies. Linkages between up-graded health facilities and communities need to be strengthened by focusing on improving referral mechanisms for obstetric emergencies, developing emergency transport systems, and introducing community saving schemes for emergencies. Finally, the community based program for MNCH needs to be supported by a healthy, vibrant and energetic national communication campaign for MNCH. 1 Tanzania DHS 2004-2005) 2 Borda 2007 5 I. INTRODUCTION The USAID/Tanzania Health & Population Office (HPO) has called for a mid-term evaluation of the Mothers and Infants Safe Healthy and Alive (MAISHA) project, which constitutes a major part of the Mission’s Maternal and Newborn and Child Health (MNCH) program. The project aims to reduce the leading causes of maternal and neonatal mortality and morbidity in mainland Tanzania and Zanzibar. The evaluation was conducted during October and November 2011 on-site in Tanzania. The goals of the mid-term evaluation were to review the overall strategy employed by the project, comment on the project’s achievements of targets and objectives as of Project Year (PY) 3, and provide recommendations to guide program planning for the remainder of the MAISHA program (through September 2013), as well as make recommendations for potential programming options for the future. The audience for this mid-term evaluation is the USAID/Tanzania Health & Population Office (HPO), USAID/Tanzania HIV/AIDS team, and the MAISHA project management team. 6 II. MIDTERM EVALUATION METHODOLOGY The Scope of Work (Appendix A) called for a mid-term evaluation of the activities funded by the President’s Malaria Initiative (PMI), Maternal and Child Health (MCH) and PEPFAR monies and implemented by MAISHA. Based on the findings, the evaluation team was tasked to assess the current progress towards meeting MAISHA goals and results, assess the effectiveness of MAISHA interventions and recommend modifications and improvements in the remaining years of the project and future direction. A team of four consultants was engaged to conduct the evaluation: two MCH experts (one international and one local); one international Project Management and M&E expert; and one local Health System Strengthening and Capacity Building expert. Prior to arrival in-country in Tanzania, the evaluation team reviewed relevant national documents including Ministry of Health and Social Welfare (MoHSW) strategic documents, national demographic health surveys, MoHSW policies, and guidelines supplied by the USAID HPO. The team also reviewed a broad set of MAISHA program documents, including the original MAISHA Cooperative Agreement, MAISHA annual work plans, the Project Monitoring Plan (PMP), annual and quarterly reports, as well as technical reports, training curricula and technical guidelines for programmatic components. The evaluation team also reviewed project monitoring data, QI data, sentinel site health facility service delivery data, and results from the baseline Quality of Care study. Upon arrival in Tanzania, the evaluation team was briefed over three days by the MAISHA Chief of Party and key MAISHA staff (senior staff in M&E, QI, and Training). MAISHA sub-grantees (White Ribbon Alliance-Tanzania [WRA-TZ], Save the Children, Interfaith Medical Alliance [IMA] World Health, and the Tanzania Capacity and Communication Project [TCCP]) also presented their achievements to the evaluation team. The team held in-briefing meetings with USAID HPO including members from the HIV/AIDS and Malaria teams, and met with the Head of the Safe Motherhood Unit, one of the key MoHSW units collaborating with the MAISHA project. Finally, the team met with WHO, and several of MAISHA’s Prevention of Maternal to Child Transmission (PMTCT) Plus Up Partners. Over the course of three weeks, the team conducted site visits in five regions including Pemba and Unguja (Zanzibar), Iringa, Morogoro, and Mtwara. These sites were selected to allow the team to see the two key MAISHA program interventions in action: the Focused Ante Natal Care (FANC) and Basic Emergency Obstetric and Newborn (BEmONC) service delivery components in regional and district hospitals and selected health centers/dispensaries, and the accompanying support to quality improvement efforts. Sites were also selected to allow the team members to observe the Cervical Cancer Program (CECAP), Kangaroo Mother Care (KMC), Infection Prevention Control (IPC), Post-natal Care (PNC) and community interventions in operation in the field. The team developed detailed questionnaires for each MAISHA stakeholder to be interviewed, including members of Regional Health Management Teams (RHMT), District Council Health Management Teams (DCHMT), clinical staff trained in Focused Ante Natal Care (FANC) and Basic Emergency Obstetric and Newborn Care (BEmONC) from MAISHA-supported health facilities. Questionnaires were also developed for regional training institute staff, and midwifery/nursing school faculty (See Appendix J). Upon completion of the site visits, the evaluation team conducted formal debriefings with the MoHSW/ Reproductive and Child Health Section, the USAID/HPO team and MAISHA senior staff to discuss initial findings and discuss draft recommendations. 7 BACKGROUND The United Republic of Tanzania has an estimated population of 44 million people of which almost 75% of live in rural areas. Tanzania’s life expectancy at birth is 52 years, and the population is primarily young (21% of women and 26% of men are between 15-19 years of age). 3 Although there has been an overall downward trend in fertility in Tanzania (5.4 children per woman), at current growth rates, Tanzania’s population will exceed 50 million by 2025. Data from the Demographic and Health Survey (DHS) in Tanzania (2009/10) shows that 27% of women of reproductive age use modern contraception and 25% of married women have an unmet need for family planning. Consistent with the health Millennium Development Goals (MDGs) Tanzania‘s national development priorities address public health and health care challenges. Tanzania faces a generalized HIV epidemic on the mainland, a concentrated HIV/AIDS epidemic in Zanzibar and other widespread communicable diseases such as tuberculosis (TB), malaria, respiratory infections and diarrheal diseases. Malaria is the leading cause of death for children in Tanzania and is a major indirect cause of maternal mortality. Complications from pregnancy and childbirth are major causes of death for women of reproductive age and newborns in Tanzania. Despite high coverage rates of antenatal care (96% attend at least once), and approximately 50% of women delivering in a health facility, the maternal mortality remains unacceptably high at 454 deaths per 100,000 live births (accounting for approximately 8,000 maternal deaths per year). The major causes of maternal mortality include: hemorrhage, sepsis, hypertensive disorders of pregnancy, obstructed labor, and abortion complications. Indirect causes include malaria, anemia, HIV and cardiovascular diseases4. For every woman that dies during pregnancy and delivery, 30 others are likely to have health complications during childbirth. Over 60 percent of all women stated that they face major barriers to accessing health care when they are sick (DHS 2004/5). The most common barriers to accessing care were: getting money for treatment (40 percent); the distance to the health facility (38 percent); having to take transportation (37 percent); and not wanting to go alone (24 percent). Women with little or no education and women from poorer households had greater barriers to overcome in order to access care. Every year at least 51,000 Tanzanian newborns die and 43,000 babies are stillborn. Up to two￾thirds (34,000) of newborn lives could be saved if essential care reached mothers and babies during pregnancy, delivery and the postpartum period. Although under-five mortality rates have dropped by 40 percent (from 137 deaths per 1,000 births in the mid-1990s to only 81 for the period 2006-105), the TDHS 2010 estimates neonatal mortality rates at 26 per 1,000 live births, and account for 47% of the infant mortality rate.6 During antenatal visits, lifesaving interventions are provided, including addressing malaria and syphilis in pregnancy. The success of malaria prevention programs focusing on pregnant mothers during the antenatal period can be seen by the recent increase in bed net ownership and use by pregnant women. The percentage of pregnant women who slept under an insecticide treated net (ITN) the night before the DHS survey increased from 18 percent in 2004-05 to 25 percent in 2007-08 and to 68 percent in 2010. Unfortunately the provision of Intermittent Preventive Treatment in Pregnancy (IPTp) has 3 Tanzania Demographic and Health Survey, 2009/10. 4 The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 - 2015 5 Preliminary results from Demographic and Health Survey, 2009/10. 6 The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 - 2015 8 been less successful with essentially no change from 2004/05. Currently, only 27% of pregnant women receive IPTp2. Reduction of maternal, newborn and child deaths is a priority for the United Republic of Tanzania, as reflected by the national MDG targets, Vision 2025, the National Strategy for Growth and Reduction of Poverty (NSGRP-MKUKUTA), and HSSP (2009-2015). The National Road Map Strategic Plan to Accelerate Reduction of Maternal, Newborn and Child Deaths lays out strategies to guide implementation on all MNC interventions in Tanzania under the MoHSW. The goal is to accelerate the reduction of maternal newborn and childhood mortality in line with MDGs 4&5. The target for the National Strategic Plan is to reduce maternal mortality from 578/100,000 live births to193/100,000 and the neonatal mortality from 32/100 to 19/1000 live births. Critical challenges in reducing maternal and newborn mortality and morbidity include inadequate implementation of existing strategies, weak health infrastructure, limited access to quality Emergency Obstetric and Newborn Care (EmONC) services, inadequate human resources, lack of equipment and supplies, low utilization of modern family planning services, and inadequate monitoring and evaluation of health services. A national survey showed that only 65% of the hospitals provided Comprehensive Emergency Obstetric and Newborn Care (CEmONC) and only 6% of HC/dispensaries performed all the signal functions for BEmONC. Availability and access to quality EmONC services are therefore key interventions in addressing the high Maternal Mortality Ratio (MMR) and Newborn Mortality Rate (NMR)7 7 MoHSW Assessment of Emergency Obstetric and Newborn Care (EmONC) equipment and training needs in Tanzania Mainland (2010) 9 III. USAID ASSISTANCE TO DATE USAID supports national efforts to prevent the spread of HIV/AIDS and mitigate its impact; combat malaria; and increase the use of reproductive and child health services (RCHS). By the end of 2009, U.S. assistance had provided HIV prevention services to over 5 million Tanzanians; treatment to more than 200,000 HIV+ individuals; and care for 750,000 people living with or affected by HIV/AIDS, including 330,000 orphans and vulnerable children. In March 2010, the United States and Tanzania signed a Partnership Framework to scale up prevention efforts while maintaining support for care and treatment. USAID is also a key partner supporting national malaria prevention and case management strategies through education, bednet distribution, indoor spraying, and strengthening of the health care system, especially for children and pregnant mothers. With USAID support, Zanzibar has essentially halted malaria transmission, with prevalence at less than 1 percent, and shifted its focus to sustainability. On the mainland, malaria deaths have dropped by half, from 120,000 in 2005 to 60,000 in 2008, but prevalence8 remains high. USAID has played a role in reducing infant mortality by 32 percent since 1999, but the rate remains high (51 deaths per 1,000 live births), as do the rates for maternal mortality and fertility (454 per 100,000 and 5.4 children per woman). Over the next five years, USAID will focus on working to control malaria, Prevention of Mother to Child Transmission (PMTCT) of HIV, provide 6-8 million children with life-saving nutritional supplements, train health workers, improve maternal health care, and scale up family planning services. In addition, the continued goal of reducing maternal and newborn mortality depends upon identifying and improving those services that are critical to the health of Tanzanian women and girls, including focused antenatal care (FANC), basic emergency obstetric and newborn care (BEmONC), postpartum care for mothers and newborns, and access to PMTCT. With this goal in mind, USAID-Tanzania has identified the improvement of maternal and newborn health and the reduction of maternal and newborn mortality as key concerns of the MoHSW and Sector Wide Approach (SWAp) partners. As a Global Health Initiative (GHI) Round Two country, Tanzania’s USG-supported MNCH interventions include pre- and in-service training for BEmONC, the improvement of service quality through provider supervision and mentoring, equipment and supplies procurement, advocacy for national policies to support emergency obstetric care at lower-level facilities, and supporting community-level communications.9 8 Tanzania DHS 2010 9 Tanzania Global Health Initiative Strategy 2010-2015 10 IV. MAISHA PROJECT OVERVIEW PROGRAM DESIGN AND FUNDING USAID/Tanzania developed the MAISHA project in collaboration with the MoHSW to support Tanzania’s and Zanzibar’s efforts to reduce maternal and newborn morbidity and mortality. The primary focus of the project is on the provision of BEmONC, identified as a major need in Tanzania’s Road Map Strategy Plan to Accelerate Reduction of Maternal, Newborn and Child Death. The project is funded under a cooperative agreement (ACCESS program associate award) from October 2008 through September 2013 for $40 million. Funding is primarily from MNCH ($25 million), including Child Survival (CS) and the President’s Malaria Initiative (PMI) funds. However, the President’s Emergency Plan for AIDS Relief (PEPFAR) funds a significant portion ($15 million) of the project as well. The project is implemented by Jhpiego and its collaborating sub-grantees: WRA-TZ/ Futures, IMA World Health, Save the Children, T-MARC Company, which was replaced by the TCCP in PY3. The MAISHA program has been designed to deliver evidence-based health interventions on a national scale, contributing to the achievement of the national targets for Millennium Development Goals (MDGs) Four and Five.10 The program addresses the following objectives:  Reduction of maternal mortality due to major direct causes of mortality;  Reduction of newborn mortality due to infection, hypothermia and asphyxia through immediate newborn care;  Reduction of low birth weight, stillbirth and newborn mortality due to malaria and congenital syphilis; and  Reduction of transmission of HIV infection from mother to child and increase of HIV free survival. The MAISHA program overlapped with Jhpiego’s earlier project, ACCESS (FY2004 to FY2009). ACCESS provided national guidelines and training in FANC, with particular focus on malaria and syphilis in pregnancy. ACCESS also developed a tool for quality improvement, incorporated FANC into pre-service nursing/midwifery curricula and initiated a program for Infection Prevention. Under MAISHA, the FANC activities have continued to roll out on a national level, but the major program focus changed to strengthening of BEmONC. Essential Newborn Care (ENC) also includes a special focus on low birth weight babies, using the Kangaroo Mother Care (KMC) principles. With PEPFAR funding, the MAISHA program has also been tasked with strengthening service delivery in PMTCT of HIV/AIDS to address gaps in the integration of MNCH services for HIV positive women and children. PEPFAR funding has expanded into a wide range of activities including cervical cancer screening and treatment, an integrated PMTCT postnatal/community model package, Infection Control Prevention, and pre-service training. 10 MDG 4 – reduce neonatal mortality from 32 to 19 per 1,000 live births (mainland) and from 29 to 23 per 1,000 live births (Zanzibar); MDG 5 – reduce maternal mortality from 578 to 193 per 100,000 live births (mainland) and from 377 to 251 per 100,000 live births (Zanzibar). 11 PROGRAM STRATEGY According to the project proposal, the MAISHA conceptual framework is designed to: • Improve the policy environment for FANC, BEmONC, KMC and PMTCT through advocacy • Improve skills of providers for FANC, BEmONC, KMC and PMTCT through in-service and pre-service training, supervision and quality improvement • Improve the availability of equipment and supplies for FANC, BEmONC, KMC and PMTCT • Increase demand for quality services through behavior change communication and community mobilization In practice, the conceptual framework for the major service delivery components of the program (provider skills and key equipment/supplies) has been translated to the following model: • Support development of national resources (guidelines, learning packages) • Strengthen the regional hospital as a BEmONC training site in each region, and as a site for KMC • Strengthen 2-3 health centers /dispensaries per district in each region for BEmONC/FANC service delivery, with training, critical equipment and supplies and Quality Improvement (QI). The QI element of the program has incorporated a revised version of Jhpiego’s Standards-Based Management and Recognition (SBM-R) tool, addressed primarily to regional hospital and health center level staff. A supportive and facilitative supervision effort has been directed to regional and district supervisory staff, as well as to supervisory staff in the selected health facilities. The project plans to achieve national scale by rolling out the above program model to all 21 regions and districts in mainland Tanzania and all districts of Zanzibar. The project has not been designed with the goal of achieving direct national impact on MDG’s 4 and 5, but rather to develop the model, the materials, and the trainers that can be used by districts and other donors to roll out FANC and BEmONC activities to other facilities. MAISHA PROGRAM IMPLEMENTATION AND REVISIONS The MAISHA program was awarded in October 2008. However, most funded activities under MAISHA did not begin until the first annual workplan was revised and approved in June 2009. Therefore at the time of the mid-term evaluation (October-November 2011), the core activities of the program - BEmONC, QI and supportive supervision - have been active for about 2.5 years. Technical Components The first three years of MAISHA have been a gradual process of increasing project complexity, from the core components of FANC, BEmONC and QI to the addition of activities primarily through transitions from other projects. During the first year of MAISHA, Jhpiego staff continued to expand the FANC program to Zanzibar and other regions with ACCESS funding. In late FY2009, FANC, BEmONC and QI activities were conducted under MAISHA with PMI and Child Survival (CS) funding. In April 2010, the Infection Prevention, the Pre-service Nursing/Midwifery and the Pre-service Medical Education components transitioned to MAISHA with PEPFAR funding. This was followed in October 2010 by the cervical cancer (CECAP) and 12 the Integrated PMTCT facility/community programs, originally with the global project ACCESS/FP. For FY2011 (year 3 of the project), MAISHA was provided funding and given the tasks of pre￾service training for Assistant Medical Officers (AMOs), assisting with a national Community Health Worker (CHW) program, PMI pre-service training, a placental parasitemia study in Zanzibar and establishing the foundation for a national Post Partum IUCD (PPIUCD) program (converting to family planning preservice education integration in FY 2012). Of these, only the parasitemia study is currently underway and some initial PPIUCD activities have taken place. Finally in FY2010, additional PEPFAR resources were made available to PMTCT implementing partners (IPs) as well as to MAISHA for increased scale-up of BEmONC, FANC, PNC and cervical cancer prevention (CECAP) activities. In these, MAISHA was asked to play a technical assistance and coordination role to the implementing partners (who are already established through PMTCT programs at facilities across the country), rather than implementing with “Plus Up” funds. MAISHA gave a presentation on FANC and BEmONC to the PMTCT partners in December 2010. However, most PMTCT partners to date have used the “Plus Up” funding for other activities, with a few exceptions. MAISHA has provided technical support to ICAP to utilize these “Plus Up” resources for BEmONC site assessments and training in the Kagera region and to ICAP and Grounds for Health for CECAP in the Kigoma region. Discussions with EngenderHealth/Iringa initiated in September 2011 will lead to rollout of BEmONC training in selected health facilities early 2012, with MAISHA technical assistance. Geographical Coverage The ambitious plan to cover all 21 regions and Zanzibar by the end of year 3 (FY2011) was revised to reflect the difficulty of rolling out in all districts. To “cover a region” requires a lengthy process of advocacy at the regional and each district level, selection with the district team of 2-3 facilities per district, training of regional and district administrative staff in supportive supervision, training for1-2 providers/facility in BEmONC and QI and a baseline assessment and provision of critical equipment and supplies for each facility. Consolidation and management of MAISHA then requires a minimum of quarterly visits to each facility to coach and mentor, conduct QI assessments and follow up, together with the district and the regional teams. The Year 3 FY2011 workplan revised the geographic rollout schedule in order to consolidate activities in Zanzibar and 16 regions. The final 5 regions were postponed from Year 3 to Year 4 and MAISHA has recently initiated the advocacy process in 3 of these regions. The original MAISHA proposal was to manage the project with 9 MNH Zonal Program Officers, but it was then determined that a Regional Program Officer was needed in each region. This was to ensure that the rollout process moved in a timely manner with sufficient level of quality and oversight and that each region had adequate coaching and followup. The shortage of skilled candidates for these positions has slowed the regional rollout, as it is logistically more difficult to manage from Dar Es Salaam. 13 V. MAISHA’S SUB-GRANTEES Most of MAISHA’s core activities have been implemented by Jhpiego directly; these activities have focused on the components for improved service delivery (provider skills, QI and equipment/supplies) and project management. Sub-grantees have been assigned specific additional tasks in the cooperative agreement, related to advocacy, establishment of KMC Units and the development of behavior change communication (BCC) materials and information, education and communication (IEC) materials. White Ribbon Alliance Tanzania (WRA-TZ/Futures) has been tasked with advocating for greater awareness and action on maternal health at the national and regional/district levels, and with work on community mobilization including the establishment of transportation networks for women in need. To this end, WRA-TZ assisted in the design of a government template for MNCH budgeting at the district level and has successfully advocated for the creation of the Parliamentary Group for Safe Motherhood, and for the annual celebration of White Ribbon Day. Through collaboration with other USAID-funded NGOs, WRA-TZ has supported the national roll out of Home-based Life Saving Skills (HBLSS) including training of 25 master trainers from Muhimbili Medical School, WRA-TZ, WellShare International, CARE International and PathFinder, Arusha Regional Health Management Team and Karatu District Council Health Management Team. Each master trainer/organization was tasked with expanding HBLSS training at the community level in the regions of Arusha, Rukwa and Iringa. The creation of the White Ribbon Alliance Core Committee has garnered support for the voluntary position of regional advocacy focal persons in 10 MAISHA regions. Through ACCESS and MAISHA, Jhpiego has funded WRA for over 7 years. Although WRA-TZ will remain a partner with MAISHA through December 2011, the Jhpiego team feels with advocacy mechanisms firmly in place at the national and regional level, it is now time for WRA￾TZ to become a true alliance with support provided by partnering agencies. As part of advocacy efforts, Interfaith Medical Alliance (IMA) World Health was commissioned to develop a methodology to garner support from faith based organizations (FBO) and their constituents for improved maternal and newborn health. During their two years as a MAISHA partner, IMA World Health has held advocacy workshops for religious leaders and facilitated coordination meetings between District Council Health Management Teams and faith-based health providers. Following a long process, IMA World Health has finalized a sermon guide which they have provided to Muslim and Christian religious leaders. During the process of adapting and translating the sermon guide, IMA also designed a monitoring and evaluation tool for training of religious leaders in the guide and to follow-up on community impact of the recommended sermons. In year 4, IMA implemented an active handover process of tools and training to the interfaith forum of religious leaders. As of the mid-term evaluation, due to excessive delays in implementing project activities, MAISHA will be terminating its cooperative agreement with IMA World Health. The sub-grantee T-MARC Company was originally tasked with the creation of a multi-media Behavior Change Communication (BCC) campaign and Information, Education and Communications (IEC) materials. The T-MARC Company did not present to the evaluation team but it was reported that they provided minimal input into the creation of job aids for project-trained health providers as originally planned. The MAISHA project has decided to end its collaboration with T-MARC and to partner instead for the remainder of the project with the Tanzania Capacity and Communication (TCCP), currently USAID’s lead implementing partner for all its BCC programs in Tanzania. This decision is in line with USAID/Tanzania’s 14 consolidation of communication programs, which is intended to increase the efficiency of these programs through integration. To date, TCCP has created BEmONC job aids and reviewed and adapted FANC job aids for a community outreach setting. A multi-media safe motherhood campaign “Mama Nipende” is currently under development with the goal of designing integrated health messages for greater impact. TCCP has conducted a baseline household survey (KPC) while piloting birth plan brochures/posters in MAISHA sites. They are also piloting BEmONC job aids in Morogoro Region and a checklist for birth planning. MAISHA partner Save the Children (SC) has primarily been responsible for the training and setup of Kangaroo Mother Care (KMC) Units in MAISHA’s planned 22 regional hospitals. Save the Children has successfully created 17 KMC centers in regional hospitals supported by MAISHA with two additional centers planned for future regional roll out. SC will continue to complete its planned activities over the next two years of the project. 15 VI. REVIEW OF TECHNICAL COMPONENTS FOCUSED ANTENATAL CARE (FANC) Component Summary Training in Focused Ante-Natal Care (FANC) was initiated in 2002 under the USAID-supported Maternal and NewBorn Health project. The project, managed by Jhpiego, developed a national cadre of FANC trainers and strengthened capacity in zonal training centers in Arusha and Iringa for FANC training in 3 regions. These activities were then transitioned and expanded under the ACCESS project (FY2004-2009), again implemented by Jhpiego, using PMI and Child Survival (CS) funding. The updated FANC approach specifically emphasized prevention of malaria in pregnancy (MIP) through use of IPTp and counseling on insecticide treated nets (ITNs). It also placed attention on the need for testing and treatment for syphilis in pregnancy (SIP). The ACCESS project left a number of achievements, including development of FANC national guidelines, training materials and FANC trainers at the district level. ACCESS also developed a tool for quality improvement and data recording, addressed the link between ANC and PMTCT, integrated FANC into pre-service nursing/midwifery curricula and initiated a program for Infection Prevention. Finally, ACCESS conducted FANC training for approximately 2,971 providers (49.5% of estimated 6,000 providers) in 1448 facilities (31% of 4,745 total). Under MAISHA, the FANC activities have continued to roll out on a national level during the first 3 years of the project. This has included both direct training with MAISHA funding and advocacy for districts to use their own or other donor partner funding to conduct local training. MAISHA also worked with the WHO, other technical leaders in the Safe Motherhood working group and the MoHSW to finalize national FANC guidelines and a learning resource package. During the process of rollout of the BEmONC component in the regions, MAISHA conducted facility needs assessments and provided equipment and supplies for ANC clinics. The MAISHA and district oversight of selected facilities for BEmONC has also included periodic QI assessments and supervision visits for the ANC clinics. Findings from Site Visits The MAISHA mid-term evaluation team visited 16 facilities in Zanzibar and 3 regions of mainland Tanzania. Fifteen of the sites had Reproductive and Child Health (RCH) clinics, which included one or two rooms for ANC visits. (Morogoro Regional Hospital had recently shifted all routine ANC visits to an urban health center and now only conducts ANC clinics weekly for high-risk referrals.) This spot check of facilities does not substitute for the more rigorous baseline survey of Quality of Maternal and Newborn Health Services in Tanzania: Findings on Antenatal Care July 2010. During that survey, interviewers were able to observe patient counseling and exams and other required FANC interventions, thus providing a better measure of quality of care. However, the field visits did give an indication of how the training, the QI assessments, the supervision visits and the health system situation are reflected in the average MAISHA-assisted facility. In all sites, the evaluation team was able to identify one or more FANC trainees or FANC trainers, and many staff reported FANC updates on the job. Clinics were generally busy, but orderly and clean. IEC materials and job aids were posted on the walls, patient examination tables had privacy screens, and hand washing facilities (often soap and a water container) were available, as well as drinking water for direct observed therapy (DOT) of sulfadoxine pyrimethamine (SP) and mebendazole. The ANC room or another room was available for confidential counseling and testing for HIV. FANC guidelines or checklists were available in the 16 room or nearby. Blood pressure cuffs and weighing machines were generally available, but not always in good condition due to continuous use. The team observed that most ANC staff could list various requirements in FANC policies and guidelines, demonstrate their prepared emergency kit, and often rapidly recited dosing and timing for FANC drugs and anti-retrovirals (ARVs) for HIV+ clients, as well as detail signs of pre-eclampsia. When asked about the FANC guidelines, many stated that they liked the step-by-step process of the checklist so that they could follow the protocol. They seemed less familiar with the FANC QI assessment tool and noted that district and regional supervisors did not normally use this format, but rather their own MoHSW checklist. ANC staff also knew there was a facility QI assessment related to BEmONC, but did not feel it was relevant to their work. The exceptions to this disengagement between ANC and BEmONC were in Pemba and Unguja, Zanzibar, where staff at the cottage or district hospital level constituted one facility team and the MAISHA Regional Program Officer encouraged use of and feedback on both assessment tools. The site visits also revealed a major health system factor that is probably affecting the quality of care and outcome for FANC more than any training or guidelines can compensate for – the inconsistent supply of drugs and test kits. (See Appendix C for FANC and BEmONC Supplies and Equipment in Site Visits, October 2011.) During the site visits, there were significant shortages of iron tablets (FE), SP, reagents for syphilis test kits, urine dipsticks, and testing supplies for Hemaglobin (HB). The shortages were especially severe in Zanzibar, where ANC clients were often asked to cost-share (i.e. pay) for tests in the facility lab or buy drugs from the private market. In the mainland, staff reported frequent stock outs, but stated that in many cases they requested the District Medical Officer (DMO) to provide funding or procure drugs locally during shortages. Findings from Data Review MAISHA committed to training 4,325 additional providers in FANC in an estimated 2,800 facilities by the end of project. A revised training target of 4,125 has been met in 2011, with a cumulative 4,164 (3966 mainland and 198 Zanzibar). In addition, 104 districts through their own funds and other donor partnerships have rolled out the training to 2,455 more FANC trainees. The expected end of project results for FANC also include:  80% of sampled MAISHA-assisted facilities comply with nationally recognized clinical performance standards for FANC and  85% of antenatal care (ANC) clients at sampled MAISHA-assisted facilities receive a syphilis test and a second dose of intermittent preventive treatment (IPT) for MIP. For objective measurement of improvement in quality of care in FANC services under MAISHA, project achievements must await the findings of the follow-up Quality of Care survey in 2012. However, a review of the MAISHA M&E Plan and PMP reveal the following status of selected FANC indicators at the end of Year 3 in Table 1: 17 TABLE 1: SELECTED MAISHA FANC INDICATORS FANC Indicator Baseline Year 3 actual EOP targets Percent/number of facilities meeting clinical performance standards for FANC* Not Available** 80% 95% Percent/number of ANC clients who received a syphilis test at USG-assisted health facilities*** 66% (Oct-Dec08 sentinel site data) 63% 90% Percent/number of ANC clients testing positive for syphilis who have received appropriate treatment at MAISHA-supported health facilities*** 100% (Oct-Dec08 sentinel site data) 94% 100% Percent/number of ANC clients at MAISHA-supported health facilities who received 2nd dose of IPT under DOT *** 54% (Oct 07-Sept 08 sentinel site data) 37% 90% Percent/number of USG-assisted health facilities experiencing stock￾outs of specific tracer drugs (FANC)*** 32% for SP; 22% for RPR kits; no data for mebendazole/ albendazole (Oct-Dec 08)** SP=73%; RPR=63%; Meb/alb=28% ; Iron=68% 10% * From QI assessments **No site assessed for FANC standards prior to MAISHA project startup ***From Sentinel data at 40 sites From QI assessments at sentinel sites, MAISHA-supported facilities appear to be meeting clinical performance standards for FANC, in line with meeting end of project targets. However for syphilis, although 94% of clients who test positive are treated appropriately, only 63% of all clients are tested. And there is bad news for the MIP program, as IPTp 2 rates remain low at 37% - a continuing major problem also confirmed nationally in the DHS 2010 (27%). The low rate of IPTp 2 could be due to several factors, including late timing of ANC visits. However, the field visits and the MAISHA sentinel data also reveal the most likely reason: high levels of stock outs for the malaria prevention drug SP. An analysis of 2011 data by MAISHA revealed that IPTp 2 rates were higher (55%) in facilities with no stock outs of SP. (See related chart in Appendix C). The mediocre syphilis testing rate above is also probably due to frequent stockouts for RPR kits. Summary of Findings and Achievements USAID, through the combination of the ACCESS and MAISHA projects, is to be congratulated for the achievements in FANC to date. National FANC guidelines approved by the MoHSW, as well as a National FANC learning resource package, are now in place. A critical mass of over 7,355 providers trained in FANC is now staffing approximately 74% of the nation’s ANC clinics. Over 100 district CHMT’s recognized the importance of FANC and rolled out further training to 2,455 staff. National, regional and district level trainers are available for future updates. FANC is now included in pre-service curricula for nurse-midwives and on-the-job FANC updates are taking place. The links for MIP, syphilis and PMTCT in antenatal visits have been made. As part of the training, ante-natal care standards have been established and in MAISHA-assisted facilities a related Quality Assessment tool is generally in use. Facilities where the ANC staff (and the FANC QI assessment tool) are better linked with the maternity ward staff (and 18 BEmONC SBM-R tool) appear to have more interest, enthusiasm and perhaps integrated quality of care for clients. Regional Program Manager (example: Zanzibar) can foster this link during the joint supervisory visits. However, the excellent work in setting standards and achieving high levels of training may not result in the desired outcomes vis-a-vis malaria, HIV, syphilis and other threats to health in pregnancy unless related health system issues are resolved. The high levels of drug and test kit stock outs are undercutting any gains in staff capacity building. The JSI/Deliver logo “No product, no program” very much applies to the FANC (and BEmONC) efforts. Recommendations  MAISHA and USAID should support policy and other initiatives to ensure a consistent supply of drugs and test supplies required for FANC quality of care.  Continued supportive supervision by the MAISHA project is necessary to consolidate the FANC service delivery gains of the past years. However, more importantly, the district and regional teams will be key to continued supervision, focusing especially on supply factors and adherence to FANC guidelines.  There is no further need for FANC training within the MAISHA project activities. However, the BEmONC SBM-R tool can be revised to include FANC to better link pregnant women with delivery in facilities. In addition, MAISHA staff (e.g., Regional Program Officers) can help make the vital link between the ANC clinic and the maternity ward by ensuring staff jointly process feedback from FANC and BEmONC QI assessments.  MAISHA and future programs should focus more on community mobilization to increase the number and improve the timing of ANC visits. The pilot activities in Iringa, Morogoro and Pemba are to be commended, yet the impact of an excellent FANC clinic will be less wherever pregnant women do not come at the right time or often enough to gain the benefits before delivery. BASIC EMERGENCY OBSTETRIC AND NEWBORN CARE (BEmONC) Component Summary The MAISHA project was conceived in large part to assist the MoHSW to strengthen the platform for service delivery for BEmONC as a key intervention to address the major causes of maternal and neonatal mortality. The MAISHA strategy is to build the capacity of health care providers in BEmONC through in-service and pre-service education and to support these capacity building efforts through the use of a Quality Improvement approach. Overall, the goal is to strengthen the clinical skills of providers in key signal BEmONC functions, create a strong service delivery environment for BEmONC and provide continuing support for providers to consolidate and strengthen their skills and improve the quality of care by quality improvement activities. To this end, MAISHA technical staff has worked in collaboration with Safe Motherhood partners (e.g., WHO and other technical stakeholders) to support the development of national resources (guidelines, training packages, trainers, supervision tools and clinical performance standards) for BEmONC. The main activity has been to review and update the existing national Life Saving Skills curriculum, with a focus on the strengthening of the competency-based content, including the use of the partograph to identify and manage prolonged labor, the introduction of the principles of Active Management of the Third Stage of Labor (AMSTL) and the treatment of Post Partum Hemorrhage (PPH). The updates also focused on the counseling of mothers on the importance of postnatal care within two days of childbirth and on Essential Newborn Care 19 (ENC), including newborn resuscitation, cord care, treatment of sepsis, and immediate warming and drying. To update the knowledge and skills on BEmONC for providers in-service, MAISHA has created a group of National BEmONC trainers by providing supplementary training for the existing national Life Saving Skills (LSS) trainers. A cascade for BEmONC training has been envisaged, i.e. national trainers are to train trainers at the regional and district level. District trainers are to train providers from Health Centers/Dispensaries initially with MAISHA support, but subsequently using other funds. In addition, the plan has been for MAISHA to collaborate with Zonal Training Centers to roll out training at the district level, as was successfully done for FANC. The plan also envisaged designating and strengthening the maternity wards of regional hospitals (one per region) to serve as a model facility, a training site for districts and a referral center for complicated obstetric cases. Within each district in all 21 regions, MAISHA has identified 2-3 facilities to be designated as MAISHA-supported sites. Facilities were selected in consultation with District Health Management Teams based primarily on their high delivery case load and their potential for being accessible by as many women as possible. One staff member at each site receives a 12-day training course in the BEMONC curriculum. To create an enabling working environment within these facilities (which also include maternity wards within Regional Hospitals), MAISHA has provided essential equipment and supplies for BEmONC service delivery based on an extensive on-site assessment of needs. To support the effort for improving quality of BEmONC services within these facilities, MAISHA has introduced Jhpiego’s Quality Improvement methodology, the Standards Based Management￾Recognition (SBM-R) approach into the selected facilities for use by staff after training, and into the Regional and District Health Management Team Supervisory System. The SBM-R methodology encourages health care providers to conduct internal assessments within their own facilities, to assess their adherence to established standards of care for BEmONC, to monitor performance, to identify gaps in their services, and to make action plans to address identified gaps. The SBM-R methodology has also been used to strengthen the supportive supervisory skills of Regional and District Health Management Teams who are responsible for the external supervision of BEmONC service providers in designated facilities. For the pre-service nursing education, MAISHA has built on work implemented by the ACCESS project. In partnership with WHO, MAISH has worked with the MoHSW Human Resource Development directorate to review the curriculum for certificate and diploma midwifery programs, to ensure the integration of evidence-based midwifery practices, to update midwifery tutors and clinical preceptors, to strengthen schools with skills laboratories to build an enabling environment for quality teaching, and to introduce the standards-based quality improvement approach at clinical training sites. ACCESS integrated FANC into the curriculum of 51 nursing and midwifery schools, and with core and USAID/Africa Bureau funding, MAISHA continues to work with selected schools to improve the BEmONC component of the curriculum. Findings from Site Visits MAISHA intervention sites have included a regional hospital maternity unit and 2 - 3 lower level facilities per district in each region on the mainland. In Zanzibar, the MAISHA interventions have been directed at the main referral hospital on Unguja and its associated maternity unit, three district hospitals in Pemba and four lower level district facilities (cottage hospitals) at both Unguja and Pemba. 20 At the facilities visited, MAISHA had trained one provider from each of the two facilities and one from the regional hospital in the mainland, while in Zanzibar an average of three to four providers per facility have been trained in BEmONC. The trainees were mainly nurse/midwives, including registered and enrolled nurses. At the regional hospitals, many staff had been trained earlier on Life Savings Skills, as well as on LSS/BEmONC organized by the region and or the Zonal training center (for example there are ongoing BEmONC trainings by the Iringa PHCI supported by USAID). The Iringa PCHI coordinates BEmONC training for the Southern zone regions including Iringa and Ruvuma and according to the RRCHCO, to date they have trained providers from 238 facilities in the southern zone. Despite the stated plan of having one trained provider per designated facility, the evaluation team noted substantial staff turnover at the MAISHA facilities, especially at the health centers and dispensaries in the mainland. During the field visits to district health facilities, only two out of the five Health Centers/Dispensaries visited, had staff available on site that had been updated on BEmONC by MAISHA. The other trained health care providers had either been transferred or were on study leave. At the regional hospitals visited, notably in Morogoro, Mtwara and Iringa, some of the trained staff were on site, but others have been posted to other departments in the hospital. For the trained staff that the team was able to interview, most stated that the BEmONC training provided by MAISHA has been useful. Specifically, they noted the improved knowledge and skills imparted on management of PPH and eclampsia and on the use of the partograph. According to the project plan, the selection criterion for facilities within each district was to be a high delivery case load. The evaluation team noted however that many of the lower level facilities, both on the mainland and in Zanzibar were under-utilized, with very few deliveries being conducted on a monthly basis. On the contrary, the labor wards, delivery rooms and post-natal wards at regional hospitals were extremely crowded and busy, with severe shortages in staff. Interviews with available staff at these crowded sites, cited a host of constraints to the provision of quality obstetric care including over-worked staff, frequent stock-outs of essential supplies and medicines (particularly at Mnazimmoja Hospital in Zanzibar where uterotonic drugs had not been available in the previous three months), and overcrowding of patients in the maternity area (labor ward, antenatal and postnatal wards) due to inadequate space. The team observed ongoing MAISHA quality improvement efforts at the health facilities visited. Overall, staff who had been trained by MAISHA and were present on the day of our site visit stated that the SBM-R tool was useful in identifying specific areas of clinical care that need improvement. Numerous sites had the Quality Improvement Assessment results posted on their walls and were able to identify areas of increasing improvement and areas that still required work. However during interviews with staff at both the regional and the lower level facilities, many interviewees they stated that the tool was very time consuming and difficult to complete on their own. Due to time constraints and staff shortages, the facility-based QI teams were often unable to perform the QI assessments on their own without the support of MAISHA technical staff. In places where trained providers were not on site, Regional and District RHC Coordinators were conducting the facility-based quality improvement assessments themselves. Regional and District Supervisors also noted in interviews that due to competing supervisory duties, the SBM-R tool was burdensome to complete, and that that it would be useful if elements of the tool could be integrated within existing MoHSW supervisory systems and checklists. The team observed during the site visits that most of the basic equipment supplied by MAISHA was available and on site (see Appendix C for list of Supplies and Equipment). The team noted 21 the availability of most of the BEmONC job aids and clinical guidelines and protocols (including on AMTSL and use of MgS04), as well as the presence of emergency trays, properly packed sterile delivery kits, hand-washing areas and plastic bins for the proper disposal of waste. At two regional facilities (Ligula and Morogoro) the staff informed us that the sterilizer supplied by MAISHA (dry heat oven) was not appropriate for their needs. But again, in sites where staff was not present and there were few patients, the equipment and supplies were under-utilized, and in facilities with extremely high case loads, the supplies and equipment were insufficient. Despite the availability of BEmONC services at in lower-level facilities, it was noted that many women still are unable to avail themselves of services due to the lack of transport. The team also observed that many women bypass some of the recently developed MAISHA facilities and go directly to the regional/district hospital, as was observed at Mwembeladu hospital in Zanzibar and Mikindani dispensary in Mtwara. At Mwembeladu hospital, because the hospital lacked a doctor on site, many women preferred to go to the regional hospital where they felt that a complication could be better dealt with. To learn more about the Pre-service training component, the evaluation team visited two nurse/midwife training schools: Mkomaindo in Masasi and Edgar Marantha in Ifakara. In interviews, the school principals and tutors informed the team that at least one midwifery tutor from each school had been trained in the MAISHA organized BEMONC and FANC training. The schools had been provided with a laptop and LCD and they had participated in a seminar organized by ITEC on use of computers in midwifery education. At Mkomaindo, at the time of the evaluation, the school had identified two clinical instructors in the ward, but they had not yet received training on BEmONC. Now that the in-service training package has been completed, the planned training for clinical preceptors will take place in Year 4 of the Project. At both Mkomaindo and Edgar Marantha School in Ifakara, a room had been set aside for a skills training laboratory, but the skills lab had not yet been established. Overall the tutors interviewed were not aware of the planned curriculum review or of any updated BEmONC content being integrated into the curriculum that they were using. During MAISHA team presentations to the evaluation team, some of the achievements mentioned in pre-service nursing/midwifery education included FANC training of tutors, interpersonal communication training for199 tutors, assessment of space for skills labs in 8 schools and a tracking survey of nursing/midwifery graduates. Apparently, none of the schools visited were among those assessed for laboratory space by MAISHA. Findings from Data Review MAISHA committed to training 1,145 providers (mainland) and 78 (Zanzibar) on BEmONC and ENC, in 287 facilities by the end of the project. The training targets were revised in year 3 with a slowdown in geographic coverage, and the project at end of year 3 has now met 41% (of 967 for mainland) and 63% (of 78 for Zanzibar) of its BEmONC training targets. A complete list of MAISHA PMP indicators and progress to end Year 3 is in Appendix G. The table below notes the official progress (or lack of progress) on a few key indicators. 22 TABLE 2: BEmONC INDICATORS AT THE END OF YEAR 3 COMPARED TO END OF PROJECT TARGET* MAISHA PMP Indicator Actual Year 3 End Project Target #1 - Percent/number of facilities of USG assisted facilities meeting clinical performance standards for safe delivery (BEmOC) 0% 75% #2 - Percent/number of USG assisted facilities achieving at least 80% clinical performance standards for newborn care (ENC)/resuscitation 0% 90% #3 - Percent/number of USG-assisted health facilities experiencing stock-outs of specific tracer drugs (BEmONC) Oxytocin, 44%; Ergometrine, 56%; Misoprostol, 75%; MgSO4 , 47% 10% Indicators #1 and #2 which report on progress towards meeting clinical standards in BEmONC across six technical areas are shown as 0%, as none of the facilities have met a QI average score of 80%. After discussions with the MAISHA team during the debriefing, intermediate indicators have been proposed as these will show the QI SBM-R score improvements over time and better display the actual progress observed in the field. For instance, 91% of regional hospitals have shown significant improvements in QI scores. Please see Appendix H with a Proposal for New Indicators to Monitor Quality Improvement Efforts. Indicator #3 on stock-outs reflects a factor which is negatively impacting the quality of BEmONC services, but over which MAISHA has little control. Another indicator, the MAISHA PMP case fatality rate – is highly relevant, but is more appropriately collected from regional hospitals which have a high complication case load, rather than from sentinel sites. Finally, many of the BEmONC indicators will not provide any measures of progress in improving the quality of care until the follow up survey in 2012 for the Quality of Maternal and Newborn Health Care Study. Quality Improvement Data At all MAISHA facilities, baseline, first and second internal quality of care assessments have been conducted using the MAISHA SBM-R QI tool. Available data shows gradual improvement in the individual category scores, as well as the overall average score, in almost all MAISHA sites. Figure 1 shows the QI scores for the baseline, first and second internal assessment for mainland regional hospitals. Facility teams had developed action plans to address the quality gaps and discussed their needs with the CHMT and HMT as appropriate. However, due to budget constraints, the CHMT and HMT have not always been able to adequately address the gaps. The main identified gaps included stock outs of drugs such as oxytocin and MgS04, consumables such as cotton wool and gauze, and the inadequate number of staff. 23 Figure 1: QI BASELINE, FIRST AND SECOND INTERNAL ASSESSMENT SCORES, MAINLAND Summary of Findings and Achievements Overall, MAISHA has successfully supported the MoHSW in developing a national BEmONC Learning Resource Package (LRP), including a facilitator guide, participant’s handbook and a skills check list, which is used to update provider knowledge and skills. MAISHA has trained providers according to plan and has provided basic BEmONC equipment to the selected facilities (including delivery kits, episiotomy kits, BP machines, Doppler, sterilizers and neonatal resuscitation equipments, caesarean section kits to some of the regional maternity units) after conducting an extensive facility-based assessment. Clinical guidelines, job aids and standard performance tools are in place in MAISHA facilities to monitor the quality of care. Now that these project components are in place, MAISHA needs to provide continuing support and supervision to sustain improvements in the quality of care over the remaining life of the project in order for these changes to become institutionalized. The issue of rapid staff transfers is one that needs to be looked into by the MAISHA Regional Program Officers. It is clear that the quality of BEmONC service delivery in MAISHA-supported facilities will be negatively affected if trained staff is not onsite. If trained staff is not expected to return to designated facilities, then arrangements should be made to train new staff members to ensure the presence of providers with up-dated BEmONC skills. Additionally, MAISHA needs to look again at selected sites where there is an excessively low patient load where resources are potentially under-utilized, and those sites with excessive patient loads and resources are potentially insufficient. Addressing any such imbalances is 24 important to improve resource allocation, but also to ensure that the quality of care is not being compromised in understaffed, over-crowded labor wards.11 Overall MAISHA has been successful in developing the technical resources necessary to upgrade provider skills in regional and lower level facilities, and has developed a model that could be replicated, if additional resources were to become available. Of course, this model needs to be extended into all facilities in all districts throughout the country in order to have a real impact on maternal and newborn mortality rates. It is also important to acknowledge that there are grave challenges within the health system that, if not dealt with, will continue to affect the quality of Emergency Obstetric and Newborn Care throughout Tanzania, regardless of the strength of the MAISHA model. These include severe shortage of clinical staff nationwide and the frequent stock outs of critical BEmONC supplies and equipment. Recommendations:  Clinicians (doctors) should be included in the BEmONC update training, and more staff per facility needs to be trained if feasible.  At regional and district levels, MAISHA should identify and train clinicians (doctors and midwifes) who could be clinical instructors during MAISHA trainings and other trainings organized by the region/district or by PMTCT partners  Trained clinical instructors could be co-opted by CHMTs to do clinical supervision, mentoring and coaching on BEmONC at the health center and dispensary facilities. At dispensaries and some health centers where there is a very low delivery caseload, regular coaching is critical for maintaining provider skills  Establish the skills training laboratory and train clinical preceptors for the planned nursing schools to ensure graduates have adequate practical midwifery skill upon graduation.  MAISHA should strengthen the district level maternity services since the district is the first referral level. Women referred from a HC/dispensary should be assured of quality Emergency Obstetric care.  MAISHA support to create “model” training centers at regional hospitals should include support for training resources and equipment for CEmONC.  MAISHA should support zonal training centers, e.g., Iringa PHCI, with training resources including establishing skills laboratories to facilitate roll-out of in-service BEmONC training at district level to non-MAISHA facilities. KANGAROO MOTHER CARE (KMC) Kangaroo Mother Care (KMC), an evidence-based intervention aimed at contributing to the reduction of newborn deaths due to pre-term complications, has been introduced into the MAISHA project as part of its hospital-based interventions. Under the MAISHA cooperative agreement, Save the Children has been specifically tasked with the development of KMC units in selected regional hospitals in the mainland and Zanzibar by improving skills of providers through training, mentoring and coaching and providing critical supplies. To facilitate the scale up of KMC nationwide, MAISHA has supported the MoHSW to train a national core group of KMC master trainers (in collaboration with Save the Children, WHO and 11 It is true that the evaluation team was not tasked with judging the adherence to clinical practice (this information will come from the Quality of Care study follow-up). But it was clear that in very busy, understaffed wards that standards of care were not being sufficiently adhered to, despite MAISHA training. i.e. partographs were not being used appropriately, protocols for PMTCT during delivery were not being followed, and steps for AMTSL were not being taken. 25 other partners); MAISHA has supported the MoHSW to establish KMC units in regional/ hospitals (including training and equipping units). As of the mid-term evaluation, 326 health workers from 17 established sites have attained additional knowledge to implement quality KMC services. Three hundred and forty six (346) health care providers (RHMT & HMT) have acquired additional skills to effectively provide supportive supervision of KMC services at respective regions. Forty two (42) focal persons have acquired new knowledge and have attained skills in data management through in service training for three days. Findings from Site Visits The evaluation team visited KMC Units in Zanzibar, Iringa, and Mtwara established by Save the Children. Overall the sites were well managed by trained staff. In one instance, in Zanzibar, the KMC unit was empty because of the low number of deliveries been conducted in the particular facility visited. In another instance, in Iringa region, despite the establishment of KMC units at the regional hospital, high-risk, underweight babies and mothers were sharing beds with other postnatal mothers and babies, including sick ones, because of lack of space. Recommendations Save the Children needs to ensure close follow-up of units in collaboration with HMTs to ensure adherence to KMC standards of care and address the issues of over-crowding on post￾natal wards. Linkages between KMC services and the community must also be established in order to identify low birthweight babies born at home who would benefit from referral to a facility for KMC, as well as for follow-up of mothers and newborns who “graduate” from KMC back into their community. QUALITY IMPROVEMENT AND SUPPORTIVE SUPERVISION The Standards-Based Management and Recognition (SBM-R) approach, a quality improvement methodology developed by Jhpiego, has been an important tool for assessing, monitoring and ensuring quality of BEmONC care in MAISHA-supported health facilities. The SBM-R QI methodology has also been introduced to Regional and District Management Teams in order to strengthen supportive supervision for health facility staff and to facilitate on-site coaching and mentoring. This approach, first developed for FANC services strengthened under the ACCESS project, focuses on establishing and monitoring achievable evidence-based standards and recognizing the efforts of those facilities that are able to perform accordingly. Findings from Site Visits Overall this tool is appreciated by health facility staff and district and regional health management teams, and has been used to great effect allowing staff and managers to focus on key elements of ante-natal care, labor and delivery and newborn care, and to work towards the improvement in the quality through an internal assessment and scoring mechanism. The internal assessment process has provided some facilities with the information necessary to improve planning, prioritize areas of need, and advocate for changes in policy and budget. The SBM-R is however, very complex and time-consuming and currently requires very heavy support by the MAISHA program team. In facilities with staff shortages or in facilities that do not have a provider trained by MAISHA on site because of staff transfers, the assessment exercise is very difficult to complete. The Regional and District RCH Coordinators who have been trained in supportive supervision and the SBM-R QI methodology are active participants in MAISHA-facilitated QI visits and on- 26 going supportive supervision. They realize the importance of QI and appreciate the useful and comprehensive nature of the tool, but there is an overall consensus among those interviewed that the use of the QI tool at the regional and district levels for continued monitoring and supervision may not be practical in its current form. Interviewed District and Regional RCH Coordinators noted that the tool took too much time to administer (2-3 days) and was not practical given their need to also complete the MoHSW and other donor checklists during supportive supervision visits. Recommendations The scaling-up of the SBM-R methodology for use in other facilities is dependent upon the acceptance of the tool by the MoHSW as part of their standardized national program of supportive supervision and quality improvement. The QI process needs to be stream-lined and simplified, and integrated within the Ministry current supervisory system in order to ensure its continued use after end of the MAISHA project. The coaching and mentoring component of the QI and supportive supervision activities is an important mechanism for the expansion of BEmONC training to additional staff and facilities. If coaching and mentoring is to remain a part of the QI and supportive supervision process, clinical mentors, in additional to regional and district supervisors, should be trained in the methodology to provide appropriate on the job training. PEPFAR FUNDED ACTIVITIES In Year Three, the MAISHA program has continued strengthening the platform of prevention of mother to child transmission (PMTCT) of HIV/AIDS established by USAID partners to address gaps in integrating MNCH services for HIV positive women and children (activities were started during the MAISHA timeframe, but under the ACCESS/FP and ACCESS programs). This PEPFAR funding has expanded into a wide range of activities including an integrated PMTCT postnatal/community model package; cervical cancer screening and treatment; Infection Prevention and Control; and support for pre-service education for Assistant Medical Officers (AMO) and nurse/ midwife training schools. A Comprehensive Facility/Community Approach to Integrated PMTCT/ PNC Services Although PMTCT services in terms of counseling, testing and prophylaxis appear to be well￾established within existing reproductive and child health clinics in many sites throughout Tanzania, there is an overall lack of available comprehensive MNCH and RCH services for HIV￾positive postpartum women and their infants, which limits opportunities to provide integrated care for this population. With PEPFAR funding, Jhpiego had established the Integrated Facility/Community PMTCT Program. Since the program’s inception in October 2008, Jhpiego has been actively working with the MoHSW to establish a model for provision of comprehensive postnatal care (PNC) services for HIV-positive and HIV-negative postpartum women and their infants. The program goal is to provide a continuum of comprehensive PNC services, inclusive of postpartum family planning (PPFP) and PMTCT for women living with HIV and their HIV-exposed infants, through an integrated facility/community approach. This model is being piloted in six districts of Morogoro Region (Kilombero, Kilosa, Mvomero, Morogoro District Council, Morogoro Urban and Ulanga). 27 At the facility level, the MAISHA project has worked on building the capacity at selected district hospitals and health centers, including improved pre-discharge counseling, linkages between under five/immunization clinics and care and treatment services, PNC and PPFP. As of the mid￾term evaluation, the project has conducted PNC training for 85 providers in all six districts in Morogoro region, and three districts in Iringa region with plans for expansion to Lindi, Mtwara and 2 other regions this year. MAISHA supported MoHSW to develop the postpartum care learning resource package and Training of Trainers (ToT) materials to accompany the PPC LRP and scale-up effort. At the community level, the program plans to develop a cadre of community health workers to provide health education on PMTCT, FANC, preventing malaria, infant feeding, PNC services, PPFP and cervical cancer prevention. As of the mid-term evaluation, MAISHA has completed the development of the integrated community maternal and newborn care guidelines and training package for CHWs (now have been translated into Kiswahili, and are at pretesting stage). The community package also includes an antenatal piece that covers from antenatal through postnatal and child care. In collaboration with D-tree, MAISHA is also working on the development of facility- and community-based electronic modules using mobile technology, although this activity had not begun as of the mid-term evaluation. Site Findings In the health facilities visited where the post natal care package had been introduced, trained staff clearly understood the importance of monitoring women in the immediate post-partum period for danger signs for the mother and newborn. Staff interviewed stated that more time was time spent after training on counseling mothers on breastfeeding, hygiene, danger signs for the mother and baby postpartum, and the critical need to return for post-natal check-ups as of days 7, 28 and 42. They also stated that they were providing immunizations and Vitamin A supplementation pre-discharge. Staff also emphasized the need for testing and counseling of mothers whose HIV status was unknown, and the need to link mothers whose tests were positive to CTCs. They also noted that they were careful to provide special attention to the counseling of HIV+ mothers on breastfeeding and the need to return day 7 for post-natal check-ups for the mother and infant. It was noted during the site visits, that due to restricted space on post-natal wards, many mothers were leaving the wards sooner than the recommended 24 hour period, which poses the greatest danger for mothers and newborns. Recommendations The introduction of the Integrated Facility/Community Model is a very important initiative as it seeks to strengthen services in the critical post natal period both for newborns and mothers but also for ensuring strong PMTCT linkages in the post natal period. The development of a community health worker component is a critically important intervention, without which the project will not reach the many women who currently do not avail themselves of facility-based health care services for ANC, labor and delivery and for post-natal care services, including post￾partum family planning. The post natal care training package and services need to be amplified and consolidated both in other regional hospitals, facilities and the community. Cervical Cancer Prevention Program The overall strategy of this component has been for MAISHA to provide technical assistance to the MoHSW to establish a National Cervical Cancer Prevention Program (service delivery guidelines, training package, IEC materials, and M&E tools). MAISHA has worked at expanding 28 the continuum of care for women to include cervical cancer screening and treatment for HIV￾positive women, and specifically, has established cervical cancer screening sites and access to VIA and cryotherapy services through single visit approach (SVA) at selected MAISHA supported health facilities. The program has been introduced at the regional level in Morogoro where services were established at the Regional Hospital, 5 district hospitals, and 2 health centers (total of 8 sites) and in Iringa region at the Iringa Regional Hospital and Mafinga District Hospital (total of 2 facilities). MAISHA has succeed in training 34 providers and 12 regional trainers in cervical cancer screening and treatment techniques and 43 CTC providers have been oriented by the program on CECAP guidelines and protocols for HIV+ women. In addition a LEEP machine has been installed at Morogoro Regional Hospital for treatment of women with larger lesions. Findings from Site Visits In the three sites that the evaluation team visited, the CECAP program seemed to be functioning well. HIV+ women of all ages coming from Care and Treatment Centers (CTC), women ages 30-50 from Reproductive and Child Health Clinics and from the general medical wards were encouraged to come for screening. Service providers were well versed in the clinical protocols for visual inspection using acetic acid as well as the use of on-site cryotherapy treatment. Guidelines and visual aids were available. All CECAP staff had a tracking system in place with the screening and treatment results, placing particular attention on the tracking of results for HIV+ women. As of the mid-term evaluation, 856 HIV+ women have been screened and received services and 34 providers have been trained. Recommendations The CECAP program needs to take a more systematic approach to developing community messaging and hospital-based communication on the availability of the screening service in order to truly expand the service. The program also needs to strengthen the referrals system for women with suspected cancers and those with large lesions, to ensure better tracking of the treatment compliance and palliative care options for women who chosen not to avail themselves of chemotherapy or radiation therapies, and those who cannot afford treatment. The issue of continuing supplies of cryotherapy after the end of the MAISHA project needs to be addressed by the MoHSW for the future. Infection Prevention and Control (IPC) The goal of this component, which has been in place since 2005 under ACCESS, is to improve the quality of IPC practices at facilities by supporting 1) the development of national IPC resources including guidelines, training package and trainers, 2) developing a national recognition system for quality improvement of health services 3) introducing the SBM-R approach for IPC at six referral hospitals associated with medical schools 4) address gaps in IPC supplies for maternity wards. As of the mid-term evaluation, the MAISHA project has developed and disseminated National IPC Guidelines, an IPC Pocket Guide, an Orientation Guide and IPC Standards. MAISHA has also instituted the SBM-R for IPC at 6 training hospitals and conducted on-site training in IPC for labor ward staff from 9 DSM facilities. 29 Other PEPFAR Funded Activities For FY2011, year 3 of the project, MAISHA was provided funding and the tasks of preservice training for Assistant Medical Officer (AMO), assisting with a national Community Health Worker (CHW) program, and establishing the foundation for a national PPIUCD program (converting to family planning preservice education integration in FY 2012). Of these, the parasitemia study is currently underway and some initial PPIUCD activities have taken place. 30 VII. PMTCT PLUS UP The commitment of the USAID Mission to the continued roll out of BEmONC, FANC and CECAP has been solidified by the provision of additional Partnership Framework Funds (Plus Up funds) in FY 2011 for Tanzania’s approximately seven (7) PMTCT implementing partners (IP). The PMTCT program is one of the USGs most successful health interventions with services available in all health facilities providing RCH services. At the national level, the structure for the provision of PMTCT services is based on a partitioning of the nation’s 21 regions with each implementing partner responsible for 2-3 regions. Currently, PMTCT funding provides a wide￾range of services including, but not limited to training, HIV counseling and testing, provision of ARV prophylaxis and ART treatment to HIV+ mothers, follow up of HIV exposed infants and Early Infant Diagnosis (EID), renovations to existing health structures, support referrals and linkages to Care and Treatment Centers, and supporting the broader maternal and child health services. With the provision of additional PMTCT Plus Up funding, Program Officers for CDC, USAID and US DoD-funded IPs have the mandate to ensure all their partners have identified activities in line with the “Illustrative list of activities to be supported using PF Funding (11 point PF package)” (see Appendix I). USAID reports that the illustrative list has been provided to IPs over the past two years during multiple partner meetings and that these activities must be included in their work-plans and implemented according to the guidance. MAISHA has been identified as the technical lead in MNCH programming and has received additional funding to provide assistance to IPs including: conducting BEmONC/FANC/CECAP integration planning and workplan review meetings with RHMT and regional PMTCT partners; providing support to regional PMTCT partners for facility assessments, training, supervision and M&E; providing BEmONC trainers; conducting advocacy meetings; and, providing technical assistance in the follow-up of quality improvement efforts, supervision of FANC/BEmONC/CECAP and the monitoring of FANC/BEmONC/ CECAP activities through Regional Health Management Teams and District Council Health Management Teams that have already been trained by MAISHA. Findings from Site Visits/Meetings Meetings were held with technical staff and administrators of Engender Health/Iringa Regional Office, International Center for AIDS Care and Treatment Program (ICAP), AIDS Relief, and African Medical and Research Foundation (AMREF). These IPs reported that they had been oriented by the USAID Mission to the Plus Up guidance and had received formal notification through the “Illustrative list of activities to be supported using PF Funding (11 point PF package).” The illustrative list included various components of a comprehensive MCH/PMTCT package that IP are expected to support. The guidance also included the caveat that IPs needed to select and implement activities in close collaboration with the RHMT and DCHMT. This means that there is no guarantee that partnering regions and districts will consider BEmONC/FANC-related activities as their top priority. During the first year of implementation, MAISHA has provided technical support to the ICAP and Grounds for Health for the establishment of cervical cancer services in Kigoma Region. They have also provided technical assistance and support to ICAP for BEmONC site assessments and training in Kagera Region. Engender Health /Iringa has also sought technical assistance from MAISHA and has identified health facilities in their project area for the roll out 31 of BEmONC with their first training scheduled for early 2012. AMREF has begun discussions with MAISHA staff for the implementation of CECAP services in their project areas. It is clear to the evaluation team that although BEmONC/FANC activities are paramount to the Mission’s GHI MNCH intermediate results, the guidance provided to IP was not clear. CECAP activities are gaining popularity among partner agencies and will continue to expand as funding is available. Recommendations To ensure the expansion of BEmONC/FANC activities through this funding mechanism it must be very clear to the IP and the Regional and District health leaders that this is the top priority. The achievements of PEPFAR in the provision of PMTCT services is an example of successfully leveraging existing base services for the expansion of essential health activities which must include MNCH. This will be critical to the roll out of BEmONC. Although MNCH funds do not compare to funding levels of PEPFAR, the provision of funding through the Plus Up mechanism is essential to the roll out of MAISHA’s program now that district level funding has been eliminated. It must be recognized that many IP have the experience and qualifications needed to implement BEmONC/FANC/CECAP programs without technical assistance from Jhpiego. As long as Ministry approved trainers, training materials and QI methods are used these IP can expand the program with or without engaging Jhpiego. 32 VIII. PROJECT MANAGEMENT MAISHA PROJECT MANAGEMENT Management and Organizational Structure The MAISHA project management structure has been built on a framework established under the previous ACCESS project. Jhpiego, registered as an NGO in 2005, is the prime managing and implementing partner in this cooperative agreement. Jhpiego hires all project staff, submits all project reports and budgets and determines project technical and management direction. The MAISHA project is located in Jhpiego’s main office in Dar es Salaam which also houses Jhpiego’s other Tanzania activities. There were four original collaborating partners or subgrantees assigned to very specific and limited roles in MAISHA program. WRATz/ Futures is still housed at the Jhpiego office and has served to advocate for MNCH at the national level, regional and district levels. WRATz MAISHA management has decided that it can manage regional rollout advocacy with Jhpiego staff and, with USAID’s concurrence, support by WRATz will no longer be needed for advocacy related to MAISHA project implementation. For various reasons, including inability to meet project expectations and replacement by the TCCP (see Project Partners), T-MARC Company is no longer with the project. IMA World Health has not met its commitments to MAISHA on a timely basis and will be phased out by mid-Year 4. This leaves Save the Children, that manages their own activity from their offices and is on track to complete its required obligations to set up the agreed number of KMC units in a timely manner. The MAISHA project organization chart (See Appendix F) has a Chief of Party, who currently also has the dual role of Jhpiego Country Director responsible for other Jhpiego activities in Tanzania. There is no Deputy Chief of Party. There are 6 senior managerial staff members for key technical areas, Finance and M&E, with one position vacant. There are approximately 130 Jhpiego/Tanzania staff, with 60 full- or part- time technical and managerial staff assigned to the MAISHA project and many more staff in support positions (HR, reception, drivers, etc). Short term technical assistance to various project components and senior management oversight has been provided from Jhpiego headquarters in the US. At the time of this report, there are also 4 positions vacant in the Dar Es Salaam office (See Project Organogram in Appendix F). Project management in the field (ie, regions and districts) has primarily been by staff based in Dar Es Salaam. As part of project design, MAISHA identified the need for Regional Program Officers (RPOs) to be placed in the regions. RPO’s are now in place for 8 regions and Zanzibar; however, there have been project delays, initially due to differences of opinion with the MoHSW on regionally located staff and later due to shortage of appropriate candidates. Currently there are 11 out of 22 planned Regional Program Officers (including 5 for the final new regions in Year 4) still to be hired. The MAISHA project began in FY2009 while Jhpiego was still completing activities under the ACCESS project. Since that time, there has been a constant process of project buildup to complete the FANC component and to develop materials for BEmONC with its related QI, pre￾service training and supportive supervision. The process of rollout of these core activities to the 21 regions is intense and lengthy, with advocacy and project discussions with regional and district MoHSW staff on decisions at all points to ensure real partnership. The process of supervision and followup is also complex; e.g., MAISHA visits to health facilities must include 33 official invitations to regional and district supervisory staff to join for 2-3 day visits. This is important to ensure involvement and capacity-building for supervisory staff and hopefully establish an on-going process for the future. However, it is also another reason to have Regional Program Officers in place and to recognize the time and effort required to consolidate quality gains. Over the course of the three years, there has also been a continuing process of addition of components requiring significant project management time and effort. These components include cervical cancer screening, an integrated PMTCT postnatal/community model package, PMI pre-service medical education, HIV pre-service medical education, AMO education, a PMI placental parasitemia study, preparation of a national training program for CHWs, integration of FP into pre-service (FY2012) and, most recently, the “PlusUp” PMTCT partners funding for rollout of MAISHA core activities. At least 4 of these components have not yet begun activities. Infection Prevention and nursing/midwifery education were begun under the previous project ACCESS, and have continued into MAISHA. Financial Management The MAISHA project core activities of FANC, BEmONC and QI in FY 2009 were primarily implemented through CS and PMI funding. CS and PMI still fund approximately 70% of MAISHA’s activities and staffing. At project startup, there was also some PEPFAR funding for pre-service nursing/midwifery curricula and Infection Prevention, transitioned from ACCESS. In the following two years, more activities were added with PEPFAR and FP funding. Currently at the beginning of Project Year 4, FY12, the project is managing and reporting on budgets and spending for activities under 13 different funding streams. The quarterly “burn rate” has increased since January 2009 from $320,000 to $1.3 million in October 2010 to over $2.3 million per quarter inFY2011. At the start of Project Year 4, with two years left in the project, approximately 60% of the total $40 million remains to be utilized. However, the project is now working at almost full capacity and the current rate of project spending will use all funding allotted if activities are rolled out as planned. Summary of Findings from Site Visits and Project Documents The MAISHA project today is a very complex entity with multiple components and multiple funding streams. What began with a clear focus on consolidation of FANC achievements and introduction of BEmONC has now become a project framework asked to implement a wide range of significant but sometimes tangential undertakings. Jhpiego as an organization probably has the technical expertise necessary to conduct most of these activities, e.g., the pre-service curricula and training and PMI placental study. As MAISHA staff has proven capable of handling such activities well, they may be a natural choice for more work. However, as a project, MAISHA is straining under a major management burden and HR limitations to finding appropriate staff to implement so many different activities. During the field site visits, some of this strain was revealed when expected protocol with a regional RCH Coordinator was bypassed, several sites were not notified of the evaluation team visit, and the two pre-service training institutions MAISHA selected for the evaluation team visit had apparently not received any project input in the last 1.5 years. There was a clear difference between the regions with a Regional Program Officer and those managed only from the main office in Dar Es Salaam. The ideal version of this regional manager is reflected in the Zanzibar Program Officer who works closely with the Zanzibar Reproductive and Child Health Section and the seven designated health facilities to provide both clinical mentoring for BEmONC and assistance with staff and facility QI and supervision activities. 34 In all project documents and meetings and in field visits, it was apparent that the MAISHA project is primarily a Jhpiego activity. The three partners already or in process of phase out – WRATz, IMA World Health and T-MARC had limited assigned roles and not much to present for their 3 years of efforts under MAISHA. Presumably this was due to relative weakness of the partners and not MAISHA management as the lead. Save the Children has a relatively important role in MAISHA in training and setup for 22 KMC units in regional hospitals to address morbidity in low-birth weight infants. However, it appears that from a MAISHA/Jhpiego perspective, this activity is a stand alone effort with little evident effort to integrate BEmONC and KMC. Some examples: MAISHA reports that Save the Children is developing its own KMC QI (no MAISHA/Jhpiego input) and conflicting reports on how many and which KMC units are completed through MAISHA. Constraints at National Level To leave a legacy for future national impact through national guidelines and learning resource packages, MAISHA needs to have input and approval from the MoHSW as well as other donor and technical partners such as WHO and UNICEF. This takes much time and effort at the national level, but can lead to materials and processes (e.g. supportive supervision) to be used by many implementing groups over the next 5-10 years. Integration of FANC, HIV, BEmONC and PNC training materials into pre-service curricula can also have lasting impact, but require focused attention by high level technical staff at the national level in Dar Es Salaam. This includes building relationships in the MoHSW in other departments beyond RCH and with other institutions. Constraints at Field Level The project rollout of the core activities of BEmONC, QI and supportive supervision to all 21 regions, including regional hospitals and two to three facilities in every district, is a major management and logistical challenge. It is also a costly exercise, especially when technical expertise and management are provided in such a large country primarily from the head office site. More local strategic planning and decision-making by Regional Program Officers can address many issues on the spot and lead to more effective supervision. Recommendations for Project Management MAISHA staff are to be commended for their sustained level of effort and commitment to keep this complex project moving towards project targets. The visible principles of their project management, such as including taking the time to develop counterpart agreement and involvement before rolling out training packages and placing staff in the regions, will strengthen eventual project outcomes. The quarterly financial burn rate shows a project in full action. However, with the complicated nature of the primary focus (BEmONC, QI) plus all the many additional activities, it is too easy to forge rapidly ahead yet lose focus and miss the necessary detail. The mid-term evaluation team recommends the following:  USAID and MAISHA should consider consolidation of BEmONC in the current 16 regions before or instead of rollout to the final five regions. This will help secure solid on-ground results and provide close supervision and follow up to selected facilities over the remaining time in the project.  A one-year extension of the project is also an option, in line with the slow project start up. Again, a slower pace will allow the project to consolidate current on-ground activities and provide adequate time for partnerships and facility strengthening in the 35 final 5 regions. It will also allow time for Plus Up PMTCT partners and others to roll out FANC, BEmONC and QI activities and for pilot activities such as the facility/community outreach to demonstrate results.  The change in management structure to Regional Program Officers is a positive step towards decentralization of technical assistance and decision-making and should be continued. This may prove more efficient in terms of logistical costs and also in strong relationships with local government partners. RPOs with a combination of management and clinical skills are preferred choices.  The complexity of the MAISHA project requires a full time COP and Jhpiego’s recent decision to separate the COP and Country Director positions is a timely step. In addition, Jhpiego may wish to review the current MAISHA management structure to better reflect the multiple project components. Examples may be addition of a deputy COP and/or separation of national level (guidelines, pre-service curricula) staff and field level (QI assessments, community models) technical and supervisory staff.  KMC units for low birth weight newborns are an important addition to ENC. The activities might be better integrated at the field level if Jhpiego and SAVE encourage staff to share information and followup.  USAID and MAISHA should discuss prioritization of activities within the project and also consider transition of some activities to other (Jhpiego) projects. 36 IX. MONITORING AND EVALUATION MONITORING AND EVALUATION PLAN M&E Activities The MAISHA program proposal contained an illustrative Monitoring and Evaluation Framework, which was later developed into a Monitoring and Evaluation Plan approved at the end of Project Year 1 in October 2009. The program outcomes were to be measured through several approaches: 1) training data using Jhpiego’s Training Monitoring Information Systems (TIMS) database, 2) a sentinel site surveillance system, expanded from ACCESS to include new facilities and data for BEmONC, 3) the QI Assessments for MAISHA-supported facilities, and 4) a baseline and follow up survey of quality of maternal and newborn health services using clinical observations for FANC and BEmONC. For pre-service education, MAISHA planned (M&E Plan October 2009) to measure capacity building through 1) a tracking system for graduates of nurse-midwifery schools and 2) a study conducted with a sample of recent graduates to assess retention of FANC, PPH and ENC clinical skills and knowledge. The skills assessment will include questions to evaluate the current work environment and its impact on skill retention. The latter study is planned now for 2012 and only for FANC skills, as PPH and ENC will only enter preservice curricula in project Year 4. The MAISHA Project Monitoring Plan (PMP) tracks indicators including standardized USAID operating indicators such as numbers trained, service delivery indicators focused on maternal and newborn health and some (PMI) relevant indicators. A separate set of standard HIV indicators are reported for PEPFAR. Training and service indicators are tracked quarterly and reported quarterly and annually, as are QI assessment scores. The clinical observation survey will only be reported for the years conducted. (See Appendix G for the MAISHA indicators and results to end Year 3.) During the 3 years of the project to date, MAISHA has tracked and reported on training data and the indicators in its PMP. See Appendix G for PMP annual targets and results for FANC, BEmONC and KMC indicators. The baseline survey of quality of health care (QOC) was conducted in July and August 2010 (almost 2 years after project start). The QOC survey was conducted in 52 facilities in 12 regions and additional facilities in Zanzibar. The survey included all regional hospitals and lower-level MAISHA-supported facilities conducting at least one delivery per day. It included facilities active in years 1 and 2 of the project, as well those truly at baseline. The reports have been completed and disseminated. Although the baseline was late, the follow-up is still planned for mid-2012. In April 2011, MAISHA conducted a pilot tracking survey by phone and found 984 (68%) of 1440 nurse/midwifes who graduated in 2009. The survey found that 95% were employed in a health facility, 65% in government facilities and less than 30% in rural areas. High or very high job satisfaction was reported by 86%. This is a very positive sign for URT as long as these graduates can provide high quality of care. MAISHA plans future tracking to be more frequent and in collaboration with the Tanzania Nurses and Midwives Council together with license renewal, as this may lead to better followup than a telephone survey. However, the most important item in MAISHA’s M&E approach is the recent decision to track indicators for all MAISHA supported sites, beginning FY2012. This change will mean tracking 37 results from approximately 280 facilities by end of project, rather than the current 40-50 sentinel sites. Summary of Findings from Document and Data Review The MAISHA training data is regularly reported to USAID in quarterly and annual reports. The Jhpiego TIMS database appears to easily reflect the multiple ongoing training activities. As these are standard operational indicators, the only item to note is that there have been minor revisions in the expected training targets for the project. The PEPFAR indicators are also primarily related to providers trained and clients served, which the Jhpiego data collection system is well capable of validating and reporting. The major issue comes in the reporting and interpretation of the FANC and BEmONC related indicators as measure of project progress. The first item is that the data has been primarily from sentinel facility sites, of which only 8 have been BEmONC delivery sites. The change from sentinel system to complete site data collection will be time-consuming and may produce some major differences in the reported project results. The fact that the data must be collected now at the district level will also confuse the numbers for some time. However the data will be more reflective of actual project activities and is a more sustainable approach. At the time of this report, MAISHA is already beginning the transition collecting data from all MAISHA sites via district information and expects to be ready by end December 2012. The key problem the team observed is that several of the indicators only tracked situations in the field that were not really a reflection of project efforts. Other indicators were designed such that site visits identified real improvements in the project facilities, yet the project PMP indicator showed no change. Several examples will illustrate the situation: 1. Two indicators track stock-outs for FANC and BEmONC related drugs and test kits. The results are informative for planning purposes. However, they are reflective of the continuing health systems problem of supply – over which MAISHA has no real control – and do not measure program efforts. 2. Another example is “% of ANC clients …who received 2nd dose of IPT under DOT”. Since stock out of SP is a major problem, MAISHA might use instead its analysis of IPTp 2 coverage in facilities with SP to measure the effect of program training and QI. 3. The BEmONC Quality Improvement Assessment is the summary of 6 complex elements and facility staff find it difficult to meet the clinical performance standards for an overall score of 80%. However, review of QI scores on field visits and MAISHA tracking charts show improvements over time. This BEmONC QI indicator has remained at 0% for 3 years and MAISHA needs to find some intermediate measures to reflect progress in specific areas of clinical care. Finally, a review of the baseline Quality of Maternal and Newborn Health Care survey raises a few issues for analysis and measurement of change over the project lifetime. The survey was conducted in July 2010 rather that in the beginning year of MAISHA. It therefore included facilities where staff had received FANC training previous to MAISHA, some that had already been in the MAISHA program for Years 1 and 2, and some that had not yet received MAISHA (and possibly no FANC/ACCESS) inputs. In other words, it is not a clear baseline to measure the differences between project start and project end, between no input vs full input. Fortunately (or unfortunately) there is plenty of room for significant improvement: a mean of 23% of ANC clients were screened for pre-eclampsia and eclampsia and AMSTL was correctly 38 performed in only 26% of all deliveries observed. The follow up survey in 2012 will measure changes in quality of care for MAISHA supported facilities and not changes or situation at a national level. Recommendations for M&E The MAISHA M&E plan is clearly organized and well thought out. However, in order to better reflect project program and outcomes related especially to FANC and BEmONC, there are several changes recommended for the remainder of the project. Jhpiego may wish to review and resubmit its M&E Plan and PMP for the remainder of the project.  The change from data tracking at sentinel sites to all sites will better reflect project results and is to be encouraged. This will also tie in better with the government reporting system, an important issue for sustainability. At the same time, the project may wish to review the amount of data collected to ensure there is timely use for each piece of data to avoid project overload in collection and analysis.  MAISHA should re-examine the indicators used to monitor project progress, especially in FANC, BEmONC and QI to ensure they are within MAISHA’s ability to impact and that they better reflect project progress over time. o For example, reporting % facilities with stockout of SP is helpful as a contribution to country monitoring, but reporting % IPTp 2 in facilities with SP as a PMP indicator will reflect MAISHA’s program efforts. o For QI, reporting on improvements in any of the 6 BEmONC QI sub￾components will show more progress in quality improvement over time. Right now, it is “0”. See a better indicator in the draft SBM-R example in Appendix G (PMP) developed by MAISHA staff at end of mid-term evaluation. o For media reach with MNH messages, this indicator no longer reflects MAISHA, but rather TCCP and can be deleted.  In selection of sites and analysis of the Followup 2012 Quality Survey with clinical observations, MAISHA must be able to report on changes in quality for MAISHA-supported sites. Depending on baseline survey site selection, there may be significant differences in quality at sites in just 2 years or it may be possible to compare with non-supported sites. Finally, if the project time is extended, the survey should be delayed to reflect more time for changes. 39 X. CONCLUSIONS Performance Monitoring Plan Intermediary project results as of October 31, 2011 are below. Results were gathered using the TIMS database, selected indicator data collected from sentinel health facilities, and Quality Improvement Assessments. Table 3: MAISHA EXPECTED RESULTS: YEAR 3 COMPARED TO END OF PROJECT TARGETS (SEPTEMBER 2013) TRAINING Expected Result End Year 3 Status Original EOP Target Revised EOP Target Additional Providers Trained in FANC, including malaria 4,186 providers in 2073 facilities 4,475 providers in 2800 facilities 4,275 providers (4125 mainland; 150 ZnZ) Providers trained in BEmOC and ENC 450 providers 1,145 providers 1,123 providers Providers trained in KMC 364 providers 225 providers 301 OTHER INDICATORS Expected Result End Year 3 Actual Status End of Project Target # Facilities strengthened to deliver BEmONC , ENC 239 in 16 regions and ZnZ (230 mainland; 9 ZnZ) 287 facilities in 21 regions and Zanzibar % MAISHA facilities comply with clinical standards 80% for FANC* 0% for BEmOC* 0% for ENC* 80% FANC 80% BEmONC % Women in MAISHA facilities receive AMTSL 26% 70% % ANC clients at MAISHA facilities receive syphilis test and second dose IPT 63% syphilis test* 37% IPTp 2* 85% syphilis test 85% IPTp 2 # KMC units established 17 22 units (one per region); including 3 from SNL # individuals reached with FANC, BEmONC and PMTCT messages through media outreach 0 5 million individuals * MAISHA sentinel site data As of the Midterm evaluation in Oct-Nov 2011, the revised FANC training target of 4,125 has been met with a cumulative total of 4,164 providers trained (3,966 mainland and 198 Zanzibar). In addition, 104 districts through their own funds and other donor partnerships have rolled out the training to 2455 more FANC trainees. For BEmONC, MAISHA has trained a total of 401 providers from 230 facilities on the mainland and 48 providers from 9 facilities in Zanzibar. In addition, 360 supervisors from the mainland and 37 supervisors from Zanzibar have been trained in BEmONC. 40 Two hundred and thirty nine (239) facilities in 16 regions (230 on the mainland and 9 in Zanzibar) have been strengthened to deliver BEmONC and ENC against a target of 287 facilities in 21 regions and Zanzibar. For Kangaroo Mother Care, 364 providers have been trained in KMC against a EOP target of 301, and 16 Units have been established against an EOP target of 22 unites (1 per region) including 3 units From Saving Newborn Lives Initiative. For results related to the quality of antenatal care, 80% of those MAISHA-supported facilities that requested an external verification visit in Year 3 comply with clinical standards for FANC. For results related to the quality of BEmONC, as of the mid-term evaluation, none of the MAISHA-supported facilities meet clinical standards for BEmONC and ENC, although intermediary results show that 91% of the hospitals and 63% of the Health Centers/Dispensaries showed an increased compliance to clinical standards. For the indicator on the percentage of women in MAISHA-supported facilities receiving Active Management of the Third Stage of Labor (AMTSL) by trained birth attendants, data is not available as of the mid-term evaluation. This data will be available after the follow-up study Quality of Maternal and Newborn Health Study is completed in 20012. According to sentinel site data sources, 63% of ANC clients received a syphilis test against an EOP target of 85% and 37% of ANC clients received IPTp2 against an EOP target of 85%. For results related to Information Education and Communication (IEC) activities, MAISHA has experienced some delays due to performance issues with one of its sub-contractors, T-MARC. As of the Mid-term evaluation, the project had not yet developed messages on FANC, BEMONC and PMTCT as planned for in the PMP. MAISHA has now forged a relationship with T-CCP, the organization leading IEC activities for all of USAID programs in Tanzania, and should be in a better position to achieve IEC goals in the second half of the project. CONCLUSIONS ON KEY TECHNICAL COMPONENTS Overall, MAISHA’s conceptual framework is solid and in line with international recommendations on key interventions to reduce maternal and newborn mortality. The MAISHA Project has aligned its interventions within the National Road Map Strategic Plan to Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 – 2015 and has developed substantial collaboration and buy-in by the MoHSW, which is vital for the overall sustainability of the interventions after the end of the project. The MAISHA Project has been at the forefront in supporting national, regional and district based advocacy efforts to place Maternal and Newborn Health as a priority health intervention. MAISHA serves as the Secretariat for the Safe Motherhood Working Group and MAISHA has single-handedly supported the advocacy efforts of White Ribbon Alliance of Tanzania. Regarding the main technical component of the MAISHA Project, improving skills of providers, MAISHA has been successful in developing top quality training packages for FANC, BEmONC, CECAP and PNC. The use of evidence-based training materials that establish standards of care and steer health facility staff towards the use of established clinical protocols, is an important contribution of the MAISHA project. 41 The SBM-R Quality Improvement methodology introduced by the MAISHA project in facilities and within Regional and District Health Management Teams has been used to great effect, allowing staff and managers to focus on key elements of ante-natal care, labor and delivery and newborn care, and to work towards the improvement in the quality through an internal assessment and scoring mechanism. The facility-based assessment tool of SBM-R is an important innovation, which allows staff to improve planning, prioritize areas of need and advocate for changes in policy and budget. The process allows staff to take ownership for the process of quality improvement by diagnosing challenges in service delivery, implementing changes and monitoring their success in meeting higher standards of care. The SBM-R is however very complex and time-consuming and currently requires very heavy support by the MAISHA program team. In facilities with staff shortages or in facilities that do not have a provider trained by MAISHA on site because of staff transfers, the assessment exercise is very difficult to complete. In order for the SBM-R tool to be scaled-up for use in other facilities throughout the country, the process needs to be stream-lined and simplified and integrated within the current MoHSW quality improvement and supportive supervision framework. MAISHA has trained key staff in BEmONC in selected health facilities and hospitals according to the project implementation plan. The evidence-based trained packages coupled with the SMB-R Quality Improvement methodology is an effective and powerful combination of resources for strengthening the quality of maternal and newborn care services in MAISHA-supported facilities. The gains made in improving provider skills however, need to be sustained through periodic updates for staff in key technical areas, and through a system of on-site mentoring and coaching which will support staff in maintaining their skills. In sites where MAISHA- trained staff is no longer present due to staff transfers, MAISHA needs to ensure the training of new providers. It was beyond the scope of the mid-term evaluation to a make a definitive determination on the extent and depth of qualitative changes in the practice of BEmONC and FANC in supported facilities after training. This information will be available from the follow-up to the Quality of Care Study. What the evaluation team is able to say is that in the MAISHA-supported facilities visited during the evaluation, where trained staff was practicing on site, where supplies and essential drugs were available, and where regional and district HMTs were providing strong supervisory support, then FANC and BEmONC services were indeed improved. In order to have a national impact in reducing maternal and newborn mortality, the network of BEmONC-trained providers needs to be expanded beyond MAISHA’s current project strategy of supporting two facilities per district. This is a huge undertaking and beyond the scope of the current MAISHA project and beyond current MCH funding levels. Ideally, the model would be taken up and funded by district budgets. However, given the recently announced moratorium on training by districts, the MAISHA project and USAID will need to spend more time thinking about and developing alternative mechanisms for rolling out trainings, such as the proposed plan of working with PMTCT implementing partners and building upon existing platforms to roll out the BEmONC training packages and other technical resources developed for CECAP, IPC, and PNC. Overall MAISHA has been successful in identifying and supplying critical MNCH equipment for hospitals, health centers /dispensaries, midwifery/nursing schools, and training Institutes. However, the overall issue of drug stock-outs and shortages for key ANC and BEmONC services (such as SP and uterotonics) is a major problem in the ensuring the quality of MNCH care in the long-run. This is a national health systems issue, but USAID/HPO should bring all its resources to bear in helping solve this critical problem by leveraging other USAID funded initiatives such as JSI’s DELIVER and MSH’s SCMS. 42 As of the mid-term evaluation, MAISHA has not yet achieved planned results in increasing demand for quality services through behavior change communication and community mobilization due partially to the delayed performance by one of the sub-grantees (T-MARC) and the desire of USAID to house all its IEC/BCC work with the Tanzania Capacity and Communication Program (T-CCP). Under the leadership of T-CCP and with technical assistance from MAISHA, the project should now be making headway on developing a strong communications program for improving community awareness and community mobilization on MNCH issues. The communication component is critical for expanding the project reach beyond the interventions in health facilities and will help address the need to scale-up services and information for the many women and newborns who do not avail themselves of health services at the time of labor and delivery, and the post-natal period. MAISHA has initiated work in Zanzibar with Community Health Workers and is piloting community activities in Morogoro region which are new, but extremely important initiatives within the project. Findings of these pilot activities need to be closely examined in order to provide solid recommendations to the Government as to the best model for working on community mobilization, communications and the creation of linkages with facilities. 43 XIII. RECOMMENDATIONS FOR CHANGES IN MAISHA FOR NEXT TWO YEARS The MAISHA project should continue its strong leadership as an advocate for greater investments in MNCH nationwide, and continue to work with the Ministry in finding solutions for health systems problem such as drug supply and health worker resources shortages through other funding mechanisms such as DELIVER and SCMS. Its work on integrating changes to medical curriculum and strengthening medical and nursing/midwifery pre-service and in-service programs are important initiatives and should be vigorously pursued in the last two years of the project. In terms of project implementation, MAISHA should work hard on consolidating the changes and improvements in maternal and newborn heath service delivery at the regional/district management and health facility levels over the next two years of the project. The project has introduced many new initiatives through its training programs and these need close monitoring and support in order for these improvements to be sustained. The introduction of the Integrated Facility/Community Model is a very important initiative as it seeks to strengthen services in the critical post natal period both for newborns and mothers but also for ensuring strong PMTCT linkages in the post natal period. The development of a community health worker component is a critically important intervention, without which the project will not reach the many women who currently do not avail themselves of facility-based health care services for ANC, labor and delivery and for post-natal care services, including post￾partum family planning. The post natal care training package and services need to be amplified and consolidated both in other regional hospitals, facilities and the community. The project needs to find mechanisms to strengthen on-going support and supervision of staff after initial training at the district and health facility-level. The introduction and strengthening of clinical mentoring on site for health care providers would be in important step in this direction. In order to ensure the very important work introduced with the QI methodology as a way of assessing, monitoring and ultimately improving the quality of care of maternal and newborn services, the project needs to work closely with the Ministry to integrate the principles and tools of the SBM-R into existing Ministry QI methodologies. In terms of MAISHA’s plan for use of PMTCT Plus funds to roll out the BEmONC, PNC and CECAP training packages and tools, better planning and clearer guidance for partners is required if this is to be the primary mechanisms for rolling out BEmONC, PNC, CECAP and other services to achieve national scale. Overall, better collaboration with other MNCH partners would create important efficiencies and would facilitate roll-out for services to other regions. 44 APPENDICES A. SCOPE OF WORK B. CUMULATIVE TRAINING C. SITE VISIT – SUPPLIES AND EQUIPMENT D. PERSONS CONTACTED E. REFERENCES F. ORGANIZATIONAL CHART G. PROJECT MONITORING PLAN INDICATORS AND PROGRESS TO END OF YEAR 3 H. POTENTIAL INDICATOR CHANGES WITH REGARD TO SBM-R UNDER MAISHA I. ILLUSTRATIVE LIST OF ACTIVITIES TO BE SUPPORTED USING PMTCT PLUS UP FUNDS J. INTERVIEW INSTRUMENTS MAISHA Midterm Evaluation Report 45 APPENDIX A. SCOPE OF WORK I. Activity: Mid-Term Evaluation of Mothers and Infants Safe Healthy Alive (MAISHA) USAID/ Tanzania Maternal Newborn Health Program Contract: Global Health Technical Assistance Project (GH Tech), Task Order No. 1 II. PERFORMANCE PERIOD The assignment will start o/a Oct 3 - mid Dec 2011 (In country: Oct. 10 - Nov. 15) All field work must be completed no later than end November 2011 due to GH Tech close out in Mid December 2011. III. OBJECTIVES AND PURPOSE OF THE ASSIGNMENT: The USAID/Tanzania Health & Population Office (HPO) will engage a team of consultants to conduct a mid-term evaluation of the Mothers and Infants Safe Healthy and Alive (MAISHA) project, which constitutes a major part of the Mission’s Maternal and Newborn Health (MNCH) program. The project aims to reduce the leading causes of maternal and neonatal mortality and morbidity in Tanzania mainland and Zanzibar. The evaluation will focus on the major MNCH interventions. The evaluation will also include any health system strengthening (HSS) activities underway, given that these are fundamental to achieving progress and sustainability in the health interventions. The evaluation will be conducted in 2011, in order to make in-course corrections and to guide program planning for the remainder of the MAISHA program (through September 2013). IV. BACKGROUND AND PROJECT INFORMATION Program Title: Mothers and Infants Safe Healthy Alive (MAISHA) Implementing Partner: Jhpiego Sub-grantees: WRATz/ Futures; IMA World Health; Save the Children; T-MARC Company Cooperative Agreement Number: 621-A-00-08-00023-00 Project Budget: $40 million ($25 million for MNCH, $15 million for PMTCT) Project Duration: October 2008 – September 2013 Tanzania has a population estimated at nearly 43 million, almost 75% of which lives in rural areas. The percentage of people living in poverty has decreased only marginally over the past 10 years, while continued rapid population growth has increased the absolute number of Tanzanians living in poverty by more than a million persons, overwhelming an already fragile social service system. Tanzania relies heavily on foreign aid, and roughly one-third of the national budget is financed by direct budget support. Effective lack of basic healthcare especially at more rural and poorer communities persists as a major challenge to development. Tanzania’s life expectancy at birth is 52 years, slightly lower than the regional average of 53 years, and the population is primarily young (21% of women and 26% of men are between 15-19 years of age). 12 Although there has been an overall downward trend in fertility in Tanzania (5.4 children per woman), at current growth rates, Tanzania’s population will exceed 50 million by 2025. Data from the Demographic and Health Survey (DHS) in Tanzania (2009/10) show that 24% of women of reproductive age use modern contraception and 30% of married women wanted no more children (of which 3.6% are sterilized). Not surprisingly, Tanzania has unacceptably high maternal and infant mortality rates, ranking 19th and 32nd in the world respectively.13 12 Demographic and Health Survey, 2009/10. 13 UN State of the World’s Children, 2009. 46 Complications from pregnancy and childbirth are major causes of death for women of reproductive age and newborns in Tanzania. Despite high coverage rates of antenatal care (96% attend at least once), and approximately 50% of women delivering in a health facility, the maternal mortality remains high at 45414 deaths per 100,000 live births (accounting for approximately 8,000 maternal deaths per year). The major causes of maternal mortality include sepsis, hemorrhage, hypertensive disorders of pregnancy, obstructed labor, and abortion complications; indirect causes include malaria, anemia, HIV and cardiovascular diseases15. For every woman that dies during pregnancy and delivery, 30 others are likely to have health complications during childbirth. Over 60 percent of all women stated that they face major barriers to accessing health care when they are sick (DHS 2004/5). The most common barriers to accessing care were: getting money for treatment (40 percent); the distance to the health facility (38 percent); having to take transportation (37 percent); and not wanting to go alone (24 percent). Women with little or no education and women from poorer households had greater barriers to overcome in order to access care. Every year at least 51,000 Tanzanian newborns die and 43,000 babies are stillborn. Up to two￾thirds, or 34,000, newborn lives could be saved if essential care reached mothers and babies during the pregnancy, delivery and postpartum period. Although under-five mortality rates have dropped by 40 percent (from 137 deaths per 1,000 births in the mid-1990s to only 81 for the period 2006-1016), TDHS 2005/05 neonatal mortality rates are 32 per 1,000 live births, and account for 47% of the infant mortality rate.17 During antenatal visits, lifesaving interventions are provided, including addressing malaria and syphilis in pregnancy. The success of malaria prevention programs focusing on pregnant mothers during the antenatal period can be seen by the recent increase in bed net ownership and use by pregnant women. The percentage of pregnant women who slept under an ITN the night before the DHS survey increased from 18 percent in 2004-05 to 25 percent in 2007-08 and to 68 percent in 2010. Unfortunately the provision of Intermittent Preventive Treatment in Pregnancy (IPTp) has been less successful with essentially no change from 2004/05. Currently, only 27% of pregnant women receive IPTp2. Complications from delivery and childbirth are major causes of death and disability for women of reproductive age and newborns in Tanzania mainland. “The critical challenges in reducing maternal and newborn mortality and morbidity include inadequate implementation of existing strategies, weak health infrastructure, limited access to quality Emergency Obstretic and Newborn Care (EmONC) services, inadequate human resources, lack of equipment and supplies, low utilization of modern family planning services, and inadequate monitoring and evaluation of health services. Availability and access to EmONC services are therefore key interventions in addressing the high Maternal Mortality Ratio (MMR) and Newborn Mortality Rate (NMR)”.18 Disparities in access to quality services between rural and urban populations are an additional challenge. Poorer, less educated, and rural households consume less health care, access lower quality services and have worse health outcomes. Women from wealthier households are three times more likely to receive skilled birth attendance and nine times more likely to give birth by Caesarean section. One analysis found that even the poorest women in urban areas have a skilled birth attendance rate that is higher than all but the richest women in rural areas. 14 Preliminary results from Demographic and Health Survey, 2009/10. 15 The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 - 2015 16 Preliminary results from Demographic and Health Survey, 2009/10. 17 The National Road Map Strategic Plan To Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania 2008 - 2015 18 MOHSW Assessment of Emergency Obstetric and Newborn Care (EmONC) equipment and training needs in Tanzania Mainland(2010) MAISHA Midterm Evaluation Report 47 Mainland Tanzania has five medical schools and 66 nursing schools; however, there continues to be a severe shortage of health-care workers at all levels, especially in rural districts, and the expansion of facilities will exacerbate this shortage. The number of mid-high level health professionals per 10,000 persons is 6; there are pilot programs training community health workers (CHW) and plans to extend this cadre, though presently there are few formal community health workers available to extend the work of health professionals. V. USAIDs Maternal Newborn Health Program (MAISHA) Improving maternal and newborn health and reducing maternal and newborn mortality are key concerns of the Ministry of Health and Social Welfare (MOHSW) and Sector Wide Approach (SWAp) partners. Tanzania has committed to the reduction of maternal mortality by 75% by 2015, through its Road Map Strategy Plan to Accelerate Reduction of Maternal Newborn and Child Death. The roadmap has set ambitious targets for improving the availability of EmONC, and MAISHA was developed in collaboration with the MoHSW, specifically the Reproductive and Child Health Section (RCHS), to address these needs. The MAISHA project is designed to deliver critical, evidence-based health interventions on a national scale, contributing to the achievement of the national targets for Millennium Development Goals (MDGs) Four and Five19. As such, the MAISHA program is supporting the advancement of local and national human and material capacity to achieve the following objectives:  Reduction of maternal mortality due to major direct causes of mortality;  Reduction of newborn mortality due to infection, hypothermia and asphyxia through immediate newborn care;  Reduction of low birth weight, stillbirth and newborn mortality due to malaria and congenital syphilis; and  Reduction of transmission of HIV infection from mother to child and increase of HIV free survival. MAISHA is implemented by Jhpiego, in partnership with:  Futures/ White Ribbon Alliance of Tanzania which was to support with advocacy to leadership and decision makers at the national level and synchronized efforts at the regions as MAISHA rolled out across the country.  Interfaith Medical Alliance (IMA) which committed to provide grassroots awareness of the importance of the interventions provided at ANC and delivery through the use of sermon guides; and to improve advocacy efforts at the District Health Management Team (DHMT) to enable improved provision of MNCH supplies to faith based facilities.  Save the Children’s Kangaroo Mother Care (KMC) program was supported by MAISHA in order that a KMC program could be rolled out into each regional hospital where Basic Emergency Obstetric and Newborn Care (BEmONC) programs were being trained by MAISHA.  T-MARC Company was engaged to develop appropriate mass media and interpersonal messaging in regions as the MAISHA program is being rolled out in order to ensure that pregnant women are aware of and access the services established. MAISHA is consolidating the work of the previous Jhpiego ACCESS Program, which supported quality Focused Antenatal Care through nationwide trainings. MAISHA’s new activities included strengthening provision ofBEmONC20 and Essential Newborn Care (ENC) to prevent and treat 19 MDG 4 – reduce neonatal mortality from 32 to 19 per 1,000 live births (mainland) and from 29 to 23 per 1,000 live births (Zanzibar); MDG 5 – reduce maternal mortality from 578 to 193 per 100,000 live births (mainland) and from 377 to 251 per 100,000 live births (Zanzibar) 20 BEmONC program includes rolling out the Helping Babies Breathe program supported by USAID 48 key contributors to maternal and newborn mortality. The program is designed to strengthen the mid-level health facilities (mostly Health Centers and Dispensaries that have the highest burden of deliveries) given that we were not including the interventions associated with hospitals (C-section and blood transfusion) in the Basic Emergency Obstetric Care (BEmOC) program. MAISHA activities in support of these objectives include:  Development of national, regional and district resources (guidelines, training package, trainers, supervision tools, regional training sites) to roll out BEmOC and ENC with national stakeholders  Advocacy and coordination with District Health Management Teams to ensure funding is allocated for support of quality ANC, BEmOC and ENC service delivery, including training of providers at district level, provision of necessary equipment, supplies and drugs, availability of transport for women and newborns in need of referral and conducting routine supportive supervision  With President’s Emergency Plan for AIDS Relief (PEPFAR) support, the program will strengthenthe platform of prevention of mother to child transmission (PMTCT) of HIV/AIDS to address gaps in integrating MNCH services for HIV positive women and children.  By the end of the program, MAISHA will be implemented and consolidated in all twenty-one regions of Tanzania. In each region, MAISHA strengthens the regional hospital as a service delivery and clinical training site for BEmOC, Helping Babies Breathe (HBB)/ENC and KMC. MAISHA committed to support improvements in BEmOC and ENC service delivery in at least two health centers or dispensaries in each district with high delivery caseloads, by supporting the training of providers, the provision of equipment and supplies, and strengthening the processes of coaching/mentoring and supportive supervision at the district level. As the ANC trainings funded under the ACCESS program were taking place and MAISHA programs were being planned, it became clear that the trainings seemed to have had little impact in light of the major service provision indicator of malaria in pregnancy, IPTp. The DHS 04/05 had shown IPTp rates of 22% and subsequent ones showed increase to 30% (07/08) and then 27% (09/10). For this reason, the team decided to shift the focus of the ANC component of the program, moving away from conducting increasing numbers of trainings to establishing strong systems of internal quality assurance at facilities and empowering the DHMT members to provide external mentorship and supervision. These improvement efforts were designed to cover both ANC and EmONC activities in MAISHA sites, and lessons learned would be extended to non-MAISHA sites by the empowered DHMT members. MAISHA is currently implementing Quality Improvement efforts at the facility level in regional hospitals and in health centers and dispensaries. Their approaches at the facility level include:  Jhpiego’s Standards-Based Management and Recognition—series of modular trainings and meetings  Performance standards, introduced during Focused Antenatal Care (FANC) and BEmONC service provider training – developed and in place  Baseline and follow-up assessments  External verification visits  Recognition by MoHSW\ MAISHA’s supportive supervision interventions are based on the use of standards and include training of zonal/regional/district RCH coordinators and regional/district nursing officers in facilitative supervision skills (including use of standards, recording/reporting, coaching, and MAISHA Midterm Evaluation Report 49 mentoring). They also include support for supervision visits in each region to the MAISHA￾targeted sites The outcome of MAISHA’s facility-based program is to be evaluated through a Quality of Care assessment covering all major areas of intervention in facilities that are intended to be supported by MAISHA (baseline conducted 2010) and a follow up assessment in 2012 will provide a measure of improvements and assist with intervention re-design. MAISHA committed to achieve the following results by the close of the program:  1,145 providers trained in BEmOC and ENC, including newborn resuscitation  287 facilities strengthened to deliver BEmOC and ENC services and developed as model sites;  Outside of ACCESS, 4,325 providers trained in FANC, including malaria during pregnancy treatment or prevention from an estimated 2,800 facilities;  80% of sampled MAISHA-assisted facilities comply with nationally-recognized clinical performance standards for FANC, safe delivery and ENC;  70% of women in MAISHA-assisted facilities receive active management of the third stage of labor (AMTSL) by skilled birth attendants  85% of antenatal care (ANC) clients at sampled MAISHA-assisted facilities receive a syphilis test and a second dose of intermittent preventive treatment (IPT) for MIP  22 KMC units established (one per region)  At least 5 million individuals reached with FANC, BEmOC, ENC and PMTCT messages through media outreach Changes in the Program Jhpiego proposed to roll out BEmONC training across the 21 regions of Tanzania in the following manner:  Year 1: Lindi, Mtwara and Zanzibar (design of program for scale up)  Year 2: Arusha, Iringa, Kigoma, Kilimanjaro, Manyara, Morogoro, Pwani, Ruvuma and Tabora  Year 3: Dar es Salaam, Dodoma, Kagera, Mara, Mbeya, Mwanza, Rukwa, Shinyanga, Singida and Tanga  Years 4 & 5: Continued support to all regions to promote high quality antenatal, delivery and newborn services The pace of this roll out has proved to be a challenge to Jhpiego’s team, and the following more moderate approach was submitted in the Year 3 workplan:  Year 1: Lindi, Mtwara and Zanzibar (design of program for scale up)  Year 2: Arusha, Iringa, Kigoma, Kilimanjaro, Manyara, Morogoro, Pwani, Ruvuma and Tabora  Year 3: Mbeya, Rukwa, Shinyanga and Tanga  Year 4: Dodoma, Kagera, Mara, Mwanza and Singida  Year 5: Continued support to all regions to promote high quality antenatal, delivery and newborn services In the 3rd implementation year, MAISHA was allocated additional PEPFAR funding to work with regional PMTCT partners in scaling up improved BEmONC, FANC, Post-Natal Care (PNC) and Cervical Cancer Program (CECAP) service delivery. MAISHA’s Year 3 workplan proposes a technical leadership and coordination role, while the PMTCT partners will spend their separate PEPFAR funding to support activity implementation in any of the four service delivery areas. 50 The workplan states: This additional funding will allow the MAISHA program to expand nation-wide this year and reach more facilities than originally anticipated. MAISHA will continue its direct support for service delivery and quality improvement in the Year One and Year Two regions (Zanzibar, Lindi, Mtwara, Arusha, Kilimanjaro, Iringa, Kigoma, Morogoro, Manyara, Tabora, Ruvuma and Pwani) at the facilities that have already received some support from the program for BEmONC and FANC service strengthening. MAISHA will also provide technical guidance to the regional PMTCT partners in these regions to assist them in scaling up BEmONC, FANC, PNC and CECAP improvements beyond those facilities already supported by MAISHA. In the remaining ten regions, MAISHA will work directly with regional hospitals to strengthen their BEmONC and FANC practices, establish KMC services and develop their capacity to function as a BEmONC training site. MAISHA will also begin district-level implementation following the usual program approach in Dar es Salaam (in collaboration with CCBRT), Mbeya, Rukwa, Shinyanga and Tanga. The last five regions (Dodoma, Kagera, Mara, Mwanza and Singida) will start with district-level implementation in Year Four. In all regions, with its PEPFAR integration funds, MAISHA will work with the regional PMTCT partner to introduce the integrated program, provide technical support in assessing additional sites selected by the RHMTs for strengthening, provide technical guidance to the partner and the RHMT in developing a workplan for site strengthening and assist with training, supervision and monitoring and evaluation. VI. Purpose and Objectives of the Midterm Evaluation Purpose: Conduct a mid-term evaluation of the activities funded by President’s Malaria Initiative (PMI), Maternal and Child Health (MCH) and PEPFAR monies and implemented by MAISHA. Objective: Based on the findings, assess current progress towards MAISHA goals and results, including effectiveness of MAISHA interventions. Recommend modifications and improvements in the remaining years of the project and future direction, based on the successes and challenges of MAISHA’s approach, for a new MCH program. Audience: HPO Team, HIV team, MAISHA team VII. Key Evaluation Questions The evaluation team will review the following illustrative questions. Evaluation questions will be refined and finalized by the team during the Team planning meeting and discussed with USAID/Tanzania prior to proceeding with work. Evaluation priority focus areas by region will be:  FANC, BEmONC, Quality improvement and supervision system (all regions);  Pre-service nursing/midwifery training (Iringa and Mtwara);  Community program (Morogoro and Zanzibar);  Post natal care program (Morogoro and Iringa);  Cervical cancer program (Morogoro and Iringa). 1) Are the various objectives and results on track to deliver the interventions specified in the Cooperative Agreement in terms of both geographical coverage and technical content? MAISHA Midterm Evaluation Report 51 a) What has been the project’s progress to date in relation to planned results and performance indicators (provided in the Results Framework and the projects’ Performance Monitoring Plans)? b) Are the subcontractors achieving the results in their workplans? c) Determine whether development of facility based quality improvement systems and external supervision systems adequately support the objective of improving quality of ANC and BEmONC services. 2) Does the rollout of in-service training together with provision of regional support for implementation of change (support and mentorship to the District and Regional Reproductive and Child Health Coordinator and facility) produce the intended results? a) What is the extent and depth of qualitative changes in the practice of BEmONC and FANC in supported facilities? b) How many individuals have been trained? How many per facility? c) Are clinicians who participated in the training using what they learned? Are they being retained on labor wards and practicing their new skills, or are they being rotated to different wards? d) What is the method of training rollout? e) Are MAISHA’s pre-service training activities having the desired results on HIV and MCH? For HIV, the team should review successes in establishment of skills labs and strengthening the quality of tutors/ preceptors. For MCH, the team should review data collected on graduates from ANC training up-dates, if available. Progress and data may be sparse on BEmONC. f) What are the achievements to date? g) What are the future needs in pre-service training? 3) What planned result targets are not on track to be met or exceeded? Why? a) What have been the greatest constraints to achieving results? b) What are the main causes of delays in the rollout of the BEmONC training? 4) Is the Quality Improvement approach undertaken by the MAISHA project scalable, and has it had an impact on the health facilities? a) Are District Health Management Teams using Quality Improvement (QI) methods of supervision? b) In supported facilities, are QI methodologies being applied and implemented in the delivery of FANC and BEmONC services? 5) Were there specific project management policies, structure or practices that contributed to either success or failure of intervention implementation? a) How well is the project (Jhpeigo and its partners) working together to coordinate planning and implementation of activities, avoid duplication and support each other? How well is the program working in collaboration with other programs, including USG/PMTCT funded partners? b) Are the staffing structure and capacity sufficient to achieve project goals? 6) What changes should the MAISHA project make in the second half of implementation and what recommendations for programmatic change should the HPO make for the success of future maternal health programs in Tanzania? a) Identify lessons learned, successful interventions for continuation or replication, best practices, significant products and tools of the above projects for possible dissemination and replication. b) Are the content and structures of MAISHA’s approach sustainable? Are they embedded in national structures? 52 c) Does MAISHA’s approach empower District teams to proactively address MCH needs? In the process of the evaluation alternative approaches that may contribute to host country ownership should also be noted. d) Based on the findings related to Questions 1 and 3, identify problems and/or issues that appear to impede timely progress and suggest potential courses of action to meet stated objectives in maternal health. e) Based on the findings related to Questions 2 and 4, identify needed changes in delivery of the training and QI aspects of the program. f) Based on the findings related to Question 5, identify possible changes to improve coordination between partners or to improve management structures, if needed. g) In the final year of programming, should MAISHA focus on expansion or would the consolidation of current activities be more meaningful? 7) How has USG informed and enabled MAISHA and other recipients of PMTCT Acceleration Plan funds to use MAISHA as the Technical / Content expert with which they are to plan their additional funds to support the objective of expanding BEmONC in the regions in which they work? a) What do the PMTCT implementing partners understand regarding their expanded role given the 2010-2011 Partnership Framework Plus up funds? In what way were the list of interventions (BEmONC, FANC, CECAP, Postnatal Care) prioritized by guidance they recieved? How did they decide what area to address first? b) What do PMTCT implementing partners understand regarding the role of MAISHA as the Technical / Content expert as they start to expand their PMTCT role into the new content areas of BEmONC, FANC, CECAP, Post natal Care? Has the TA relationship been established/ worked? How have they benefited from working with MAISHA? VIII. Methodology  The revised/final SOW will be shared with the evaluation team members for comment and clarification. Evaluation objectives and questions may be shared with the Jhpiego as part of the planned explanation of the evaluation activity.  Background Reading: Review relevant national documents including MOHSW strategic documents, national surveys, policies, and guidelines supplied by HPO prior to team’s arrival in country. Review project documents, including proposals, implementation plans, monitoring and evaluation plans, progress reports, review/evaluation reports, and training curricula provided by MAISHA prior to team’s arrival in country. Please see Section XI for a list of suggested documents.  The evaluation team will conduct a 3 day team planning meeting (TPM) upon arrival in Tanzania and before starting the in-country portion of the evaluation. The TPM will review and clarify any questions on the evaluation SOW, draft an initial work plan, develop a data collection plan, finalize the evaluation questions, develop the evaluation report table of contents, clarify team members’ roles, and assign drafting responsibilities for the evaluation report. HPO will participate in relevant sections of the TPM and the TPM outcomes will be shared with USAID/Tanzania.  Conduct field visits to selected project sites. A representative number of Year 1–Year 3 sites may be selected in consultation with the HPO and MAISHA teams. The study should include at least one region where CECAP is being conducted. The evaluation team should verify program achievements and collect both quantitative and qualitative data that demonstrate program performance. The evaluation team will conduct secondary data analysis as feasible based on the descriptive data available from implementers. The evaluation team will not conduct primary data collection. USAID/Tanzania will advise on in-country MAISHA Midterm Evaluation Report 53 travel required –cities, modes of transportation, duration of trips—prior to the assignment’s inception.  Interview key informants including USAID Mission management and staff (HPO and PEPFAR), Jhpeigo and sub-partner staff (IMA, WRATz, STC, TMARC); key Ministry of Health staff (including RCHS program staff, regional and district Health Management Teams, Hospital and Health center staff); key health partners (UNFPA, WHO); and relevant NGO partners (Comprehensive Community-based Rehabilitation in Tanzania). USAID/Tanzania will provide a list of interviewees and key stakeholders prior to the assignment’s inception.  Conduct a debriefing with the HPO, HIV, and PMI teams upon completing in-country work and prior to departing Tanzania. The team should present preliminary findings and get input before completing the report.  Analyses in the report should include, but are not limited to, - Quantitative comparison of achievements vs. targets for key output indicators to date - Assessment of training and QI programs and tools based on evidence collected during field visits and interviews Illustrative Tasks and Level of Effort TASK Team Leader (Int’l) International Team Member Local Technical Specialists (2) Document Review (prior to arrival for international consultants) 5 days 5 days 5 days Travel to Tanzania (international team members) 2 days 2 days - Team Technical Planning Meeting (TPM) including Meeting with HPO and introductory meeting with USAID Mission Director and senior management 2 days 2 days 2 days Meetings with key partners in and around Dar es Salaam 3 days 3 days 3 days Field Visits- data collection 16 days 16 days 16 days In-country travel days 3 days 3 days 3 days Preparation for and presentation of mid-point debrief to HPO 1 day 1 day 1 day Data analysis/team discussion; Write draft report 4.5 days 4.5 days 4.5 days Debrief MAISHA .5 day .5 day .5 day Debrief USAID/Tanzania and HPO and partners 1 day 1 day 1 day Complete draft report/ incorporate comments from debrief in report/ submit draft report prior to departure 2 days 2 days 2 days Depart Tanzania (international team members) 2 days 2 days - HPO will have one week to review the draft report and return it to the team leader with comments - - - Team members review HPO’s comments and provide feedback to the Team Leader/ Team Leader finalizes report, submits to USAID/Tanzania 5 days 3 days 3 days 54 TOTAL LOE 47 days 45 days 41 days A six-day work week is authorized during the consultancy. IX. Team Composition The mission will seek the services of a four person consultant team: one expatriate team leader, one expatriate team member, and two local technical consultant team members. All team members should have the following characteristics:  Master’s degree or higher level of education in a relevant technical area  Knowledge, skills, and experience with USAID contracting and reporting requirements; policies and initiatives; and tools, such as performance monitoring plans (PMPs) and results frameworks  Advanced written and oral communications skills in English  Expertise working in developing countries with decentralized health systems  Strong quantitative and qualitative analysis skills Additionally, the team members should together include the following individual levels of expertise:  Team Leader/ MCH Expert (International): One person with a minimum of 10 years of experience in public health, with technical knowledge and experience with maternal health, reproductive health, and newborn health programs implemented at scale. The team leader will be identified by USAID prior to the start of evaluation activities. The team leader will be responsible for (1) managing the team’s activities, (2) ensuring that all deliverables are met in a timely manner, (3) serving as the primary liaison with the Mission teams, and (4) leading briefings and presentations. In addition the team leader must have these characteristics:  Excellent skills in planning, facilitation, and consensus building;  Demonstrated experience leading an evaluation team;  Excellent interpersonal skills;  Excellent organizational skills and ability to keep to a timeline.  2nd MCH Expert (Local): A second person with a minimum of 8 years of experience in maternal and child health, with extensive technical knowledge and experience with maternal health, reproductive health, and newborn health programs implemented at scale. This should include extensive technical knowledge and experience with interventions, policies and programs in maternal and child health.  Project Management/ Monitoring and Evaluation Expert (International): One person with relevant education and at least 5 years of experience in USAID project and organizational management. This person should have a minimum of 10 years of experience in designing, implementing, and managing international health programs at scale. This person should also have strong knowledge, skills and experience in qualitative and quantitative evaluation tools.  Health Systems Strengthening/ Capacity Building Expert (Local): One person with knowledge and at least 10 years of experience in quality management, quality improvement, pre-service training, and capacity building in the health sector. NOTE: At least two of the technical team members should be local consultant with at least 10 years of experience designing, implementing, and evaluation MNCH programs in Tanzania. MAISHA will provide a Local Logistics Coordinator for handling the travel related logistics and providing administrative support to the technical team members. The Logistics Coordinator will also be responsible for setting up meetings with USAID and stakeholders. MAISHA Midterm Evaluation Report 55 X. Relationships and Responsibilities The HPO team will organize a tentative schedule of meetings and field visits before the evaluation team arrives, in collaboration with the local logistics coordinator. HPO will facilitate and coordinate field visits with local provincial and district officials, where necessary. The local logistics coordinator will coordinate all other interviews. The schedule will be finalized by the team in the TPM during their first days of the consultancy in coordination with HPO. Selected HPO staff will accompany the evaluation team, as required. The evaluation Team Leader may ask the USAID staff person(s) to recuse themselves from some key informant interviews when the topic is USAID program management or other sensitive administrative topics. HPO will send letters of introduction informing key MOHSW staff and partners of the nature, timing, and scope of the evaluation and of the evaluation team members. GH Tech will be responsible for arranging assignment logistics, including electronic country clearance requests, international travel, vehicle rental, and lodging arrangements. Client Roles and Responsibilities: Before In-Country Work 1. Documents. Identify and prioritize background materials for the consultants and provide them, preferably in electronic form. 2. Local Consultants. Assist with identification of potential local consultants and provide contact information. 3. Site Visit Preparations. Provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line items costs. Missions can protect scarce budgets by using their in-country knowledge to suggest the travel calendar (i.e. number of in-country travel days required to reach each destination, and number of days allocated to interviews at each site). 4. Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in￾country travel (i.e., car rental companies and other means of transportation) and identify a person to assist with logistics (i.e., visa letters of invitation etc.) 5. USAID-Supplied Participants. If relevant, provide guidance regarding participation in the assignment by Mission and USAID/W staff (i.e., who will participate, how long, source of funding for their participation). During In-Country Work 6. Mission Point of Contact. Throughout the in-country work, ensure constant availability of the Mission Point of Contact person(s) and provide technical leadership and direction for the team’s work. 7. Meeting Arrangements. While local consultants typically will arrange meetings for contacts outside the Mission, support local consultant(s) in coordinating meetings with stakeholders. 8. Formal and Official Meetings. Assist the team in arranging key appointments with national and local government officials and accompany the team on these introductory interviews (especially important in high-level meetings). 9. Other Meetings. If appropriate, assist in identifying and helping to set up meetings with local professionals relevant to the assignment. 10. Facilitate Contacts with Partners. Introduce the team to project partners, local government officials and other stakeholders, and where applicable and appropriate, prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. After In-Country Work 56 11. Timely Reviews. Provide timely review of draft/final reports and approval of the deliverables. XI. Deliverable & Report Timeline 1. Evaluation Framework: Present USAID with the framework and work plan for the evaluation on Day 4 of the in country work for this assignment. This will include the materials produced during the Team Planning Meeting described in Section VII above. HPO staff will be available for consultation during the TPM. 2. Debriefings: The team will conduct one mid-point and two final debriefings. The mid￾point debriefing will take place after the field visits to discuss preliminary findings and report outline with USAID. The first final debriefing will be with USAID and the second will be with HPO and the four partners. The debriefings should present key findings and recommendations in a PowerPoint format. 3. Draft Report: The first draft of the final evaluation report will be due electronically prior to the Team’s departure from Tanzania. The draft report will include key findings and recommendations for mission review. The report will present findings, conclusions/lessons learned and recommendations for the four project(s) and program components evaluated. USAID will provide one set of written comments on the draft report within 5 working days of receiving the document. 4. Final Report: The Team will submit a final report that incorporates responses to Mission comments and suggestions no later than five working days after USAID/Tanzania provides written comments on the Team’s draft evaluation report (see above). This report should not exceed 30 pages in length (not including appendices, lists of contacts, etc.). The format will include an executive summary, table of contents, glossary, methodology, findings, recommendations useful towards development of follow-on programming, and conclusions. The final report will be submitted in English, electronically, and then disseminated within USAID/Tanzania for final approval. All procurement sensitive information will be removed from the final report and submitted to as an Internal USAID Memo which will not be distributed outside the Mission. NOTE: GH Tech will be unable to complete final editing/formatting and preparation of a 508 compliant USAID branded version of the public report due to project closing on December 22, 2011. The final report shall be direct and brief but still provide a sufficiently detailed presentation. Appendixes shall be included with the final report. The report should be structured as follows:  Executive summary  Introduction – purpose, audience, summary of statement of work  Background – a brief overview of MOHSW, infectious diseases and health issues in Tanzania, including the main trends and challenges to increasing service uptake, to improving quality, and to expanding access to essential services in MOHSW /MNCH  USAID assistance to date - a description of the USAID program strategy and activities implemented in response to the problem (coverage, implementing partners, funding levels) as well as strategic directions  Methodology  Findings/Conclusions/Recommendations - based on the objectives and questions outlined in Sections V and VI above, and presented across each of the technical domains, these should include but not necessarily be limited to: - Key findings and issues identified in management and implementation of the projects; MAISHA Midterm Evaluation Report 57 - Project achievements; - Project gaps, issues and areas needing improvement; - Recommendations for improving implementation of MOHSW (national, regional and district) as well as health system strengthening activities (supervision, QI/QA, pre-service training) - Recommendations towards future maternal newborn health interventions that will take up from and build on the MAISHA program  Appendixes - List of persons contacted - Activity timeline / schedule - Bibliography - Other, as needed XIII. MISSION AND/OR WASHINGTON CONTACT PEOPLE/PERSON The Evaluation Team will work under the direction and guidance of USAID/Tanzania Senior Maternal Child Health Advisor, Dr. Ráz Stevenson, and in collaboration with the HPO team. Ráz Stevenson, MD MPH Senior Maternal Child Health Advisor Health and Population Office USAID Tanzania 58 APPENDIX B. CUMULATIVE TRAINING MAISHA Achievements Year 2008/09 Year 2009/10 Year 2010/11 Providers trained 30 (Mainland), 0 (Zanzibar) 116 (Mainland), 34 (Zanzibar) 255 (Mainland), 15 (Zanzibar) Cumulative total providers trained 30 (Mainland), 0 (Zanzibar) 146 (Mainland) 34 (Zanzibar) 401 (Mainland), 49 (Zanzibar) Supervisors trained 0 (Mainland) 140 (Mainland) 12 (Zanzibar) 420 (Mainland) 25 (Zanzibar) Cumulative total supervisors trained 0(Mainland), 0 (Zanzibar) 140 (Mainland) 12 (Zanzibar) 360 (Mainland) 37 (Zanzibar) # facilities covered 27 (Mainland), 0 (Zanzibar) 71 (Mainland), 9 (Zanzibar) 132 (Mainland), 0 (Zanzibar) Cumulative total facilities covered 27 (Mainland), 0 (Zanzibar) 98 (Mainland), 9 (Zanzibar) 230 (Mainland), 9 (Zanzibar) ACCESS/MAISHA Combined Achievements ACCESS 2004- 2008 MAISHA Up to Oct 2011 Cumulative total providers trained 2971 7,157 (6,959 Mainland; 198 Znz) Proportion of estimated total ANC providers (6,000) 49.5% 115% Mainland Cumulative total facilities covered 1448 3,521 Proportion of estimated total ANC facilities (4,745 – Mainland and Zanzibar) 31% 74% MAISHA Midterm Evaluation Report 59 APPENDIX C. SUPPLIES/EQUIPMENT- SITE VISITS Regions Visited FANC Unguja Pemba Iringa Morogoro Mtwara Total (n=15) Iron/Iron Folate Tablets 3/4 2/3 3/3 1/1 4/4 13/15 Sulfadoxine pyrimethamine (SP) 1/4 0/3 3/3 1/1 3/4 8/15 Mebendazole/ Albendazole 4/4 3/3 3/3 1/1 4/4 15/15 Syphilis test kits 0/4 0/3 2/3 1/1 4/4 7/15 HIV test kits 0/4 3/3 3/3 1/1 3/4 10/15 Urine dip stick 3/4 2/3* 1/3 0/1 3/4 9/15 Hb measure (i.e. hemaccue) 0/4 3/3* 2/3 1/1 4/4 10/15 Blood Pressure Cuff 4/4 3/3 3/3 1/1 4/4 15/15 *services/supplies available for a fee Regions Visited BEmONC Unguja Pemba Iringa Morogoro Mtwara Total (n=15) Uterotonics (any available) Oxytocin Misoprostol Ergometrine 4/4 2/4 0/4 3/4 3/3 0/3 3/3 0/3 2/2 2/2 2/2 0/2 2/2 2/2 2/2 0/2 4/4 3/4 1/4 1/4 15/15 9/15 8/15 4/15 MgSO4 4/4 3/3 2/2 2/2 4/4 15/15 Partographs 4/4 3/3 2/2 2/2 4/4 15/15 BP machine 4/4 3/3 2/2 2/2 4/4 15/15 Newborn resuscitation equipment 4/4 3/3 1/2 2/2 4/4 14/15 60 Service Jan – Mar 10 Apr – June 10 Jul – Sept 10 Oct – Dec 10 # ANC Clients % ANC Clients # ANC Clients % ANC Clients # ANC Clients % ANC Clients # ANC Clients % ANC Clients IPTp 1 8,266 55% 6,922 57% 6,774 53% 8,466 55% IPTp 2 4,154 28% 5,509 45% 6,046 48% 5,894 38% ITN vouchers 9,042 60% 6,250 51% 8,971 71% 7,125 46% MAISHA Midterm Evaluation Report 61 APPENDIX D. PERSONS CONTACTED Date Meetings Organization 1. Jhpiego-MAISHA 10/10/2011- 10/14/2011 Maryjane Lacoste, Chief of Party (MAISHA), Country Director Jhpiego Natalie Hendlen, Senior Advisor, CECAP, IPC Rose Mnzava, QI Advisor Aprila Them, Pre-service Marya Plotkin, Monitoring and Evaluation Muthoni Magu-Kariuki, Senior Program Officer 2. Partner Presentations 10/12/2011 Joel Lubebe, Program Coordinator IMA World Health Veronica Mukusa, Program Coordinator IMA World Health Jenifer Orkis, Program Manager TCCP Ali Ali, Program Officer TCCP Rachel Makunde, Program Manager Save the Children Rose Mlay, Country Manager White Ribbon Alliance/Tanzania 3. USAID 10/11/2011 Dr. Raz Stevenson, Senior MNC Advisor USAID-Health and Population Office Dr. Jessica Kafuko, Senior Malaria Advisor USAID-President’s Malaria Initiative Dr. Jema Bisimba, Public Health Specialist, Pediatric AIDS USAID-PEPFAR Moses Besigye, Monitoring and Evaluation USAID 4. Field interviews 16/10/11 Ahmada Omar, Assistant Project Officer Jhpiego-Pemba 17/10/11 Sharifa Hamud Rashid, Zonal RCH Coordinator MoH-Pemba 17/10/11 Dr. Rashid D. Mkasa, DMO MoH-Micheweni 17/10/11 Pili Juma, District RCH Coordinator MoH-Micheweni 17/10/11 Suleman Saidi, District Health Officer MoH-Micheweni 17/10/11 Saidi Mmanga-District Administrative Officer MoH-Micheweni 17/10/11 Khalfan Maffar Nassor-District Materials Officer MoH-Micheweni 17/10/11 Faki Ismail Baker-District Surveillance Officer MoH-Micheweni 17/10/11 Dr. Ahmadi Omadi Ahmadi-Hospital I/C Micheweni Cottage Hospital 62 Date Meetings Organization 17/10/11 Hamadi Masood-Laboratory Tech Micheweni Cottage Hospital 17/10/11 Fathu Mohammed-Senior Nurse/MW Micheweni Cottage Hospital 17/10/11 Fatma Shazzal-DMO MoH-Wete 17/10/11 Mena Khatis-District RCH Coordinator MoH-Wete 17/10/11 Ali Rashid-District PH Officer MoH-Wete 17/10/11 Ali Bakari-Zonal Materials Officer MoH-Wete 17/10/11 Dr. Kulthun Abazar-Medical Director, OB/GYN Wete Hospital 17/10/11 Bikombo Abdalla-Nurse I/C Wete Hospital 17/10/11 Subira Ali-Nurse/MW Wete Hospital 17/10/11 Said Self-Laboratory Officer Wete Hospital 17/10/11 Ali Baker-Pharmacist Wete Hospital 17/10/11 Tahiya Sulum-RCH I/C Wete Hospital 17/10/11 Dr. Sauda Kassim-DMO MoH-Chake 17/10/11 Dr. Yussuf H. Iddi-Hospital I/C Chake Hospital 17/10/11 Seif Alawi-Nursing Officer (Patron) Chake Hospital 17/10/11 Fatia Hiya M. Khatib-Maternity I/C Chake Hospital 17/10/11 Dr. Sauda S. Alli-AMO-Maternity Chake Hospital 18/10/11 Dr. Fahki Yussuf Hamadi-Zonal Medical Officer MoH-Chake 18/10/11 Asma Ramadhan, Project Officer Jhpiego-Zanzibar 19/10/11 Valeria Rashid Haroub-Matron Mnazimoja Hospital 19/10/11 Mbiki Iddi-ANC I/C Mnazimoja Hospital 19/10/11 Sister Tatu Fundi Salim-Maternity I/C Mnazimoja Hospital 19/10/11 Raya Zaba Ran-Data (Maternity) Mnazimoja Hospital 19/10/11 Sister Razina Said-RCH I/C Mwembeladu Health Center 19/10/11 Zainab Jaddi-Nurse/MW Mwembeladu Health Center 19/10/11 Dr. Hanuni Hogora-RCH Program Manager MoH-Ugunja 20/10/11 Abbas T. Makame-District Health Admin Officer MoH-Kivunge 20/10/11 Hadji Jabi R.-District Environmental Officer MoH-Kivunge 20/10/11 Ghamina Shehe-Facility I/C Makunduchi PHC Center 20/10/11 Zainab M. Othman- HIP-Z and Hospital Manager Makunduchi PHC Center MAISHA Midterm Evaluation Report 63 Date Meetings Organization 20/10/11 Dr. Amour Muhsin Burhan-AMO and DMO (acting) Makunduchi PHC Center 20/10/11 Khalfa Makame-Clinical Officer Makunduchi PHC Center 22/10/11 Dr. Koheleth Winani- National Safe Motherhood Coordinator MoHSW, DSM 23/10/11 Hilda Nyerembe, Maternal and Newborn Health Advisor Jhpiego, Dar es Salaam 24/10/11 Dr. Joseph Tatuba-DMO MoHSW-Mafindi, Iringa 24/10/11 Rose Nyanjali-CECAP MoHSW-Mafindi, Iringa 24/10/11 Moses Tawete-RCH Coordinator (acting) MoHSW-Mafindi, Iringa 24/10/11 Msofe Christopher Joseph-District Pharmacist MoHSW-Mafindi, Iringa 24/10/11 Dr. Victor Kalinga-AMO, HC I/C MoHSW-Kasanga, Iringa 25/10/11 Dr. Gabone Oscar-RMO (acting) MoHSW-Iringa 25/10/11 Miriam Mohammed-Regional RCH Coordinator MoHSW-Iringa 25/10/11 Samania Manangwa-Nurse I/C Iringa Hospital 25/10/11 Isabela Kayage-I/C Ngome Health Center 25/10/11 Sezaria Andrew-Municipal RCH Coordinator Iringa Urban 25/10/11 Dr. John S. Mosha-USAID Focal Person PHC Institute-Iringa 25/10/11 Dr. Haji SS Shemhilu-Public Health Specialist PHC Institute-Iringa 25/10/11 Dr. Ezekiel Simon Mabwai-Regional Program Manager Engender Health-Iringa 26/10/11 Meshak Lubeleje-Nurse Tutor Maranta Nursing School￾Ifakara 26/10/11 Mary Magome-Principal Maranta Nursing School￾Ifakara 27/10/11 Dr. Mokiti-RMO MoHSW-Morogoro 27/10/11 Carl Lajime-RHO MoHSW-Morogoro 27/10/11 John Makimbila-Regional Health Secretary MoHSW-Morogoro 27/10/11 Dr. Mtole Mwakibete-Medical Officer/CECAP MoHSW-Morogoro 27/10/11 Margaret Wapalila-Regional RCH Coordinator MoHSW-Morogoro 27/10/11 Dr. Abraham Makizo-Municipal Medical Officer I/C MoHSW-Morogoro 27/10/11 Immaculata Mhagame-Municipal RCH Coordinator MoHSW-Morogoro 28/10/11 Anastasia Melis-Program Manager CCBRT, Dar es Salaam 30/10/11 Dr. Daimon Issack, Regional Program Officer Jhpiego-Mtwara 64 Date Meetings Organization 31/10/11 Dr. Moloe Gregory-Deputy RMO/Hospital In-Charge MoHSW-Mtwara 31/10/11 Ahmed Chibwana-Regional Nursing Officer MoHSW-Mtwara 31/10/11 Vivian Kilimba-Regional RCH Coordinator MoHSW-Mtwara 31/10/11 Musa Nassor-Regional Pharmacist MoHSW-Mtwara 31/10/11 Dr. Ndungumu Eligions-Municipal Health Officer MoHSW-Mtwara 31/10/11 Veronica Kamwenda-Municipal Nursing Officer MoHSW-Mtwara 31/10/11 Deogratious Dotto-District AIDS Coordinator Mtwara Municipal 1/11/11 Dr. Harry Mwanibe-DMO (acting)/MO In￾Charge MoHSW-Mkamaindo 1/11/11 Swaleh Christa-District RCH Coordinator MoHSW-Mkamaindo 1/11/11 Omar Jongo-District Pharmacist MoHSW 1/11/11 Said Mfaume-District Health Officer MoHSW 1/11/11 Mohammed Bakani-District Laboratory Tech MoHSW 1/11/11 Sister Zenonia-Clinical Officer I/C Lukuledi Dispensary 1/11/11 Dorothy Chikawe-Nurse/MW Lukuledi Dispensary 3/11/11 Lillian Andrews- Reproductive Health Care Specialist ICAP, Dar es Salaam 3/11/11 Dr. Milembe Panya- PMTCT/Pediatric Technical Advisor ICAP 4/11/11 Alisa Mautner Cameron- Team Leader, Office of Health and Population USAID, Dar es Salaam 4/11/11 Tumaini Mikindo- Country Program Manager AIDS Relief, Dar es Salaam 4/11/11 Dr. Sekela Mwakyusa- Country Director, AIDS Relief, Dar es Salaam 4/11/11 Euphrasia Telwa- Program Officer AIDS Relief 4/11/11 Dr. Festus Ilako Country Director AMREF, Dar es Salaam 7/11/11 Dr. Neema Rusibamayila, Director, RCH Services MoHSW, Dar es Salaam 8/11/11 Patrick Swai, PEPFAR USAID, Dar es Salaam 8/11/11 Michael Mushi, Office of Health and Population USAID, Dar es Salaam APPENDIX E. REFERENCES Document Source 1. Government of Tanzania Reference Documents Health Sector Strategic Plan (III) MoHSW (2009) “One Plan” - The National Road Map Strategic Plan to Accelerate Reduction of Maternal, Newborn and Child Deaths in Tanzania (2008-2015) Reproductive and Child Health Section, MoHSW (April 2008) Roadmap to accelerate the reduction of maternal, newborn and child mortality in Zanzibar (2008-2015) MoHSW/Zanzibar (draft) Situation Analysis of Newborn Health in Tanzania MoHSW (2009) Human Resources for Health Strategic Plan, 2008-2013 MoHSW (2008) National Supervision Guidelines for Quality Health Care Services MoHSW (2008) Kangaroo Mother Care Guidelines MoHSW MNCH TWG – Terms of Reference MoHSW (2009) Controller and Auditor General (CAG) Performance Audit on District Level Health Services United Republic of Tanzania (March 2010) Tanzanian Demographic and Health Survey 2010 National Bureau of Statistics (April 2011) Tanzania Service Provision Assessment Survey 2006 National Bureau of Statistics (Nov 2007) National Supportive Supervision Guidelines for Quality Healthcare Services MoHSW, (2010) National Postpartum Care Guidelines MoHSW, Reproductive and Child Health Section Guidelines for Continuous Quality Improvement in Health Training Institutions MoHSW (June 2010) PMTCT National Guidelines-3 rd Edition MoHSW (July 2011) 2. Development Partner Reference Documents Health Basket Fund Update 2010 Irish Aid (2010) WHO Country Strategy for Tanzania WHO (2010) 3. NGO/FBO Reference Documents Maternal Health Partner Mapping Jhpiego (September 2010) Fair’s Fair – Health Inequities in Tanzania Ifakara / Women’s Dignity Project (2006) 4. MAISHA Key Activity Documents MAISHA Project Proposal Jhpiego (2008) Annual Workplan Year 1-July 2009 Jhpiego (2009) Annual Workplan Year 1-Annex 2 rev. June 2009 Jhpiego (2009) Annual Workplan Year 2 rev. April 2010 Jhpiego (2010) Annual Workplan Year 3 rev. 4 Jhpiego (2010) MAISHA Monitoring and Evaluation Plan Jhpiego (2009/10) MAISHA Quarterly Reports 2008 Jhpiego (2008) MAISHA Quarterly Reports 2009 Jhpiego (2009) MAISHA Quarterly Reports 2010 Jhpiego (2010) 66 Document Source MAISHA Quarterly Reports 2011 Jhpiego (2011) Annual report Year 1 Jhpiego (2009) Annual report Year 2 Jhpiego (2010) Mid-year report Year 3 Jhpiego (2011) Audit of USAID/Tanzania’s Ongoing Activities Under The President’s Malaria Initiative Office of Inspector General (April 2011) MAISHA Indicator Report FY 2009 Jhpiego (2009) MAISHA Indicator Report FY 2010 Jhpiego (2010) MAISHA Indicator Report FY 2011 Jhpiego (2011) Partner Mapping 21 October 2011 Jhpiego (2011) MAISHA CS-PMI-PEPFAR Quarterly Trend Analysis￾rev. Jhpiego (Oct. 2011) Standards Based Management and Recognition Guidelines for Hospitals Jhpiego Standards Based Management and Recognition Guidelines for Health Centers and Dispensaries Jhpiego 5. Tanzania MNCH Research Documents Literature Review of Tanzanian Specific Literature on Maternal Mortality and Morbidity Sheriff, Ismat (2005) Community and health system factors associated with facility delivery in rural Tanzania: A Multilevel analysis Kruk M, Rockers P, Mbaruka G et. al. (May, 2010) Use pattern of maternal health services and determinants of skilled care during delivery in southern Tanzania: Implications for achievement of MDG-5 targets Mpembeni RN, Kilewo, JZ, Massawe SN, et. al. (2007) Making EmOC a reality – CARE’s experiences in areas of high maternal mortality in Africa Kayongo M, Rubardt M, Butera J et. al. (2005) Bypassing primary care facilities for childbirth: a population-based study in rural Tanzania Kruk ME, Mbaruku G, McCord CW et. al. (2009) Women’s Preferences for Place of Delivery in Rural Tanzania: A Population-Based Discrete Choice Experiment Kruk ME, Paczkowski M, Mbaruku G et. al. (2009) An assessment of health facilities maternal delivery services in Mtwara Rural and Masasi Districts in Mtwara Region Kilima, P, EGPAF (2007) Enhancing survival of mothers and their newborns in Tanzania Mbaruku (2005) Dar es Salaam Perinatal Care Study: Needs assessment for quality of care Nyamtema A, Urassa DP, Massawe S et al (2008) Tanzania Global Health Initiative Strategy 2010-2015 USAID/Tanzania, 2011 Audit of USAID/Tanzania’s Ongoing Activities Under The President’s Malaria Initiative Office of Inspector General (April 2011) Literature Review of Tanzanian Specific Literature on Maternal Mortality and Morbidity Sheriff, Ismat (2005) Document Source Community and health system factors associated with facility delivery in rural Tanzania: A Multilevel analysis Kruk M, Rockers P, Mbaruka G et. al. (May 2010) Use pattern of maternal health services and determinants of skilled care during delivery in southern Tanzania: Implications for achievement of MDG-5 targets Mpembeni RN, Kilewo, JZ, Massawe SN et. al. (2007) Making EmOC a reality – CARE’s experiences in areas of high maternal mortality in Africa Kayongo M, Rubardt M, Butera J et. al. (2005) Bypassing primary care facilities for childbirth: a population-based study in rural Tanzania Kruk ME, Mbaruku G, McCord CW et. al. (2009) Women’s Preferences for Place of Delivery in Rural Tanzania: A Population-Based Discrete Choice Experiment Kruk ME, Paczkowski M, Mbaruku G et. al. (2009) An assessment of health facilities maternal delivery services in Mtwara Rural and Masasi Districts in Mtwara Region Kilima P, EGPAF (2007) Enhancing survival of mothers and their newborns in Tanzania Mbaruku (2005) Dar es Salaam Perinatal Care Study: Needs assessment for quality of care Nyamtema A, Urassa DP, Massawe SN et. al. (2008) Where did they go? Tracking Human Resources for Health from Pre-service Nursing – Midwifery Graduates in Tanzania, April 2011 Jhpiego (April, 2011) Quality of Maternal and Newborn Health Services in Tanzania, Report 1: Findings on Antenatal Care Jhpiego (2010) Quality of Care for Prevention and Management of Common Maternal and Newborn Complications, Report 2: Findings on Labour, Delivery and Newborn Care Jhpiego (2010) PMTCT “Plus Up” Partner Guidance USAID 68 APPENDIX F. JHPEIGO ORGANIZATIONAL CHART MAISHA Midterm Evaluation Report 69 APPENDIX G. PROJECT MONITORING PLAN INDICATORS AND PROGRESS TO END OF YEAR 3 Summary indicator progress MAISHA Program Reported indicators - FY09, FY10 and FY11 (as of June 11) Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 1 Percent/number of assessed facilities meeting clinical performance standards for safe delivery (BEmONC) 6% of facilities provided BEmONC signal functions (2006 TSPA) 25% 0% 35% 0% 45% 0% 75% From QI assessment data 2 Percent/number of women delivering in USG￾assisted health facilities receiving AMTSL by skilled birth attendants 7% (2006 POPPHI study) N/A N/A 10% 8.20% N/A N/A 70% From clinical observations (2 times over LoP) 3 Percent/number of USG￾assisted health facilities experiencing stock-outs of specific tracer drugs (BEmONC) Uteronic stockouts at 87- 97%, mag sulph stockouts at 94% (2006 NIMR EmONC report) 85% Oxytocin=47%; Ergometrine=16%; Misoprostol=63%; MgSO4=56% 60% Oxytocin=73%; Ergometrine=46%; Misoprostol=91%; MgSO4=74% 45% Oxytocin=44%; Ergometrine=56%; Misoprostol=75%; MgSO4=47% 10% From sentinel site data 70 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 4 Case fatality rate (maternal health) in MAISHA-supported health facilities 3.3% (2006 NIMR EmONC report) 3% 1.58% 2.50% 1.40% 2.50% 2.00% 1.00% From sentinel site data (reported annually) 5 Percent/number of people trained in malaria treatment or prevention with USG funds 2,988/50% (mainland, as of 30 Sept 08) 875 - mainland; 100 - Zanzibar 1,425 - mainland; 123 - Zanzibar 1300 - mainland; 50 - Zanzibar 1224 - mainland; 75 - Zanzibar 1950 - mainland; 0 - Zanzibar 1,317 - mainland; 0 - Zanzibar Please see revised target table (from year 3 workplan) Using denominator of 6,000 ANC providers (for mainland) until further clarification provided by RCHS; 147 is estimated denominator for ZNZ 6 Percent/number of facilities meeting clinical performance standards for FANC N/A 30% 10% 50% 50% 65% 80% 95% From QI assessments 7 Percent/number of ANC clients who received a syphilis test at USG￾assisted health facilities 66% (Oct-Dec08 sentinel site data) 70% 71% 75% 61% 80% 63% 90% From sentinel site data 8 Percent/number of ANC clients testing positive for syphilis who have received appropriate treatment at MAISHA-supported health facilities 100% (Oct-Dec08 sentinel site data) 100% 87% 100% 97% 100% 94% 100% From sentinel site data MAISHA Midterm Evaluation Report 71 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 9 Percent/number of ANC clients at MAISHA￾supported health facilities who received at least 90 tablets of iron over course of ANC visits N/A 50% 49% 55% 74% 60% 66% 80% From sentinel site data (denominator is 1st ANC visit clients, not all ANC clients) 10 Percent/number of ANC clients at MAISHA￾supported health facilities who received one dose of mebendazole or albendazole 63% (Oct-Dec08 sentinel site data) 65% 65% 70% 71% 75% 75% 85% From sentinel site data 11 Percent/number of ANC clients at MAISHA￾supported health facilities who received 2nd dose of IPT under DOT 54% (Oct 07-Sept 08 sentinel site data) 60% 38% 85% 40% 85% 37% 90% From sentinel site data; each quarter, MAISHA will report quarterly data as well as compiled data for the previous 4 quarters 12 Percent/number of ANC clients at MAISHA￾supported health facilities who received at least 2 injections of tetanus toxoid 69% (Oct-Dec08 sentinel site data) 70% 62% 85% 65% 85% 69% 90% From sentinel site data 72 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 13 Percent/number of USG￾assisted health facilities experiencing stock-outs of specific tracer drugs (FANC) 32% for SP ; 22% for RPR kits; no data for mebendazole/albendazole (Oct-Dec08 sentinel site data) 30% SP=49%; RPR=49%; Meb/alb=27% ; Iron=59% 25% SP=80%; RPR=75%; Meb/alb=55% ; Iron=88% 20% SP=73%; RPR=63%; Meb/alb=28% ; Iron=68% 10% From sentinel site data (this number represents facilities which have had any stock out, even 1 day) 14 Percent/number of pre￾service education nursing/midwifery graduates with sustained clinical skills in maternal and neonatal service provision N/A N/A N/A N/A N/A N/A N/A 75% To be reported in Year 4 15 Percent/number of newborns put to the breast within one hour of birth at USG-assisted health facilities 59% (2004/05 DHS) N/A N/A 60% 47.20% N/A N/A 80% From clinical observations (2 times over LoP) 16 Percent/number of newborns dried and wrapped immediately after birth at MAISHA￾supported health facilities N/A N/A N/A 20% 86.90% N/A N/A 80% From clinical observations (2 times over LoP) 17 Percent/number of newborns that received clean cord care at USG￾assisted health facilities N/A N/A N/A 20% 69.70% N/A N/A 80% From clinical observations (2 times over LoP) MAISHA Midterm Evaluation Report 73 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 18 Number of people trained in maternal and/or newborn health and nutrition care through USG-supported programs 875 (FANC main); 100 (FANC ZNZ); 105 (BEmONC); 15 (KMC) 1425 (FANC main); 123 (FANC ZNZ); 30 (BEmONC main); 35 (KMC) 1300 (FANC main); 50 (FANC ZNZ); 187 (BEmONC); 82(KMC) 1224 (FANC main); 75 (FANC ZNZ); 116 (BEmONC main); 34 (BEmoNC ZnZ); 135 (KMC) 1950 (FANC main); 0 (FANC ZNZ); 300 (BEmONC main); 48 (BEmONC ZNZ); 74 (KMC main); 10 (KMC ZNZ) 1,317 (FANC main); 0 (FANC ZNZ); 270(BEmONC 255 main,15 ZnZ); 194(KMC all Main) Please see revised target table (from year 3 workplan) 19 Percent/number of USG￾assisted facilities achieving at least 80% of clinical performance standards for essential newborn care (ENC)/resuscitation N/A 25% 0% 40% 0% 60% 0% 90% From QI assessments 20 Number of KMC units established 2 1 12 8 22 16 22 To be reported annually; 22 is for 21 regions of mainland plus Zanzibar, but SNL supports 4 of those regions - so MAISHA should be responsible only for 18 74 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 21 Number of individuals reached with MNH messages through media activities and community outreach 2 million 2.8 million 3 million 2.8 million 3.8 million 0 5 million To be reported bi￾annually; this was a T￾MARC indicator, should now be deleted as TCCP is covering under their project 22 Number of individuals reached with MNH/PMTCT messages through outreach activities N/A N/A 1,000 0 2000 0 5,000 HIV/AIDS Office indicator; to be reported under ACCESS/FP in Year 1 and then under MAISHA in subsequent years 23 Percent/number of regional RCH coordinators who performed standardized supervision visits in each district in the region at least once in the previous year with USG support N/A 40% 0% 25% 0% 30% 75% (12 of 16 regions) 80% Program reports MAISHA Midterm Evaluation Report 75 Indicator Baseline Year 1 target Year 1 actual Year 2 target Year 2 actual Year 3 target Year 3 actual EOP targets Comments 24 Percent/number of districts including line item for MHN activities/supplies in annual comprehensive council health plan (CCHP) budgets N/A 10% Data not yet available 15% Data not yet available 25% Data not yet available 75% To be reported annually; MAISHA attempted to conduct an exercise with MoHSW in Year 3 to determine status of this indicator, but was told MoHSW would do the exercise themselves and provide feedback when completed Note: OP indicators highlighted in blue 76 APPENDIX H. POTENTIAL INDICATOR CHANGES WITH REGARD TO SBM-R UNDER MAISHA Indicator Definition Source Frequency Number of MAISHA-supported facilities demonstrating increased overall compliance with clinical standards over baseline Number of MAISHA-supported facilities hospitals that are implementing standards-based management and recognition that demonstrate increased compliance with standards over baseline. Denominator is number of facilities for which assessment data is available during the program year. Numerator is number of facilities that showed increase in compliance with standards as compared to baseline. This will be disaggregated by type of facility (e.g., regional hospital and health center/dispensary) Quality improvement assessment tool, supervision visit reports Annual For year 3, the results would be:  10/11 for regional hospitals (91%)  12/19 for health centers/dispensaries (63%) Number of MAISHA-supported facilities demonstrating increased compliance with clinical standards for normal labor and delivery care over baseline Number of MAISHA-supported facilities hospitals that are implementing standards-based management and recognition that demonstrate increased compliance with standards for normal labor and delivery care over baseline. Denominator is number of facilities for which assessment data is available during the program year. Numerator is number of facilities that showed increase in compliance with standards as compared to baseline. This will be disaggregated by type of facility (e.g., regional hospital and health center/dispensary) Quality improvement assessment tool, supervision visit reports Annual MAISHA Midterm Evaluation Report 77 Indicator Definition Source Frequency For year 3, the results would be:  7/11 for regional hospitals (64%)  8/19 for health centers/dispensaries (42%) Number of MAISHA-supported facilities demonstrating increased compliance with clinical standards for managing complications of labor and delivery care over baseline Number of MAISHA-supported facilities hospitals that are implementing standards-based management and recognition that demonstrate increased compliance with standards for managing complications of labor and delivery care over baseline. Denominator is number of facilities for which assessment data is available during the program year. Numerator is number of facilities that showed increase in compliance with standards as compared to baseline. This will be disaggregated by type of facility (e.g., regional hospital and health center/dispensary) Quality improvement assessment tool, supervision visit reports Annual For year 3, the results would be:  8/11 for regional hospitals (73%)  10/19 for health centers/dispensaries (53%) 78 Indicator Definition Source Frequency Number of MAISHA-supported facilities demonstrating increased compliance with clinical standards for postnatal care over baseline Number of MAISHA-supported facilities hospitals that are implementing standards-based management and recognition that demonstrate increased compliance with standards for postnatal care over baseline. Denominator is number of facilities for which assessment data is available during the program year. Numerator is number of facilities that showed increase in compliance with standards as compared to baseline. This will be disaggregated by type of facility (e.g., regional hospital and health center/dispensary) Quality improvement assessment tool, supervision visit reports Annual For year 3, the results would be:  11/11 for regional hospitals (100%)  11/19 for health centers/dispensaries (58%) MAISHA Midterm Evaluation Report 79 APPENDIX I. ILLUSTRATIVE LIST OF ACTIVITIES TO BE SUPPORTED USING PMTCT PLUS UP FUNDING The Package of PMTCT services that Implementing Partners will support includes (but not limited to): • Scale-up of PMTCT services to 80% of clinics. These services include: – HIV testing (in ANC, LW), including partner testing and the transfer of mom’s information to the baby’s health card – Counseling on FP, infant feeding (IF) and referrals – Prevention with Positives (PwP) – Clinical staging, CD4 determination and linkage to CTC – Use of more effective regimen – Provision of ART to eligible pregnant women – Screening for opportunistic infections (OIs) including TB – Use of cotrim in prophylaxis and the management of OIs • Follow up of mother and infants pairs • Scale-up of EID to 50% of RCH clinics – Increase number of sites able to take DBS – Collect and transport DBS, get results back to the parent(s) – Link HIV+ babies and orphans and vulnerable children (OVC) to care and treatment, TB/HIV services and EID • Engage with community support groups such as Mom 2 Mom (M2M), psychosocial support groups, CHWs, child psychosocial support groups and community outreach • Train nurses, nurse midwives and other cadres in PMTCT, IF, MCH, EID, ART – Mentoring – Consider and support of health care workers retention package • Use expert patients to provide education, provide adherence counseling and carry out non-medical chores • Engage regions/district in dialogue and program to retain staff • Ensure guidelines and M & E tools are available, robust data collection system, electronic data base, monitor and track HIV+ mothers and babies • Ensure quality of service is improved through feedback mechanism 80 ILLUSTRATIVE LIST OF ACTIVITIES TO BE SUPPORTED USING PF FUNDING (11 POINT PF PACKAGE) The following is an illustrative list of the various components of a comprehensive MCH/PMTCT package that IP are expected to support. Because each district has a unique situation, IP will work closely with regional and district authorities and Reproductive Health partners to identify the specific bottlenecks/priorities in their respective areas - this is where PF funding will be focused. By addressing the entire system and a comprehensive package of quality RH services, women will be encouraged to utilize available services and uptake of PMTCT should increase. All effort should be made to improve efficiencies and avoid duplication of effort. (1) Facility infrastructure improvement • PMTCT Partner will carry out minor renovations of MCH and Labor wards • Create space for cervical cancer screening and equip these clinics • Ensure water and electricity (solar if necessary) • Improve staff living conditions (2) PMTCT-ART Integration [in base package – emphasis here on purchasing machines for health centers to ensure staging at point of contact] • Supporting Hospitals, Health Centers and Dispensaries that have capacity to provide ART, MECR (in liason with MOHSW)… – Training HCP in MCH in ART and pediatric HIV management – Ensure availability of guidelines and job aids – Provide CD4, Biochemistry, Hematology Machines – Ensure availability of M and E tools – Ensure availability of and supplement commodities and ARVs drugs and OI drugs (3) PMTCT partner will complement FP and Focused Antenatal Care (FANC) package Link/liaise with districts authorities and Reproductive Health programs/partners that support FP, FANC in their region and establish needs (this is an illustrative list; and please note: partners will not buy contraceptives) • Complement training in FP, FANC through national training of trainers (TOT) to roll out FP and FANC service in respective districts • Complement procurement of Tests and reagents: RPR Test kits and reagents; drugs: Iron, Folic Acid, Mebendazole, antiseptics, weighing scales, BP machine; (Malarial interventions) IPT prophylaxis, Insecticide-treated Nets (ITN), Indoor Residual Spraying (IRS) (4) PMTCT partner will complement Emergency Obstetric Care (EmOC) package (please note that this is an illustrative list) • Link/liaise with districts authorities and health programs that support EmOC and establish needs • Complement training in EmOC through national TOT to roll out EmOC services in respective districts • Complement procurement of Tests and reagents: for HB, estimation; drugs to control bleeding: (Misoprostol /oxytocin), Iv fluids, antiseptics, weighing scales, BP machine, pulse oximeter, delivery bed, delivery kit, mackintosh, gloves, scissors, forceps • Consider support in the facility of systems to provide blood and blood transfusion equipment (liaise with blood safety) MAISHA Midterm Evaluation Report 81 (5) PMTCT partner will complement Newborn Health package • Link/liaise with districts authorities and Reproductive Health programs/Partners that support Neonatal Health and establish needs • Complement training in newborn health through national TOT to roll out Neonatal Health training in respective districts • Complement procurement of Tests: HB, and reagents; drugs: antibiotics, IV fluids, Vitamin K, antiseptics, weighing scales, blood pressure machines, pulse oximeters, baby cots, mackintosh, gloves, blood and blood transfusion equipment (liaise with blood safety), suction machines, respirators, neonatal resuscitation equipment • Revisit/support Baby Friendly Health Facilities Initiatives (BHFI) (6) Additional activities • Set up cancer screening service • Renovations and construction – Partner will carry out minor renovation and they will work with District authorities to identify needs for construction and major renovations. For major constructions, they will engage RPSO e.g building, MCH, LW/D, maternity waiting homes, • Support staffing – Deploy staff retention initiatives, top up allowances, deployment of retired but not tired nurses and other critical staff • Support Public-Private Partnerships (PPP): – Work with USG to set-up EID sentinel sites and track HIV negative kids from the EID program – Support and engage more private facilities to engage in the provision of MCH services – Work with Local Government Authority (LGA) to second staff to Private (FBO) sector and vice versa based on needs – Work with Phones for health at facility level to report PMTCT data to district – Work with Mobile phone vendors engaged in communication between facility and community-based workers including TBAs (7) Strengthen monitoring and evaluation and BPE; document lessons learned (8) Carry out PMTCT costing studies • EID transportation of samples including DBS • In supporting food by prescription for severely malnourished patients using vouchers system (9) Community and demand creation 82 APPENDIX J. INTERVIEW INSTRUMENTS Regional Health Management Team (RHMT)  Regional Medical Officer  Regional RCH Coordinator  Regional Nursing Officer  Regional Pharmacist Date:_________________ Region:_________________________________ Time:_________________ Interviewer:_____________________________ Location:______________ Interviewee:_____________________________ 1. General Introduction of the Team and purpose of the Midterm evaluation 2. What are the key MAISHA activities supported in this region? (awareness, ownership) 3. How have the key MAISHA activities been integrated into the Regional Health Plan and the CCHPs? Has the region conducted any regional ToT activities (FANC, QI, BEmONC)? How have these trainings assisted in the “roll out” to other non-MAISHA facilities? Have other partners provided support to MNH activities (i.e. LSS, AMTSL training, etc.)? 4. What impact has the MAISHA project had on the provision of MNH services in the Region? Strengths Challenges 5. How has MAISHA assisted in addressing these challenges? a. How responsive has MAISHA been to the needs/requests of the Region? 6. Recommendations for MAISHA and future MNH programming? MAISHA Midterm Evaluation Report 83 District Health Center/Dispensary FANC  Health Facility I/C  Nurse Midwife Date:_________________ District/Region:__________________________ Time:_________________ Interviewer:_____________________________ Facility/Type: __________________ Interviewee:_____________________________ 1. General Introduction and purpose of the Midterm evaluation 2. What kind of support/interventions have been provided by the MAISHA project? 3. Provision of Services at the District level by this health facility Number/type of staff in ANC ___________________ FANC # trained _______________ # on site________________ Year of FANC training: ___________________ 4. Clinical services – FANC a. Guidelines/protocols available? _____ Yes _____No b. Evidence of IEC materials ____ Yes _____No c. Type (i.e. TV, posters, etc.) ____________________________ d. Area for physical exam? ______ Privacy? _________ e. Handwashing area? _____ Yes _____No f. Confidential counseling/testing area? _____ Yes ______No g. Emergency kit prepared? _____ Yes _____No h. Fee for service/cost share? _____ Yes ______No Drugs? Lab tests? Overall Comments (probe for changes in quality of services due to FANC training of MAISHA equipment/supplies; increase in service utilization of ANC): 5. FANC Critical Equipment and Supplies (observation) Comment(s) Fe Tablets (#/client) Sulfadoxine pyrimethamine Mebendazole/Albendazole Syphilis test kits HIV test kits Urine dip stick Hb measure BP Machine 84 District Health Center/Dispensary FANC  Health Facility I/C  Nurse Midwife 1. QI a. Have you used FANC QI Assessment tool? _____ Yes _____ No b. Role with BEmONC QI Team? c. Baseline BEmONC Assessment complete? ____ Yes ____ No d. Internal BEmONC Assessment: ______________ Number of assessments completed ______________ Month/Year of last assessment ______________ Score of most recent assessment e. How has the information from the internal assessment(s) been used? f. Example(s) of something that has changed because of the assessment(s). 2. Supportive Supervision a. Have you seen any changes in the way supervision is conducted since supportive supervision has been introduced? b. Who comes to supervise your work here? District _________________________ Region__________________________ MAISHA Staff ____________________ c. How often? _______________________ d. When was the last visit? __________________________ e. What Supervision tool/checklist did they use (SBM-R FANC; MoHSW checklist; other)? 3. What do you need to provide better ANC services? MAISHA Midterm Evaluation Report 85 Regional Hospital FANC  Hospital In-Charge  ANC I/C  Nurse Midwife Date:_________________ District/Region:__________________________ Time:_________________ Interviewer:_____________________________ Facility/Type: __________________ Interviewee:_____________________________ 1. General Introduction and purpose of the Midterm evaluation 2. What kind of support/interventions have been provided by the MAISHA project? 3. Staffing and training at RCH Clinic (staff and training) Number/type of staff in ANC ___________________ FANC # trained _______________ # on site________________ Year of FANC training: ___________________ 4. Clinical services – FANC i. Revised FANC Guidelines available? _____ Yes _____No j. Evidence of IEC materials ____ Yes _____No k. IEC Type (i.e. TV, posters, etc.) ____________________________ l. Area for physical exam? ______ Privacy? _________ m. Handwashing area? _____ Yes _____No n. Confidential counseling/testing area? _____ Yes ______No o. Emergency kit prepared? _____ Yes _____No p. Fee for service/cost share? _____ Yes ______No Drugs? Lab tests? Overall Comments (probe for changes in quality of services due to FANC training of MAISHA equipment/supplies; increase in service utilization of ANC)? 86 Regional Hospital FANC  Hospital In-Charge  ANC I/C  Nurse Midwife 5. FANC Critical Equipment and Supplies (observation) Comment(s) Fe Tablets (#/client) Sulfadoxine pyrimethamine Mebendazole/Albendazole Syphilis test kits HIV test kits Urine dip stick Hb measure RCH card BP cuff Other 6. QI a. Have you used FANC QI Assessment tool? _____ Yes _____ No Probe for comments on tool b. Role with BEmONC QI Team? Baseline BEmONC Assessment complete? ____ Yes ____ No c. Internal BEmONC Assessment: ______________ Number of assessments completed ______________ Month/Year of last assessment ______________ Score of most recent assessment Who is involved in the formal internal assessment (specifically)? d. How has the information from the internal assessment(s) been used? What has been the role of the FANC team member to help improve the BEmONC QI score? e. Example(s) of something that has changed because of the assessment(s). 7. Supportive Supervision a. Have you seen any changes in the way supervision is conducted since supportive supervision has been introduced? Probe for understanding of the term and ask who is using supportive supervision. b. Who comes to supervise your work here? How often? _______________________ c. When was the last visit? __________________________ d. In the last visit, what Supervision tool/checklist did they use (SBM-R FANC; MoHSW checklist; other)? MAISHA Midterm Evaluation Report 87 Regional Hospital FANC  Hospital In-Charge  ANC I/C  Nurse Midwife 8. What do you need to provide better ANC services? 88 Pre-Service Training Sites  Principal  Nurse Tutor Date:_________________ District/Region:__________________________ Time:_________________ Interviewer:_____________________________ Facility/Type:__________________ Interviewee:_____________________________ 1. General Introduction of the Team and purpose of the Midterm evaluation 2. What type of support has your school received from the MAISHA/Jhpiego project? 3. Please describe the process of the following: a. Curriculum review for nurse midwives Have you completed the process of curriculum review? What major areas have been revised/integrated into the new curriculum? How many tutors have been oriented in the use of the new curriculum? _________ Have you implemented the reviewed curriculum? Did Jhpiego/MAISHA provide resources to enable the implementation of the reviewed curriculum for the learner and the trainers? If yes, what were these resources. How many cohorts of graduants have been trained using the revised curriculum? In your opinion what are the major differences between the skills of graduants (new vs. older curriculum). b. Skills laboratory established _____ Yes ______ No c. If yes, how well equipped it the laboratory? What supplies/materials were provided by Jpiego/MAISHA? What impact has the skills laboratory had on training activities? d. Has there been any capacity building for the tutors? If yes, in what area? How many tutors were involved? _____________________ e. Have there been any benefits to the clinical practicum sites with the revised curriculum? If yes please explain 4. What recommendations would you make on the future role of the Regional Training Center? MAISHA Midterm Evaluation Report 89 District Council Health Management Team (District CHMT)  District Medical Officer  District RCH Coord.  District Nursing Officer  District Pharmacist Date:_________________ District/Region:__________________________ Time:_________________ Interviewer:_____________________________ Location:_____________________ Interviewee:_____________________________ 1. General Introduction of the Team and purpose of the Midterm evaluation 2. What are the key MAISHA activities supported in this district? (awareness, ownership) a. Have these activities been incorporated into the District Council Comprehensive Health Plan? 3. How many MAISHA-supported facilities are in your district? What was the selection criteria for these sites? Have the key MAISHA activities been rolled out to additional health facilities? If yes, how many? a. Have additional providers been trained since the initial MAISHA training? b. If additional training has taken place, how were these trainings supported (i.e. District funds, MoHSW, other donors, etc.) 4. What has been the primary impact of the MAISHA-supported activities? a. How has MAISHA impacted on the provision of quality MNH services at district sites? b. How has MAISHA impacted on the availability of supplies/equipment needed for the provision of services? c. What have been the challenges of providing quality MNH services? How has MAISHA addressed/responded to these challenges? d. How has MAISHA impacted on the conducting of QI and supportive supervision? e. Do you use the SBM-R QI tools? Strengths/Challenges? 5. Recommendations for MAISHA and future MNH programming? 90 Regional Training Center  Regional Medical Officer  Regional RCH Coordinator  Medical Director￾Hospital Date:_________________ Region:_________________________________ Time:_________________ Interviewer:_____________________________ Location:______________________ Interviewee:_____________________________ 1. What support has Jhpiego/MAISHA provided to the Regional Training Center? 2. What role has the Regional Hospital (as the Regional Training Center) played in the improvement of MNH services at MAISHA district level facilities? Non-MAISHA facilities?  Training: Number of trained Resource persons: ______________ Number of trainings conducted: FANC _______________ BEmONC ______________ QI/Supportive Supervision ________________ Other _______________________  QI/Supportive Supervision: Frequency of visits: _____________________ 3. What is the current plan to “roll out” BEmONC training at additional health facilities in the region? a. Are these plans part of the Regional comprehensive health plan/council comprehensive health plan? b. Is funding available to support these activities? If yes, where is funding coming from? 4. What are your plans for utilizing the training center? MAISHA Midterm Evaluation Report 91 Regional Hospital BEmONC  Hospital In-Charge  Maternity I/C  Nurse Midwife Date:_________________ Region:_________________________________ Time:_________________ Interviewer:_____________________________ Facility/Type: __________________ Interviewee:_____________________________ 1. General Introduction and purpose of the Midterm evaluation 2. What kind of support/interventions have been provided by the MAISHA project? 3. Staffing and training at Maternity Ward (staff and training) Number/type of staff in ANC ___________________ BEmONC # trained _______________ # on site________________ Year of BEmONC training: ___________________ LSS Training # trained _______________ # on site________________ Year of LSS training: ___________________ Other 4. Clinical services – BEmONC q. Guidelines/job aids available? _____ Yes _____No PMTCT KMC HBB Post natal care Post natal FP EBF Other __________________________ r. Evidence of IEC materials ____ Yes _____No s. IEC Type (i.e. TV, posters, etc.) ____________________________ t. Area for physical exam? ______ Privacy? _________ u. Handwashing area? _____ Yes _____No v. Confidential counseling/testing area? _____ Yes ______No w. Fee for service/cost share? _____ Yes ______No Drugs? Lab tests? x. Wards Antenatal Labour and Delivery Post natal Overall Comments (probe for changes in quality of services due to FANC training of MAISHA equipment/supplies; increase in service utilization of ANC): 92 Regional Hospital BEmONC  Hospital In-Charge  Maternity I/C  Nurse Midwife 5. BEmONC Critical Equipment and Supplies (observation) Comment(s) Uterotonics (Oxytocin, Ergometrine, Misoprostol) MgSO4 IM/IV Partographs BP machine Newborn resuscitation equipment (ambu bag/mask) Delivery kits Doppler C-section kits Suction pump Wall clock Sterilizer Episiotomy kit Chlorine Other 6. QI a. Have you used BEmONC QI Assessment tool? _____ Yes _____ No b. Role with BEmONC QI Team? MAISHA Midterm Evaluation Report 93 Regional Hospital BEmONC  Hospital In-Charge  Maternity I/C  Nurse Midwife c. Baseline BEmONC Assessment complete? ____ Yes ____ No d. Internal BEmONC Assessment: ______________ Number of assessments completed ______________ Month/Year of last assessment ______________ Score of most recent assessment Who is a member of the QI Internal Assessment Team? e. How has the information from the internal assessment(s) been used? f. Example(s) of something that has changed because of the assessment(s). g. What do you think of the tool? How useful/practical is this tool to your work? 7. Supportive Supervision a. Have you seen any changes in the way supervision is conducted since supportive supervision has been introduced? Probe understanding of the term and ask who is using supportive supervision. b. Who comes to supervise your work here? c. How often? _______________________ d. When was the last visit? __________________________ e. What Supervision tool/checklist did they use (SBM-R FANC; MoHSW checklist; other)? 8. What do you need to provide better obstetrics services? 94 District Health Center/Dispensary BEmONC  Health Facility I/C  Nurse Midwife Date:_________________ District/Region:__________________________ Time:_________________ Interviewer:_____________________________ Facility/Type: __________________ Interviewee:_____________________________ 1. General Introduction and purpose of the Midterm evaluation 2. What kind of support/interventions have been provided by the MAISHA project? 3. Provision of Services at the District level by this health facility Number/type of staff in ANC ___________________ BEmONC # trained _______________ # on site________________ Year of BEmONC training: ___________________ 4. Clinical services - BEmONC y. Guidelines/protocols available? _____ Yes _____No z. Evidence of IEC materials ____ Yes _____No aa. Type (i.e. TV, posters, etc.) ____________________________ bb. Area for physical exam? ______ Privacy? _________ cc. Handwashing area? _____ Yes _____No dd. Wards: Antenatal Labour and Delivery Post natal Ward KMC ee. Emergency transport available? Overall Comments (probe for changes in quality of services due to FANC training of MAISHA equipment/supplies; increase in service utilization of ANC): : 5. BEmONC Critical Equipment and Supplies (observation) Comment(s) Uterotonics (Oxytocin, Ergometrine, Misoprostol) MgSO4 IM/IV Partographs BP machine Newborn resuscitation equip (ambu bag/mask) Kangaroo Mother Care (KMC) Other MAISHA Midterm Evaluation Report 95 District Health Center/Dispensary BEmONC  Health Facility I/C  Nurse Midwife 6. QI a. Have you used BEmONC QI Assessment tool? _____ Yes _____ No b. Role with BEmONC QI Team? c. Baseline BEmONC Assessment complete? ____ Yes ____ No d. Internal BEmONC Assessment: ______________ Number of assessments completed ______________ Month/Year of last assessment ______________ Score of most recent assessment e. How has the information from the internal assessment(s) been used? f. Example(s) of something that has changed because of the assessment(s). 7. Supportive Supervision a. Have you seen any changes in the way supervision is conducted since supportive supervision has been introduced? b. Who comes to supervise your work here? District _________________________ Region__________________________ MAISHA staff _____________________ c. How often? _______________________ d. When was the last visit? __________________________ e. What Supervision tool/checklist did they use (SBM-R FANC; MoHSW checklist; other)? 8. What do you need to provide better ANC services? For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com