External Evaluation of March 2013 Saving Mothers, Giving Life INTERIM REPORT PHASE 1 External Evaluation of Saving Mothers, Giving Life: Interim Report Principal Investigators Margaret E. Kruk, MD, MPH Sandro Galea, MD, DrPH Co-investigators Karen Grépin, PhD Miriam Rabkin, MD, MPH Study Team Tsitsi Masvawure, DPhil Katherine Austin-Evelyn, MSc, MPH Dana Greeson, MPH Emma Sacks, PhD (candidate) Daniel Vail, BA Design: Jon Kalish LLC © 2013 3 Executive Summary .................................................................. 5 Acronyms & Abbreviations .......................................................... 9 1 Introduction...................................................................... 11 1.1 SMGL background.............................................................. 11 1.2 Aims of the external evaluation and content of this report........................ 11 1.3 Methods ....................................................................... 12 2 Context........................................................................... 13 2.1 Global context.................................................................. 13 2.2 Country setting................................................................. 13 2.3 SMGL districts ................................................................. 18 3 SMGL program theory........................................................... 22 3.1 SMGL in the maternal and newborn care continuum ............................. 22 3.2 SMGL logic model .............................................................. 22 3.3 SMGL theory of change ......................................................... 25 4 Implementation partners and process......................................... 27 4.1 Partners ....................................................................... 27 4.2 Planning and implementation................................................... 29 4.3 Use of PEPFAR platform ........................................................ 30 4.4 Funding ........................................................................ 31 5 Mid-point results ................................................................ 34 5.1 Methods ....................................................................... 34 5.2 Results ........................................................................ 34 6 Successes, challenges, lessons learned ...................................... 39 6.1 Successes ..................................................................... 39 6.2 Innovations .................................................................... 42 6.3 Challenges ..................................................................... 44 6.4 Synergies and unintended consequences........................................ 47 6.5 Country ownership and sustainability ........................................... 49 6.6 Role of the private sector ....................................................... 53 7 Preliminary assessment and recommendations for Phase 2 ................ 55 7.1 Assessment of SMGL’s first six months ......................................... 55 7.2 Recommendations for Phase 2.................................................. 56 7.3 Next steps ..................................................................... 59 8 Appendix ......................................................................... 61 A National health and health system indicators.................................... 62 B Methods ....................................................................... 64 C Acknowledgements ............................................................ 67 D References..................................................................... 70 Contents Saving Mothers, Giving Life External Evaluation / Interim Report 5 Executive Summary Maternal mortality reduction is a major unfinished agenda in global health. More than 250,000 women die every year during pregnancy, childbirth and the postpartum period. Saving Mothers, Giving Life (SMGL) is a $200 million, five-year initia￾tive of the United States Government (USG), the Government of Norway, Merck for Mothers, Every Mother Counts, and the American College of Obstetricians and Gynecologists, in collaboration with the Governments of Uganda and Zambia which seeks to dramatically reduce maternal and newborn mortality by simultaneously addressing health system and community-level constraints to survival. SMGL focuses on improving women’s access to high quality obstetric care during the highest risk period: the 24 hours around delivery. In its first year, the initiative aims to reduce maternal mortality by 50% in eight rural pilot districts in Zambia and Uganda. SMGL activities focus on increasing demand for care (e.g. through community mobilization and birth planning), access to care (e.g. transport, expanding obstetric services), quality of care (e.g. hiring, training health workers, ensuring drug supply), and general health system strengthening (e.g. improving health records and maternal death audits). To support this ambitious program, the US government provided substantial funding. The approximately $31.5 million USD budgeted for SMGL implementation translates to $7.4 and $20.8 per person in the Ugandan and Zambian SMGL districts, which represent substantial propor￾tions of existing health budgets in the two countries. The external evaluation of SMGL was commissioned by the SMGL Global Secretariat and is led by a team of health system researchers from Columbia University. The evalua￾tion is a strategic implementation evaluation that aims to: 1) Measure the extent and fidelity of implementation of SMGL interventions; 2) Assess functioning of the partnership and engagement of stakeholders; and 3) Identify lessons learned to inform the scale-up of SMGL to other districts and coun￾tries. This report summarizes progress in implementation and lessons learned from the first six months of the initiative. Planning for SMGL implementation in Zambia and Uganda began in June 2011, spearheaded by the US Department of State, the US Agency for International Development (USAID), and the US Centers for Disease Control and Prevention (CDC). Working on an accelerated timeline, implementation of activities began in eight districts in Zambia and Uganda in early 2012. Such rapid implementation was made possible by the existence of a robust platform for health system support established for the President’s Emergency Plan for AIDS Relief (PEPFAR). This platform included a range of non-governmental organizations with expertise in service delivery and community mobilization and existing contractual relationships with USG agencies; strong relationships with Ministry of Health plan￾ners; and a bevy of tried and tested approaches for improving training, supply chain, and other health system inputs. Data for this report were collected between October 2012 and January 2013. We used a mixed methods approach: we conducted 143 interviews with US Government, Ministry of Health, implementing partners, and district health officials and collected quantitative data regarding program startup and information on 202 indicators to measure the “dose” of the intervention that had been delivered to SMGL districts as of December 1, 2012. The evaluation team visited 50 sites, trav￾eling to all eight districts in the two countries, and reviewed over 500 documents collected in country. Preliminary findings were shared with in-country partners for review and correc￾tions. This report will be followed by a second wave of data collection that will focus on the response of communities and health providers to SMGL. These data will be presented in a final report in fall of 2013. Mid-point implementation results Both countries made important progress on implementing core SMGL activities, with a slower start for procurement of transport vehicles and infrastructure improvements. Both Uganda and Zambia trained large numbers of commu￾nity mobilizers (Village Health Teams in Uganda and Safe Motherhood Action Groups in Zambia) to promote facility deliv￾eries and birth planning. These activities were supplemented with media campaigns and in Zambia through advocacy of community “Change Champions.” Both countries extensively distributed “mama kits:” packages of basic newborn supplies, such as soap and blankets, to encourage facility deliveries. Activities to increase access focused on improving transpor￾tation and upgrading health facilities to handle emergencies. Both countries purchased ambulances—motorized vehicles as well as bicycle ambulances—although procurement of ambulances and utility vehicles was delayed in Zambia. The innovative use of vouchers for transport and private delivery care in Uganda, which involved collaboration with private health providers and motorcycle drivers’ associations, was well-received by communities and providers with demand outstripping supply of available vouchers. 6 MARCH 2013 Considerable efforts were made to improve quality in obstetric care in both countries. Notably, SMGL imple￾menters in Uganda hired over 130 new health workers to ease the human resource shortages in districts. In the two countries, over 400 health workers were trained in clinical skills, particularly emergency obstetric and neonatal care (EmONC). Supervision and mentoring on the wards by senior clinicians was introduced to sustain high quality of care and staff morale. Some SMGL activities were intended to strengthen the health system and assist in coordination of this “big push” initiative. Dedicated SMGL coordinators were hired in districts to assist District Medical Officers in managing logistics and daily operations. Personnel were also trained in supply chain management, health worker supervision, and management of health information systems to promote more accurate and timely information feedback to facilities on health indicators and outcomes. Successes and challenges Although the health impact of SMGL is not yet known, there was a unanimous perception among the over 150 respondents interviewed that SMGL has already succeeded on multiple fronts. Specifically, respondents perceived an increase in facil￾ity deliveries, increased demand for obstetric services, and an improvement in the quality of services at health facilities. Community engagement, health worker trainings, improved availability of transport, improved data management, provi￾sion of equipment and supplies and facility refurbishment were also considered the major early successes. Two key success factors cited by Zambian and Ugandan stakeholders were the strong alignment between SMGL and national health policies and well-functioning national partnerships—between Ministry of Health, US Government, and a range of implementing partner organizations—that formed to implement SMGL. Early support from the Ministry of Health was essential in selecting districts and motivating district health staff to take on SMGL. Although SMGL funds were “off-budget” (not channeled through national govern￾ments), SMGL worked largely through the existing health system, rather than pursuing stand-alone solutions. While it was noted that some district health and medical officers were overwhelmed with the speed and intensity of SMGL implementation, all agreed that the buy-in, commitment and dedication of district officers was crucial to successful launch of the initiative. Several SMGL activities emerged as particularly innova￾tive. In Uganda, the involvement of the private sector allowed for rapid expansion of transport and obstetric services for pregnant women. The high level of community participation and the connection between community groups and the health system were credited with building demand and supporting many expectant mothers in reaching health care. Finally, innovative technologies were deployed to support quality care and link women to the health system, including, for example, obstetric “tackle boxes” to ensure availability of supplies by every delivery bed and mobile phones to promote communication between community health work￾ers, transport providers, and women. SMGL also encountered some major challenges, several related to the short time frame for the first phase. Launching so many activities in a short time required a huge organi￾zational effort and participation of many partners—driving up transaction costs and making coordination difficult. As a result there was duplication of some activities, and delays in funding and logistics that prevented timely delivery of some inputs. The pressure to deliver results in 12 months prevented implementers from tackling deep and urgent health system constraints, such as human resource short￾ages and poor transport, beyond implementing quick fixes. Respondents recognized that while the “sprint” nature of the first phase of SMGL generated excitement, this level of energy cannot be maintained over the long run and cannot be sustained by USG and its implementing partners alone. Crucially, the rapid pace hindered meaningful involvement of the Ministries of Health and their few senior reproduc￾tive health staff. Respondents from both USG agencies and Ministries of Health emphasized that government support of the program was not the same as government ownership: in both countries, governments were strongly supportive of SMGL but were not in the driver’s seat. Many agreed that ownership would have been greater had national govern￾ments made an explicit commitment to SMGL at the outset, for example, tackling issues such as roads, human resources, and infrastructure that cannot be adequately addressed by donors. Positive and negative synergies The introduction of complex new programs is often accom￾panied by a host of secondary effects—positive and negative. SMGL clearly energized national efforts in maternal and newborn health, nationally and at district levels. Somewhat more unexpectedly, it also spurred important actions to strengthen health systems. A prominent example is the government decision to double the salaries for physicians working in health centers in Uganda, for which a key cham￾pion was a parliamentarian from an SMGL district who cited the dearth of qualified health workers and the success SMGL had in recruiting doctors by paying higher salaries. At the district level, “positive spillover” was noted in the diffusion of improved clinical skills, record-keeping and data management from SMGL to non-SMGL staff. Increased demand for facility services led to reported greater uptake of HIV and syphilis testing. Several negative unintended consequences were noted, such as the “crowding out” of other health priorities, namely family planning and postnatal care on national agendas, EXECUTIVE SUMMARY Saving Mothers, Giving Life External Evaluation / Interim Report 7 the redirection of USG resources to the focus districts, and the diversion of district health team time from non-SMGL activities. Over time, these diversions may have serious implications for other national health priorities. The short timeframe also contributed to difficulties in sequencing activities; for example, demand creation inter￾ventions began to generate new patients before facilities were expanded and new health workers hired. As a result, health workers reported report being overloaded with patients, and complained of water and electricity shortages, and stock-outs of medications. The many off-site trainings took health workers away from facilities, exacerbating persistent health worker shortages. Finally, some community leaders began imposing fines on women who delivered at home. Such unsanctioned actions are likely to affect the most vulnerable women—the poor and women living farthest from facilities—disproportionately. Assessment and recommendations Overall, our interim evaluation found a tremendous amount of enthusiasm for SMGL in both countries. In both Uganda and Zambia, the support for continuing and expanding SMGL was strong across the board and largely undiminished by the assortment of challenges encountered in the start-up phase. The major challenges for SMGL as it scales up are how to sustain the early implementation wins, how to address endemic health system weaknesses, and how to plan for a transition to national ownership. Our primary recommenda￾tions address the content and implementation of SMGL. We recommend that SMGL stay the course on the following core interventions: 1. Engage communities as change agents: Working with community-level groups to address birth planning and encourage facility deliveries and timely referrals has been a game-changer in both countries. To make this sustainable, community workers should be compensated or supported in finding income generating activities. 2. Invest in the health system triad: Training staff, equipping and improving facilities. This combination is highly synergistic in promoting quality care and staff retention. These activities represent core health system strengthening that will require a greater contribution from national governments. We recommend the following modifications to the SMGL package: 3. Expand the focus beyond 24 hours: SMGL is missing a crucial opportunity to save lives by failing to include family planning, postnatal care, and HIV health services. These activities not only make good health sense, they are also efficient to deliver using the same health system platform, e.g. facilities, staff, and health records, as delivery services. In particular, there is a large unmet need in both countries for family planning, which reduces women’s risk of maternal death. 4. Develop and test solutions to address the bottlenecks of human resource shortages and transport: Health worker shortages and poor geographic access to facilities are structural constraints for improving maternal and newborn survival. These will require innovative solutions: ideas such as rural incentives, task shifting, telemedicine, and maternity waiting homes should be tested in Phase 2. Finally, we suggest several ways to improve the implementation of SMGL going forward: 5. Coordinate around a single plan: For Phase 2, SMGL should strive for greater harmonization of program activ￾ities in each country. One useful model is the “Three Ones approach used in HIV:” one action framework, one coordinat￾ing authority and one monitoring and evaluation system. 6. Commit to five years—with a clear transition plan: Future SMGL commitments should be for three to five years to permit more coherent and efficient implementation and better integration with the health system. Future commit￾ments should include a clear plan to transition responsibility for implementation to countries, clearly spelling out the continuing role of external funders. 7. Collect (a few) standard measures to optimize implementation: Reliable and timely data is essential to manage the program and make adjustments. A few “tracer” indicators, collected as part of countries’ health informa￾tion systems, can be augmented by special studies into mission-critical issues such as care quality and the function￾ing of the rest of the health system. 8. Harness the private sector: Reducing maternal death requires an “all hands on deck” approach. The private sector can contribute expertise in management, marketing, logistics and procurement, and human resource management as well as raise funds for and awareness of the maternal and child death. Next steps In the next stage of the external evaluation we will work with research partners in Uganda and Zambia to explore: 1) response of the community and providers to SMGL 2) extent of implementation at endline and 3) functioning of the SGML partnership. We will produce a final report in fall of 2013. EXECUTIVE SUMMARY 9 ANC ................. Antenatal Care ANC4 ................ Four or more ANC visits BEmONC ............ Basic Emergency Obstetric and Newborn Care CDC ................. Centers for Disease Control and Prevention CEmONC ............ Comprehensive Emergency Obstetric and Newborn Care CIDRZ ............... Center for Infectious Disease Research in Zambia CKIs ................. Community Key Informants CSH ................. Communications Support for Health (Chemonics, Zambia) DAQS ................ Demand, Access, Quality and Health Systems Strengthening DHO ................. District Health Office DHS ................. Demographic and Health Survey EGPAF ............... The Elizabeth Glaser Pediatric AIDS Foundation EmONC .............. Emergency Obstetric and Newborn Care GDP ................. Gross Domestic Product HIPS................. Health Initiatives for the Private Sector HIV .................. Human Immunodeficiency Virus IDI .................. Infectious Diseases Institute (Makerere University, Uganda) IHME ................ Institute for Health Metrics and Evaluation IP.................... Implementing Partner JICA ................. Japan International Cooperation Agency JSI .................. John Snow, Inc. JTI................... Japan Tobacco International MCDMCH ............ Ministry of Community Development, Mother and Child Health MCHIP ............... Maternal and Child Health Integrated Program MDG ................ Millennium Development Goal MMR ................ Maternal Mortality Ratio MNCH................ Maternal, Newborn and Child Health MoH ................. Ministry of Health MTN ................. Mobile Telephone Network NESH................ National Expansion and Strengthening of HIV/TB Services (Uganda Protestant Medical Bureau) PACE ................ Program for Accessible Health, Communication and Education (PSI, Uganda) PEPFAR ............. President’s Emergency Plan for AIDS Relief PMTCT .............. Prevention of Mother to Child Transmission (of HIV) PPP ................. Purchasing Power Parity SDS ................. Strengthening Decentralization for Sustainability SMAGs .............. Safe Motherhood Action Groups (Zambia) SMGL ............... Saving Mothers, Giving Life SPEAR............... Supporting Public Sector Workplaces to Expand Action and Responses Against HIV/AIDS U5MR ............... Under Five Mortality Rate UCP ................. Uganda Community Project UHMG ............... Uganda Health Marketing Group UNAIDS.............. Joint United Nations Program on HIV/AIDS UNFPA............... United Nations Population Fund UNICEF .............. United Nations Children’s Fund USAID ............... United States Agency for International Development USG ................. United States Government VHT ................. Village Health Team (Uganda) WHO ................ World Health Organization ZCAHRD ............. Zambia Center for Applied Health Research and Development (Boston University) ZHECT ............... Zambia Health Education and Communications Trust ZISSP................ Zambia Integrated Systems Strengthening Project (Abt Associates) Acronyms & Abbreviations Saving Mothers, Giving Life External Evaluation / Interim Report 11 1 Introduction 1.1 SMGL background Saving Mothers, Giving Life (SMGL) is a $200 million, five-year initiative of the United States Government (USG), in collaboration with the Governments of Uganda, Zambia, and Norway, Merck for Mothers, Every Mother Counts, and the American College of Obstetricians and Gynecologists. The first phase—the proof of concept—aims to reduce maternal mortality by 50% within one year in eight focus districts in Uganda and Zambia. In its second phase, the program will expand to up to 10 additional countries with high maternal mortality.1 SMGL’s focus is on reducing maternal and newborn deaths in the 24 hours around delivery by simultaneously increasing community demand for obstetric care and improving the availability and quality of health services. As a public-private partnership, SMGL engages governments, non-governmental groups, and private sector organizations.1 1.2 Aims of the external evaluation and content of this report The external evaluation is a strategic implementation evaluation, commissioned by the SMGL Global Secretariat to enable course corrections in Phase 1 and to inform design of Phase 2. The specific aims of the external evaluation are to: 1) Assess the extent and fidelity of implementation of SMGL interventions; 2) Assess functioning of the partnership and engagement of stakeholders; and 3) Identify best practices and barriers to success in order to improve effectiveness in Phase 2. The external evaluation complements the internal evaluation led by CDC and USAID, which will measure program outcomes, particularly changes in maternal deaths and program costs. The external evaluation consists of two stages and will result in two reports. This report summarizes our findings from the first stage in which we: 1) Assessed the planning and first six months of SMGL implementation in Zambia and Uganda; and 2) Identified early lessons learned in order to provide feedback to stakeholders in country and at the global level. It largely reflects the voices and experiences of SMGL implementers in Uganda and Zambia. Specifically, we discuss program context, program theory, the implementation process to date, mid-point results, emerging lessons, and assessment and recommendations for Phase 2 of SMGL. The second stage of the external evaluation will focus on the response of the community and health providers to SMGL. We will further assess implementation at endline in country and explore the functioning of the global and national part￾nerships. In the second stage of our work, we will collaborate with local research partners in Uganda and Zambia. This will culminate in a final report in fall of 2013. Family and motorcycle driver in Nyimba District, Zambia 12 MARCH 2013 1.3 Methods The external evaluation team is led by Dr. Margaret E. Kruk from the Department of Health Policy and Management and Dr. Sandro Galea of the Department of Epidemiology at Columbia University’s Mailman School of Public Health. They work closely with faculty co-investigators, Dr. Miriam Rabkin (Medicine and Epidemiology) and Dr. Karen Grépin (New York University, Health Economics), and a dedicated team of five researchers with expertise in public health, health systems, anthropology, and quantitative and qualitative methods. Data for this report were collected over a total of 134 person-days in country between October 2012 and January 2013. The majority of the time was spent in the eight SMGL districts. Our team conducted 143 interviews with 152 individu￾als, including 89 in-depth interviews focused on “lessons learned.” Respondents included all United States Government (USG) SMGL leaders and key Ministry of Health focal points, and a wide range of implementing partners. We interviewed all eight SMGL district medical officers, all district SMGL coordinators and a number of clinical staff involved in SMGL service delivery. We visited over 50 sites—including district health offices, implementing partner program offices, US Embassies and USG offices, statistical offices, public and private hospitals and clinics, and mothers’ shelters—and reviewed over 500 documents collected in country. The data collected for the interim report were exempted from human subjects review by Columbia University as they concerned program implementation activities. Qualitative data were entered, cleaned, and analyzed using the NVivo Software Package (Version 10). Transcripts of all interviews and notes were independently coded by two analysts using pre-existing codes and a process of constant comparison as described by Glaser.2 Discrepancies were reconciled through a practice of consensus building between the two coders. A second level of open, iterative coding was then conducted, creating new codes around major themes, which were then reviewed by multiple team members to synthesize and summarize. These summaries were the basis of the emerging lessons and recommendations in this report. The summaries also provided data for several graphics in this report, including the logic model, partner network, and imple￾mentation calendar. For our quantitative analysis, we assessed the “dose delivered” of SMGL activi￾ties at the mid-point of Phase 1. Dose delivered refers to the activities and inputs actually implemented in the districts, as reported by the district medical officers and implementing partners in the districts. Using the logic model we created with SMGL partner input in country (see Exhibit 5 on page 24), we collected over 200 data elements on the extent of implementation of 28 SMGL activities by December 1, 2012, the SMGL program mid-point. We identified a subset of tracer indicators that reflect performance of core SMGL activities for presentation in the report. The data were cross-checked against program documents provided to the evaluation team by the Ministries of Health and implementing partners and shared with in-country partners at district level for their review and additional input. Detailed methods used for this evaluation are outlined in Appendix B. Dose delivered data collection tool 1 INTRODUCTION Interview at Ntara HC IV, Kamwenge District, Uganda Saving Mothers, Giving Life External Evaluation / Interim Report 13 2.1 Global context High rates of maternal mortality remain a major challenge in global health. More than 250,000 women die every year during pregnancy, childbirth, and the postpartum period,3, 4 with the vast majority occurring in low-resource settings in sub-Saharan Africa and South Asia. Many more women experience ill-health and disability after delivery, ranging from postpartum depression to obstetric fistula. In addition, approximately 3 million infants die within the first 28 days of life each year,3, 5 representing approximately 40% of deaths of children under the age of five.5 Most of these newborn deaths occur during the early neonatal period (0 to 6 days).3 The primary causes of maternal deaths are postpartum hemorrhage (i.e. excessive bleeding during delivery and the immediate postpartum period) and hypertensive disorders of pregnancy.6 These conditions are treatable with medical interventions, such as the use of uterotonic drugs, magnesium sulfate, blood transfusions and Caesarean section. Unfortunately, many women in need never receive the appropriate medical services, barred by lack of information and social status, difficulties reaching health facilities due to poor roads and inadequate transport, and the shortage of quality emergency obstetric services. Delays in deciding to seek medical care, in accessing health facilities, and in receiving the appropriate care and treatment once at a health facility are the three delays that drive high rates of maternal and newborn mortality in low￾and middle-income countries. In order to avert maternal deaths, all pregnant women must have available and affordable emergency obstetric care since up to 15% of deliveries are expected to have some complication and most cannot be predicted during antenatal care screening.7 Quality obstetric care and newborn interventions provided after deliv￾ery can also save many newborn lives. SMGL tackles the 24-hour period around delivery—the riskiest period for mother and baby.1 2.2 Country setting Uganda Uganda is a landlocked country in East Africa with a history of political strife and continuing conflict in the northern region and along the Congolese border (see Exhibit 1). It has maintained relative political stability since the election of President Yoweri Museveni in 1986. With an economic growth rate of 6.7% per annum, Uganda received approximately $1.7 billion in net official development assistance in 2010.8 The country has a gross national income of $510 USD per capita and the government spends $10 per person on health in current US dollars, with 50% of total expenditure on health being paid out-of-pocket.9, 10 External resources for health are equivalent to 26% of total expenditure on 2 Context Staff housing in Nyimba District, Zambia PHOTO: Dr. Christopher Nyambe Sinyange, Nyimba District DMO 14 MARCH 2013 health.10 The country has a crucial shortage of health workers with 1.4 doctors, nurses, and midwives per 1,000 population in 2005, far below the international recommendation of 2.5 per 1,000.10, 11 Recent estimates of the maternal mortality ratio* in Uganda range from 274 (CI: 206.2-352.3 [IHME]) to 438 (368-507 [DHS]) deaths per 100,000 live births.3, 4, 12 The total fertility rate is 6.1.12 Fewer than 20% of Ugandan women have more than a primary school education and the median age of women at first marriage is 17.9 years.12 Access to health facilities for obstetric care is limited by cost, transport, and distance. According to the 2011 Uganda Demographic and Health Survey (DHS), 57% of deliveries were attended by a skilled birth attendant (doctor, nurse or midwife) and only 5% of births were Caesarean sections (compared to the expected rate of 10-15%).12 In the same survey, 65% of respondents reported difficulties accessing care when needed.12 Uganda faces other pressing health challenges. It has one of the highest recorded malaria transmission rates in the world and 33% of children are nutri￾tionally stunted.12 HIV continues to be a major challenge: the HIV prevalence among adults 15 to 49 in Uganda is 7.2%, although this has declined substantially since the peak of 15% in 1991.12-14 Additionally, Uganda has had outbreaks of viral hemorrhagic fevers (Ebola and Marburg Fever) in the past three years, including an Ebola outbreak in Kibaale in July 2012 and a Marburg outbreak in October 2012 in Kabarole, both SMGL districts. In 2007, Uganda launched a Road Map for accelerating the reduction of mater￾nal and neonatal mortality, in order to meet the Millennium Development Goals for health. The stated goals for 2007 to 2015 are: ■■ To increase the availability, accessibility, and utilization of quality skilled care during pregnancy, childbirth, and the postnatal period at all levels of the health care delivery system; ■■ To promote and support appropriate health-seeking behavior among pregnant women, their families and the community; and ■■ To strengthen family planning information and service provision for women, men, and couples who want to space or limit their childbearing thus preventing unwanted and/or untimely pregnancies that increase the risk of maternal death.15 The first two of these goals mirror the focus of SMGL, suggesting that the initiative is consistent with national policy. Zambia Zambia is a large, sparsely populated, landlocked country in Southern Africa (see Exhibit 1). President Michael Sata assumed office after a peaceful transfer of power in 2011. Copper, Zambia’s major export, suffered a price decline in the 1970s, which made Zambia more reliant on foreign aid. Since the 2000s, Zambia’s economy has stabilized and now enjoys a 6.5% economic growth rate per year (2011). Zambia received $900 million in net official development assistance in 2010.8 * The maternal mortality ratio measures the deaths of women in pregnancy and up to 42 days postpartum due to issues related to pregnancy per 100,000 live births. Given that most deaths occur around delivery, it is primarily a measure of the safety of childbirth. 2 CONTEXT Cover of Uganda’s MNCH Road Map Cover of Zambia’s MNCH Road Map Saving Mothers, Giving Life External Evaluation / Interim Report 15 Sources 1 WHO, Global Health Observatory Data Repository, 2013 2 World Bank Data , 2011 3 DEMOGRAPHIC AND HEALTH SURVEY, UGANDA: 2011, ZAMBIA: 2007 Uganda PopulatioN (IN MILLIONS)1 33.4 Total fertility Rate1 6.1 Gross natIONAL income $510 per capita (USD)2 Total health expENDITURE $47 per capITA (USD)1 HIV prevalence rate 7.2 among adults 15-491 Percentage of live births 57% in health facilities3 ZAMBIA PopulatioN (IN MILLIONS)1 13.1 Total fertility Rate1 6.3 Gross natIONAL income $1,160 per capita (USD)2 Total health expENDITURE $73 per capITA (USD)1 HIV prevalence rate 12.5 among adults 15-491 Percentage of live births 48% in health facilities3 EXHIBIT 1 SMGL focus countries Mpika Solwezi Serenje Chama Mkushi Mumbwa Kasempa Chinsali Kazungula Namwala Isoka Kasama Mungwi Samfya Kapiri Mposhi Chibombo Mbala Chongwe Mporokoso Mpongwe Luwingu Choma Kafue Milenge Mwense Lufwanyama Kawambwa Chilubi Chipata Mambwe Monze Katete Siavonga Mazabuka Gwembe Nakonde Luangwa Itezhi-Tezhi Sinazongwe Nchelenge Masaiti Chadiza Kabwe Kitwe Chingola Kalulushi Kafue Ndola Mufulira Luanshya Chililabombwe Petauke Lusaka MALAWI ZIMBABWE DEMOCRATIC REPUBLIC OF THE CONGO MOZAMBIQUE Lundazi Kalomo Mansa Nyimba ZAMBIA DEMOCRATIC REPUBLIC OF THE CONGO Kibaale Kyenjojo Kamwenge Kabarole Lake George Lake Albert Mbarara Rakai Mubende Kiboga Hoima Kasese Bushenyi Sembabule Bundibugyo UGANDA 20 miles 20 miles ZAMBIA Lusaka UGANDA Kampala EXHIBIT 2 Map of health facilities in SMGL focus districts Mpika Solwezi Serenje Chama Mkushi Mumbwa Kasempa Chinsali Kazungula Namwala Isoka Kasama Mungwi Samfya Kapiri Mposhi Chibombo Mbala Chongwe Mporokoso Mpongwe Luwingu Choma Kafue Milenge Mwense Lufwanyama Kawambwa Chilubi Chipata Mambwe Monze Katete Siavonga Mazabuka Gwembe Nakonde Luangwa Itezhi-Tezhi Sinazongwe Nchelenge Masaiti Chadiza Kabwe Kitwe Chingola Kalulushi Kafue Ndola Mufulira Luanshya Chililabombwe Petauke Lusaka MALAWI ZIMBABWE DEMOCRATIC REPUBLIC OF THE CONGO MOZAMBIQUE Lundazi Kalomo Mansa Nyimba ZAMBIA DEMOCRATIC REPUBLIC OF THE CONGO Kibaale Kyenjojo Kamwenge Kabarole Lake George Lake Albert Mbarara Rakai Mubende Kiboga Hoima Kasese Bushenyi Sembabule Bundibugyo UGANDA 20 miles 20 miles ZAMBIA Lusaka UGANDA Kampala SUMMER FALL 1-100 0 RAINFALL IN MM AVERAGE SEASONAL RAINFALL, 1983-2013 101-200 201-300 301-400 401-500 501-600 WINTER SPRING Mansa Nyimba Lundazi ZAMBIA Kalomo UGANDA Kibaale Kyenjojo Kamwenge Kabarole Provincial or regional hospitals Receive referrals from district hospitals District hospitals Receive referrals from health centers and provide inpatient care Health centers Capable of handling vaginal deliveries Health posts Not capable of handling deliveries District health offices Roads Note: Kalomo district boundaries were recently redrawn—some of the health facilities that are included in the Kalomo SMGL intervention now lie outside the district. 18 MARCH 2013 Zambia has a gross national income of $1,160 USD per capita and the govern￾ment spends $44 per person on health in current US dollars, with 27% of total expenditure on health being paid out-of-pocket.10 External resources for health are equivalent to almost 40% of total expenditure on health.10 Zambia faces a severe health worker shortage with 0.8 doctors, nurses, and midwives per 1,000 in 2006, far below the level of 2.5 required to provide near-universal delivery coverage to meet the child and maternal health Millennium Development Goals.10, 11 Zambia’s maternal mortality ratio is variously estimated at 293 (223.7-386.7 [IHME]) 3 and 440 (CI:220-790 [UNFPA]) 4 deaths per 100,000 live births. Despite the wide range, these estimates are both lower than the 2007 Zambian DHS estimate of 591 deaths per 100,000 live births.16 The total fertility rate is 6.3.16 According to the DHS, only 16% of Zambian women have more than a primary school education and the median age at first marriage for women is 18.2 years.17 Many Zambian women have limited access to formal health care, especially in rural areas where the population is sparser and transport is more difficult. The 2007 DHS reports that 47% of deliveries are attended by a skilled birth attendant and only 3% of women receive Caesarean sections. Over 77% of DHS respondents reported difficulties accessing care when needed.16 As in Uganda, Zambia faces many urgent health challenges. Approximately 45% of children are considered nutritionally stunted and 18% of children under five years reported fever (suspected malaria) in the two weeks preceding the survey.16 HIV is a major health threat in Zambia, although HIV prevalence among adults has declined from 14% in 2007 to 12.5% in 2011.16-18 Zambia has recently published its Maternal and Newborn Road Map. The aim of the policy is to accelerate the reduction of maternal, newborn and child mortality to enable attainment of the MDGs in Zambia. The specific objectives in the Road Map are to: 1) Provide skilled attendance during pregnancy, childbirth, and the postnatal period, at all levels of the health care delivery system; and 2) Strengthen the capacity of individuals, families, and communities to improve maternal, newborn and child health (MNCH).19 This strategy influenced the focus and choice of interventions for SMGL in Zambia. 2.3 SMGL districts The SMGL districts selected for the SMGL pilot phase in Uganda—Kibaale, Kabarole, Kyenjojo and Kamwenge—are contiguous districts located in the Western part of the country. Their combined population of 1,750,000 people is roughly 6% of the national total.9 The western region is rural and includes the Rwenzori Mountains, Kibaale National Forest, and the Crater Lakes. The major languages spoken in the region are Rutooro and Runyoro, although many resi￾dents also speak English and Kinyarwanda. In contrast, the Zambian SMGL districts—Kalomo, Lundazi, Mansa and Nyimba—were selected from different regions of the country.20 While Nyanja and Bemba are spoken throughout the country and English is the official language of business, Zambia recognizes 73 languages. Residents of Mansa, in the north, are largely Bemba-speakers. Kalomo residents, in the South, are mostly Chitonga￾speakers, while those in the Eastern Province districts of Lundazi and Nyimba speak Chitumbuka, a language also spoken by their Malawian neighbors. Each of these districts share borders with different countries. Exhibit 2 is a map of the SMGL districts in Uganda and Zambia, indicating health facilities capable of providing basic and emergency obstetric and neonatal care. The map demonstrates the paucity of comprehensive obstetric care (i.e. Caesarean section and blood transfusion) in SMGL districts. It also highlights the logistical challenges in delivering SMGL interventions to remote areas in Zambia. “Women walk many kilometers— like 7 to 20—to access health centers, so we thought [transport] was a very important intervention…” — Zambia District Level MoH Official Typical housing in Lundazi District, Zambia Flooded road in Kamwenge District, Uganda 2 CONTEXT Saving Mothers, Giving Life External Evaluation / Interim Report 19 In both countries, the Ministries of Health selected the districts with input from the United States Government (USG) development partners. Selection was based on high levels of maternal mortality, low socio-economic status, the absence of other large maternal health programs, and the presence of strong district lead￾ership and commitment (i.e. from local government and district health offices). In Uganda, the Ministry of Health, US Government, and implementing partners decided to work in contiguous districts to promote regional health system capac￾ity development and ease implementation. Existing US Government relationships with two strong PEPFAR partners with experience in programming for complex interventions (i.e. the Infectious Disease Institute [IDI] and Baylor College of Medicine Children’s Foundation Uganda [Baylor]) in the western region also influ￾enced the selection of SMGL districts. In contrast, the Zambian Ministry of Health focused on areas that were difficult to reach with routine programming and had high unmet health needs, selecting districts that were geographically dispersed throughout the country. The SMGL districts in Uganda and Zambia are predominantly rural with poor access to obstetric facilities. Long distances, dirt roads that become impassable during the rainy season, limited transport options, and inconsistent or unreliable communication, were cited as factors impeding access to delivery and newborn care. More generally, the agrarian economies of the districts provide relatively few income generating opportunities for communities. Poverty was frequently cited in interviews as a determinant of the poor health of the population. Exhibit 3 summarizes the socioeconomic and health context of the eight SMGL districts using data from recent DHS surveys,† SMGL country operational plans, and district health management information systems. In Uganda, between 72 and 89% of SMGL district residents are farmers. In Zambia, farmers comprise between 14 and 56% of inhabitants. In Uganda SMGL districts, between 2 and 15% of residents have electricity. In Zambia, less than 4% of residents have elec￾tricity SMGL district, except for Mansa, where 26% of the people have access to electricity. The HIV prevalence in SMGL districts ranges from 8% to 15%. Health systems suffer from workforce shortages and poor infrastructure. In Kibaale, the largest of the SMGL districts (population 646,500), there are 5 medical officers, 32 clinical officers, and 34 nurse/midwives. Kibaale reports a staff vacancy rate of over 50%. Similarly, Lundazi, the largest Zambian SMGL district (population 314,300), has 5 medical officers and 12 clinical officers; however it has 119 nurse/midwives. There are relatively few obstetric facilities in the SMGL districts. Within the focus districts, Uganda has 17 facilities that are expected to provide compre￾hensive emergency obstetric and neonatal care‡ (CEmONC), however a baseline survey found that only 5 of these facilities were able to provide CEmONC; Zambia has only 4. Uganda’s SMGL districts have 58 basic emergency obstetric and neonatal care (BEmONC) facilities and those in Zambia have 91. Coverage of facility deliveries is low in all districts. In the western region of Uganda, approx￾imately 56% of deliveries occur in facilities.21 In Zambia, coverage of facility deliveries in SMGL districts ranges from 20% in Nyimba to 46% in Lundazi.16 † NB: DHS surveys are not sampled to be representative at the district level. DHS data should thus be interpreted as illustrative. ‡ Basic emergency obstetric care is comprised of: administration of antibiotics, oxytocics, and anticonvulsants, manual removal of the placenta, removal of retained products following miscarriage. or abortion, assisted vaginal delivery, preferably with vacuum extractor, and newborn care. Comprehensive emergency obstetric and neonatal care also includes Caesarean section, blood transfusion, and care for sick and low-birth weight newborns. SMGL billboard in Kamwenge District, Uganda. 2 CONTEXT “Infrastructure is too much of a barrier in our country. This ranges from the road the mother has to travel, to the waiting rooms, the buildings etc. Deliveries are even happening in clinics which are not equipped.” — Uganda District Level MoH Official UGANDA ZAMBIA Kabarole Kamwenge Kibaale Kyenjojo Year SOURCE Kalomo Lundazi Mansa Nyimba Year SOURCE Demographic Population 409,400 324,400 646,500 369,700 2011 1 254,211 314,281 217,603 101,616 2010 1 Literacy rate (%) 69.2 47.1 61.2 55.2 2011 2 62.8 41.5 62.5 31.0 2007 2 Secondary education or higher (%) 24.0 5.8 32.9 14.9 2011 2 29.5 15.5 44.2 0.0 2007 2 Farmers (%) 71.9 86.6 78.3 88.5 2011 2 56.4 38.5 14.5 37.9 2007 2 Own a telephone (landline) (%) 0.0 2.3 1.5 3.5 2011 2 0.0 0.0 1.9 6.9 2007 2 Have electricity (%) 12.0 2.3 15.3 3.7 2011 2 1.4 3.6 25.9 0.0 2007 2 Improved water supply (%) 57.3 51.7 83.1 27.6 2011 2 38.6 20.2 60.5 10.3 2007 2 Geographic Annual rainfall, 1983-2012 (mm) 942 1069 1072 1033 AVG 3 710 830 919 818 AVG 3 Health HIV prevalence (%)* 8.2 2011 4 15.2 15.0 10.0 7.7 2010 6 Maternal deaths, annual** 80 63 126 72 2010 1 58 72 50 23 2010 5 Newborn deaths, annual*** 550 436 869 497 2010 5 1,136 1,460 973 472 2010 5 Deliveries, annual 18,341 14,533 28,963 16,563 2011 1 13,219 16,343 11,315 5,280 2010 1 Health System Medical officers 21 0 5 3 2011 1 n/a 5 1 3 2011 7 Clinical officers 51 17 32 22 2011 1 n/a 12 7 9 2011 7 Anaesthetic officers 3 0 1 1 2011 1 n/a 1 0 0 2011 7 Nurse/midwives 168 37 34 37 2011 1 n/a 119 98 50 2011 7 Staff vacancy rate (%) 37.0 22.0 > 50.0 32.0 2011 1 n/a 17.5 61.0 n/a 2011 7 Provincial/regional/referral hospitals 4 0 1 0 2011 1 0 0 1 0 2011 1 District hospitals 2 4 5 2 2011 1 1 1 0 1 2011 1 Health centers 17 9 17 15 2011 1 29 23 27 12 2011 1 Health posts 24 14 20 15 2011 1 4 19 1 5 2011 1 Total CEmONC facilities **** 6 4 5 2 2011 1 1 1 1 1 2011 1 Total BEmONC facilities 17 9 17 15 2011 1 29 23 27 12 2011 1 Utilization Women who completed at least 4 ANC visits during pregnancy (%) 51.7 43.6 40.0 33.3 2011 2 72.4 61.9 66.0 72.7 2007 2 Women who gave birth by caesarean section (%) 6.9 3.6 6.3 1.8 2011 2 3.5 2.2 11.4 0.0 2007 2 Households with bednets (%) 82.7 75.9 77.1 88.5 2011 2 79.5 73.7 89.1 79.3 2007 2 Women who received a vitamin A dose within 2 months of delivery (%) 48.3 21.8 20.8 26.3 2011 2 44.8 55.6 60.7 36.4 2007 2 Women who use a modern contraception method (%) 37.9 29.1 29.2 22.8 2011 2 29.5 39.2 17.0 44.8 2007 2 Health facility delivery coverage (%)***** 55.9 2011 2 27.0 46.0 42.0 20.0 2011 1 * HIV Prevalence for Uganda reported for the Western Region as a whole ** Maternal deaths in Zambia calculated based on the WHO 2010 Global Health Observatory point estimate of 440 maternal deaths per 100,000 births *** Newborn deaths calculated based on the WHO 2010 Global Health Observatory point estimate of 30 newborn deaths per 1,000 births in Uganda, and 86 newborn deaths per 1,000 births in Zambia **** While there are a total of 17 health facilities in the Ugandan SMGL districts that are designated as ‘CEmONC,’ the baseline SMGL survey found that only five of these were actually capable of providing CEmONC services. ***** Facility delivery rates for Uganda reported for the Western Region as a whole Sources 1 SMGL Country Operational Plans 2 Demographic and Health Surveys 3 TAMSAT Research Group 4 Uganda AIDS Indicator Survey 5 WHO Global Health Observatory 6 Zambia National AIDS Council 7 Dose Delivered Interviews with District Staff Note: Data from the Demographic and Health Surveys are not representative at the district level, and are limited by small sample sizes. Total sample size for Uganda across all four SMGL districts is 418. Total sample size for Zambian SMGL districts is 566. EXHIBIT 3 District-level demographic and health indicators Saving Mothers, Giving Life External Evaluation / Interim Report 21 The cultural context also influences care seeking for maternal and newborn health and, in turn, the success of initiatives such as SMGL. Several cultural norms, common in both countries, were cited as barriers to reducing maternal and newborn mortality. Early marriage puts young women at risk as maternal deaths are more common among adolescents. In Zambia a district health official noted that it is not uncommon to “see girls getting married at 13 or 14.” District managers and health workers noted that many women have a strong allegiance to traditional birth attendants (TBAs) who conduct deliveries in the home. The choice of home delivery was in some cases reinforced by poor health worker attitudes in health facilities that women encountered in antenatal care and other visits. Interview participants in both countries noted that men rarely take an active role in maternal health, especially with regard to birth planning. This leaves many pregnant women without financial or logistical support when emer￾gencies arise. Some respondents in Zambia reported a local belief that disclosing a preg￾nancy early on will result in a miscarriage, which prevents them from obtaining early antenatal care. Finally, participants in both countries mentioned that cultural and church groups sometimes speak out against family planning and in some cases against other modern maternal health care, including facility delivery. The existence of cultural traditions that conflict with efforts to promote maternal and newborn survival point to the importance of involving communities in SMGL, and the need for explicit communication and advocacy strategies. 2 CONTEXT “Early marriage is not addressed and is a key factor which contributes to high risk pregnancies.” — Uganda District Level Implementing Partner Family traveling by canoe to Luembe Clinic, which is not accessible by road, Nyimba District, Zambia PHOTO: Dr. Christopher Nyambe Sinyange, Nyimba District DMO 22 MARCH 2013 “…without linked programs, this cannot [continue to] work. It is too narrow a point of view. Are we giving the message: ‘it’s ok to just come for delivery’ by focusing on 24 hours?” — USG Uganda OFFICIAL 3.1 SMGL in the maternal and newborn care continuum Maternal and neonatal survival is achieved through a chain of individual and household actions and health service interventions that begin before pregnancy and end one month post-delivery. SMGL interventions tackle the highest risk period: the 24 hours around delivery when 60% of maternal deaths and 50% of newborn deaths occur.4, 6, 21, 22 Exhibit 4 situates SMGL activities within the larger continuum of maternal and newborn care. As Exhibit 4 shows, improving chances of maternal and newborn survival begins before pregnancy. Avoiding unwanted pregnancies reduces the number of maternal deaths. A recent study estimated that providing contraception for all women who want it but lack access would reduce maternal deaths globally by 104,000, or 29% of the current total.23 Once women are pregnant, antenatal care focused on high-impact services can improve newborn birth weight and provide other essential protective functions, such as tetanus prophylaxis and prevention of HIV infection.6 While most deliveries are a normal life event, delivery and the postpartum period remains a high-risk period for mother and baby as complications can arise suddenly and kill quickly. The majority of these can be treated with basic and comprehensive emergency obstetric and neonatal care—a package of services long known and implemented in developed and many middle-income countries. Provision of emergency obstetric and neonatal care in the 24-hour period around birth is the focus of SMGL. SMGL’s concentration on the peripartum period is both a strength and a weak￾ness. It focuses scarce resources on the most critical period for maternal and newborn mortality and seeks to strengthen a key weakness in many low-income country health systems: poor quality of obstetric care. However, maternal complica￾tions and many neonatal infections also occur after the first 24 hours. Additionally, the reduction of HIV-related maternal deaths—an important cause of maternal mortality in both countries—requires high quality antenatal care. The omission of family planning is also a missed opportunity to avert maternal deaths of women who did not want to be pregnant. The latter point in particular was echoed through￾out interviews with SMGL leads, implementing partners, health system managers, and facility-level providers, most notably in Uganda. 3.2 SMGL logic model SMGL simultaneously tackles both supply-side (health system) and demand-side (community and user) barriers to the provision of effective obstetric and neonatal care, in what can be characterized as a whole-of-health systems approach. SMGL activities explicitly address all three of the delays that contribute to maternal mortality: delay in seeking care (through community activation and birth planning), delay in reaching care (through transport, more obstetric facilities closer to 3 SMGL program theory Tetanus toxoid immunization Screening and treatment for HIV and sexually transmitted infections (STIs) Intermittent preventive treatment (IPT) for malaria Treatment of asymptomatic bacteriuria Antiretroviral therapy for HIV-infected, eligible women Emergency care (ectopic pregnancy, post￾abortion care) Folic acid/iron and vitamin supplementation Prevention of mother-to-child HIV transmission (PMTCT) Family planning Birth spacing Folic acid/iron and vitamin supplementation Screening and treatment for HIV and sexually transmitted infections (STIs) Prevention of mother-to-child HIV transmission (PMTCT) Corticosteroids for preterm labor Labor surveillance to detect complications (partograph) Identification of breech, transverse lie, or multiple gestation pregnancy Basic emergency obstetric care: ■■ Parenteral antibiotics for infection/premature rupture of membranes (PROM) ■■ Parenteral anticonvulsants for pre-eclampsia/ eclampsia ■■ Uterotonic drugs for treatment of hemorrhage ■■ Active management of labor and clean delivery ■■ Assisted vaginal delivery (vacuum extraction) ■■ Manual removal of the placenta ■■ Manual removal of retained products Comprehensive emergency obstetric care: ■■ Caesarean section for obstructed labor, etc. ■■ Blood transfusion for severe hemorrhage MATERNAL Antiretroviral therapy for HIV-infected, eligible women Infection prevention at surgical sites (Caesarean incision, episiotomy) Identification and treatment of sepsis NEONATAL Basic neonatal resuscitation (bag and mask) Management of hypothermia Promotion of exclusive breastfeeding Detection/treatment of pneumonia/sepsis Antiretroviral therapy for HIV-exposed infants Extra care for low birth weight and preterm infants Skin to skin contact Early detection and referral of complications MATERNAL Postpartum family planning Screening for postpartum depression General nutritional counseling Antiretroviral therapy for HIV-infected, eligible women NEONATAL Case management for pneumonia and tuberculosis Treatment for malaria, use of bednets Testing and treatment for pediatric HIV Basic childhood immunizations Treatment for diarrhea (oral rehydration therapy/zinc) Growth monitoring and micronutrient supplementation Integrated management of childhood illness (IMCI) SMGL FOCUS PRE-PREGNANCY PREGNANCY BIRTH POSTPARTUM / EARLY NEONATAL LATE POSTPARTUM / LATE NEONATAL EXHIBIT 4 Evidence-based clinical interventions to avert maternal and neonatal deaths along a continuum of care OUTPUTS Utilization Increase in: ■■ facility deliveries ■■ four antenatal care visits ■■ postnatal care visits ■■ referrals for complications ■■ EmONC ■■ neonatal resuscitation Quality Reduction in: ■■ maternal facility case fatality rate ■■ perinatal mortality in facilities System strengthening ■■ Stronger supply chains and health information systems ■■ New operating theaters ■■ More mothers’ shelters ■■ Improved transport and communication OUTCOMES ■■ Reduction in maternal mortality ■■ Reduction in newborn mortality ■■ SMGL program plan ■■ Funding ■■ USG, MoH, and implementing partners ■■ Health workers ■■ Trainers and training curricula ■■ Equipment (e.g. laboratory, vehicles, scales, delivery equipment, etc.) ■■ Supplies (e.g. essential medicines, blood supply) ■■ Basic infrastructure (e.g. water, electricity) ■■ Guidelines (clinical, data, supply chain) ■■ Information and communication technology INPUTS ACTIVITIES Demand ■■ Train and supervise community groups to promote facility delivery and birth preparedness (Village Health Teams [VHTs], Safe Motherhood Action Groups [SMAGs]) ■■ Identify and engage community influencers in safe motherhood (Change Champions) ■■ Provide basic newborn/birth supplies to incentivize facility delivery (Mama Kits, Mama Packs) ■■ Run mass media campaigns on safe motherhood, engage community drama groups Access ■■ Build new operating theatres with CEmONC capacity ■■ Expand/refurbish maternity wards, labs, pharmacies ■■ Renovate mothers’ shelters near hospitals for high risk women ■■ Provide service delivery vouchers and vouchers for transport to facilities and referral to higher-level private facilities ■■ Buy emergency obstetric and newborn care (BEmONC) equipment ■■ Buy ambulances, motorcycles, motorbikes, and communication equipment for transportation and referrals ■■ Form district-level transport committees to improve referral ■■ Contract with private providers for EmONC Quality ■■ Hire new doctors, nurses, and midwives in collaboration with the Ministry of Health ■■ Train health workers in EmONC ■■ Train health workers in newborn resuscitation (“Helping Babies Breathe”) ■■ Train doctors in surgical obstetric care and anesthetic officers in anesthesia ■■ Provide regular supportive supervision to frontline health workers to maintain and improve skills in obstetrics, newborn care and anesthesia ■■ Provide training and oversight for maternal death reviews ■■ Provide essential medicines to health facilities System strengthening ■■ Build provincial and district health team capacity with SMGL staff ■■ Strengthen supply chains through training on procurement and stock management ■■ Build capacity of facility staff to supervise community health workers ■■ Provide computer-based medical records (SmartCare) ■■ Strengthen pharmacy, lab, and blood supply ■■ Support roll-out of electronic district health information system ■■ Train health workers in data collection and health information systems ■■ Both countries ■ UGANDA ■ ZAMBIA EXHIBIT 5 SMGL logic model 24 Saving Mothers, Giving Life External Evaluation / Interim Report 25 3 SMGL PROGRAM THEORY population), and delay in receiving appropriate care (through training, mentoring, equipment, drugs, referral).24 To create the SMGL logic model, we have categorized the interventions into Demand, Access, Quality, and Health Systems Strengthening (DAQS) activities. The logic model (Exhibit 5) is based on our review of ongoing and planned activities in each country. The logic model summarizes SMGL efforts to increase awareness of the need for care, facilitate transportation to reach care, provide equipment, medi￾cines, and training to improve the quality of care, and ensure that the health system develops and sustains approaches to maintain accessible high quality services. While the DAQS domains, the programmatic approach, and the majority of inter￾ventions are consistent across the two countries, several specific activities and means of implementation differ. One major difference between the two countries is the role of the private sector. Uganda has involved physician and nurse professional associations in training and mentorship and has contracted with private transport and health care providers. In Zambia, there is little involvement of the private sector, reflecting the minor role that private providers play in the overall health system. While the logic model describes SMGL activities, we sought to understand which activities were most visible at the district level. To this end, we measured how frequently activities from the logic model were mentioned in all of our district￾level interviews. Exhibit 6 below summarizes this graphically. The word size gives an indication of the salience of these activities to implementers in the district. In Uganda, vouchers were most frequently mentioned by interview participants. Trainings were emphasized in both countries, although more commonly noted in Zambia. The prominence of trainings is likely a function of the frequency and intensity of trainings as well as the logistic complexity in implementing them. Finally, community outreach efforts—Volunteer Health Teams (in Uganda) and Safe Motherhood Action Groups (in Zambia)—were also highlighted. 3.3 SMGL theory of change The theory of change underlying SMGL holds that it is the combination of increased information and transport at the community level coupled with availability of better services at health facilities that will dramatically expand health system utilization and maternal and newborn survival. Central to this thesis is that neither demand nor supply is sufficient to transform the possibility of survival—hence the whole￾of-health systems approach. This is an innovative approach in the field of maternal and newborn health where many global initiatives tackle either demand-side25-28 or supply-side29, 30 interventions. SMGL’s approach is supported by the experiences of countries that accomplished large reductions in maternal deaths historically—from Sri Lanka to Egypt to China.31-33 The whole-of-health systems approach also echoes and in key ways seeks to replicate the successful experience of PEPFAR programs, which simultaneously worked at the community level to overcome stigma around HIV and built up the health system to provide high quality clinical interventions. A secondary feature of SMGL is its role as a catalyst for health system strength￾ening. There are two mechanisms by which SMGL can strengthen health systems. First, in both countries SMGL provides concrete support for core health system functions: e.g. improving management skills, health information functions, phar￾macy and other supply chains, and blood banks. These activities can benefit not only mothers and newborns, but other users as well. Second, by rapidly improving the quality of services and with it, the number of facility deliveries, resuscitations for asphyxiated newborns, and averted deaths, the initiative demonstrates what is possible to achieve in the existing health system, raising the enthusiasm of provid￾ers and managers and the confidence of women and communities. Ideally, these efforts can infuse other health services. “The Ministry of Health sees SMGL as a learning opportunity for the rest of the system— performance￾based financing, ambulance management, and better health data.” — Uganda MoH Official 26 MARCH 2013 EXHIBIT 6 Frequency of SMGL activities mentioned at the district level ZAMBIA UGANDA Saving Mothers, Giving Life External Evaluation / Interim Report 27 4.1 Partners The implementation of SMGL is a collaborative effort led by USAID, CDC, and the Ministries of Health in Uganda and Zambia. The newly formed Ministry of Community Development, Mother and Child Health (MCDMCH) in Zambia is another key SMGL stakeholder. As described below, CDC and USAID worked with reproductive and child health leaders at the Ministries of Health to design SMGL at the country level and select intervention districts. Ministry of Health officials were invited to coordinate SMGL planning and implementation meetings, although the frequency of the meetings and numerous competing priorities limited senior officials’ ability to participate. In both Uganda and Zambia, CDC and USAID played a central role in contracting implementing partners for SMGL activities, coordinating the work of the partners, and hosting frequent partner meetings to share progress. CDC, together with the countries’ statistical offices and in-country research institutions, is also charged with leading the monitoring and evaluation activities including baseline facility surveys and maternal mortality surveillance. In the first year, other global SMGL partners were in the planning and early implementation stages of their activities. Following the PEPFAR model, the majority of SMGL funding—which came from the US Government—was awarded to non-governmental organizations who had existing relationships with USG agencies and most of whom were already working in the eight SMGL districts. These “implementing partners”§ were largely responsible for day-to-day program activities in Phase 1, including: 1) Designing interventions, recruitment, training, supervision, and mentoring of health workers and community advocacy groups; 2) Outreach, advocacy, and communications; 3) Procuring medicines, equipment, and supplies; 4) Refurbishing and renovating health facilities; and 5) Collecting data. As additional global partners joined the initiative, new implementing partners have been added. For example, in Uganda, Merck for Mothers began work with local NGOs to engage private sector partners, and Every Mother Counts supported transportation vouchers. In Uganda, a single organization—Baylor-Uganda—is the lead implementing partner in three of the four SMGL districts where they are responsible for the majority of program activities, while IDI is the main implementing partner in the fourth district. Some organizations work across multiple districts. For example, Marie Stopes Uganda manages distribution of Healthy Baby vouchers, the Health Care Improvement Project (URC) supports quality improvement activities, and SURE provides technical oversight and training for supply chain management. In Zambia, CIDRZ is the lead implementing partner in two of the four SMGL districts, while ZCAHRD and MCHIP are lead partners in the remaining two § NB In this report we use the term “implementing partners” to refer to non-governmental implementers. Ministry of Health implementers will be separately identified. Ambulance in Kamwenge District, Uganda 4 Implementation partners and process Ministry of Health ■■ Kabarole District Health Office MAIN IMPLEMENTING PARTNER ■■ Baylor-Uganda OTHER IMPLEMENTING PARTNERS ■■ STRIDES and SURE (Management Sciences for Health) (MSH) ■■ Association of Obstetricians and Gynaecologists of Uganda (AOGU) ■■ Uganda Pediatric Association (UPA) ■■ Uganda Society of Anaesthesia (USoA) ■■ Marie Stopes Uganda (MSU) ■■ Peace Corps ■■ Uganda Health Marketing Group (UHMG) ■■ Health Care Improvement Project (HCI) (University Research Co., LLC) KABAROLE Ministry of Health ■■ Kamwenge District Health Office MAIN IMPLEMENTING PARTNER ■■ Baylor-Uganda OTHER IMPLEMENTING PARTNERS ■■ STRIDES and SURE (Management Sciences for Health) (MSH) ■■ Association of Obstetricians and Gynaecologists of Uganda (AOGU) ■■ Uganda Pediatric Association (UPA) ■■ Uganda Society of Anaesthesia (USoA) ■■ Marie Stopes Uganda (MSU) ■■ Peace Corps ■■ Uganda Health Marketing Group (UHMG) ■■ Health Care Improvement Project (HCI) (University Research Co., LLC) KAMWENGE Ministry of Health ■■ Kyenjojo District Health Office MAIN IMPLEMENTING PARTNER ■■ Baylor-Uganda OTHER IMPLEMENTING PARTNERS ■■ STRIDES and SURE (Management Sciences for Health) (MSH) ■■ Association of Obstetricians and Gynaecologists of Uganda (AOGU) ■■ Uganda Pediatric Association (UPA) ■■ Uganda Society of Anaesthesia (USoA) ■■ Marie Stopes Uganda (MSU) ■■ Peace Corps ■■ Uganda Health Marketing Group (UHMG) ■■ Health Care Improvement Project (HCI) (University Research Co., LLC) KYENJOJO Ministry of Health ■■ Kibaale District Health Office MAIN IMPLEMENTING PARTNER ■■ Infectious Disease Institute (IDI) OTHER IMPLEMENTING PARTNERS ■■ Association of Obstetricians and Gynaecologists of Uganda (AOGU) ■■ Uganda Pediatric Association (UPA) ■■ Uganda Society of Anaesthesia (USoA) ■■ Marie Stopes Uganda (MSU) ■■ Peace Corps ■■ Uganda Health Marketing Group (UHMG) ■■ Health Care Improvement Project (HCI) (University Research Co., LLC) KIBAALE ■■ Ugandan Ministry of Health (MoH) ■■ National Medical Stores (NMS) ■■ National Blood Bank ■■ Association of Obstetricians and Gynaecologists of Uganda (AOGU) ■■ Uganda Pediatric Association (UPA) ■■ Uganda Society of Anaesthesia (USoA) ■■ Uganda Bureau of Statistics (UBOS) Uganda ■■ United States Government ■■ U.S. Agency for International Development (USAID) ■■ CENTERS FOR DISEASE CONTROL AND PREVENTION (CDC) ■■ U.S. Peace Corps ■■ U.S. Department of State ■■ U.S. Department of Defense ■■ U.S. Department of Health and Human Services ■■ OGAC (OFFICE OF THE GLOBAL AIDS COORDINATOR) ■■ National Institutes of health (NIH) ■■ Government of Norway ■■ Merck for Mothers ■■ Every Mother Counts ■■ American College of Obstetricians and Gynecologists (ACOG) GLOBAL ■■ Zambian Ministry of Health (MoH) ■■ Zambian National Blood Transfusion Services (ZNBTS) ■■ Ministry of Community Development, Mother and Child Health (MCDMCH) ■■ Central Statistics Office (CSO) ■■ Zambian Defence Force (ZDF) ■■ University of Zambia (UNZA) Zambia Ministry of Health ■■ Kalomo District Health Office MAIN IMPLEMENTING PARTNER ■■ Zambia Center for Applied Health Research and Development (ZCAHRD) (Boston University) OTHER IMPLEMENTING PARTNERS ■■ Zambia Integrated Systems Strengthening Program (ZISSP) (Abt Associates) ■■ John Snow, Inc. (JSI) ■■ Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) ■■ Communications Support for Health (CSH) (Chemonics) ■■ Maternal and Child Health Integrated Program (MCHIP) (Jhpiego) ■■ Society for Family Health (SFH) (Population Services International) KALOMO Ministry of Health ■■ Lundazi District Health Office MAIN IMPLEMENTING PARTNER ■■ Centre for Infectious Disease Research in Zambia (CIDRZ) OTHER IMPLEMENTING PARTNERS ■■ Zambia Integrated Systems Strengthening Program (ZISSP) (Abt Associates) ■■ John Snow, Inc. (JSI) ■■ Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) ■■ Communications Support for Health (CSH) (Chemonics) ■■ Maternal and Child Health Integrated Program (MCHIP) (Jhpiego) ■■ Society for Family Health (SFH) (Population Services International) SUPPORTING ORGANIZATION ■■ Thandizani LUNDAZI Ministry of Health ■■ Mansa District Health Office MAIN IMPLEMENTING PARTNER ■■ Maternal and Child Health Integrated Program (MCHIP) (Jhpiego) OTHER IMPLEMENTING PARTNERS ■■ Zambia Integrated Systems Strengthening Program (ZISSP) (Abt Associates) ■■ John Snow, Inc. (JSI) ■■ Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) ■■ Communications Support for Health (CSH) (Chemonics) ■■ Society for Family Health (SFH) (Population Services International) ■■ Zambia Prevention, Care & Treat￾ment Program (ZPCT II) (FHI360) SUPPORTING ORGANIZATION ■■ Groups Focused Consultants MANSA Ministry of Health ■■ Nyimba District Health Office MAIN IMPLEMENTING PARTNER ■■ Centre for Infectious Disease Research in Zambia (CIDRZ) OTHER IMPLEMENTING PARTNERS ■■ Zambia Integrated Systems Strengthening Program (ZISSP) (Abt Associates) ■■ John Snow, Inc. (JSI) ■■ Elizabeth Glaser Pediatric AIDS Foundation (EGPAF) ■■ Communications Support for Health (CSH) (Chemonics) ■■ Maternal and Child Health Integrated Program (MCHIP) (Jhpiego) ■■ Society for Family Health (SFH) (Population Services International) NYIMBA EXHIBIT 7 SMGL Partner Network Saving Mothers, Giving Life External Evaluation / Interim Report 29 districts. In addition to the lead implementing partners, the following partners provide SMGL services in all four districts: ZISSP (SMGL coordinators, EmONC and SMAG trainings), MCHIP (Helping Babies Breathe trainings), EGPAF (comput￾ers, supporting solar panels, SmartCare and rapid syphilis test trainings), CSH (birth plans and other community outreach materials), and JSI (essential medi￾cines and logistics improvement trainings). Since SMGL activities are focused at the district level, district medical or health officers and their teams, who oversee all district health activities, are the pillars of SMGL program implementation. The eight district health and district medical officers (DHOs/DMOs) oversee and approve implementation activities. To assist them in coordinating such a complex program, USG agencies assigned an SMGL coordinator funded by the initiative, to manage the day-to-day imple￾mentation of activities as well as to provide a link between DHOs/DMOs and implementing partners. At the facility level, SMGL services are primarily provided by public sector health workers. In Uganda, nearly 200 additional health workers and health system managers were hired to augment the workforce. In addition to the public sector workforce, private service providers in Uganda have been contracted to address transport and delivery care. In contrast, the private health sector has not played a role in Zambia, where private providers have a smaller presence in rural areas. 4.2 Planning and implementation SMGL was conceptualized during a visit to Zambia by former Secretary of State, Hillary Rodham Clinton, in June 2011. Initial planning began at the US Department of State, in partnership with USAID and CDC. Planning in Zambia began shortly thereafter with the creation of an SMGL technical task force and steering committee. Frequent meetings between CDC, USAID, and the MoH shaped the Zambian vision for SMGL, drawing from Zambia’s National Maternal and Child Health Road Map, as well as the Zambian adaptation of the Campaign to Accelerate the Reduction of Maternal Mortality in Africa (CARMMA). During these initial planning meetings, the details of the SMGL logic model were expanded and neonatal health was included as a program focus. By October of 2011, US imple￾menting partners had been identified and contracts were awarded. Implementing partners conducted planning activities and facility assessments for the remain￾der of 2011; implementation of SMGL activities began in January 2012. In Uganda, planning began in October 2011 as CDC, USAID, and the Ugandan MoH met to shape the program. Uganda’s national Road Map provided a key start￾ing point; SMGL was envisioned as the operationalization of the main components of the strategy. In December 2011, SMGL leadership convened a large meeting of national and district MoH officials and implementing partners, including private and faith-based facility managers, in the regional capital of Fort Portal. Contracts and initial work plans for SMGL Uganda were finalized in December 2011. Preparatory work began in January 2012, including staff recruitment and the development of baseline assessment tools. Baseline facility assessments and the training of existing staff were completed by February 2012. In April 2012, SMGL Uganda implementation began in earnest with new staff reporting for duty, ambulances being deployed to the districts, and facility renovations underway. Exhibit 9 is a planning and implementation calendar, noting SMGL meetings and delegation visits in both countries. This calendar highlights the great inten￾sity of the initiative for funders and implementers, who are simultaneously managing their regular portfolio of activities. For district health teams, SMGL was added to numerous existing health programs in the districts, managed by a variety of local and global, public and 4 IMPLEMENTATION PARTNERS AND PROCESS A typical mothers’ shelter in Mansa District, Zambia A renovated mothers’ shelter in Mansa District, Zambia PHOTOS: ZPCT II 30 MARCH 2013 private actors. To illustrate this point, Exhibit 8 shows the crowded health partner landscape that SMGL joined in each district. The Exhibit shows the organizations engaged in health activities in the SMGL districts, in addition to core Ministry of Health programs. As of December 2012, each of the eight SMGL districts had between 13 and 28 organizations. 4.3 Use of PEPFAR platform The intensity displayed in the implementation calendar (Exhibit 9) alludes to the highly competent teams involved in the planning and implementation of the program. Another facilitator of SMGL’s quick launch was the ability to leverage PEPFAR partnerships. As PEPFAR focus countries, Uganda and Zambia have longstanding and substantially funded programs supporting HIV prevention, care, and treatment. In order to expedite program start-up, the USG largely built on existing partnerships, grants, and contracts rather than initiating new funding mechanisms or competitive bidding for SMGL activities. CDC, for example, added supplemental funding to existing cooperative agree￾ments, expanding the remit of selected PEPFAR implementing partners to include SMGL activities. USAID, in contrast, shifted funding and adjusted scopes of work Kabarole Kibaale Kyenjojo Kamwenge AOGU NESH UCP USOA AOGU USOA AOGU USOA HIPS EMESCO SDS AOGU USOA SPEAR EXHIBIT 8 Organizations active in health initiatives in SMGL districts Uganda 4 IMPLEMENTATION PARTNERS AND PROCESS Saving Mothers, Giving Life External Evaluation / Interim Report 31 Kalomo Mansa Lundazi Nyimba MUMUNI PROJECT LUFAID GROUPS FOCUSED CONSULTANTS ANGLICAN DIOCESE OF MANSA CATHOLIC DIOCESE OF MANSA ACTION FOR SOCIAL DEVELOPMENT MANSA BUSINESS ASSOCIATION to permit current implementing partners to focus on SMGL activities and districts. This approach, described by one USG officer as “bureaucratically nimble,” enabled very rapid start-up of program activities by implementing partners, most of whom were already at work in the focus districts. In this way, SMGL leveraged the lessons, resources, partners, and programs developed by PEPFAR. 4.4 Funding Although a costing analysis of SMGL is outside the purview of this evaluation (a separate costing study is being conducted by USAID), we will present an estimate of the funds expended thus far on SMGL to highlight the magnitude of health system investments and, in turn, the scope and ambition of the initiative. The funding for Phase 1 of SMGL came almost exclusively from the US Government. As of December 2012, the USG had allocated approximately $37 million USD—$31.5 million for implementation and an additional $5.5 million for monitoring and evaluation activities. USAID attributed $6 million in Uganda and $12 million in Zambia. These were not new funds; rather they were monies that had been budgeted for activities related to PEPFAR, family planning, maternal and child health, nutrition, and malaria that were now concentrated on SMGL districts. Zambia 4 IMPLEMENTATION PARTNERS AND PROCESS 32 MARCH 2013 CDC contributed approximately $9.5 million USD in new money to each country: $6.5 to 7 million for implementation activities and $2.5 to 3 million for monitoring and evaluation. The US Department of Defense committed a smaller amount for facility renovations in Zambia and the Peace Corps contributed the services of Peace Corps Response Volunteers in all four focus districts in Zambia and two districts in Uganda. In terms of other SMGL global partners, Merck for Mothers is in the early stages of funding activities in Zambia and Uganda. In Zambia, Merck for Mothers has issued a request for proposals for two to three grants of $200,000 to $250,000 each for planning projects to determine the feasibility and design of a maternity waiting home model that can ensure high quality services in a financially self-sus￾taining manner. Merck for Mothers plans to award an additional $3 million for one to two projects in Phase 2 to conduct implementation research and test the model. In Uganda, Merck for Mothers has commited $9 million in the initial four SMGL districts and several other districts in year 1 and approximately 30 districts over the three-year course of the project to strengthen the private health sector. Merck for Mothers is working with a consortium led by PACE, a local NGO, to address several aspects of the Demand-Access-Quality-System Strengthening framework: bolster the private provider network (e.g. providing training and business loans to private clinics), improve women’s access to supplies and information (e.g. work with drug shops, Mama Ambassadors, and care providers to market and distribute Mama Kits), make care more affordable (e.g. community based health insurance and savings schemes), and improve emergency transport (e.g. training taxi drivers 4 IMPLEMENTATION PARTNERS AND PROCESS COUNTRY LEVEL ACTIVITY JUN JUL AUG SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC UGANDA Central CDC, USAID, & implementing partners Partners meetings • • • • • • • • • • • • • • • • • • • • • • • • • • Hosting visiting delegations • • • • • District Health offices & implementing partners Partners meetings • • • • • • • • • • • • • • • • • • • • • ZAMBIA Central CDC, USAID, & implementing partners Partners meetings • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • Hosting visiting delegations • • • • • • • District Health offices & implementing partners Partners meetings • • • • • • • • • • • • • • • • • • • • • • • 2011 2012 Planning month Implementation month Indicates a single event (i.e. meeting or visiting delegation) EXHIBIT 9 Implementation calendar Saving Mothers, Giving Life External Evaluation / Interim Report 33 EXHIBIT 10 SMGL budget in the context of national health expenditures who transport women in labor). Recruitment of private clinics into the private franchise network, and clinical training activities, began in early 2013. Every Mother Counts provided $41,000 USD to support transportation vouchers in Uganda. The Government of Norway will provide funding in Phase 2, as origi￾nally planned, and the American Congress of Obstetricians and Gynecologists is in the process of finalizing its support for activities in the two countries. Both the Government of Uganda and the Government of Zambia made addi￾tional investments that directly or indirectly support the goals of SMGL. For example, the Government of Uganda plans to increase physician salaries and the Government of Zambia hired more than 2,000 new health workers nationwide in 2012, of which 75% were nurses. The US Government funding for SMGL translates into an approximate investment of $7.4 per person in the Ugandan focus districts and $20.8 per person in the Zambian focus districts.¶ To put these figures into context, the total health spending in Uganda is $46.7 per person, of which $10.0 comes from the Ugandan government. In Zambia, the total per capita health spending is $72.9, of which $43.9 is government-funded.10 As Exhibit 10 shows, SMGL funding represents 16% and 29% of all health spending—and 74% and 47% of government spending—in Uganda and Zambia, respectively. This is a substantial investment in maternal and newborn survival compared to countries’ current resource envelopes. ¶ Figures obtained by dividing funds for implementation (approximately $13 million in Uganda and $18.5 million in Zambia) by the combined population of the four SMGL districts per country. Population figures were taken from the SMGL operational plans. Uganda Zambia $0 $10 $20 $30 $40 $50 $60 $70 $46.7 $10.0 $7.4 $72.9 $43.9 $20.8 Total health spending per capita Government health spending per capita SMGL spending per capita “Quite a lot of investment has gone into the focus districts in terms of extra dollars reaching the ground... and the partners who are implementing the program have received extra funds to increase their capacity.” — Zambia MoH Official 34 MARCH 2013 5.1 Methods Our analysis of the “dose delivered” assessed the extent to which SMGL activities had been implemented in the intervention districts as of December 1, 2012—the mid-point of Phase 1 of the program. The data in Exhibit 11 thus provide a district￾level perspective on the extent of implementation of SMGL at six months. Our analysis excludes any planned or future activities. In interpreting the figures, it is important to note that some activities were sequenced for later implemen￾tation, and that in some cases, the lack of activities may reflect lower need for those inputs in a particular district rather than slow implementation. An additional caveat is that there may be some peripheral or pilot activities that were not included, as we sought to assess only the major activities reflected in the country SMGL operational plans. Our dose delivered instrument consisted of 202 indicator variables representing the 28 core SMGL activities from the logic model. To ensure that we captured all relevant activities, we reviewed the SMGL operational plans for each country, and data from interviews with USG, MoH, and implementing partners in Zambia and Uganda. Once all data were collected and vetted with in-country partners, we then selected 33 (Uganda) and 32 (Zambia) tracer indicators that captured implemen￾tation of the core SMGL activities. Tracer indicators provide a useful summary of overall SMGL implementation from the district perspective. These indicators were selected based on three criteria: 1) The extent to which the indicator variable was representative of the larger activity it was nested under; 2) The availability of data for the indicator across all SMGL districts; and 3) The reliability of the available data for the indicator. Specific details on the methods for collecting, vetting, and analyzing the dose delivered data can be found in Appendix B. 5.2 Results Uganda There was little variability in the extent to which activities had been implemented across districts in Uganda in the first six months of the program. Some variation in activities implemented was noted in Kibaale, which has a different implement￾ing partner (IDI) than the other three districts (Baylor-Uganda). Upwards of 600 Village Health Team (VHT) members were trained in each district, with Kibaale, the most densely populated district, training over 1,900. This was the major demand-building activity in the four Uganda districts and it was widely reported to have facilitated community-level data collection and demand generation for maternal health services. Activities to increase access to maternal and newborn care included facility upgrades to CEmONC capacity in each district (11 facilities), with all but Kibaale 5 Mid-point results “Kit boxes” for obstetric emergencies in Kalomo District, Zambia PHOTO: Dr. Davidson Hamer, ZCAHRD Saving Mothers, Giving Life External Evaluation / Interim Report 35 receiving facility upgrades to BEmONC capacity (11 facilities). The four districts had a total of 17 refurbished maternity wards (with refurbishments defined as any upgrades that fell short of the construction of a new building). Kibaale was the only district that reported completed renovations of mothers’ shelters (4). All three of the Baylor districts implemented a voucher system and private provider contracts for EmONC services. Additionally, Marie Stopes Uganda distributed transport and delivery vouchers in all four districts. Transport was another major activity aimed at increasing access. A total of 30 vehicles were distributed across the four districts with each district receiving at least one vehicle ambulance and at least two motorcycle ambulances. Furthermore, each district established a transport committee responsible for coordinating vehicle use and maintenance. Many activities aimed at increasing the quality of maternal and newborn health care were implemented, including the hiring and training of health providers. A total of 20 additional doctors were hired across SMGL districts, while 20 addi￾tional nurses were hired in Kyenjojo. The largest stimulus to human resources, however, was the addition of 94 midwives across the four districts. One of the most intensive activities was the training of health workers in EmONC and newborn resuscitation. Doctors and nurses also received trainings in surgical obstetric care and anesthesia, respectively, and substantial numbers of frontline health workers in all districts received regular supportive supervision. A total of 250 health workers received SMGL-sponsored clinical trainings. Maternal and perinatal death review (MPDR) trainings were also conducted in all four districts. Several SMGL activities were aimed at strengthening health systems within the focus districts. Health workers were trained in data collection and health infor￾mation systems. Kibaale had the highest number of health workers trained in supply chain strengthening (30). Baylor and IDI each hired an SMGL coordinator to manage project logistics in the districts. Zambia In Zambia, there was considerable variation in implementation status between different activities. Demand creation activities, especially Safe Motherhood Action Group (SMAG) trainings and birth plan dissemination, were implemented in all four districts. Lundazi, the district with the largest population, trained six SMAGs (120 individuals), whereas Nyimba, with approximately one-third the population, trained twelve (210 individuals). Exhibit 12 is a video clip showing a training of a SMAG in Zambia. District-level respondents reported that neither the planned documentary about safe motherhood nor a mass media campaign had been completed, although a radio campaign was noted to be in production. All districts engaged community influencers as “Change Champions” for safe motherhood, but only Lundazi and Kalomo reported the use of Mama Packs (i.e. gift packages containing a combination of a traditional fabric, diapers, and soap) to create demand. Several of these activities, such as SMAGs and Mama Packs, predate SMGL and were mostly scaled-up and intensified under SMGL. Activities to improve access to maternal and newborn health services included facility upgrades to BEmONC capacity in all districts but Mansa. It is possible that most facilities in Mansa, a provincial capital, already had sufficient BEmONC capacity. No facilities were upgraded to CEmONC capacity in any of the districts. Mansa and Kalomo were the only districts to see maternity wards refurbished, while Nyimba was the only district to report a refurbished pharmacy. Renovations were in progress for five mothers’ shelters in Kalomo and twelve in Mansa. Districts reported substantial hurdles in procuring emergency and utility vehi￾cles. Nyimba still had no SMGL vehicle as of December 1, 2012, while Lundazi 5 MID-POINT RESULTS Operating theatre before renovation, Kyenjojo District, Uganda Operating theatre after renovation, Kyenjojo District, Uganda Train community groups to promote delivery/birth preparedness (VHTs/SMAGs) Individuals trained as VHT members 692 626 1,912 750 Provide basic newborn/birth supplies to pregnant women (Mama Kits) Mama Kits distributed to pregnant women ** ** ** ** Run mass media campaigns on radio, engage community drama groups Radio spots broadcast 27,391 Drama skits conducted 500 Increase facility EmONC capacity Health facilities upgraded to BEmONC capacity 6 2 3 Health facilities upgraded to CEmONC capacity 3 2 5 1 Expand/refurbish maternity wards, labs, pharmacies Maternity wards with infrastructure improvements 10 2 4 1 Expanded/refurbished pharmacies 0 0 0 0 Renovate mothers’ shelters near hospitals for high risk women Renovated mothers' shelters 0 0 4 0 Buy ambulances, motorcycles, and motorbikes for transportation and referrals Vehicle ambulances provided to district 2 2 2 1 Motorcycle ambulances provided to district 3 3 8 2 Provide vouchers for delivery, EmONC, transport, postnatal care in private facilities Vouchers distributed by Baylor to women 17,192 5,340 7,360 Baylor vouchers redeemed by women 8,649 2,207 4,128 Healthy baby vouchers distributed to women 16,631 Private facilities enrolled in voucher program 42 Form district-level transport committees to improve referral Transport committees established in district 1 1 2 1 Buy emergency obstetric and newborn care (EmONC) equipment EmONC equipment procured ** ** ** ** Contract with private providers for EmONC Private providers contracted to provide EmONC 7 6 6 Hire new doctors, nurses, and midwives in collaboration with the Ministry of Health New doctors hired in district 5 3 9 3 New nurses hired in district 0 0 0 20 New midwives hired in district 19 12 48 15 Train health workers in EmONC Health workers trained in EmONC 90 44 29 60 Train health workers in newborn resuscitation Health workers trained in newborn resuscitation 90 44 29 60 Supervision of frontline workers to improve obstetrics and newborn care skills Health workers who received supportive supervision 120 110 41 Provide training and oversight for maternal death reviews Health workers trained in maternal death reviews 12 8 29 10 Provide essential medicines Provide essential medicines to health facilities ** ** ** ** Train doctors in surgical obstetric care and nurses in anesthesia Doctors trained in surgical obstetric care 7 3 ** 5 Nurses trained in anesthesia 7 2 3 Strengthen supply chains through training on procurement and stock mgmt Workers trained in procurement and stock mgmt 7 0 30 14 Build provincial and district health team capacity with SMGL staff SMGL coordinators in district* 1 1 1 1 Strengthen pharmacy, lab, and blood supply Blood banks 1 0 5 2 Equipment provided to strengthen blood supply ** ** Train health workers in data collection and health information systems (HIS) Workers trained in data collection and HIS 213 206 11*** 214 ACTIVITY TRACER INDICATORS FOR ACTIVITY Uganda DEMAND ACCESS QUALITY STRENGTHENING 409,400 324,400 646,500 369,700 KABAROLE KAMWENGE KIBAALE KYENJOJO POPULATION DISTRICTS EXHIBIT 11 “Dose delivered:” SMGL activities implemented in districts as of December 1, 2012 Data not available * A single SMGL coordinator oversees the Kabarole, Kamwenge, and Kyenjojo districts ** Activity under way, district-level data not available *** This figure only refers to the record clerks and assistants trained in data collection— it does not include the number of nurses and midwives trained in data collection ACTIVITY TRACER INDICATORS FOR ACTIVITY Zambia Train community groups to promote facility delivery and birth preparedness (VHTs/SMAGs) Safe Motherhood Action Groups trained 13 6 24 12 Individuals trained as SMAG members 220 120 240 210 Improve birth preparedness at home through VHTs/SMAGs Birth plans distributed ** ** 56,000 16,000 Produce documentary about safe motherhood using traditional leaders Documentary airings 0 0 0 0 Run mass media campaigns on radio, engage community drama groups Campaigns conducted 0 0 0 0 Identify and engage community influencers in safe motherhood Change Champions trained 11 35 13 21 Provide basic newborn/birth supplies to pregnant women (Mama Packs) Mama Packs distributed to pregnant women 6,000 350 0 0 Increase facility EmONC capacity Health facilities upgraded to BEmONC capacity 28 34 0 7 Health facilities upgraded to CEmONC capacity 0 0 0 0 Expand/refurbish maternity wards, labs, pharmacies Expanded/refurbished maternity wards 5 0 12 0 Expanded/reburbished pharmacies 0 0 0 1 Renovate mothers’ shelters near hospitals for high risk women Mothers' shelters under renovation 5 0 12 0 Buy emergency obstetric and newborn care (EmONC) equipment EmONC equipment procured ** ** ** ** Buy ambulances, motorcycles, and motorbikes for transportation and referrals Vehicle ambulances provided to district 1 1 3 0 Motorcycle ambulances provided to district 0 0 12 0 Hire new doctors, nurses, and midwives in collaboration with the Ministry of Health New doctors hired 0 2 0 0 New nurses hired 0 7 0 0 New midwives hired 12 0 0 0 Train health workers in EmONC Health workers trained in EmONC 38 40 80 21 Train health workers in newborn resuscitation Health workers trained in newborn resuscitation 22 24 80 20 Train health workers in rapid syphilis testing Health workers trained in rapid syphilis testing 36 39 43 22 Supervision provided to maintain/improve skills in obstetrics/newborn care Health workers who received supportive supervision 24 46 20 Provide essential medicines Provide essential medicines to health facilities ** ** ** ** Provide training and oversight for maternal death reviews Health workers trained in maternal death reviews 0 24 9 Strengthen supply chains through training on procurement and stock mgmt Workers trained in procurement and stock mgmt 35 56 25 30 Build provincial and district health team capacity with SMGL staff SMGL coordinators in district 1 1 1 1 Build capacity of facility staff to supervise community health workers Facility staff trained in supervising health workers 13 0 3 26 Provide computer-based medical records (SmartCare) Computers provided to health facilities 35 39 30 17 Staff trained in computer literacy and SmartCare 73 38 78 40 Strengthen pharmacy, lab, and blood supply Blood banks 1 0 1 1 Equipment provided to strengthen blood supply ** ** ** ** Train health workers in data collection and health information systems (HIS) Workers trained in data collection and HIS 95 48 80 30 DEMAND ACCESS QUALITY STRENGTHENING 254,211 314,281 217,603 101,616 KALOMO LUNDAZI MANSA NYIMBA POPULATION DISTRICTS Data not available ** Activity under way, district-level data not available 38 MARCH 2013 had one vehicle ambulance of the three they were expecting. Kalomo received one vehicle ambulance and Mansa received three in addition to twelve motorcycle ambu￾lances. The slow pace of vehicle procurement was a major source of frustration among district health officials and implementing partners in all districts. In contrast to facility refurbishment and transport, implementing partners in Zambia made rapid and substantial progress on activities related to quality improve￾ment. A total of 179 health workers in the four districts were trained in EmONC and newborn resuscitation (“Helping Babies Breathe”), with the goal to have at least one newly trained skilled provider per facility. Following the trainings, implementing partners provided onsite supportive supervision. With the exception of Lundazi, all districts reported trainings in maternal death reviews (MDRs). Compared to Uganda, few facility-level providers were hired by SMGL partners, an issue that was mentioned frequently by district respondents. Kalomo, however, received 12 retired midwives who were affectionately referred to as “the twelve disciples” in the district. Lundazi was the only other district to report additional human resources through SMGL, consisting of two doctors and seven nurses. Implementing partners carried out a number of health systems strengthening activities in the four districts. Each district has a dedicated SMGL district coordinator to oversee activity coordination. Between 25 and 56 health workers in each district attended an extensive five-day training on logistics and supply chain management through the Essential Medicines and Logistics Improvement Program (EMLIP) provided by John Snow, Inc. (JSI). Providers in each district were also trained in computer literacy, SmartCare, and rapid syphilis testing by the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF), which also provided computers and supporting solar panels to every SMGL facility. Additionally, some providers received trainings in health worker supervision and health information systems. Synthesis Trainings and mentorship were the two most strongly implemented activities across the eight SMGL districts. VHT and SMAG trainings, EmONC and newborn resusci￾tation trainings, and onsite mentoring were also implemented extensively in both countries. This was likely due to the ability of implementing partners to rapidly recruit qualified trainers and mentors and may have been facilitated by partners’ existing relationships with district health teams. Although it is not reflected in the table due to a lack of specific data, Uganda, like Zambia, provided facility delivery incentives—Mama Kits containing some basic newborn supplies (e.g. a baby blan￾ket)—in some districts. Some mothers’ shelters were in the process of being upgraded in both countries. Although shelters and housing for health workers topped the wish lists of many of the district medical officers interviewed, new construction is not permitted under USG foreign assistance regulations. Upgrades to pharmacies and laboratories were also lacking in the first six months of the project. Notable differences between activities implemented in Uganda and Zambia were the lack of CEmONC upgrades, additional health care workers, and emergency transport vehicles in Zambia. Some of these activities may be pursued in the second half of Phase 1. The substantial increase in human resources in Uganda helped to allay the absenteeism experienced during off-site trainings, a common complaint in Zambia, which is discussed in Section 6.4 of this report. Another point of difference was that only Uganda used a voucher system and only Zambia reported the use of community influencers as Change Champions. While both countries provided train￾ings on supply chain management and logistics, this was implemented with more intensity in Zambia, perhaps because it was streamlined across the four districts by JSI, one of Zambia’s main procurers of medical supplies. 5 MID-POINT RESULTS EXHIBIT 12 Video of SMAG training in Nyimba District, Zambia SMGL health worker training, Nyimba District, Zambia Saving Mothers, Giving Life External Evaluation / Interim Report 39 6.1 Successes Perhaps the most notable findings of this evaluation is that although the effect of the SMGL on maternal mortality is not yet known there was a unanimous percep￾tion among the over 150 respondents we interviewed that SMGL was already a success on many fronts. While we heard a range of thoughtful criticisms of the program and many ideas for improvement, no interviewees considered SMGL a failure. The respondents interviewed for this interim report were policymak￾ers, funders, and implementers involved in SMGL in Zambia and Uganda. The perspectives of communities and health providers will be captured in the next wave of our data collection and included in the final report. In listing key SMGL achievements, participants noted that early data from routine health information systems showed increases in institutional deliveries in intervention districts. They also reported improved awareness of and demand for maternity services from women in communities, and the perception that the level and quality of obstetric and newborn services at facilities had improved markedly since the start of the program. This perspective was shared by interviewees at central and district levels in both countries and by respondents with different roles in SMGL from policy to clinical care. The activities deemed most successful spanned the full range of the Demand￾Access-Quality-System Strengthening framework. These included Village Health Teams and Safe Motherhood Action Groups in Uganda and Zambia, respectively, and Mama Kits/Packs in both countries (Demand); transportation vouchers and hiring of new health workers in Uganda and building new maternity wards and operating theaters (Access); new equipment, emergency obstetric and newborn resuscitation training and mentoring (Quality); and community data collection and improvement in health information systems (System Strengthening). In short, those interviewed agreed that in the first six months of operations SMGL had achieved notable successes in both the community and the health system. Respondents were also asked to comment on the drivers of SMGL’s success to date. One of the most frequently cited success factors in both Zambia and Uganda was the strong national partnerships—between Ministry of Health, US Government, and a range of implementing partner organizations—that formed around SMGL in each country. Respondents pointed to the support for the initiative from the Ministry of Health, particularly at the outset in selecting interventions and SMGL districts. They also noted that despite SMGL funds being “off-budget” (i.e. not channeled through national governments) SMGL worked largely through the existing health system, rather than pursuing stand-alone solutions. Many participants noted that although there were points of tension, the “unusually collegial” relationship between CDC and USAID in both countries was an essential ingredient to the early success of such a large program. An imple￾menting partner respondent in Lusaka commented on the “fabulous interagency “SMGL did not create parallel structures like other projects in the past have: SMGL was designed to work within the existing [health system] structure and communities.” — Uganda District Level Implementing Partner 6 Successes, challenges, lessons learned 40 MARCH 2013 collaboration” on SMGL as a key driver of success. This sentiment was echoed by a senior Ministry of Health official in Uganda, who observed that the collaboration among USG agencies in SMGL was “the best I’ve ever seen” of any US-funded health program. USG respondents in country also pointed to the important role of CDC and USAID headquarters—and “USG big-wigs”—in supporting programming. Respondents agreed that implementation within two months of the original SMGL plan would have been impossible without the presence in both countries of organizations with existing relationships and, crucially, contractual arrange￾ments with USG agencies. Most of these relationships were originally developed for PEPFAR programs. These implementing partners’ experience delivering complex HIV and other programs at district level and their strong relationships with Ministry of Health counterparts at central and district levels greatly facili￾tated their ability to deliver interventions quickly. In this way the large PEPFAR programs in the two countries served as a launching pad for SMGL. Frequent meetings among partners, weekly at the outset, enabled input and participation and promoted a sense of common purpose. While some noted that district medical officers were sometimes overwhelmed with the speed and intensity of SMGL implementation amidst other health prior￾ities, all agreed that the buy-in, commitment, and in many cases the personal commitment of district medical officers in intervention districts were essential preconditions for success. Respondents noted that the large financial commitment by the US Government in SMGL captured stakeholders’ attention and elevated maternal health efforts— at least temporarily—on the agendas of policymakers and implementers. While many pointed to the tremendous challenges presented by the compressed time￾line, US government respondents in particular noted that the “sprint” nature of Phase 1 of SMGL generated excitement among national actors and their develop￾ment partners that one rarely sees in maternal health or health system initiatives. At the same time, respondents pointed out that this level of energy cannot be maintained over the long run and that the program must adapt to permit integra￾tion into the health system. Interviewees from the USG and Ministries of Health acknowledged, however, that the need to produce results within 12 months hindered country ownership. This is discussed in more detail below. Finally, the emphasis on community involvement from the outset of SMGL was cited as a critical driver of the early success of SMGL. Safe Motherhood Action Groups (Zambia) and Volunteer Health Teams (Uganda) were credited not only with promoting facility births but also with changing community views about the inevitability of maternal deaths. Exhibit 13 below summarizes the SMGL program activities that were the most frequently cited successes by showing the proportion of interviewees who mentioned each item in response to a question on key successes in the two coun￾tries. Below we briefly discuss these. Community engagement through VHTs and SMAGs While Safe Motherhood Action Groups (SMAGs) and Volunteer Health Teams (VHTs) existed in Zambia and Uganda before SMGL their role was greatly intensified for SMGL. This was done through both additional recruitment and training of existing participants. Both groups were also charged with new tasks, notably community-level data collection on pregnancies and maternal and newborn deaths. In Uganda, respondents reported that VHTs helped engender “community commitment” and “increased confidence in providers.” SMGL “brought hope to a lot of people in the community” who were said to have received the project it with “[It is] very special in my experience to see organizations work this closely and this well.” — USG Zambia Official “[Before SMGL] there were many mothers dying in silence. At least now when mothers die, people notice and they try to learn from it. It’s a big issue. Now when a mother dies we know before lunch.” — Uganda District Level Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 41 0% 10% 20% 30% 40% Refurbishing operating theaters Equipment & supplies Improved data management Mentoring Transport Obstetric & newborn care trainings Community engagement (VHTs/SMAGs) 32% 15% 25% 6% 15% 17% 43% 30% 11% 25% 20% 13% 8% “open arms.” Community dialogue meetings facilitated by VHTs were mentioned as a particularly powerful means to mobilize community members. In Zambia, SMAGs and community leaders were credited with improving the communities’ acceptance of SMGL and building trust in the health system. Chiefs and other community leaders participated in brainstorming sessions for SMGL, for example, identifying poor provider attitudes as a key barrier to women deliv￾ering in facilities. SMAG members were praised as the true demand creators, with a number of respondents reporting facility deliveries were higher at facili￾ties with trained SMAGs. Health provider training and mentoring in obstetric and newborn care The extensive trainings in emergency obstetric care, neonatal resuscitation (“Helping Babies Breathe”), and specialized training in obstetric surgery in Uganda were highly praised in both countries. District medical officers and providers said that the trainings improved skills and the confidence and morale of doctors, nurses, midwives, and clinical officers. In health systems where providers are frequently overworked and do not have the skills or tools to do their job, we heard that SMGL trainings and mentoring made providers feel “invested in instead of policed.” In Zambia, district respondents noted that staff could now manage abortion complications, postpartum hemorrhage, and sutur￾ing on the ward rather than in the operating theater. Participants also noted the high levels of enthusiasm among clinical staff for the initiative. Improved availability of transport The success of transport investments was mixed, with a much more positive assessment in Uganda than Zambia. The investment in transport was seen as “Health workers have been empowered [by the trainings] and have risen to the occasion in a way that they haven’t with many projects.” — USG ZAMBIA OFFICIAL EXHIBIT 13 Frequency of reported successful activities by country Uganda Zambia 42 MARCH 2013 a key contribution of SMGL in Uganda. Specifically, respondents in Uganda felt that the provision of emergency vehicles and motorcycles (boda bodas) was one of the most important successes of SMGL. A district MoH respondent noted that, “boda bodas for mothers is a huge success because it integrates services and norms that already exist in our community to encourage maternal health. Vehicle ambulances are also a hugely important success and have increased our capacity.” An implementing partner in the district observed, “Transport was a big problem solved and the mothers have been enjoying coming to the health units much more than before.” In Zambia district health officials expressed frustration that few ambulances had been delivered six months into the program. Improved data management In both countries MoH respondents and implementing partners cited improve￾ments in health information systems that resulted in better and more timely data on mothers and newborns. This was particularly the case in Zambia where there was a substantial investment in electronic health cards that linked with facility patient databases. Also in Zambia, community key informants trained by the Central Statistics Office to conduct verbal autopsies were cited as key to improved understanding of deaths in the community. In Uganda, an implementing partner commented that as a result of SMGL efforts data utilization in SMGL districts was higher than he had witnessed in other districts. Another benefit observed in Uganda was that data from private sector facilities is now better integrated into district information systems. Provision of equipment and supplies and facility refurbishment Facility improvements were another important SMGL success. In Uganda, respondents enthusiastically reported that refurbished operating theaters were functional for the “first time in years.” The expansion of maternity wards was another important success given that lack of privacy was a frequently mentioned barrier to women wanting to deliver in a facility. Although facility improvements in Zambia were at an earlier stage than in Uganda, respondents noted that new supplies (e.g. delivery kits, IV tubing, mattresses, and blankets), medicines, and equipment (e.g. manual vacuum aspi￾rators, ultrasound machines) helped to improve provider morale, the quality of obstetric care, and encouraged women to use facilities. 6.2 Innovations Reducing maternal mortality has proved exceedingly difficult for low- and middle-income countries around the globe. Most regions, with the exception of East and Central Asia, will not meet Millennium Development Goal 5—reduction of the maternal mortality ratio by 75% between 1990 and 2015.34 It is clear therefore that business as usual will not be sufficient to bring about meaningful improvements in maternal survival and that there is a need for new ideas on how to implement what we know works. In our implementation evaluation of SMGL we were particularly interested in identifying interventions, service delivery approaches, and organizational strategies that provided new approaches for tackling maternal and newborn mortality. To this end, we asked SMGL stakehold￾ers in country what they felt were the most innovative aspects of SMGL. We identified three themes on innovation: new technologies, improved ways of working, and new ideas for community mobilization. It was notable that the innovations mentioned were not related to provision of clinical services per se. Boda boda in Kyenjojo District, Uganda Newly refurbished Maternity Ward, Kamwenge District, Uganda 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 43 Instead, respondents spoke about new ways to link women to services, change community perceptions, and organize health services. The results are summa￾rized in Exhibit 14. Respondents in both countries described the intense interagency collabora￾tion between CDC and USAID as a new way of working on a shared agenda for US agencies in global health. The use of SMGL coordinators in the districts to support district Ministry of Health structures and maintain communication was also cited, as was the use of VHTs and SMAGs to collect community-level data. Engaging non-standard actors in the fight for maternal survival was another innovation. This included collaboration with private facilities in Uganda to expand safe delivery options for women. Churches in Zambia were encouraged to deliver safe motherhood messages to congregants on Sundays. Traditional community leaders were engaged as Change Champions in several Zambian districts, acting to motivate women to use services and to communicate barriers to access to medical staff. Another new way of working was the explicit effort to deliver services to women who could not get to the health system. In Kibaale, mobile clinics attended to pregnant women living in remote areas who may otherwise have no access to services. Finally, mentorship by experienced clinicians onsite was cited as an extremely effective innovation that permitted nurses to stay in their facili￾ties and hone their skills in “real-world” settings. Ways of working ■■ Onsite mentoring ■■ SMGL district coordinators ■■ Improved referral networks ■■ High-level partner coordination ■■ Data collection by VHTs & SMAGs ■■ Mobile clinics for maternal health services ■■ Involvement of private facilities ■■ Churches deliver safe motherhood messages ■■ Local leaders as Change Champions ■■ Phased implementation approach ■■ Emergency drills for mentoring ■■ Referral & feedback pilot ■■ Mothers shelter logs Community mobilization ■■ Safe motherhood day/week ■■ Mama Kits/Packs ■■ Transport committee ■■ ANC4 incentive system ■■ Transport & service vouchers ■■ Income generating activities for VHTs ■■ Male involvement promoted at soccer ■■ Partnership with village savings association ■■ Incentives for postnatal care ■■ Culturally appropriate birth plans ■■ Incentives for traditional birth attendants ■■ Drumming group promotes safe motherhood ■■ Recognition meetings for exceptional facilities Technology ■■ Local radio programs ■■ Solar lighting ■■ Phones for VHTs ■■ Bodas for mothers ■■ BABIES matrix for facility monitoring of newborn indicators ■■ Zambulances-terrain appropriate vehicles ■■ Kit (tackle) boxes for obstetric emergencies ■■ SMS pilot for following pregnant women in the community UGANDA BOTH ZAMBIA EXHIBIT 14 SMGL innovations by country “[The husband] has the bicycle or the money. Bring men on board from the onset—this is critical—or you do all this [awareness raising] and he will say ‘stay, don’t go.’” — Zambia District Level Implementing Partner 44 MARCH 2013 Prominent community mobilization innovations were Mama Kits and Mama Packs in Uganda and Zambia, respectively, to create demand for facility deliv￾eries. Districts also held a Safe Motherhood Day in Uganda and were planning a Safe Motherhood Week in Zambia, to galvanize women to come for services and increase community awareness. In Uganda, one district organized soccer matches using the half time break to encourage men to become informed about and involved in their partners’ pregnancies. One district in Zambia spoke about traditional drumming and drama groups that used performances to inform community members about maternal and newborn health. Solar lighting installed in facilities was an innovative technology used in Uganda and Zambia. Providers were relieved that they no longer had to deliver babies at night by the light of their mobile phones. In Zambia, one implementing partner created “kit boxes” from fishing tackle boxes after they observed that providers frequently left a woman’s bedside during an obstetric emergency to obtain supplies. Kit boxes for postpartum hemorrhage, eclampsia, and other obstetric emergencies are stocked with the necessary supplies on maternity wards in this district. In Uganda, mobile phones were provided to VHTs to improve their communication with implementing partners, pregnant women, and drivers. An implementing partner in Zambia is conducting a pilot using mobile health technology to send text messages between facilities and community volunteers to assist in tracking pregnant women within their zones and promote ANC atten￾dance and facility delivery. 6.3 Challenges Although the overall perception of SMGL was positive, respondents were frank in noting a number of challenges and weaknesses in program design and imple￾mentation. In analyzing the interviews we repeatedly found that some successes were also mentioned as challenges—sometimes by the same respondent. This is unsurprising given that large, complex programs, however helpful, create heavy burdens on policymakers and implementers, creating new problems and exacer￾bating existing bottlenecks. Respondents listed a number of challenges that they felt impeded SMGL imple￾mentation and its potential for achieving and sustaining reductions in deaths of women and newborns. These ranged from organizational issues related to SMGL itself to deep-rooted problems endemic to development efforts. They are discussed below. Short timeframe While in many ways a motivator, the one-year timeframe of Phase 1 was a major challenge for implementation and an even larger one for sustainability. The pres￾sure to deliver was tremendous, making it difficult to meaningfully involve the Ministry of Health in implementation. As one respondent noted, “The short time￾frame … incentivizes ‘short cuts’ and approaches that do not promote capacity building or sustainability. For example, it is very tempting to just hire short-term staff to go to the districts and do things directly, instead of doing the right thing and working through the MoH.” SMGL’s rapid start-up required nimble use of existing USG funding and procurement mechanisms. New funds were made available to CDC-funded imple￾menting partners within a very short time and existing funds were leveraged to support USAID-funded implementing partners, enabling the swift launch of SMGL in both Zambia and Uganda. However, many of the awards were limited to one year. Multiple implementing partners concurred that “running a long term program on a short term budget” posed important implementation problems, “We are moving at such a fast pace that one actually needs to sit down and think, ‘out of what we have done, what really is practical and sustainable and what can continue to be done?’” — Zambia Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 45 most notably when it came to hiring staff and sequencing interventions: “We were trying to cobble things together on the fly, sometimes not as efficiently as you would want things to be.” Some activities were explicitly modified due to concerns that funding may not be there after the end of Phase 1. For example, recently in Uganda voucher distri￾bution was limited to women in their third trimester of pregnancy to ensure that funds would be available when women actually deliver. The rapid pace and short timeframe of the SMGL program also occasioned frustration among USG agency staff, who were attempting to accelerate tradition￾ally slow processes such as procurement and approval of no-cost extensions in Atlanta and Washington. Weaknesses in coordination and duplication of activities Despite the predominantly positive view of the SMGL partnership, respondents in both countries pointed out that coordinating the large portfolio of partners and interventions presented major challenges. Respondents were concerned about a lack of clarity about the roles of CDC and USAID in the leadership of the SMGL program. The involvement of both US agencies and their respective grantees (i.e. implementing partners) created a large group of stakeholders, each with their respective agendas and programmatic approaches. For example, many implementing organizations already had maternal care interventions “on the shelf” and were understandably reluctant to change these, particularly for a one-year program—despite acknowledging the value of a coherent intervention package across districts for implementation and assessment of effects. Although respondents agreed that frequent partners’ meetings were helpful for sharing action plans, many wished for more explicit guidance from the US government leads and/or the Ministry of Health on the content of SMGL activi￾ties—an arbiter of the SMGL approach. Respondents noted that only a mandate “from above” would convince implementing partners to change their existing approaches to activities like training, mentorship, supervision and support for procurement and infrastructure. Respondents also spoke about the lack of clear roles for different implement￾ing partners. Implementing organizations and district MoH leaders in Uganda pointed to duplication and competition among partners, most notably with the voucher program in which several partners offered different versions of vouch￾ers at different prices in SMGL districts. Private health providers involved in Uganda mentioned incurring higher operational costs due to a lack of harmoni￾zation and coordination, again in the voucher system. Zambian respondents also noted the duplication of some activities, such as training for Safe Motherhood Action Groups, which was done by several groups, and complained about confu￾sion about who is doing what. Communication between capital and district was also not optimal. District offi￾cials in Zambia reported that partners did not always inform the district health office about SMGL activities they were conducting in the communities, and that decisions were sometimes made with the MoH centrally, without discussion with the implementing districts. Funding and logistic gaps While the use of existing contractual mechanisms between USAID and CDC and their respective implementing partners enabled rapid provision of start-up funds and swift initiation of SMGL services, the one-year funding stream created difficulties for several implementing partners, whose grants ended at the end of “It was surprising to be so uncoordinated even though we were all working on the same thing in the same district.” — Uganda District Level Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED 46 MARCH 2013 September 2012. The slow pace of acquiring no-cost extensions meant that at least one Zambian partner had to suspend activities and pull staff out of the field for several months. USG respondents in Zambia mentioned that budgetary issues led to a scaling back of their initial transportation and refurbishment goals. USG procurement regulations also markedly limited the speed with which partners could purchase vehicles for use as ambulances; this took more than a year in some cases. As of December 1, 2012 some districts were still waiting for renovated mother’s shelters, additional facility-level midwives, updated radio systems, and essential medical equipment. District-level MoH respondents in Uganda noted that inadequate blood supply was a remaining barrier to safe motherhood that had yet to be addressed by SMGL. Finally, respondents cited ambiguity around roles and responsibilities for equipment training, delivery, and maintenance, which contributed to underutiliza￾tion of equipment that was available. Difficulties with transport The same difficult geography and weak transport infrastructure that contributed to the low rates of facility delivery in SMGL districts proved formidable barriers to SMGL implementation. The most common issue noted by district-level respon￾dents in both countries was the ongoing transportation barriers for both SMGL implementers and women seeking care, due to rough terrain, distance, and inac￾cessibility of health facilities during the rainy season. Although the issues had been anticipated, and SMGL included interventions designed to address them— in particular different types of vehicles—they continued to undermine successful implementation and facility delivery rates. Indeed, SMGL transport interventions were sometimes not suited to the terrain. For example E-rangers (three-wheeled motorcycle ambulances) were not able to traverse waterlogged dirt roads. In Zambia, frustration with transport was high due to procurement delays for badly needed ambulances and motorbikes. Human resources Staffing shortages were overwhelmingly cited as a persistent challenge in SMGL implementation at all levels in both countries. The baseline health work￾force levels were very low—even compared to other countries in Africa—with an estimated 0.48 skilled health workers (e.g. doctors, nurses, midwives, clinical officers) per 1,000 population in Zambian SMGL districts. There were 0.25 health workers per 1,000 in Uganda’s SMGL districts although these figures do not include all private sector providers. These numbers are one-fifth and one-tenth of the recommended minimum health worker numbers to provide essential maternal and child health services, respectively.35 Respondents in Uganda noted that the 134 additional doctors, nurses, and midwives hired by SMGL implementing partners—representing a 25% increase in overall health workforce—were essential to scaling up obstetric services. However, they noted that even this number was quickly proving insufficient as more and more women came to health clinics for antenatal care and delivery. Because the new staff were offered higher salaries to boost recruiting, Ministry of Health respondents at central and district levels were concerned that they would be unable to retain the staff once the project was over. Zambia reportedly received 21 providers across two districts—far lower than was initially planned—and concerns over insufficient health workforce were voiced in all interviews in the districts. As one district-level informant noted, the scarcity of human resources “will undermine everything we do if it is not addressed in some way.” “Low staffing cannot keep up with the increasing demand brought on by SMGL; we are swamped and overworked.” — Zambia District Level MoH Official 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 47 “SMGL has helped with our ability to gather local data, which has helped us with planning in other areas of our health system.” — Uganda Implementing Partner Finally, as described in greater length in the following section (6.4), several respondents in Zambia in particular felt that the proliferation of off-site training in Zambia had exacerbated facility-level staffing shortages by taking workers away from their clinics. 6.4 Synergies and unintended consequences Because health systems are complex adaptive systems, the introduction of new programs is often accompanied by a host of secondary effects—including synergies and unintended consequences. In some cases, these may be a surprise to program implementers, while in others, they are predictable enough to antici￾pate and plan around. In both Uganda and Zambia, the launch of SMGL activities resulted in outcomes that were not primary program objectives. These “secondary” effects—both positive and negative—are discussed below. Positive synergies and positive unintended consequences At the national level, respondents in both countries described a range of positive changes due to SMGL. SMGL clearly increased the enthusiasm for action on maternal and newborn health among national policymakers and their devel￾opment partners. However, in Uganda, the program also sparked action to strengthen the health system more generally. The high level of parliamentary interest in SMGL, most notably in Uganda, was cited as a key factor for prompting lawmakers to conduct a review of the national health budget and increase salaries for physicians nationwide. At the district level, Ministry of Health respondents in Zambia observed a “spill over” of skills and knowledge from SMGL to non-SMGL staff. There were positive effects on staff morale, as health care workers saw and appreciated the improvement in the quality of care provided to patients. Some respondents felt that the increased attention to record-keeping and data management would also be likely to spill over to non-SMGL programs, to beneficial effect. There were also synergies with HIV and syphilis programs. Increased demand for facility-based delivery led to reports of increased uptake of HIV testing, PMTCT services, and testing for syphilis and other sexually transmitted infections. At the community level, the inclusion of traditional birth attendants (TBAs) in SMAGs and VHTs was described as an important and positive secondary effect of SMGL in both Zambia and Uganda, shifting their role from an impediment to safe motherhood to an ally in promoting facility deliveries. Negative synergies and negative unintended consequences Concerns were expressed about maternal health overwhelming other health sector priorities. One implementing partner characterized this risk by observing that “HIV has become stale. This [SMGL] seemed tangible and new.” Similarly, dozens of respondents expressed concern that the exclusion of family plan￾ning from SMGL had not only created a missed opportunity to reduce maternal mortality, but had de-emphasized the importance of family planning as a health strategy in the country more broadly. Respondents also noted that the SMGL districts had, to some extent, benefited at the expense of other districts, with implementing partners in both countries noting that some activities and programs were redirected from non-SMGL to SMGL districts. Similarly district level MoH officials noted the diversion of time and district-level staff away from non-SMGL activities: one District Medical Officer noted that 60% of his time was spent managing SMGL, which necessitated 6 SUCCESSES, CHALLENGES, LESSONS LEARNED 48 MARCH 2013 O F F S I T E O N S I T E ZAMBIA NYIMBA DISTRICT O N S I T E O F F S I T E UGANDA KIBAALE DISTRICT “Alarm” trainings are a series of trainings on emergency obstetric and newborn care (including CEmONC, BEmONC and “Helping Babies Breathe”), conducted by the Association of Obstetricians and Gynaecologists of Uganda. Graphs are based on a 250 day work year. One district was selected from each country based on data availability. Shaded areas represent SMGL￾related trainings and mentorings experienced by a representative SMGL health worker in a Ugandan HC III and a Zambian Health Facility. EXHIBIT 15 A “year in the life” of a typical SMGL health worker 5 Days First alarm training 5 Days Fourth alarm training 4 Days Second alarm training 231 Days Work days with no SMGL training 5 Days Third alarm training 20 Days Emergency obstetric and newborn care 5 Days Essential medicines logistics improvement program 191 Days Work days with no SMGL training or mentoring 4 Days Helping Babies Breathe 5 Days Maternal death review 5 Days Medical records 20 Days Mentoring delegating other health concerns to subordinates. Although the SMGL logic model addresses the full range of Demand, Access, Quality and Systems strengthening interventions, the sequencing of activities was not optimal and created unintended negative effects. For example, trainings of health workers and community groups (SMAGs and VHTs) were launched months and in some cases more than a year before improvements in infrastruc￾ture, supply chains, and transportation. The community level activities generated demand for services before the expansion and improvement of supply. This led to surging numbers of pregnant women coming to facilities before facilities were expanded, upgraded, or staffed with new workers. Respondents were concerned that poor experiences in the facility would lead to disappointed mothers and communities and a decline in trust in the health system. There were also complaints of burnout and overload for “overwhelmed” staff at some SMGL facilities as well as stock outs of medi￾cines and supplies, power shortages, and water shortages that were attributed to the marked increase in demand for services. SMGL provided extensive training to facility-level clinicians. Because the bulk of this training was provided off-site and not at the health facility, the training removed health workers from clinics and thus itself reduced availability of services. Off-site training exacerbated health worker shortages particularly in Zambia where only a handful of new health workers were hired for SMGL. Exhibit 15 illustrates a “year in the life” of a typical SMGL health worker. Using a base of 250 work days, we estimated how many days per year a typical midwife at a health center spent in SMGL-related trainings (off-site) and mentoring (on-site), based on actual trainings delivered in the district. In Nyimba, a midwife could be expected to spend 39 work days at off-site trainings. District officials in Uganda observed that the higher salaries of workers hired as part of SMGL created resentment among “non-SMGL staff caring for other patients.” A few respondents were also concerned about the “poaching” of staff from other health programs. Two potential unintended consequences were noted at the community level. In Uganda, we repeatedly heard concerns that efforts to improve access to and quality of facility-based deliveries were actually incentivizing women to have more babies, although no concrete data were provided to support this assertion. In Zambia, district health officials and implementing partners reported that some village leaders had begun imposing penalties (e.g. a fine of a chicken or a goat) on the husbands of women who decided to deliver at home instead of a health facility. These efforts, which were not sanctioned by SMGL and maybe perceived as coercive, are concerning as they may create a punitive instead of a supportive environment for women and are likely to affect the most vulnerable women —the poor and women living farthest from facilities—disproportionately. Their effects on women and communities should be carefully studied as SMGL goes forward. Exhibit 16 synthesizes the primary and secondary effects—both positive and negative—that were most apparent to SMGL stakeholders by the six-month mark and that we discussed above. 6.5 Country ownership and sustainability Respondents in both countries reported that SMGL was highly congruent with national maternal health strategies both in terms of policy focus and specific activities, with the notable exception of family planning. All agreed that this congruence was a key prerequisite for country ownership. From the outset, USG leads engaged their Ministry of Health counterparts in planning for SMGL implementation and made a concerted effort to include them in implementation “I think that we must ensure that our communities should own their health. No one should feel that their health is in the hands of the US government.” — Uganda District Level MoH Official 6 SUCCESSES, CHALLENGES, LESSONS LEARNED 50 EXHIBIT 16 Matrix of reported primary and secondary effects PRIMARY EFFECTS SECONDARY EFFECTS POSITIVE NEGATIVE ■ ■ ■ ■ ■ ■ ■ ■ ■ ■ Increased community commitment Increased community confidence in providers ■ ■ ■ ■ ■ Increased male engagement ■ ■ ■ ■ ■ ■ ■ ■ ■ Increased demand for facility-based delivery Increased advocacy for maternal health Enhanced infrastructure Improved quality of BeMONC, EmONC, HBB services Community maternal deaths captured due to community key informants in Zambia Enhanced supply chain Increased uptake of HIV and syphilis testing Increased interest from new/other donors Improved overall coordination between CDC, USAID, DoD, PC Spillover of skills from SMGL￾trained staff to other staff Increased advocacy for health in general Increased national health budget, higher salaries for doctors (in Uganda) Spillover data management skills to outside programs Shift of TBA role from safe motherhood impediment to facility delivery advocate ■ Improved health worker morale Improved and more timely data on mothers and newborns Major time commitment for DMOs; of SMGL Heavy workload for staff, stock outs, and shortages due to increased demand Health programs diverted from non-SMGL districts Penalties imposed by chiefs for non-facility deliveries in Zambia Tension amongst SMGL and non-SMGL health workers Staff shortages due to off-site trainings Crowd-out of non-MCH services and programs in districts ■ Saving Mothers, Giving Life External Evaluation / Interim Report 51 meetings and decisions. In both countries, the ultimate selection of districts was a Ministry of Health decision. The Ministries’ visible role during the planning process, including in the development of district work plans, in turn generated support for the program among district health officials and community lead￾ers. Respondents in both countries also pointed to the support of First Ladies, high-ranking MoH staff, and parliamentarians as examples of high-level buy-in. Informants highlighted the key distinction between government ownership versus government support of the program. The fact that SMGL was almost entirely donor￾funded and followed an externally-set timeline raised questions about the extent of true government ownership at the national level as well as the sustainability of the initiative. Many respondents pointed out that for true ownership national govern￾ments should have made a financial commitment supporting the aims of SMGL at the outset, particularly in areas such as poor roads, human resource shortages, and crumbling infrastructure, that cannot be addressed by donors. Hands-on engagement with the program was felt to be highest at the district level. Respondents in both countries highlighted the important role that local politi￾cians and traditional leaders had played in generating district- and community-level support and facility-based deliveries. “Ownership is at the district level because this is where implementation occurs,” said one informant. Below, we report some specific findings for each country. Uganda Although the majority of respondents at both district and central levels rated MoH ownership of the SMG program relatively high, with most giving it an average rating of 7 out of 10 on a scale where 10 was excellent, respondents were quick to point out that this only referred to the MoH’s “enthusiasm” for the program rather than a guiding role in implementation. Some noted that the program was initially viewed as “made in the USA” and that “you don’t see the Ministry involved in planning meetings, at the Kampala and district level” to the extent they should be involved. A general sentiment voiced by central and district level partners was that government ownership was lower than it could have been due to the short timeline for the project: “The MoH…are willing to participate [but] one year is too short. It’s not how government works.” However, respondents noted despite their limited role in design and oversight of SMGL, the MoH demonstrated its commitment to the goals of SMGL in numerous ways, including supporting construction and refurbishment projects, and, above all, investing in health workers: “No one knew that government could be pushed as far as they were on human resources. And now the government is trying to sustain the program.” Respondents pointed to the doubling of doctors’ salaries at the Health Center IV level (health facilities with inpatient care) that was approved by Uganda’s parliament in fall of 2012. The Honorary Victoria Rusoke, Member of Parliament in the SMGL district, Kabarole, was mentioned numerous times for her skill in advo￾cating for the salary increase as a means to bring health workers to SMGL and other rural districts. Her role as a champion for women’s health tremendously energized SMGL district implementers. Most respondents agreed that, “at the districts, country ownership has been high. Honestly, they are doing everything they should do and it is working.” District health officials were especially commended for being aware of all the SMGL partners and activities taking place in their districts, being responsive to the needs of SMGL part￾ners, and for making time to attend and coordinate meetings among the different stakeholders in their districts. “[A maternal death] is a big big deal, even the DHO and other local politicians are talking about it and they are now vigilant.” — Uganda District Level Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED 52 MARCH 2013 A number of respondents highlighted the role that the Director General of the MoH had played in chairing SMGL meetings in the districts and explained that this showed that the program was important to the government. An implementing partner had this to say about district ownership: “I would say whenever we have needed the district to do something for us they have done their best to get it done. The political will is there and the technical people have gone out of their way to make sure services flow seamlessly.” District health officials, in turn, stated that their interests had been adequately taken into account. One district health official had this to say: “We are really involved in these activities, they have taken us into account as they have planned, designed, and implemented”. A commonly expressed sentiment across all respondents was that the level of district ownership was being greatly hampered by the fact that they have not committed funding: “The districts can’t be 10 out of 10 because they do not have financial resources. The next phase should be the government commitment.” Zambia Many respondents commended the MoH for being committed to reducing mater￾nal mortality prior to SMGL, citing as an example that the President’s wife was a physician who promoted facility deliveries during public appearances. The general view among respondents in Zambia was that ownership of SMGL at MoH level was “good,” but that actual participation in planning and implementation lagged the expressed interest. Multiple respondents attributed this largely to the fact that the MoH was “stretched thin,” both financially and in terms of human resources, and was thus not always able to participate in the many meetings and planning activities required to launch SMGL. Some respondents noted that although the MoH had been actively involved in the planning phase, including the selection of SMGL districts, the organization of various trainings, and visits to the districts, this level of involvement had waned during actual implementation. The rapid pace of SMGL did not fit well into existing Ministry decision-making processes. One respondent noted that, “this is a fast moving freight train and it was difficult for them to jump on board.” Several respondents pointed out that the elections and subsequent creation of the Ministry of Community Development Mother and Child Health further hindered effective engagement of national officials: “Between December 2011 and December 2012 we had a great lack of clarity as to which Ministry was going to be responsible for what. In December 2012 they finally laid out what each ministry would do.” Positive examples of MoH support for maternal survival included introduction of programs with direct entry into midwifery training, introduction of a quality improvement curriculum for nursing schools, and the fact that many SMGL train￾ings had been initiated and organized at national level. The majority of respondents in Zambia rated district ownership of SMGL very highly and described the districts as being “in the driver’s seat.” Implementing partners reported that district health officials were highly invested in the initia￾tive and regularly participated in SMGL activities taking place in their districts. The District Medical Officer of Mansa even became a mentor in the SMGL mentor￾ship program, supporting nurses and midwives in providing quality obstetric care in the clinics. An implementing partner noted: “I can just call the district office and they will give me a vehicle, because they know what I’m doing is not for [this implementing partner], but it is for Mansa District”. Respondents in Zambia noted the involvement of key local leaders, such as the chiefs and district commission￾ers, as positive, and important, examples of SMGL ownership at district level. “For us the district has been very instrumental to coordinate and provide everything that we need to do. They’re excited like everybody else... They’ve been involved in everything that is being done.” — Zambia District Level Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 53 “The private sector brings in innovative ways of dealing with [equipment] like ultrasound machines, for example. The public sector is very poor in maintenance and the private sector has the ability to play an important role.” — Uganda District Level Implementing Partner Sustainability Two major opinions emerged among respondents in both countries regarding the sustainability of SMGL moving forward. A large group of respondents expressed concern that all the gains and progress made in addressing maternal mortality in the districts would be threatened by the end SMGL funding without a clear transition plan to the national government. These respondents cautioned that, “if we don’t think about sustainability “we are just ‘fixing pipes’ but not seeing where there are leaks.” The lack of a long-term plan for SMGL created “a wave of uncertainty [for managers and health workers] because they have seen so many programs come and go. It creates anxiety because they are worried the tap will be turned off.” There were clear tensions between the short period of time that partners had to implement activities and planning for sustainability. One respondent noted “It would take five years of sustained inputs for this program to be anywhere near sustainable.” In Uganda, Ministry of Health respondents worried about the country’s ability to retain SMGL facility staff whose higher salaries would be difficult to “absorb by a government with limited resources.” District health officials and district-level implementing partners felt strongly that putting the community in the driver’s seat was important for sustainability. For example, one partner noted that a year into the project villages should start running the community dialogue meetings themselves and contribute to them without continued support from implementing partners. Another noted that the community should be empowered to look after certain costs themselves, such as purchasing a community motorcycle to bring women to facilities, to prevent the attitude of “letting others do things for them.” There was recognition that this would require income-generating activities to create capital in the community. Most implementing partners felt that volunteer health team members should receive meaningful monthly stipends if they were to continue their work into the future. In Zambia, an implementing partner respondent in one district worried that the level of help their organization was providing the DHO (e.g. bringing in equip￾ment, providing support by phone during obstetric emergencies) may be creating a dependence that would prevent the district from running the program once external funders withdraw from SMGL activities. Certain activities were consid￾ered more sustainable included maternal death reviews and trainings, which had occurred to varying degrees before SMGL. A smaller group of respondents in both countries was more positive about SMGL sustainability moving forward. These respondents noted that the SMGL program had made substantial investments in strengthening district health systems and that these investments would ensure continuity of SMGL. These included infrastructure investments such as operating theater renovations and health information systems. They also pointed to the existing maternal health expertise at the Ministry of Health and the involvement of key leaders and stakeholders as an important investment that would ensure SMGL sustainability moving forward. 6.6 Role of the private sector Private sector involvement was substantially greater in Uganda than in Zambia, reflecting the larger role of the private sector in health care delivery. The private sector in Uganda was involved in transport and delivery care. Pregnant women were given vouchers that could be redeemed for motorcycle and taxi transport to facilities for delivery and antenatal care as well as for ambulance services from lower- to higher-level facilities if an obstetric emergency arose. Further, SMGL 6 SUCCESSES, CHALLENGES, LESSONS LEARNED 54 MARCH 2013 implementers contracted with private clinics and hospitals to provide antenatal and delivery care to quickly expand the range of facilities that women could deliver in using delivery vouchers. One of the implementing partners in Uganda also provided performance-based contracts in which investments were made in equipment and private providers were paid bonuses for meeting and exceeding targets for four antenatal care visits and deliveries. Staff from private facilities also provided mentorship and supportive supervision at lower level facilities. The private sector was also involved in other components of SMGL in Uganda. Corporate contributions to SMGL were noted, including discounted phones and airtime (from Warid and MTN), fuel (from Total Gas) and discounted radio airtime. Ugandan professional associations of obstetricians, pediatricians, and anesthe￾siologists were contracted to conduct training and mentoring of health staff to increase technical competence in life saving maternal, surgical, and newborn resuscitation. There was broad consensus among Ugandan implementing partners and district government officials that the public/private partnerships in the SMGL focus districts enhanced service delivery and quality of care for women in the districts. However, some expressed concern that SMGL’s encouragement of private facilities to expand their capacity might draw patients away from the public system. Subsequently, direct support to the private sector should be balanced with overall goals of strengthening the public health system—the main means for sustaining SMGL services for the foreseeable future. One specific flaw that some pointed to in the model was that private-sector vouchers were not usable in public sector facilities, even in the case of a medical emergency. Overall, private sector respondents reported a positive experience with the SMGL program and noted several benefits to participation in the project—partic￾ularly the ability to upgrade their equipment and expand their client base. Private facility managers suggested that standardizing vouchers would help reduce their transaction costs and that the flat fee reimbursement method for Caesarean sections needed to be adjusted for complicated cases. All respondents were concerned about sustainability: the private sector worried that they would lose patient volume if vouchers went away and public sector workers expressed fears that their private colleagues would cease to collaborate in the SMGL program once direct subsidies were reduced. Zambian respondents noted that there had yet to be engagement either at the corporate level or at the private provider level in Zambia, where there are few or no private providers in the SMGL districts. Partners were enthusiastic about the potential for private sector partners, identifying key gaps and challenges that might be best met by the private sector, including the construction of mothers’ waiting shelters and purchase of vehicles—both activities that are challenging for USG partners under their procurement regulations. Merck for Mothers has issued a request for proposals to test maternity waiting shelter models. Beyond gap-filling, respondents were eager for new thinking about how to tackle maternal mortality that they felt private sector partners could bring to the table. One suggestion was that Merck for Mothers, an SMGL lead partner with extensive experience in corporate social responsibility, could help to enlist the domestic private sector in taking a greater role in fundraising or promoting visibility of maternal and newborn health: “We need help with the private sector. We are public health people, so this isn’t our expertise…[but] if the CEO of Merck spoke to the CEO of Airtel, they could make some real progress.” “Vouchers need to be restructured so as not to kill the current system. Otherwise, they will have a negative impact on the public sectors.” — Uganda Implementing Partner 6 SUCCESSES, CHALLENGES, LESSONS LEARNED Saving Mothers, Giving Life External Evaluation / Interim Report 55 7.1 Assessment of SMGL’s first six months Our interim assessment found a tremendous amount of enthusiasm for SMGL in both countries. It was striking to hear positive sentiments about SMGL from respondents ranging from senior national government and USG officials in the capitals to maternity ward nurses in the districts. The support for continuing and expanding SMGL was strong and undiminished by the assortment of challenges encountered in the start-up phase. In synthesizing the lessons learned from the countries at mid-point of Phase 1, we found that the “big push” model—high global visibility, large investment, demand for rapid results—succeeded politically and operationally by raising the stakes for national governments and US development partners in country. SMGL catapulted maternal and newborn health to prominence on national health agendas. It gave Ugandan and Zambian health ministries a way to oper￾ationalize national commitments to reducing maternal mortality by bringing visible improvements in services for mothers and newborns in intervention districts within a short time. This large investment, combined with Ministry of Health involvement, motivated district health managers to tackle the challenge of managing a large influx of activities. In sum, the magnitude of investments and high-level attention to SMGL meant that failure was not an option—for USG or national governments. However, the imperative for rapid results combined with the nearly 100% US government funding which—as per US law, was spent by US agencies working with NGOs rather than through government budgets—diminished countries’ ability to own and direct the program. Ministry of Health respondents at central and district levels pointed out the important distinction between support and leadership: while they strongly back the goals of SMGL and, for the most part, the means to achieve them, they do not feel to be in the driver’s seat. They want this to change as SMGL moves into Phase 2. As one senior official put it, “We are very grateful [for SMGL investments] but not to be figureheads.” Another downside of the rapid nature of the initiative was the lack of a plan on how to sustain gains made. When SMGL was launched, there was no clear plan for how the training would be repeated for new entrants, how mentoring can be continued, how equipment will be serviced, maintained, and replaced, how community groups would be supported to continue their work beyond the end of the funding commitment. Some of the challenges brought on by the abbre￾viated planning period and the short timeline included problems in sequencing inputs and activities, a large training load for health workers, and budgetary and procurement delays due to off-cycle budgeting—all of which frustrated imple￾menting partners and their district counterparts. “I started working in maternal and child health 35 years ago. It’s been a dream come true to participate in a program this serious and of this magnitude.” — USG Zambia Official 7 Preliminary assessment and recommendations for Phase 2 56 MARCH 2013 7.2 Recommendations for Phase 2 As it expands to other districts and countries, SMGL will adapt to local needs, goals, and capacities. However, useful lessons have emerged from our evaluation of the experience of Uganda and Zambia that can inform program design and implementation. One important lesson for Phase 2 is that successful imple￾mentation of SMGL requires a national commitment to maternal and newborn survival at the highest level and a highly functional team of development part￾ners with a strong relationship with the Ministry of Health. Below we propose several recommendations for the content of the intervention package and its implementation in Phase 2. Content of SMGL package 1. Continue to engage communities as change agents SMGL’s work with communities to raise demand for maternal care was univer￾sally viewed as a game-changer in the effort to save maternal and newborn lives. Both community action models—the Volunteer Health Teams from Uganda staffed by community health workers and the Safe Motherhood Action Groups in Zambia, which include community members at large—were felt to be successful in bridging the gap between women and the health system. We also heard early indications—which we will confirm in our final evaluation—that the community initiatives were playing an important role in reducing the inevitability of maternal and newborn deaths in the community. A community action group model adapted to community structures of the country should be part of the intervention pack￾age in Phase 2. Community health activities require people’s time and are unlikely to be sustained on a volunteer basis indefinitely. In Phase 2, implementers, including the Ministry of Health, should consult communities on ways to compensate groups for their efforts. Ideas range from modest payments from the health system to providing seed funds and training for income-generating activities as an incentive to stay involved in the effort. In Zambia, implementers mentioned funding small mills for grinding maize or providing bicycles for personal, as well as program, use to facilitate access to markets. 2. Invest in the health system triad: train staff, improve, and equip facilities The combination of trained, confident staff and functioning facilities with reliable equipment is highly synergistic. While training on how to treat hemorrhage or to resuscitate an unresponsive newborn is crucial, health workers cannot practice these skills effectively without running water, lights, or steady supply of oxytocin or IV tubing. In addition, research shows that lack of functional equipment is a major disincentive for recent medical graduates to accepting rural postings in Africa.36, 37 The combined investment in human capacity and infrastructure in the first phase of SMGL is an excellent model for future efforts. There is no question that external donors cannot do this alone. Facility upgrades—and criti￾cally—repair and maintenance of facilities need to be part of core health system investments by national governments. One caveat is timing. It is important to invest in improving quality at the outset, before women begin arriving in large numbers. This sequencing will help to ensure women and newborns receive appropriate care and do not leave the facil￾ity disappointed with the quality of care, affecting their future health choices and potentially those of their peers. “Before we could only watch babies die. ‘Helping Babies Breathe’ [SMGL-supported training] changed everything; it showed me I can do something.” — Zambian nurse 7 Preliminary assessment and recommendations for Phase 2 Saving Mothers, Giving Life External Evaluation / Interim Report 57 3. Expand the focus beyond 24 hours While intrapartum and immediate postpartum care is an excellent starting point for reducing maternal and newborn mortality, solitary focus on the 24 hours around delivery in SMGL misses crucial opportunities to save lives and will limit its overall effectiveness in the future. Family planning, newborn care, and HIV services—in particular PMTCT— figured more prominently among national maternal and reproductive health plans than they did within SMGL. These services are also provided using shared platforms (e.g. maternity wards and outpatient clinics) often by the same health workers. The focus on 24 hours intro￾duced unnecessary cleavages between this set of essential services. Family planning, which reduces women’s risk of maternal death, should be strengthened as part of Phase 2. Three in ten women in Uganda and Zambia have an unmet need for family planning, which results in unwanted or mistimed preg￾nancies and, in turn, maternal deaths.12, 16 Care of post-abortion complications, already being implemented in some SMGL districts, is another important inter￾vention to reduce maternal deaths. Other services that require greater focus in Phase 2 include prevention of mother-to-child transmission of HIV; a necessity for decisive reductions in mater￾nal and newborn mortality in countries with high HIV prevalence. Finally, half of newborn deaths occur after the first 24 hours of life—these infants require close high quality health care after they leave the hospital with their mothers.21, 22 4. Develop and test solutions for two bottlenecks: human resources and distance to care Although SMGL made investments in human resources (particularly in Uganda) and transport to bring women to facilities, it has not produced effective or lasting solutions for these key constraints to maternal survival. This is because the size and distribution of the health workforce and extent of the road and health infrastructure are issues that lie largely outside the scope of a globally led effort, requiring national policies and investments. It is unlikely that SMGL can sustainably provide high quality obstetric care to all women in the district with the number of health workers currently on the ground—not without staff burnout and attrition. Concerns about work overload were already expressed six months into the program and will only grow with the volume of facility deliveries. Phase 2 should expand on approaches already attempted in Phase 1: contracting with private providers where they are available and hiring recently retired nurses and midwives. Other potentially effective solu￾tions include rural incentives such as salary increases, more rural rotations for senior medical students and residents, task shifting to non-physicians for obstet￾ric surgery, and remote clinical phone or radio support to frontline providers in smaller clinics. Ministries of Health should spearhead these efforts. Even with expanding health budgets, health systems in resource-constrained countries will not be able to provide high-quality clinics close to all women, particularly in large areas with low population densities. Indeed, it is not clear that all women want to deliver at primary care clinics nearby, as studies show substantial proportions favor hospitals.38 Bringing obstetric services to women will thus require scaling up of transport options—ambulances, private taxis—but also other innovative solutions, such as mothers’ shelters near hospitals. The human resource and transport approaches need to be carefully tested in Phase 2 to ensure they meet the needs of women and contribute to maternal and newborn survival. “[Lack of] family planning is a huge barrier and makes our job harder when women are coming in for their eighth child or they get pregnant too soon after another pregnancy.” — Ugandan Midwife 7 Preliminary assessment and recommendations for Phase 2 58 MARCH 2013 Implementation 5. Coordinate around one plan Initiatives to promote universal coverage with quality obstetric and newborn care are growing; SMGL is one among many initiatives to focus on this issue in low-in￾come countries. This proliferation of good intentions is an exciting new stage in the global fight for maternal survival but also presents a growing challenge for coordination. Despite the presence of SMGL district coordinators, health system managers have told us they struggle to stay abreast of activities of implementing partners and manage their regular portfolios. Similarly, at the national level, implementing partners were concerned about duplication of activities, which led to inefficiencies and confusion among stakeholders in districts. Finally, the lack of a single harmonized set of activities across districts was cited as a weakness in the design of SMGL that hampered comparison of effectiveness and development of an ultimate model for the country. For Phase 2, SMGL should strive for greater coherence in program activities, while allowing for some local adaptation. One useful precedent may be the “Three Ones” effort, which sought to harmonize the multitude of AIDS activities and in-country programs in the era of rapidly expanding antiretroviral treat￾ment.39 The “Three Ones” are: one action framework, one coordinating authority, and one monitoring and evaluation system. It would be ideal for global and national SMGL partners to agree to a single set of interventions, with implemen￾tation clearly allocated in country to avoid duplication. The global partners should then work to support key aspects of the action framework, whether through funding, implementation, advocacy, or testing of innovations. To minimize the burden of donor management for overstretched Ministry of Health senior staff, USG agencies should continue to play a key liaison role with host governments. Finally, a single standard approach to monitoring and evaluation will be critical to tracking progress and making corrections. 6. Commit to five years—with a transition plan One clear observation emerging from our evaluation of Phase 1 is that interven￾tions attempting to change health systems and the customs and behaviors of populations cannot be done on a one-year timeline and budget. While there was value to having a first year “big push” to raise attention to issue and build a cadre of champions in country, SMGL should approach the next phase with a health system legacy plan that clearly spells out how SMGL will leave health systems better off for mothers, babies, and health workers. In Phase 2, SMGL global partners should make minimum commitments of three to five years to enable appropriate planning, engagement of local ministries, sequencing of interventions, and crucially, permit the laying of the groundwork for sustainability. Future SMGL countries should also start with a three to five year plan to ensure lasting gains. Going forward, the role of national governments and district authorities should be clearly outlined in SMGL plans. From the outset this should include govern￾ment investments in core areas such as infrastructure and human resources. Beyond this, there needs to be a transition plan that lays out how countries will assume responsibility for the program beyond five years—perhaps linked to greater direct budget support from donors. This approach leverages the lessons of the PEPFAR model, which started out as emergency response managed by the US and is now transitioning to being a source of funds for national governments and their partners to use to sustain services. 7 Preliminary assessment and recommendations for Phase 2 “We can’t solve the country’s problems in only 4 districts, but at least we have a model that works. Let’s use these lessons. Let’s project how long it will take and then leave it totally to local people.” — USG UGANDA OFFICIAL Saving Mothers, Giving Life External Evaluation / Interim Report 59 7. Collect (a few) standard measures to optimize implementation SMGL is nothing less than an overhaul of maternal and newborn healthcare with consequences across the entire health system. Reliable data, collected and analyzed in a timely fashion, will be essential to inform health system managers and other stake￾holders about progress and alert them to any negative consequences and interventions going off track. In Phase 2, a small set of standard tracer indicators could be collected quarterly from facilities in SMGL districts to give a picture of utilization, mortality, referrals within the system, as well as attrition or hiring of health workers, trainings, and overall facility functioning. The indicators, some of which may already be routinely collected, will be valuable for management of the health system overall and not just SMGL. These data will only be useful to the extent they are accurate; enhancing quality of data collection should thus remain a core SMGL activity. Ideally, Phase 2 should institute similar moni￾toring systems in non-SMGL districts to understand how SMGL affects care delivery over and above the underlying secular trends affecting the entire health system. These tracer indicators could be augmented with special studies, as required. Examples of special studies include assessments of quality of care, provider motivation and intent to leave, women’s satisfaction, and causes of maternal and newborn deaths. As noted above, SMGL partners should also commission implementation science experi￾ments on what interventions are most effective and feasible to add to the SMGL portfolio. 8. Harness the private sector Although there were positive experiences in engaging the private sector in Uganda, there was a feeling in both countries that private sector expertise and capacities were underutilized. Respondents in our evaluation noted that they needed “all hands on deck” to reduce maternal mortality: public sector, private companies, and churches and faith based organizations. In particular, while the bulk of responsibility for health care in both countries lies with the government—as it does in most industrialized countries—there is a fundamental role for the private sector to play in supporting maternal survival. Three arenas for private sector action emerged from our study. First, as discussed in Section 6.6, the global and national private sector can help in areas where USG and other partners can offer limited assistance and do not have core expertise, in particular in infrastructure, transport, and communications. Second, thriving domestic industries, such as mining and tourism should be encouraged to get involved in maternal and newborn survival, by raising money and conducting advertising campaigns, for exam￾ple. Large mobile phone companies could expand networks or donate airtime for health workers and community volunteers. Private global SMGL partners could help make this case most effectively. Finally, the private sector has technical, managerial, and communications expertise often lacking in the public sector—expertise that will be essential to sustain high quality of care and community demand beyond the initial investment. From leadership training for health sector managers to assisting with supply chain, logistics, and equipment maintenance to helping community health volunteers launch small businesses, to promoting a customer service mentality in clinics—these are just a few ideas we heard about how private sector partners can strengthen the implementation of SMGL. 7.3 Next steps In the next stage of the external evaluation we will focus on three areas: 1) Response of the community and providers; 2) Extent of implementation at endline; and 3) Assessment of the SGML partnership. This work will be conducted in collaboration with local research partners in Uganda and Zambia and will take place between May and July, 2013. We will produce a final report in fall of 2013. “[Maternal and newborn mortality] is a fight for everybody. Anybody’s wife or daughter can die. Many people are looking for the government to do everything and they can’t.” — Zambia Implementing Partner 7 Preliminary assessment and recommendations for Phase 2 61 8 Appendix 62 MARCH 2013 Uganda Year SOURCE Zambia Year SOURCE Population & Economy Population 33,425,000 2010 1 13,089,000 2010 1 Number of women of reproductive age (15-49) 7,298,000 2010 2 2,893,000 2010 2 Total fertility rate * 6.1 2010 1 6.3 2010 1 Crude birth rate (births per 1,000 people per year) 45.0 2010 1 46.0 2010 1 Percent of the population living on less than $1.25 (PPP) a day (at 2005 international prices) 38.0 2009 3 69.0 2006 3 Human development index (0-1) ** 0.45 2011 4 0.43 2011 4 Gross national income per capita (PPP current international $) $1,310 2011 5 $1,490 2011 5 Gross national income per capita (current USD) $510 2011 5 $1,160 2011 5 GDP growth (annual %) 6.7 2011 5 6.5 2011 5 Gender inequality index (0-1) *** 0.58 2011 4 0.63 2011 4 Transparency International corruption perceptions index (0-100) **** 29.0 2012 6 37.0 2012 6 Health HIV prevalence rate among adults 15-49 7.2% 2011 1 12.5% 2011 1 Malaria deaths (annual number of reported deaths due to malaria) 8,431 2010 1 4,834 2010 1 Incidence of tuberculosis (estimated number of new pulmonary, smear positive, and extra-pulmonary tuberculosis cases per 100,000 people) 193 2011 1 444 2011 1 Annual number of maternal deaths (per 100,000 live births) 4,700 2010 7 2,600 2010 7 Maternal mortality ratio (MMR) (maternal deaths per 100,000 live births) 310 2010 1 440 2010 1 Annual number of neonatal deaths (deaths under 28 days of life) 43,000 2011 1 17,000 2011 1 Neonatal mortality rate (deaths under 28 days of life per 1,000 live births) 28 2011 1 27 2011 1 Infant mortality rate (deaths of infants under 1 year old per 1,000 live births) 58 2011 1 53 2011 1 Under five mortality rate (U5MR) (deaths of children under 5 years old per 1,000 live births) 90 2011 1 83 2011 1 Progress on MDG 4 – U5MR Insufficient progress 2012 8 Insufficient progress 2012 8 Progress on MDG 5 – MMR Making progress 2012 8 Insufficient progress 2012 8 APPENDIX A National health and health system indicators Saving Mothers, Giving Life External Evaluation / Interim Report 63 Sources 1 WHO Global Health Observatory Data Repository 2 United Nations, Department of Economic and Social Affairs – World Population Prospects, the 2010 Revision 3 World Bank PovcalNet 4 UNDP International Human Development Indicators 5 World Bank Databank , 2011 6 Transparency International Coruption Perceptions Index 2012 7 Trends in Maternal Mortality: 1990 to 2010. UNFPA, WHO, UNICEF and World Bank Estimates 8 Countdown to 2015 9 Global Health Workforce Alliance, 2010 10 DHS Uganda Year SOURCE Zambia Year SOURCE Health System Number of physicians 3,361 2005 1 649 2006 1 Number of nurses & midwives 37,625 2005 1 8,369 2006 1 Health workers per population (density of midwives/ nurses/doctors per 1,000) 1.4 2005 1 0.8 2006 1 Hospital beds per 10,000 population 5.0 2010 1 20.0 2010 1 Health worker vacancy rate (%) 48.0 2010 9 61.0 2010 9 Per capita total expenditure on health (USD) $47 2010 1 $73 2010 1 Per capita total expenditure on health (PPP international $) $124 2010 1 $90 2010 1 Per capita government expenditure on health (USD) $10 2010 1 $44 2010 1 Per capita government expenditure on health (PPP international $) $27 2010 1 $54 2010 1 General government expenditure on health as % of total expenditure on health 22% 2010 1 60% 2010 1 Out of pocket expenditure as a % of total expenditure on health 50% 2010 1 27% 2010 1 Private prepaid insurance plans as a % of total expenditure on health <1% 2010 1 1% 2010 1 External resources for health as a % of total expenditures for health 26% 2010 1 39% 2010 1 General government expenditure on health as % of total government expenditures 12% 2010 1 16% 2010 1 Total expenditure on health as a % of GDP 9% 2010 1 6% 2011 4 Health Care Utilization Percentage of live births in health facilities 57% 2011 11 48% 2007 11 Percentage of births attendant by skilled birth attendant 57% 2011 11 47% 2007 11 Women receiving 4+ antenatal care visits 48% 2011 11 60% 2007 11 Percentage of deliveries by Caesarean section 5% 2011 11 3% 2007 11 ART coverage among people with advanced HIV infection 54% 2011 1 82% 2011 1 PMTCT coverage (maternal ARV coverage) 50% 2011 1 86% 2011 1 Percentage of children under 5 sleeping under bednet previous night 43% 2011 11 33% 2007 11 DTP3 coverage 82% 2010 1 81% 2010 1 Contraceptive prevalence rate (any method) 24% 2011 11 30% 2007 11 Contraceptive prevalence rate (modern) 21% 2011 11 25% 2007 11 * The total fertility rate is the number of children that would be born to a woman if she were to live to the end of her reproductive life, and if she were to experience the current age-specific fertility rates for the duration of her life. ** The human development index (HDI) is a composite of life expectancy, educational attainment, and income. It ranges from a minimum of 0 (equal) to a maximum of 1 (unequal). *** The gender inequality index (GII) combines indicators for reproductive health, female empowerment, and female labor force participation into a single metric that ranges from a minimum of 0 to a maximum of 1. **** The corruption perceptions index scores countries on a scale from 0 (highly corrupt) to 100 (very clean). 64 MARCH 2013 This external evaluation is a strategic evaluation of implementation of Phase 1 of the Saving Mothers, Giving Life initiative in Uganda and Zambia. Conceptual framework We use a multi-disciplinary approach to evaluate the imple￾mentation of Saving Mothers, Giving Life. Our conceptual framework combines elements from (a) the process eval￾uation framework of Linnan and Steckler and (b) complex adaptive systems analysis.40, 41 The key components assessed in the evaluation are: context, program theory, implementa￾tion process, dose delivered, community response, fidelity, and emergent properties. These are described below: ■■ Context: baseline context and recent changes in social, political, and economic environment that may influence implementation of SMGL. ■■ Program theory: the “active ingredients” of the program and how these will deliver desired change. ■■ Implementation process: actors involved in program implementation; planning and implementation timelines; funding; obstacles/facilitators of implementation. ■■ Dose delivered: measure of intervention activities deliv￾ered to the field (e.g. supplies received, trainings given, etc.) in a given timeframe (i.e. 6, 12 months). ■■ Community response: perceptions of SMGL interventions by women in the community (i.e. users and non-users of the health system), community leaders, health workers; women’s satisfaction with SMGL-supported delivery care. ■■ Fidelity: this aspect focuses on the quality and integrity of the partnership model (extent of partner engagement; participation of stakeholders, e.g. MoH, health workers, and the private sector) and the content of the intervention (e.g. is training leading to improved obstetric knowledge and improved perceived quality by women?). ■■ Emergent properties: dynamic evolution of the inter￾vention, real-life complications, and unexpected results: unintended consequences (positive and negative); syner￾gies and extent of integration with local health systems; innovations and mechanisms of action. This interim report focuses on a subset of these constructs indicated by the previous bullets in bold text: context, program theory, implementation process, dose delivered, and some emergent properties. The findings will inform recom￾mendations for Phase 2. The final report, to be completed in fall of 2013, will address: 1) response of the community and providers 2) extent of implementation at endline and 3) func￾tioning of the SGML partnership. Setting As the focus of the interim report was on country activities, data collection occurred in Uganda and Zambia. Our teams conducted data collection in Kampala and Lusaka, primar￾ily with officials from the CDC, USAID, Ministry of Health, SMGL implementing partners and key stakeholders, such as the WHO, Central Statistical Offices, and UNICEF, among others. The teams also visited all eight SMGL districts: Lundazi, Nyimba, Kalomo and Mansa in Zambia and Kibaale, Kyenjonjo, Kamwenge and Kabarole in Uganda. Instruments We used a mixed methods design, which included both qualitative and quantitative data. We conducted in-depth interviews with key SMGL partners at central (i.e. CDC, USAID and the Ministry of Health) and district levels (i.e. district health teams, SMGL Coordinators and implementing part￾ners). The content of the interview guides is described below: a. Descriptive interview guide: descriptive information on the main roles of the different SMGL partners, and the implementation process and timeline. Interviews typically lasted 30 to 45 minutes. b. Lessons learned interview guide: this examined the successes, challenges, unintended consequences, evolution and innovation of SMGL; emerging lessons and recommendations for scaling up in Phase 2, among other issues. Interviews lasted 45 to120 minutes. We assessed the SMGL “dose delivered” by December 1, 2012 using a survey of 166 questions for Zambia and 178 questions for Uganda, enumerating all inputs and activities falling under the 28 key SMGL activities of the logic model. “Dose delivered” refers to the activities and inputs actually implemented in the districts by the mid-point of the first year, as reported by the district medical officers and implementing partners in districts. Data was entered into Excel and verified by in-country partners. APPENDIX B Methods Saving Mothers, Giving Life External Evaluation / Interim Report 65 Data collection The majority of data collection occurred between October 1 and December 20, 2012. Data collection commenced with a preliminary visit to Zambia and Uganda by the principal investigators, Drs. Kruk and Galea, who met with key SMGL partners and visited one SMGL district in each country. The main goals of the preliminary visits were to gain famil￾iarity with the SMGL program in the two SMGL countries, share the scope of work of the external evaluation and obtain feedback on it. Following the preliminary visits, three-person teams comprising a faculty member and two dedicated research￾ers visited each country. Data collection in Zambia occurred from November 5 to December 20 and in Uganda from November 25 to December 15. Two faculty advisors, Dr. Rabkin and Dr. Grépin, led data collection efforts in Zambia and Uganda, respectively. Evaluation team members were: Dana Greeson (MPH) and Daniel Vail (BA) in Zambia and Emma Sacks (PhD candidate) and Katherine Austin-Evelyn (MSc, MPH) in Uganda. The teams were assisted from New York by Program Manager, Tsitsi Masvawure (DPhil). Our team conducted a total of 143 interviews (67 Uganda, 76 Zambia) with 152 individuals. Of these, 89 were in-depth, lessons-learned interviews (47 Uganda, 42 Zambia). These are broken down as follows: 52 interviews conducted in the districts (23 Uganda, 29 Zambia) and 37 interviews conducted at central level (24 Uganda, 13 Zambia). We interviewed all key US Government SMGL leads. We also interviewed five senior Ministry of Health officials who were centrally involved in SMGL in both countries (2 in Zambia, 3 in Uganda). We interviewed representatives of 20 imple￾menting partner organizations (12 in Zambia, 8 in Uganda), at capital and district level. We met with and interviewed all eight SMGL district medical officers in the SMGL focus districts. We also interviewed all seven SMGL coordinators in both countries (2 in Uganda, 5 in Zambia, including the provincial SMGL coordinator). Lastly, the team attended a total of 6 SMGL partner meetings in both countries (5 in Zambia, 1 in Uganda). To obtain the quantitative dose delivered data, we inter￾viewed district health and district medical officers and SMGL coordinators in each country and collected data on a total of 202 indicators we developed for the 28 SMGL activities contained in the logic model. Informants were asked about inputs and activities already implemented as of December 1, 2012, not those that were planned for future implementation. We collected information on 166 SMGL activity indicators in Zambia and on 178 activity indicators in Uganda. The data for this came from two main sources: key informant interviews with all SMGL district coordina￾tors in each country (4 in Zambia, 2 in Uganda) and other key implementers in the districts, including district-level members of the Ugandan and Zambian Ministries of Health (specifically, district medical officers, chief medical officers at the district hospitals, district planners, district infor￾mation officers, district human resource officers, district equipment officers, and district nursing officers), and district-level representatives from the SMGL implementing partners. Multiple respondents were interviewed in each district, and, wherever possible, interviewers confirmed evidence of implementation firsthand. After initial data collection was completed, the evaluation team examined all documents collected from each country to verify information contained in the dose delivered dataset. A reliability score was given to each piece of data, depending on the number of people who corroborated the information, and if we were able to verify the information with docu￾mentation. All dose delivered data were then shared with in-country partners at district level for their review and correction in January, 2013. We received clarifications and corrections from all eight SMGL districts and incorporated all suggestions and edits. In addition to the dose delivered data, we also obtained geocoded maps on all health facilities in the eight SMGL districts and collected quantitative infor￾mation about the planning and implementation activities and timeline from the descriptive interviews. Our teams spent a total of 134 person-days in country (86 in Zambia, 48 in Uganda), the majority of which was time spent on the ground in the eight SMGL districts. The teams visited a total of 50 sites in the districts (26 in Zambia, 24 in Uganda). Sites visited included district health offices, provincial health offices, district hospitals, mothers’ shelters and clinics. We also collected 536 program, health system and national strategy documents from USG and SMGL imple￾menting partners and the MoH at central and district levels. Documents collected included policies, program plans, task force meeting minutes and monthly updates on SMGL activ￾ities prepared by the SMGL coordinators in each district and reports prepared by the various SMGL implementing part￾ners. District utilization and other health system statistics were collected to inform the context section of the report. Data collection in country was complemented by a review of the current literature on maternal and newborn health in Zambia and Uganda, and a review of the the Demographic Health Surveys (DHS) data for each country. The data collected for the interim report were exempted from IRB by Columbia University as they concerned program implementation activities. 66 MARCH 2013 Data management and analysis Qualitative data were entered, cleaned, and analyzed using the NVivo Software Package (Version 10). Transcripts of all interviews and notes were independently coded by two analysts using pre-existing codes and a process of constant comparison as described by Glaser.2 Discrepancies were reconciled through a practice of consensus-building between the two coders. A second level of open, iterative coding was then conducted, creating new codes around major themes, which were then reviewed by multiple team members to synthesize and summarize. These summaries were the basis of the content in this report. The summaries also provided data for the logic model, partner map, and imple￾mentation calendar. Dose delivered quantitative data were entered and verified against field notes and documents provided and assigned a reliability score that indicated our level of confidence in the results, with the most reliable being data corroborated by two people (including one at the MoH) and backed up by documentation. We sought additional input for indicators with low reliability scores. A subset of indicators, “tracer indi￾cators,” were identified, which are presented in the report. These tracer indicators reflect SMGL core activities across the two countries. All dose delivered data were then shared with in-country partners at district level for their review and correction in January, 2013. All districts responded with useful clarifications and corrections. The descriptive interviews with lead USG and implement￾ing partners in each country were reviewed individually to extract information about SMGL partners, and the timeline and key activities in the implementation process. These were shared with countries for their review ahead of publication. APPENDIX B METHODS Saving Mothers, Giving Life External Evaluation / Interim Report 67 This project was funded by a grant from Merck for Mothers to Drs. Margaret E. Kruk and Sandro Galea at Columbia University. We would like to acknowledge the important contributions made by various individuals and institutions to the produc￾tion of this report. Global SMGL Partners We would like to thank the SMGL Global Secretariat and members of the SMGL Monitoring and Evaluation Committee, including Ms. Celina Schocken (Merck for Mothers), Ms. Angeli Achrekar, Ms. Mary Ellen Stanton, Dr. Claudia Conlon, Ms. Annie Schwartz, Ms. Karen Fogg, Ms. Sandhya Rao, and Ms. Marta Levitt (USAID), Dr. Howard Goldberg, Dr. Mary Goodwin, and Mr. Paul Stupp (CDC) for their assistance in the evaluation. We would like to thank Ms. Julie Becker and Ms. Sarah Hank (Rabin Martin) for their assistance in the project’s formative phase. We would like to acknowledge the support of the SMGL Leadership Council, including Ms. Lois Quam (US Global Health Initiative), Mr. Robert Clay (USAID), Dr. Naveen Rao and Dr. Priya Agrawal (Merck for Mothers), Dr. Herbert Peterson (ACOG), Ms. Christy Turlington Burns and Ms. Erin Thornton (Every Mother Counts), and Dr. Tore Godal and Ms. Helga Fogstad (Government of Norway) National Ministries of Health The participation of the Ministries of Health of Uganda and Zambia was critically important to our project. In Uganda, we extend our gratitude to Dr. Miriam Sentongo, Senior Medical Officer, Reproductive Health Office, Dr. Anthony Mugasa, Reproductive Health Advisor, MoH, and Dr. Jennifer Wanyana, former Assistant Commissioner of Reproductive Health, MoH. In Zambia, we extend our thanks to Dr. Max Bweupe, Deputy Director of Public Health and Research, MoH, and Dr. Caroline Phiri, Acting Director of Maternal and Child Health at the Ministry of Community Development, Mother and Child Health (MCDMCH). Our conversations with them yielded rich insights into the SMGL planning and imple￾mentation process. District Health Officials We extend very special thanks to the eight SMGL district health teams for their central assistance in obtaining dose delivered data and their reflections on program imple￾mentation. We are indebted to the four District Medical/ Health Officers in Uganda, Dr. William Mucunguzi (Kyenjojo), Dr. Richard Mugahi (Kabarole), Ms. Winnie Rurangaranga (Kamwenge), and Dr. Dan Kyamanywa (Kibaale). In Zambia, we received assistance from Dr. Christopher Nyambe Sinyange (Nyimba), Dr. Mutinta Mudenda (Mansa), Dr. Allan Chisenga (Lundazi), and Dr. Kenneth Chibwe (Kalomo). We also thank Dr. Kennedy Malama, the Provincial Medical Officer of Eastern Province and Dr. Elicho Bwalya, the Provincial Medical Officer of Luapula Province. The DMOs and DHOs in both countries also facilitated our access to local leaders, including Mr. Nathan Musingusi, the Secretary for Health in Kyenjojo and Sister Harriet Kembabazi (representative for the Secretary for Health in Kamwenge). We would also like to recognize the assistance of other health officials in the districts, such as the district planners, chief medical officers, district nursing officers and medical superintendents, among others. In Uganda, they include Dr. Fred Omio (Ntara Health Centre), Sister Regina Mary Nyaga (Regional Referral Hospital in Fort Portal), Ms. Joanne Chesamg (Mentor Midwife in Kyenjojo), Ms. Zidah Najjuma (Mentor Midwife in Kibaale), Ms. Grace Akot (Mentor Midwife), and Dr. Charles Olaro (Medical Superintendent of Fort Portal Regional Referral Hospital). In Zambia, we extend our thanks to Ms. Christina Banda (Lundazi Director of MCDMCH), Mr. Joseph Sakala (Kalomo District Planner), Mr. Haswell Kapomba (Nyimba Chief Clinical Officer and former Acting Planner), Dr. Jonathan Chama (Chief Medical Officer, Nyimba District Hospital), Ms. Ireen Chiama, (Kalomo MNCH Coordinator), Ms. Agness Phiri (Lundazi MNCH Coordinator), Ms. Beatrice Ling’umbwa (Lundazi District Hospital Enrolled Midwife), Ms. Bether Muzyamba (Acting Chief Nursing Officer of Zimba Mission Hospital), Mr. Hastings Chileshe (Acting Nursing Officer at Nyimba Hospital), Ms. Numbi (Nyimba HR Officer) and Mr. William Mangimela (Chipata Information Officer), Ms. Betty Kunda (Mansa District Nursing Officer), Mr. Muleya Siakakole (Lundazi HR Officer). We would also like to acknowledge Lieutenant Colonel Joyce Mulilo (Zambia Defence Force). We are especially grateful to the following individuals in all four districts in Uganda for swiftly responding to our requests for information. Mr. Abel Katuramu generously obtained the geocodes of all thirty-seven health facilities in his district for us. Mr. Tumuhairwe Collins, Mr. Justine Mugisa, Ms. Hope Katushabe, and Mr. Robert Mugabe also provided key data. US Government Agencies We would like to extend our sincere gratitude to the U.S. Centers for Disease Control and Prevention (CDC) and the United States Agency for International Development (USAID) in the two countries for their essential help in organizing data collection and introductions to key stakeholders. We appreciate their willingness to share their insights and APPENDIX C Acknowledgements 68 MARCH 2013 reflections about the SMGL program and for inviting us to observe a range of SMGL meetings. In Uganda, we thank Dr. Frank Kaharuza (Chief, Epidemiology Branch, CDC), Dr. Jayne Tusiime-Byakikia (CDC Team Leader, Operations Research,), Mr. Prosper Behumbiize (Data Manager, CDC) and Mr. George Acire (GIS Analyst, CDC). We also thank Ms. Karen Klimowski (USAID Director, Health, HIV and Education) and Dr. Janex Kabarangira (Deputy Health Team Leader, USAID), and Dr. James Tanu Duworko (Reproductive Health Manager, USAID). In Zambia, we thank Dr. Lawrence Marum (CDC Country Director), Dr. James McAuley (Medical Officer, CDC, SMGL Focal Point), Ms. Fatma Soud (CDC, Monitoring and Evaluation Lead), and Dr. Jorge Velasco (USAID Deputy Health Team Leader). We extend our gratitude to the Peace Corps. In Uganda, we thank Ms. Loucine Hayes (Country Director), Ms. Bernadine Nganda (Coordinator), Ms. Cotious Tukashaba (Country Manager), Ms. Shiphrah Mutungi (Program Manager, Health), and Mr. Gordon Parola (Volunteer, Kamwenge). In Zambia, we thank Ms. Lauren Mamane (Director of Programing and Training) and Ms. Sally-Rose Mwachilenga (Peace Corps Response and Extension Coordinator). We also thank Peace Corps Response volunteers: Ms. Carrie Nichols (Mansa), Ms. Hazel Domanque (Kalomo), and Ms. Karen Janssen (Kalomo), with a special thanks to Ms. Carrie Vogelsang (Lundazi) and Ms. Ashley Arnold (Nyimba), who were invaluable in assisting us during and after district data collection in Zambia. We thank Ms. Patricia Ulaya (Department of Defense PEPFAR Program Assistant), Ms. Yvonne Mulenga (Project Concern International Zambia Department of Defense HIV/ AIDS Program Manager), and Ms. Mary Mfula (Department of Defense Finance Department). SMGL Coordinators We extend sincere thanks to the SMGL coordinators for helping facilitate our district visits and for graciously responding to email requests for further information after our visits. In Uganda, we thank Dr. Joseph Lubwama (Program Coordinator for Baylor-Uganda Regional Office) and Mr. Edward Kyagulanyi (IDI-Uganda). In Zambia, we extend special thanks to Ms. Rebecca Lusumpa (Eastern Province SMGL Coordinator), and district coordinators: Mr. Jonathan Kunyi (Nyimba), Ms. Esnart Juunza (Kalomo), Ms. Cathrine Konie (Lundazi), and Ms. Mutinta Maingaila (Mansa). SMGL Implementing Partners We thank the SMGL implementing partners for their enthusiasm and cordiality. On numerous occasions, partners devoted their entire mornings or afternoons to talk to us and gave us access to key staff members and planning documents. In Uganda we are grateful to the following individuals at Baylor-Uganda: Dr. Adeodata Kekintiiwa (Executive Director), Ms. Patricia Mwebaze (SMGL Program Manager), Dr. George Akol (SMGL Program Manager), Ms. Barbara Sentiba (Human Resources Director), Dr. Robert Iriso (Medical and Psychosocial Programs Director), Dr. Dan Murokora (Technical Advisor), Mr. Paul Mayende (Demand Creation), Mr. Rogers Sekabira (Logistics Management), Mr. Joseph Mukasa (Vouchers Grants Manager), Ms. Alice Asiimwe (Researcher, Community Monitoring and Evaluation), Mr. Sam Wasike (Monitoring and Evaluation Officer), Mr. Nelson Businge (Community Linkage Officer), and Mr. Thomas Kulumba (Community Linkage Office). We would also like to extend our gratitude to the following individuals at IDI-Uganda: Dr. Alex Coutinho (Executive Director), Ms. Brenda Picho (SMGL Director), Ms. Grace Oling (Community Linkage Officer), and Ms. Zida Najjuma (mentor midwife). At STRIDES, we are grateful to the following: Ms. Cecilia Kakande (Chief of Party), Dr. Henry Kakande (Deputy Chief of Party), Ms. Rita Lulua (Regional Coordinator), Ms. Manjit Kaur (Regional Operations Coordinator), Dr. Miriam Mutabazi (Technical Director), and Mr. Richard Ssewajje (Director of Grants/Contracts). We would also like to thank the following individuals from AOGU: Dr. Romano Byaruhanga (President), Dr. Jolly Beyeza, Dr. Eve Nakabembe, Prof. Florence Mirembe, Ms. Juliet Oinomugisho and Dr. Imelda Namagembe; Ms. Harriet Nambanya from UPA and the following individuals from IPA: Ms. Jessica Kiessel (Director of Country Offices), and Mr. Dan Pakerson (Country Director). We extend our gratitude to the private sector partners we met with, namely Dr. David Mwinganiza (Director, Midas Touch) and Dr. Priscilla Busingye (Director, Virika Hospital). At Marie Stopes Uganda we are grateful to Mr. Jon Cooper (Country Director), Ms. Christine Namayanja (Director, Private Sector Partnerships), Ms. Julia Mayersohn (Public Relations), and Mr. William Nyombi (Senior Manager of Franchising). In Zambia, we sincerely thank the following individuals at CIDRZ: Dr. Carla Chibwesa (Gynaecologist), Ms. Mari Katundu (Program Manager), Mr. Jameson Mubonde (Nyimba AMAI Coordinator), and Mr. Simon Mutibo (Lundazi AMAI Coordinator). At ZCAHRD, we sincerely thank Dr. Davidson Hamer (Director of Research and Evaluation), Ms. Irene Singogo (SMGL Project Director), Ms. Martha Musweu (Senior Clinical Mentor), Ms. Shari Davis (Logistics Coordinator), and Dr. Don Thea (Principal Investigator). At MCHIP, we thank Mr. Kwame Asiedu (Country Representative), Ms. Michelle Wallon (Project Director), Ms. Martha Ndhlovu (Technical Advisor), and Ms. Brenda Mutinta (MNH Technical Officer). At ZISSP, we thank Ms. Kathleen Poer (Chief of Party), Ms. Mutinta Nalubamba (MNCH Team Leader), Mr. Mukata Mulonda (Eastern Province Community Health Officer), and Mr. Timothy Silweya (Luapula Community Health Officer), in addition to the SMGL coordinators previously mentioned. APPENDIX C ACKNOWLEDGEMENTS Saving Mothers, Giving Life External Evaluation / Interim Report 69 At EGPAF, we thank Dr. Jack Menke (Technical Director), Ms. Ruth Nswana (Strategic Information Officer), and Mr. Bright Kulukulu (Strategic Information Officer). At JSI, we thank Mr. Walter Proper (Country Director), Mr. Sydney Chanda (Senior Public Health Logistics Advisor), and Mr. Derrick Nyimbili (Public Health Logistics Advisor). At ZPCT II, we thank Dr. Michael Welsh (Country Director), Dr. Thierry Malebe (Senior Advisor, PMTCT/CT), Ms. Gail Bryan,Mofya (Senior Advisor, Pharmacy and Labs), Ms. Prisca Kasonda (Director of Technical Services), and Mr. Suzgo Kapanda (Provincial Program Manager). Other Stakeholders In Uganda, we would like to thank Dr. Olive Sentumbwe (Reproductive Health Director) at the World Health Organization, Mr. Sean Blaschke (Health Systems Strengthening Coordinator, UNICEF) and the following indi￾viduals at PACE: Ms. Hanna Baldwin (Technical Advisor) and Ms. Dorothy Balaba (Country Manager, Reproductive Health). In Zambia, we also thank Mr. Nicholas Shiliya at SFH/PSI, Mr. Alexander Mkosi (SMGL M&E) at CSH, Mr. Paul Pascal Chimedza (Country Director) and Mr. Elizabeth Simwawa— both at Africare, and Kaiko Mkilolo, Results Based Financing Technical Specialist at the World Bank. APPENDIX C ACKNOWLEDGEMENTS 70 MARCH 2013 1. SMGL. Saving Mothers, Giving Life. 2012; http://www. savingmothersgivinglife.org. Accessed January 28, 2013. 2. Glaser B. The Constant Comparative Method of Qualitative Analysis. Social Problems. 1965;12(4):436-445. 3. Lozano R, Wang H, Foreman KJ, et al. Progress towards Millennium Development Goals 4 and 5 on maternal and child mortality: an updated systematic analysis. Lancet. 2011;378(9797):1139-1165. 4. World Health Organization (WHO). Trends in Maternal Mortality 1990-2010: WHO, UNICEF, UNFPA and The World Bank Estimates. Geneva: World Health Organization;2012. 5. Liu L, Johnson HL, Cousens S, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet. 2012;379(9832):2151-2161. 6. Khan KS, Wojdyla D, Say L, Gulmezoglu AM, Van Look PF. WHO analysis of causes of maternal death: a systematic review. Lancet. 2006;367(9516):1066-1074. 7. Ronsmans C, Graham WJ, Lancet Maternal Survival Series steering group. Maternal mortality: who, when, where, and why. Lancet. Sep 30 2006;368(9542):1189-1200. 8. World Development Indicators database. 2013. http://databank. worldbank.org/ddp. Accessed February 19, 2013. 9. Global Health Initiative (GHI). Saving Mothers, Giving Life Uganda: Operational Plan. January 2012 – February 2013. Washington DC: U.S. Global Health Initiative;2011. 10. World Health Organization (WHO). World Health Organization Global Health Observatory Data Repository. http://apps.who. int/gho/data/. Accessed January 29, 2013. 11. World Health Organization (WHO). World Health Report 2006: Working together for health. Geneva: World Health Organization;2006. 12. Uganda Bureau of Statistics (UBOS), ICF International Inc. Uganda Demographic and Health Survey 2011. Kampala, Uganda: UBOS and Calverton, Maryland: ICF International Inc.: Uganda Bureau of Statistics;2012. 13. Green E, Nantulya V, Stoneburner RL, Stover J. What Happened in Uganda? Declining HIV Prevalence, Behavior Change, and the National Response Washington DC: United States Agency for International Development;2002. 14. Stoneburner RL, Low-Beer D. Population-level HIV declines and behavioral risk avoidance in Uganda. Science. 2004;304(5671):714-718. 15. Government of Uganda (GOU). Roadmap for Accelerating the Reduction of Maternal and Neonatal Mortality and Morbidity in Uganda 2007-2015. Kampala: Government of Uganda, Ministry of Health;2007. 16. Central Statistics Office (CSO), Ministry of Health (MoH), Tropical Diseases Research Centre (TDRC), Univeristy of Zambia, Macro International Inc. Zambia Demographic and Health Survey 2007. Calverton, Maryland, USA: CSO and Macro International Inc.: Zambia Central Statistics Office;2009. 17. Joint United Nations Programme on HIV/AIDS (UNAIDS). Zambia: HIV and AIDS Estimates 2011. 2011; http://www. unaids.org/en/regionscountries/countries/zambia/. Accessed January 29, 2013. 18. Fylkesnes K, Musonda RM, Sichone M, Ndhlovu Z, Tembo F, Monze M. Declining HIV prevalence and risk behaviours in Zambia: evidence from surveillance and population-based surveys. Aids. 2001;15(7):907-916. 19. Government of the Republic of Zambia (GRZ). Road Map for Accelerating the Attainment of the Millennium Development Goals Related to Maternal, Newborn and Child health in Zambia. Lusaka: Government of Zambia;2007. 20. Global Health Initiative (GHI). Saving Mothers’ Lives Zambia: Operational Plan. October 2011-December 2012. Washington DC: U.S. Global Health Initiative;2011. 21. World Health Organization (WHO). Maternal, newborn, child and adolescent health; Newborn health epidemiology. 2013; http://www.who.int/maternal_child_adolescent/epidemiology/ newborn/en/index.html. Accessed February 1, 2013. 22. Lawn JE, Cousens S, Zupan J. 4 million neonatal deaths: When? Where? Why? Lancet. 2005;365(9462):891-900. APPENDIX D References Saving Mothers, Giving Life External Evaluation / Interim Report 71 23. Ahmed S, Li Q, Liu L, Tsui AO. Maternal deaths averted by contraceptive use: an analysis of 172 countries. Lancet. 2012;380(9837):111-125. 24. Freedman L, Waldman R, de Pinho H, et al. Who’s got the power? Transforming health systems for women and children. Task Force on Child Health and Maternal Health. New York: UN Millennium Project;2005. 25. Manandhar D, Osrin D, Shreshtha B, et al. Effect of a participatory intervention with women’s groups on birth outcomes in Nepal: cluster-randomized controlled trial. Lancet. 2004;364(9438):970-979. 26. Mullany BC, Becker S, Hindin MJ. The impact of including husbands in antenatal health education services on maternal health practices in urban Nepal: results from a randomized controlled trial. Health Education Research. 2007;22(2):166-176. 27. India Ministry of Health and Family Welfare. Janani Suraksha Yojana: Guidelines for Implementation. New Delhi: Government of India;2009. 28. Lim SS, Dandona L, Hoisington JA, James SL, Hogan MC, Gakidou E. India’s Janani Suraksha Yojana, a conditional cash transfer programme to increase births in health facilities: an impact evaluation. Lancet. Jun 5 2010;375(9730):2009-2023. 29. Uxa F, Bacci A, Mangiaterra V, Chiaffoni GP. Essential newborn care training activities: 8 years of experience in Eastern European, Caucasian and Central Asian countries. Seminars in Fetal and Neonatal Medicine. 2006;11(1):58-64. 30. Santos C, Diante D, Jr., Baptista A, Matediane E, Bique C, Bailey P. Improving emergency obstetric care in Mozambique: the story of Sofala. International Journal of Gynecology & Obstetrics. 2006;94(2):190-201. 31. Koblinsky M, ed Reducing maternal mortality: learning from Bolivia, China, Egypt, Honduras, Indonesia, Jamaica and Zimbabwe. Washington, DC: The World Bank; 2003. 32. Campbell O, Gipson R, Issa AH, et al. National maternal mortality ratio in Egypt halved between 1992–93 and 2002. Bulletin of the World Health Organization. 2005;83(6):462-471. 33. Hogan MC, Foreman KJ, Naghavi M, et al. Maternal mortality for 181 countries, 1980-2008: a systematic analysis of progress towards Millennium Development Goal 5. Lancet. 2010;375:1609-1623. 34. United Nations. Millennium Development Goals: 2012 Progress Chart. 2012; http://www.un.org/millenniumgoals/pdf/2012_ Progress_E.pdf. Accessed February 22, 2013. 35. Chen L, Evans T, Anand S, et al. Human resources for health: overcoming the crisis. Lancet. Nov 27-Dec 3 2004;364(9449):1984-1990. 36. Kruk ME, Johnson JC, Gyakobo M, et al. Rural practice preferences among medical students in Ghana: a discrete choice experiment. Bull World Health Organ. May 2010;88(5):333-341. 37. Lehmann U, Dieleman M, Martineau T. Staffing remote rural areas in middle- and low-income countries: a literature review of attraction and retention. BMC Health Serv Res. 2008;8(19):19. 38. Kruk ME, Mbaruku G, McCord CW, Moran M, Rockers PC, Galea S. Bypassing primary care facilities for childbirth: a population-based study in rural Tanzania. Health Policy Plan. Jul 2009;24(4):279-288. 39. Joint United Nations Programme on HIV/AIDS (UNAIDS). The “Three Ones” in action: where we are and where we go from here. Geneva2005. 40. Linnan L, Steckler A. Process Evaluation for Public Health: Interventions and Research. San Francisco: Jossey-Bass; 2002. 41. de Savigny D, Adam T, eds. Systems thinking for health systems strengthening. Geneva: Alliance for Health Policy and Systems Research, WHO; 2009.