1 END-OF-PROJECT EVALUATION OF THE USAID/ZIMBABWE FAMILY AIDS INITIATIVE (FAI) PROJECT December, 2012 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez, England & Associates.    ii An Evaluation of USAID’s Family AIDS Initiative (FAI) Project’s Efforts to Develop Capacity Within the Zimbabwe Ministry of Health and Child Welfare and Local Non-Governmental Organizations to Support Continued Delivery of PMTCT Services Beyond the life of the Project FINAL REPORT Prepared under Task Order AID-613-TO-12-00002 Under the Evaluation Services IQC Submitted to: USAID/Zimbabwe November 28, 2012 Submitted by: Eliot Putnam (Team Leader) Rumbidzai Chitombi Nyikadzino Mahachi Priscilla Mataure Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government    END-OF-PROJECT EVALUATION OF FAI PROJECT iii CONTENTS ACKNOWLEDGEMENTS .................................................................................... v ABBREVIATIONS ................................................................................................. vi EXECUTIVE SUMMARY ....................................................................................... 1 Background and Approach .................................................................................................. 1 Findings .................................................................................................................................... 2 Recommendations ................................................................................................................. 3 1.0 INTRODUCTION ....................................................................................... 5 2.0 EVALUATION PURPOSE & EVALUATION QUESTIONS ................. 6 2.1 Evaluation Purpose and Objectives .......................................................................... 6 2.2 Evaluation Team ........................................................................................................... 6 3.0 BACKGROUND ......................................................................................... 7 3.1 Goal of FAI .................................................................................................................... 7 3.2 Background on Zimbabwe’s HIV/AIDS Epidemic ................................................. 7 3.3 GOZ Focus on PMTCT .............................................................................................. 8 3.4 USAID and Other Donor Support .......................................................................... 9 4.0 EVALUATION METHODS & LIMITATIONS .................................................. 9 4.1 Field Visit Site Selection ........................................................................................... 10 4.2 Limitations .................................................................................................................. 10 5.0 FINDINGS & OBSERVATIONS ............................................................. 11 5.1 Overall FAI Performance ........................................................................................ 11 5.1.1 Background............................................................................................ 11           5.1.2 Findings and Observations ...................................................................... 12 5.2 Sustainability ............................................................................................................... 22 5.2.1 Background ............................................................................................. 22 5.2.2 Findings and Observations.................................................................... 22 5.2.3 Strengths and Challenges...................................................................... 24    END-OF-PROJECT EVALUATION OF FAI PROJECT iv 5.3 Coordination and Integration ................................................................................. 24 5.3.1 Background ............................................................................................. 24 5.3.2 Findings and Observations.................................................................... 25 5.3.3 Strengths and Challenges...................................................................... 27 5.4 Leveraging Resources ............................................................................................... 28 5.4.1 Background ............................................................................................. 28 5.4.2 Findings and Observations.................................................................... 28 5.4.3 Strengths and Challenges...................................................................... 29 6.0 KEY LESSONS LEARNED ...................................................................... 30 7.0 RECOMMENDATIONS ........................................................................... 30 ANNEXES: ANNEX 1: List of People Interviewed ANNEX 2: Evaluation Tools ANNEX 3: Evaluation Work Plan ANNEX 4: Bibliography ANNEX 5: Statement of Work    END-OF-PROJECT EVALUATION OF FAI PROJECT v ACKNOWLEDGEMENTS The evaluation team wishes to express its appreciation to Mr. Matthews Maruva, Dr. Ruth Bulaya￾Tembo, and colleagues at USAID/Zimbabwe, for their support for this evaluation. Their encouragement and suggestions throughout the evaluation process were focused and very helpful. We offer a special word of thanks to Dr. Angela Mushavi, PMTCT and Pediatric AIDS Coordinator of the Zimbabwean Ministry of Health and Child Welfare (MOHCW), for her frank and informative observations regarding the national PMTCT program and its complexities. Other senior Ministry officials also provided us with useful background. We thank senior managers responsible for the implementation of the Family AIDS Initiative (FAI) Program – EGPAF, the Kapnek Trust, OPHID and ZAPP - for the time they took to meet with us, for their excellent support during our time in the field, and for their material contributions to our data gathering process. Our work was greatly enhanced and facilitated by MOHCW and Municipal health officials at provincial and district levels. We especially thank the District Nursing Officers (DNO) who took time from their perpetually busy schedules to meet with us and share their perspectives on the PMTCT program. We thank most especially the many health service providers – nurses, midwives, matrons and others – who welcomed us to their health centers and hospitals, shared with us the frustrations and rewards of their work, and were always generous with their time and ideas. They are the individuals most directly responsible for ensuring that PMTCT services are becoming an integral element of family health care in Zimbabwe. Finally, we are grateful to Ms. Mirela McDonald, Ms. Meredith Kummings and Ms. Audra Stark of ME&A, who have been unfailingly responsive to our needs and questions throughout this assignment, and to Mr. Columbus Mavhunga who provided us with every type of support needed during our work in Zimbabwe, and did so with infectious good humor.    END-OF-PROJECT EVALUATION OF FAI PROJECT vi ABBREVIATIONS ANC Antenatal Care ART Antiretroviral Therapy ARV Antiretroviral (drug) CA Cooperative Agreement CDC Center for Disease Control CHAI Clinton Foundation HIV/AIDS Initiative CIFF Children’s Investment Fund Foundation DBS Dried Blood Spot (test for HIV in infants) DFID Department for International Development (UK) DFP District Focal Person (for PMTCT) DHS Demographic and Health Survey DHT District Health Team DMO District Medical Officer DNA Deoxyribonucleic Acid DNO District Nursing Officer EBF Exclusive Breast Feeding EID Early Infant Diagnosis (of HIV) EGPAF Elizabeth Glaser Pediatric AIDS Foundation FACE Families and Communities for Elimination of Pediatric AIDS FAI Family AIDS Initiative FGD Focus Group Discussion FP Family Planning HSS Health System Strengthening IMAI Integrated Management of Adult and Adolescent Illness IMPAC Integrated Management of Pregnancy and Childbirth IYCF Infant and Young Child Feeding ME&A Mendez England and Associates MER More Efficacious Regimen MIMS Multiple Indicator Monitoring Survey MMR Maternal Mortality Ratio MOHCW Ministry of Health and Child Welfare MTCT Mother-to-Child Transmission (of HIV) NAC National AIDS Council NGO Non-Governmental Organization OPHID Organization for Public Health Interventions and Development OVC Orphans and Vulnerable Children PCR Polymerase Chain Reaction PEPFAR President’s Emergency Fund for AIDS Relief PMTCT Prevention of Mother-to-Child Transmission of HIV POC Point of Care PPF Partnership Forum SAFAIDS Southern Africa HIV and AIDS Information and Dissemination Service UNICEF The United Nations Children’s Fund VHW Village Health Worker ZAPP Zimbabwe AIDS Prevention Project ZNFPC Zimbabwe National Family Planning Council    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 EXECUTIVE SUMMARY BACKGROUND AND APPROACH This is a report on the End-of-Project Evaluation of the Family AIDS Initiative (FAI) project in Zimbabwe. FAI was funded through a Cooperative Agreement (CA) between USAID/Zimbabwe and the Elizabeth Glaser Pediatric AIDS Foundation (EGPAF.) FAI was launched on October 1, 2007 and concluded on September 30, 2012. The project was designed to build on earlier efforts, by EGPAF and others, to create and maintain a national program for the prevention of mother-to-child transmission (PMTCT) of HIV, a disease that has for many years been an extremely heavy burden on Zimbabwean health systems. This end-of-project evaluation assessed FAI’s success in achieving its objective of building the capacity of the Zimbabwean Ministry of Health and Child Welfare (MOHCW) to develop and maintain PMTCT services of high quality on a national scale. The evaluation took place during October and November of 2012. It was carried out by a team of four specialists, three Zimbabweans and a U.S.-based consultant, recruited by Mendez England & Associates (ME&A), a Maryland-based consulting firm, under contract with USAID/Zimbabwe. Over the course of three weeks spent in Zimbabwe, the team met with principal stakeholders and traveled extensively throughout the country, visiting numerous health facilities that provide PMTCT services with support from the different FAI implementing partners. The report that follows describes the team’s process of site selection for these visits, and its approach to data gathering, including use of tools designed by the team. In conducting the evaluation, the team sought to answer two overarching questions, as specified in the evaluation’s Scope of Work:  Has FAI managed to develop capacity within the MOHCW and local non-governmental organizations (NGOs) to support continued delivery of PMTCT services beyond the life of the project?  How can these efforts be improved upon? To answer these questions, the team looked at FAI from four perspectives: 1) It assessed overall FAI project performance as well as the extent to which the project achieved the desired quantity, quality and timeliness of results; 2) It looked at the potential, from various perspectives, for the project’s sustainability, i.e., whether the MOHCW and the health system have the capacity to continue PMTCT services to beneficiaries after external support has ended; 3) It addressed issues of coordination and integration, looking at working relationships within the FAI implementing partnership and between the partners and the MOHCW, and at progress in achieving integration of PMTCT with other clinical services; and 4) It assessed the extent to which leveraging of resources other than USAID funds impacted the achievement of overall FAI project results. The evaluation took place at a time when FAI had ended, and when a follow-on program had been launched, with essentially the same mission, increased USAID funding, and a reconfigured implementing partnership. Under this new initiative the overall objective of building capacity of the MOHCW to ensure lasting availability of comprehensive PMTCT services is unchanged. Findings of this evaluation, while focused on FAI, are thus intended to be applicable to the new program as well, highlighting successes on which to build while at the same time stimulating new thinking.    END-OF-PROJECT EVALUATION OF FAI PROJECT 2 FINDINGS The Evaluation Team found that FAI has succeeded in achieving its principal objective, namely to ensure access to high quality PMTCT services for women, children and their families in urban and rural health service facilities. Project achievements can be summarized as follows and are discussed more broadly in this report:  The FAI project supports PMTCT in 60 districts (out of 62) in Zimbabwe, covering a total of 1381 health facilities (89% of all facilities).  All 1381 sites offer PMTCT services that include on-site HIV rapid testing for pregnant women and their partners and ARV prophylaxis, an increase from 290 sites offering testing in October 2007.  The number of women attending Antenatal Care (ANC) in FAI-supported sites has also risen significantly from 138,719 in 2007/8 to 426,615 in 2011/12. There has also been a corresponding increase in the number of women tested in ANC settings from 109,293 in 2007 to 410,615 in 2012.  A total of 316 PMTCT sites (22.8% of all FAI sites) provide on-site Antiretroviral Therapy (ART) while 537 provide ART through outreach to initiating centers. This implies that a majority of HIV+ women in need of ART are referred to these centers, often at considerable distance.  154 FAI -supported PMTCT sites, or 11% of the whole, are equipped with Point of Care (POC) CD4 machines, making on-site CD4 testing at these locations possible. Others must send samples in stabilized tubes to these sites, creating a bottleneck to universal access to ART for pregnant women.  66% of all HIV exposed infants are tested for HIV before 2 months of age, using DNA￾Polymerase Chain Reaction (PCR).  73% of HIV exposed infants are being initiated on Cotrimoxazole prophylaxis. There is universal availability of HIV testing at FAI-supported sites visited by the evaluation team. At least one nurse, and usually a primary care counselor, is trained in HIV rapid testing and counseling at each health facility. Data from FAI partners also shows a steady increase in the number of women in ANC being tested for HIV (see table illustrating PMTCT cascade in Section 4.1.1.). There is wide geographic coverage of PMTCT services in all 60 FAI districts. High quality, comprehensive PMTCT services, are now provided in 95% of the 1,560 health facilities in Zimbabwe. The level of PMTCT integration with other services, a principal goal of this project, differs according to the size and geographic setting of health facilities, with integration decreasing the higher one goes in the health system. However, this is a work in progress, and integration is becoming more and more evident, and is given high priority, throughout the system. A critical mass of cadres of health workers trained in PMTCT has been achieved through the project. There has also been a striking improvement over the 5 years of FAI in terms of the ready availability of medicines and supplies. No significant gaps or shortages in terms of commodities were observed at any health facility, although there are some gaps in availability of equipment of dependable quality. In terms of the sustainability of services built through FAI, management and technical capacity to lead and sustain the PMTCT program is already evident in the MOHCW, and continuing to grow. Awareness of the availability and importance of PMTCT services appears now to be close to universal, especially among women. However, male involvement remains unsatisfactory, and there is a need for new ideas for achieving greater gender balance in utilization of services. The ability of the Government    END-OF-PROJECT EVALUATION OF FAI PROJECT 3 of Zimbabwe (GOZ) to assume financial support of the PMTCT program is a looming question. A one￾item-at-a-time approach to this challenge should start with developing plans to absorb the role of District Focal Person (DFP) into district health structures after 2015. FAI’s coordination with MOHCW has been effective and supportive, ensuring Ministry ownership of the PMTCT program. Support for key personnel has given PMTCT a strong “seat at the table” with other services. The PMTCT Partnership Forum has provided a vibrant structure for coordination of PMTCT and pediatric HIV interventions among stakeholders. Establishment by EGPAF of the DFP role provided an extra cadre to reduce the burden on overworked District Nursing Officers (DNOs). Coordination with the District Health Team (DHT) was a challenge at first, but results from the field show that DFPs have become indispensable partners in strengthening PMTCT services. Expanding service integration, from rural health centers where it occurs out of necessity because staffs are small and patients often must travel long distances, to larger facilities and hospitals where family health services are often separated, should remain a major priority. To be truly accepted, PMTCT needs to be part of an integrated whole. Leveraged resources have played a big role in FAI’s success. Notably, the $60 million 5-year award from the Children’s Investment Fund Foundation (CIFF) demonstrated the positive energy new resources can bring to an enterprise. However, a question still remains: Once external funding has been used to develop a certain level of capacity, can it be maintained unassisted? Again, a key test case will be the DFPs. It should also be mentioned that the MOHCW gives the USAID/Zimbabwe Mission high marks for its close consultation with Ministry leaders at all points in the development of this and other health-related interventions. While its internal policies do not permit USAID to fund the Ministry directly, the Mission is always extremely careful to share in detail its ideas and strategies for assistance to the health sector, provided through selected NGOs such as EGPAF. RECOMMENDATIONS This is a summary of the evaluation team’s recommendations, expanded upon on pp. 29-30  Priority attention should be given to simplifying indicators and substantially reducing the burden of registers that health workers are obliged to fill out, taking precious time away from providing quality health care to patients.  There is a pressing need to increase coverage of ART initiating sites beyond those equipped with POC CD4 machines.  Integration of PMTCT with other services, at all service levels, must be a continuing priority, for reasons bearing on both economics and service quality.  With regard to training, PMTCT content in pre-service curricula for nurses needs to be strengthened. Time that clinic staff is away from their facilities for in-service training should be minimized by combining curricula.  New approaches for encouraging and mentoring nurses in initiation of ART need to be developed. One might be a “roving doctor,” for every 2-3 districts, dedicated to teaching and mentoring nurses in ART initiation.  The latter idea notwithstanding, it is important to note that the PMTCT program in Zimbabwe is nurse driven, and training of MDs in PMTCT should be scaled up to make them more informed supporters.    END-OF-PROJECT EVALUATION OF FAI PROJECT 4  The DFP model has proven to be effective in strengthening PMTCT services across the country. Discussion of whether and (if so) how this role will be absorbed into MOHCW structures and budgets when external funding of DFPs ends in 2015 should not be delayed.  A targeted approach to caring for young, HIV+ women (14-18), one that recognizes that their needs and concerns are different from more mature women, should be developed in line with Prong #2 of PMTCT guidelines.  Ways to address the seeming lack of coordination of donor support at district level, and the imbalance between those who do and do not receive support for such things as clinic renovations and furnishings, should be explored.    END-OF-PROJECT EVALUATION OF FAI PROJECT 5 1.0 INTRODUCTION This is the final report of the End-of-Project Evaluation of the Family AIDS Initiative (FAI) project in Zimbabwe, conducted in October and November 2012. FAI was funded through a CA between USAID and EGPAF. Funding at the time the CA was signed in 2007 came to $12.3 million, an amount subsequently increased by USAID to $22.3 million. The project was launched on October 1, 2007, and concluded on September 30, 2012. Field activities were implemented by EGPAF and its three partners: the J.F.Kapnek Charitable Trust, the Organization for Public Health Interventions and Development (OPHID), and the Zimbabwe AIDS Prevention Project (ZAPP) of the University of Zimbabwe. FAI was intended to build on earlier efforts, many of them supported by EGPAF and USAID, to make the prevention of PMTCT an integral part of family health services throughout Zimbabwe. It was designed to build the capacity of MOHCW to achieve its goal of “promoting delivery of comprehensive, high quality PMTCT services that are integrated and linked to treatment, care, and support.”1 The purpose of this evaluation was to determine to what extent that goal has been achieved. As a result of USAID’s investment in expanding and strengthening the national PMTCT program over FAI’s five-year life span, EGPAF was able to leverage substantial additional support from other sources. Most substantial was a 2010 award, in the amount of $60 million over five years, from the British-based Children’s Investment Fund Foundation (CIFF). This particular funding led to important additions and structural changes in the implementation of the program, the effects of which will be examined in this report. Significant FAI funding was also leveraged by EGPAF from the Department for International Development, UK (DFID) and Clinton Foundation HIV/AIDS Initiative (CHAI), and individual partners received support from The United Nations Children’s Fund (UNICEF), the Gates Foundation, and others. It is also important to note that, while FAI has ended, a follow-on program with essentially the same mission has been approved, with substantially increased USAID funding, and in fact is already in operation. The structure of the implementing partnership for the new program, Families and Communities for Elimination of Pediatric AIDS, or FACE, has been slightly changed. ZAPP has been replaced by the Southern Africa HIV and AIDS Information Dissemination Service (SAFAIDS), and OPHID has replaced EGPAF as prime contractor. But the overall objective of continuing to build the capacity of the MOHCW to ensure the availability of high quality, comprehensive PMTCT services at service points throughout the country is unchanged. Findings of this evaluation, while focused on FAI, are thus intended to be applicable to the new program, highlighting successes on which it can build and stimulating new thinking as well.                                                              1 From EGPAF FAI Program 2011 Annual Report    END-OF-PROJECT EVALUATION OF FAI PROJECT 6 2.0 EVALUATION PURPOSE & EVALUATION QUESTIONS 2.1 EVALUATION PURPOSE AND OBJECTIVES The end-of-project evaluation of the FAI project was designed to answer two key, overarching questions, as specified in the evaluation’s Scope of Work:  Has FAI managed to develop capacity within the MOHCW and local non-governmental organizations (NGOs) to support continued delivery of PMTCT services beyond the life of the project?  How can these efforts be improved upon? To answer these questions, the evaluation team was asked to look at FAI from four perspectives: 1. Overall FAI Project Performance. To what extent did the project achieve the desired quantity, quality and timeliness of results? 2. Sustainability. To what extent were processes, tools and approaches developed through the project institutionalized within the MOHCW? Has the ability of the Ministry to continue PMTCT service provision to intended beneficiaries after external support has ended been assured? 3. Coordination and Integration. How did working relationships within the FAI implementing partnership; between the partners and the MOHCW; and between the project, the Ministry and other stakeholders facilitate the achievement of desired results? In particular, how did issues of coordination and integration impact the quality and comprehensiveness of service delivery in health facilities country-wide? 4. Leveraging Resources. What was the significance of financial support leveraged from external sources to the achievement of overall FAI project results? How did the leveraging of such resources help or hinder long term sustainability of services? 2.2 EVALUATION TEAM The evaluation team was composed of four specialists: three Zimbabweans and a U.S.-based consultant. The team included a physician and a nurse-midwife, each having extensive experience with and understanding of the delivery of HIV-related services, including PMTCT. It also included members with long experience in the development and evaluation of programs designed to strengthen primary health care services in many countries, as well as extensive experience working in both the Zimbabwean public sector and with NGOs. No team member had any conflict of interest relative to the FAI project.    END-OF-PROJECT EVALUATION OF FAI PROJECT 7 3.0BACKGROUND 3.1 GOAL OF FAI In its Technical Application for funding of the FAI Project, EGPAF stated that it sought to “improve the quality of life of families infected and affected by HIV in Zimbabwe through prevention of pediatric HIV infection.” It hoped to “achieve a continuum of care that not only encompasses HIV care and support but integrates them into the overall health system.” To this end it set an ambitious national strategic goal: “To ensure delivery of comprehensive, high quality PMTCT services integrated with care, treatment, and support of families, including children living with HIV and AIDS.” This goal was linked directly to the GOZ’s National HIV and AIDS Strategic Plan (ZNASP l) 2006-2010, subsequently replaced by ZNASP II, 2011-2015. In implementing FAI, EGPAF sought to help the Government make major inroads in addressing a health issue that has plagued the country for years. 3.2 BACKGROUND ON ZIMBABWE’S HIV/AIDS EPIDEMIC Zimbabwe bears one of the heaviest burdens of HIV and AIDS in the world. Despite progress over the last decade, HIV prevalence remains unacceptably high. With a population estimated at 12.7 million (ZIMSTATS Projection, 2011), about 1.1 million people are living with HIV. Fully 10% of these are children, 90% of whom are infected through mother-to-child transmission (MTCT)2. There have been positive developments, indicating that progress can be made with concerted effort. In 2009, the number of new HIV infections in children was 14,976, a decline from 16,643 in 2008. There has also been a decline in the number of HIV- positive pregnant women, from 53,000 in 2008 to 50,000 in 20093. However, these figures also indicate that the situation remains dire. Zimbabwe has one of the highest maternal mortality ratios (MMR) in the world. The 2010-11 Demographic and Health Survey (DHS) estimated it to be 960 against a Millennium Development Goal (MDG) #4 target of 174 /100.000 (MDG 4-Reducing maternal mortality)4. MMR has increased over the last decade (the 2005/6 DHS showed an MMR of 555), a fact attributed to various reasons, prime among them being socio-economic decline over the decade 1999-2009. This resulted in rapid deterioration in health services, with major referral hospitals shutting down and the loss of skilled midwives and doctors to jobs abroad. Currently, HIV and AIDS-related mortality comprises about 25.5% of maternal mortality, making it a major cause of maternal death. Under-five mortality, often used as a proxy measure of the state of social services, has risen steadily since the 1990s. The Multiple Indicator Monitoring Survey (MIMS) estimates the rate to be 82 per 1000 live births, with HIV and AIDS-related illness accounting for 21% of deaths. Nearly 95% of HIV infections in children are transmitted vertically, i.e., during pregnancy, labor/delivery and breastfeeding, a compelling case for PMTCT scale-up.                                                              2 WHO PMTCT strategic vision 2010‐2015 3 MOHCW Annual Report 2009 4 Zimbabwe Demographic and Health Survey Report, 2010‐2011    END-OF-PROJECT EVALUATION OF FAI PROJECT 8 Early efforts to curb the spread of HIV in Zimbabwe were confined to ensuring blood safety and behavior change interventions. In 1999, the GOZ enacted its first national policy on HIV, declaring it a national emergency. By 1999, the epidemic had become generalized in the population and, therefore, required a multi-sectoral response to address specific drivers of the spread of HIV in different parts of society. Strategies were put in place, including ZNASP 1 and 2, and national plans and policies enacted dealing with ART, PMTCT (see below), HIV Testing and Counseling, and orphans and vulnerable children (OVC). The country also adopted the “Three Ones” principle and established the National AIDS Council (NAC) to coordinate a multi-sectoral response. 3.3 GOZ FOCUS ON PMTCT The PMTCT and Pediatric HIV Prevention, Treatment and Care Plan (2006-2010) of the GOZ had as its overall goal: “To provide comprehensive PMTCT services to at least 80 % of pregnant women, their babies and families, including care and treatment of pregnant women, in the context of universal access, with the aim of reducing MTCT rates to less than 10% by the end of 2010”. The number of health facilities providing PMTCT services has risen from 3 in 2001 to 1381 at the present time. Services have evolved rapidly from the early days, when single dose Nevirapine was offered to the mother and baby, to MER28, and subsequently to MER14. The GOZ is currently implementing World Health Organization (WHO) Option A for PMTCT, which requires CD4 testing for all HIV positive pregnant women before they can be initiated on ART. CD4 testing and early infant diagnosis (EID) present some of the most significant challenges to the PMTCT program, although progress has been made in ensuring access to testing through POC CD4 machines and use of dried blood spot (DBS) for DNA-PCR for EID. MOHCW and partners’ efforts are guided by goals set in ZNASP II5 Expected Outcomes The % of health facilities providing ANC services that provide both HIV testing and PMTCT on site is increased from 66% in 2010 to 95% by 2015 The % of infants born to HIV+ women who received a virologic test from 6 weeks after birth is increased from 13% in 2009 to 80% in 2015 The % of HIV+ pregnant women who received ARV prophylaxis to reduce the risk of MTCT is increased from 59% in 2009 to 95% in 2015 The % of HIV infected infants born to HIV+ mothers is reduced from 30% in 2009 to less than 5% by 2015. National PMTCT strategy is aligned with WHO PMTCT policy, which is premised on four “prongs”: 1. Primary prevention of HIV among women of child bearing age. 2. Preventing unintended pregnancies among HIV positive women. 3. Preventing HIV transmission from a woman living with HIV to her infant; and 4. Providing appropriate treatment, care and support to mothers living with HIV, their children and families.                                                              5 ZNASP II 2010‐2015    END-OF-PROJECT EVALUATION OF FAI PROJECT 9 The FAI project focused at the outset on Prongs 3 and 4, aimed at reducing HIV transmission from mother-to-child, as well as providing treatment and care for mothers, their children and families. 3.4 USAID AND OTHER DONOR SUPPORT Zimbabwe is a low-income country and, therefore, largely dependent on donor aid to support development programs and social services. The country, with an agro-based economy, experienced economic challenges over the last decade that saw a deterioration in social services nationwide. USAID is one of the leading donors in supporting Zimbabwe’s health sector through various local and international partners. Its funding of the FAI program through EGPAF represented an expansion of an already strong commitment to strengthening PMTCT services. USAID’s commitment has been further emphasized by its recent approval of substantially increased funding for FACE, the follow-on PMTCT program to FAI, launched on October 1st, 2012. Other international donors have contributed substantially to the national PMTCT effort as well. DFID contributed £2 million through EGPAF/FAI in support of: 1) furthering early infant diagnosis of HIV; and 2) strengthening national management and coordination of PMTCT services. It also provided interim funding to FAI at a point in the project when regular funding was delayed. UNICEF also plays a major role as the manager of a $200 million, multi-donor Health Transition Fund, dedicated to overall health system strengthening (HSS). Finally, as has been noted and will be discussed later in this report, USAID funding was instrumental in enabling EGPAF to leverage an award of $60 million from CIFF for FAI, which substantially changed and expanded the reach of the program. 4.0 EVALUATION METHODS & LIMITATIONS This was primarily a qualitative evaluation, designed to determine FAI’s effectiveness and efficiency in achieving its objective of expanding access and quality in the delivery of PMTCT services while building MOHCW capacity to carry such services forward in the long term. The evaluation team employed a combination of structured and open-ended interviews to facilitate its assessment, using tools that it designed to gather information from different sources at different levels. As will be evident throughout this report, the team also gathered enough quantitative data, including data on selected PMTCT indicators from visited districts, to verify its qualitative findings. Prior to convening in Zimbabwe, the team reviewed briefing materials provided by USAID/Zimbabwe and drafted a work plan for its in-country activity. Once assembled in Harare on October 15, 2012, the team had a series of briefings with USAID, EGPAF and its partners, and officials of the MOHCW, notably the National PMTCT and Pediatric HIV Care and Treatment Coordinator. The team met with many other stakeholders with strong involvement in the national PMTCT program, including NAC, DFID, CDC, UNICEF, and the Zimbabwe National Family Planning Council (ZNFPC), while at the same time finalizing its work plan and developing tools for use in data gathering. Evaluation tools included an interview guide for use in interactions with stakeholders and service providers at various levels, and a guide for focus group discussions (FGD) that the team planned to hold with beneficiaries, communities and cadres of service providers in visits to project sites around the country. The team also developed a short questionnaire to enable it to gather quantitative, program-    END-OF-PROJECT EVALUATION OF FAI PROJECT 10 related information at district level in the course of those visits, which it could use to compare with data provided in FAI project reports and other sources (see Annex 2 for Evaluation Tools.) 4.1 FIELD VISIT SITE SELECTION To conduct the evaluation, the team sought to cover a cross section of provinces and districts of Zimbabwe, and to be sure they were proportionally representative of coverage by the three FAI partners - ZAPP, OPHID, and the Kapnek Trust6. The team set an objective of visiting at least two health facilities per district, purposely selecting sites to include government and mission hospitals, and government and council clinics in both rural and urban settings. By dividing itself into two sub-groups of two members each, the team was able to visit 13 districts and 2 metropolitan provinces, representing roughly 25% of the 60 districts covered by FAI, as follows:  Matebeleland North: Lupane, Hwange  Matebeleland South: Bulilima, Mangwe, Umzingwane  Masvingo: Zaka, Chiredzi  Manicaland: Mutasa, Mutare, Chipinge, Chimanimani  Mashonaland East: Mutoko  Chitungwiza (a district of Harare, but semi-autonomous)  Bulawayo  Harare Meetings were held with District Nursing Officers (DNOs), District Medical Officers (DMOs) and other members of District Health Teams (DHT), or, in their absence, their replacements. The team made certain to have extensive contact with DFPs recruited in 2011 by EGPAF specifically to strengthen PMTCT service delivery country-wide. The team also held a number of FGDs with clinic patients, service providers, village health workers (VHWs) and community leaders. Reconvening in Harare after this series of field visits, the evaluation team reviewed and synthesized its findings in preparation for briefings of preliminary conclusions for FAI partners, the MOHCW, and USAID. It also scheduled follow-up stakeholder meetings as needed, and prepared to disband to return home and draft the evaluation report. 4.2 LIMITATIONS The evaluation process was characterized by enthusiastic participation of FAI stakeholders, from national down to health facility level. Time limitations made it hard to conduct a comprehensive quantitative analysis of FAI, but the evaluation team was able to perform relevant cross sectional analyses. In scheduling key informant interviews, we sometimes found that DMOs, DNOs or others were away at other meetings, but we were always able to meet with alternate or acting officials. Finally, changes in PMTCT indicators over the years of the project contributed to some gaps in our understanding of achievements. But overall, the team was well received and benefited from full cooperation at all levels.                                                              6 Kapnek Trust supported PMTCT sites in 38 districts, OPHID in 23 districts, ZAPP in one district (Chitungwiza) Srg egri    END-OF-PROJECT EVALUATION OF FAI PROJECT 11 5.0 FINDINGS & OBSERVATIONS 5.1 OVERALL FAI PERFORMANCE 5.1.1 Background The FAI program was launched during a difficult period in Zimbabwe’s history. In 2007-2008, the country faced unprecedented economic challenges - erosion of incomes, hyperinflation, and deteriorating social services. Experienced doctors and midwives left the country, leaving many health facilities with staff vacancies, a situation compounded by a GOZ-mandated employment freeze due to budget limitations. Consequently, the MOHCW had limited capacity, both financial and human, to implement a project of FAI’s magnitude. Nonetheless, FAI was found to have succeeded in achieving its principal objectives, namely to ensure access to high quality PMTCT services for women, children and their families in urban and rural areas. Statistical findings of the evaluation team include:  The FAI project supports PMTCT in 60 districts (out of 62) in Zimbabwe, covering a total of 1381 health facilities (89% of all facilities.)  All 1381 sites offer PMTCT services that include on-site HIV rapid testing for pregnant women and their partners, an increase from 290 sites offering testing in October 2007.  The number of women attending ANC in FAI supported sites has also risen significantly from 138,719 in 2007/8 to 426,615 in 2011/12. There has been a corresponding increase in the number of women tested in ANC settings from 109,293 in 2007 to 410,615 in 2012.  A total of 316 PMTCT sites (22.8% of all FAI sites) provide on-site ART while 537 provide ART through outreach to initiating centers. This implies that a majority of HIV+ women in need of ART are referred to these centers, often at considerable distance.  154 FAI -supported PMTCT sites, or 11% of the whole, are equipped with POC CD4 machines, making on-site CD4 testing at these locations possible. Others must send samples in stabilized tubes to these sites, creating a bottleneck to universal access to ART for pregnant women.  66% of all HIV exposed infants are tested for HIV before 2 months of age, using DNA-PCR.  73% of HIV exposed infants are being initiated on Cotrimoxazole prophylaxis. The following table reflects trends in these areas across the PMTCT cascade: Year ANC attendance # women tested & received results in ANC # women testing HIV+ in ANC #women testing HIV + in L&D # women CD4 tested in ANC # women receiving ARV prophylaxis in ANC # infants receiving ARV prophylaxis 2008 138,719 109,293 17,935 - - 15,691 10,817 2009 146,412 131,007 19,462 844 No Data 18,117 14,318 2010 224,656 205,400 28,303 2820 14,176 35,341 24,534 2011 328,181 321,617 38,594 5617 6683 42,442 - 2012 426,615 410,615 41,903 4360 27,615 49,310 54,935    END-OF-PROJECT EVALUATION OF FAI PROJECT 12 0 500 1000 1500 2000 2007 2012 FAI Goal 290 1381 1560 Figure 1. Number of Health Facilities as of Oct. 2012 2007 2012 FAI Goal The data confirms some “leakage” across the PMTCT cascade in terms of discrepancies between numbers of women tested HIV+ and those assessed for ART eligibility using CD4 count, reflecting limitations in access to CD4 testing. Also, the number of infants receiving ARV prophylaxis is, in most cases, fewer than that of mothers tested HIV+ in ANC and Labor and Delivery (L&D) combined. High numbers of women receiving ARV prophylaxis for PMTCT are explained by the fact that some are tested elsewhere. Key Definitions: According to FAI guidelines, the project adopted certain key definitions and strategies to meet its goal and objectives. An integrated approach refers to improving the availability and quality of PMTCT services through their complete integration with other key, already established health services and systems, such as ANC, HIV counselling and testing, provision of ART, breastfeeding and nutrition counselling, and family planning. A comprehensive PMTCT package indicates that health workers in a facility designated as providing PMTCT services are trained in overall PMTCT services, including HIV prevention, on site rapid HIV testing, pre- and post-test counselling, infant feeding counseling, ARV prophylaxis, short course sdNVP, quality assurance and comprehensive monitoring and evaluation (M&E). A family centered approach ensures that women, children, their partners/spouses and families can access relevant services as part of a continuum of care on one comical site. It includes ensuring that all HIV services are offered to men, women and children in an environment that encourages families to be treated and supported as a unit and not as individual entities. 5.1.2 Findings and Observations A. Coverage vs. Targets From a baseline of 290 health facilities providing PMTCT services in October 2007, as of October 2012, there were 13817 facilities offering comprehensive PMTCT services, an achievement of 89% of the FAI target of 1560 such facilities by 2012. While in some districts the program was pioneered or co-financed by other actors, continual training of staff, coordination of PMTCT activities, supervision and monitoring, were made possible by FAI. However, while most sites offer a comprehensive package of PMTCT services, as previously described, initiation of ART for eligible HIV positive pregnant mothers and infants/children is still far from universal.                                                              7 EGPAF‐FAI Quarterly progress report, July‐September 2012.    END-OF-PROJECT EVALUATION OF FAI PROJECT 13 At present only 316 sites out of the 1381 supported by FAI (23%) provide on-site initiation of ART for pregnant women. 537 sites provide ART through community outreach services. Other sites must either refer mothers to a higher-level facility, or notes have to be taken to an initiating center by the nurse. Universal access to ART at health facility level is still a distant goal and a major PMTCT program gap. B. Access to PMTCT Services Knowledge of Availability of PMTCT Services. MIMS-2009 revealed comprehensive knowledge about HIV prevention among women 15–49 years to be 55.2%, higher in urban provinces (66%) than rural provinces (48%). Knowledge about HIV prevention among young women aged 15–24 years was 53.3%. 86% of women knew about a drug being used to reduce MTCT and about the risk of transmission of HIV during breastfeeding8, compared to only 52% in the 2005-6 DHS. Strengths  Interviews with health professionals during the evaluation indicated that in 2012 as many as 95% of intended beneficiaries of the PMTCT program are knowledgeable about the importance of services and where they can be accessed, a substantial improvement in knowledge levels among women of child bearing age from the program’s inception.  Correct understanding of most of the key issues related to pregnancy and PMTCT were evident during FGDs with pregnant and postnatal women. Challenges  Despite high knowledge levels about PMTCT and benefits of early ANC booking, in practice most women continue to book late. An example is Chitungwiza, where expectant mothers book as late as 36 weeks, far later than the MOHCW/WHO recommended 14 weeks, which coincides with MER 14. The average gestational age for first ANC visits was found to be 20-24 weeks. This pattern corroborates that reported in the Zimbabwe DHS 2010-11, which states that only “19% of women have their first ANC visit within the first trimester of their pregnancy, and 5% have first visits from the eighth month onwards. The median duration of pregnancy at first visit was 5.3 months”9. Some of the reasons for the delay given by key informants included beliefs or practices that are difficult to modify, such as the following: 1. Traditional belief systems that encourage mothers to conceal pregnancies for fear of witchcraft, and/or that prevent some religious sects from accessing conventional health care. 2. Young unmarried pregnant women tend not to disclose pregnancies until very late for fear of stigma and rejection by families. 3. Avoidance of travelling long distances (to the clinic) frequently during a pregnancy. 4. Fear of being tested for HIV. 5. Inconsistent approach to community mobilization and sensitization by community leaders, Ward Health Committees and VHWs. C. HIV testing and Counseling There is universal availability of HIV testing at FAI-supported comprehensive PMTCT sites visited by the evaluation team. At least one nurse, and often a primary care counselor, are trained in HIV rapid testing                                                              8 ZDHS 2010‐2011 9 ZDHS 2011‐2012: p109    END-OF-PROJECT EVALUATION OF FAI PROJECT 14 and counseling at each health center. Data from FAI partners also shows a steady increase in the number of women in ANC being tested for HIV as shown in Figure 2. Figure 2: HIV testing in ANC over the 5 Years of FAI. However, secondary data analysis shows that many women reach labor and delivery with unknown HIV status and go on to deliver without being tested (see Figure 3, below). These women will not have attended ANC at the clinic where they deliver. They may have done so elsewhere, where they might or might not have been tested, and are not disclosing their status. They will thus miss the opportunity for health education, counseling, testing and ART prophylaxis if required. Figure 3 – Women reporting for L&D with Unknown HIV Status in FAI Partners’ Facilities The graph shows a decline in the number of women reporting for labor and delivery with unknown HIV status, which would seem to indicate that more women are in fact accessing testing through ANC. However, more work needs to be done to encourage women to access ANC in the early stages of their pregnancies. 0 100000 200000 300000 400000 500000 2008 2009 2010 2011 2012 Counseling and Testing in ANC Number of first ANC visits Number of pregnant women tested and received HIV results in ANC Number of women HIV positive in ANC 0 5000 10000 15000 20000 25000 30000 35000 2009 2010 2011 2012 HIV testing and Counseling in Labor and Delivery Number of pregnant women arriving in L&D with unknown HIV status Number of pregnant women tested in L&D Number of women HIV positive in L&D    END-OF-PROJECT EVALUATION OF FAI PROJECT 15 Strengths  HIV testing and counseling is offered in all health facilities visited under this evaluation, confirming FAI’s PMTCT service coverage data. Training Primary Care Counselors in HIV testing and counseling provides additional cadres at facility level. Challenges  High staff turnover, combined with a Government hiring freeze, leaves some health facilities with nurses inadequately trained in rapid testing.  There continues to be low direct uptake of HIV testing by male partners. D. Affordability of PMTCT Services The gradual removal of user fees in health facilities has enhanced access for poor households, leading to a notable increase in demand for PMTCT services. Implementation of the Health Transition Fund10 through the MOHCW should complete this process. However, user fees remain in place in some facilities, notably in municipality and rural council clinics, where maternity costs can range from $25 - $50, a serious obstacle for many poor households. At the same time it must be noted that, while elimination of user fees may improve access to health services for all, the GOZ needs to address the supply side of services, ensuring that adequate health personnel are in post to meet the rise in demand for health care, including PMTCT. E. Geographic Access Long distances from homes and villages to health facilities remain a barrier for pregnant women, their children and families in accessing PMTCT services. For example, in Chiredzi District, women in some cases must travel 110km to reach the nearest facility, a challenge also noted in Bulilima District and resettlement areas. Many communities have constructed maternity waiting homes, or “shelters,” adjacent to health facilities for pregnant women due to deliver, but such shelters are not available at every facility. F. Male Involvement with ANC/PMTCT Services Male participation in PMTCT requires that men accompany their partners to health facilities for ANC booking and services. In so doing, the male partner also receives HIV counseling and testing, health education on the implications of a positive result for the couple and unborn child and on the treatment and care required for him, the mother and infant. In such cases, men are more likely to take full responsibility for support and adherence to treatment and care for the whole family. Despite the desirability of greater gender balance, male participation in ANC and PMTCT remains low across FAI-supported districts, although some successes have been noted. Key informant interviews and FGDs with women revealed various reasons for low male participation:                                                              10 A pooled multi-donor contribution of approximately $80 million dollars per year for 5 years, managed by UNICEF and given to MOHC&W to help revitalize Zimbabwe’s health sector. The plan is to abolish user fees for women and children within one year of establishing the fund, save more than 30,000 lives among children under five and pregnant women, and accelerate progress on the health related Millennium Development Goals.    END-OF-PROJECT EVALUATION OF FAI PROJECT 16  Due to high unemployment in the country, many Zimbabweans (especially men) have migrated to neighboring countries for employment and return home infrequently;  Imbedded patriarchal practices and societal norms remain influential;  Men are at work during the day and may not have time off to accompany partners for ANC;  Some men are tested separately at their work places;  Fear of stigma and unwillingness to disclose an HIV positive status to partner;  Fear of men in knowing their HIV status which makes them use their partners as “proxy” for determining their own HIV status; and  “Men say that they don’t want to stress themselves or embarrass themselves by going for an HIV test” – Quote from woman during a focus group discussion. Strengths  Anecdotal findings reveal that men are increasingly allowing their partners to enroll for ANC and get tested for HIV. Health centers report that many pregnant women now say their partners support the PMTCT program and agree for them to get tested, an evolution from when women told clinic staff they would have to consult their husbands before undergoing HIV testing.  Media campaigns for male participation have encouraged men to get tested at workplaces, new start centers and in OI/ART services in health facilities. Health facilities have also tried to accommodate men at ANC through approaches such as exempting them from waiting in queues for consultation, or altering visiting hours for men. Such innovations have shown promise, such as in Mutoko and Chitungwiza districts, and should be scaled up throughout the country. Challenges  Direct participation and involvement of men in PMTCT remains limited, rarely exceeding 10%. Exceptions were a reported 30% participation in Mutoko and Seke South clinics. Suggestions for achieving greater gender balance with regard to accessing PMTCT services appear later in this report. G. Focus on Young Pregnant Women According to the Zimbabwe DHS 2010-2011, 24% of women age 15-19 have begun childbearing. Most of these are rural teenagers who tend to be less educated and of low socio-economic status.11 HIV prevalence among women in their teens is 7.5%.12 Prevailing economic conditions have increased women’s vulnerability to HIV and created conditions that foster transactional sex and early or forced marriages. Young women especially (15-19 years of age), may have unwanted or untimed pregnancies. By virtue of their inexperience, they belong to a subgroup that is at risk and requires attention to its special needs, ideally through a tailored package of services linked to PMTCT services, such as peer support and access to youth-centered counselling. Pregnancies among young women were reported to be common in most health facilities visited by the evaluation team, although statistics were not gathered for this evaluation. In FGDs that the team conducted with pregnant women, these teenagers displayed their shyness and immaturity. More than older women, they are uneasy about inviting their partners for HIV testing and counseling, and afraid of knowing their own status, for fear of stigmatization and discrimination. This brings to the fore the need to address specific youth pregnancy and childbearing concerns through the PMTCT program.                                                              11 ZDHS 2010-2011, p.67 12 Ministry of Health and Child Welfare, Zimbabwe National HIV and AIDS Estimates. 2009.    END-OF-PROJECT EVALUATION OF FAI PROJECT 17 Interview with a Young Mother Mary (not her true name) is a very shy patient enrolled in the PMTCT program in a Bulilima District health facility. (It took a while for her to relax for the interview). Mary gave birth to her first healthy baby boy four months ago at the clinic. Mary is exclusively breast feeding her baby according to instructions given by nurses. However, she asked the interviewer whether the baby would truly survive on breast milk only for six months! Mary tested HIV positive when she was 16 years old and in Grade 5 for schooling. She is on ART. She has recently turned 20 years old. Her parents died when she was in Grade 2 and has been looked after by her aunt since then. She met the father of the baby who was employed as a “herd-boy” in her area, but coming from Nkayi district. They had casual sex but the man moved back to his home district when he realised that Mary was pregnant because he was not prepared to take responsibility. When asked whether she was now on a family planning method, Mary said that she was offered an implant at the clinic but refused since she currently doesn’t have a boyfriend. H. Access to ART for HIV+ Pregnant and Postnatal Women and Children Background. FAI has ensured universal access to prophylaxis for mothers and infants in all PMTCT facilities following MOHCW guidelines. For these guidelines the Ministry adopted WHO’s Option A, whereby CD4 testing is a pre-requisite for the initiation of ART in a pregnant woman. The procurement and deployment of POC CD4 count machines, through leveraged funding from CIFF, has increased access to CD4 testing across districts. POC machines facilitate early access to treatment for eligible pregnant women by reducing waiting periods for ART initiation. A total of 154 POC machines have been made available, primarily to higher volume health facilities. This means, however, that only 11% of FAI supported facilities (154 of 1381) have a POC machine on site. Other sites collect samples using stabilized tubes for transport to facilities where machines are in place. Limited availability of CD4 testing capacity thus presents a significant barrier to universal access to ART for expectant mothers. ART initiation is generally doctor-led. To try and expand this capacity, the Global Fund has supported the training of nurses in ART initiation through its Task-Sharing Initiative. These nurses initiate ART through OI/ART outreach services from district hospitals to rural health centers. PMTCT has been integrated into these outreach services, the challenge being that they are offered only at intervals of 1-2 months. The following table (next page) shows a steady increase in numbers of women receiving ART in ANC settings across FAI supported districts. This can be attributed to increased coverage of PMTCT services including HIV testing, an important entry point to treatment, care and support, as well as training of staff in OI/ART. But the data also shows a gap in terms of HIV + women accessing CD4 testing and being initiated on ART compared with the actual numbers of such women. Limits on numbers of POC CD4 machines and ART initiating sites contribute to these discrepancies.    END-OF-PROJECT EVALUATION OF FAI PROJECT 18 Year 2008 2009 2010 2011 2012 Nr. of women initiated on ART - 415 1,914 5,128 6,827 Nr. of HIV + women in ANC 17,935 19,462 28,303 38,594 41,903 Nr. CD4 tested No data No data 3,102 6,683 27,615 Strengths  Improved access to CD4 testing across all districts supported by FAI, and availability of prophylaxis in all health facilities visited. The supply of cotrimoxazole prophylaxis, which was erratic in early phases of the project cycle, has become significantly more dependable.  Sound knowledge of protocols for HIV prophylaxis and treatment among all nurses interviewed. 100% of sites sampled by the evaluation team provide EID using DBS for DNA PCR testing. The challenge, which has been acknowledged in EGPAF reports, remains with the turnaround time for DBS results to be returned to site and patient (see below.) Challenges  Many nurses in districts surveyed are uncomfortable with initiating pediatric ART, due primarily to a lack of confidence in dealing with small children. Nurses cite lack of experience despite their training, and fear of complications or drug reactions in children. This is a major impediment in the program. If pediatric initiation remains doctor centered, mothers and babies must be referred to initiating centers as much as 100-200 kms distant (Chiredzi and Bulilima districts.) Or, nurses must take patients’ files to the initiating center to get the patient starter pack (Mutare district.)  The fact that establishment of new initiating centers has not kept up with demand is another impediment to universal access to ART.  Delays in turnaround time for DNA PCR results due to delays in transporting samples. While it should be acknowledged that there has been a general decrease in turnaround time, this remains a challenge for early initiation of ART in infants. In some cases, mother/baby pairs are lost to follow-up while waiting for results. I. Quality of PMTCT Services Given the preeminent importance of quality in the delivery of health services, the evaluation team assessed this aspect of FAI from several perspectives: Comprehensiveness. The FAI program was designed to provide high quality, comprehensive PMTCT and pediatric HIV prevention, treatment and support services. It has evolved over the years from the use of sdNVP to the introduction of the 2006 WHO guidelines for ART (MER 28, adopted in 2009) and then to the adoption of the 2010 WHO guidelines (MER 14.) These further improved the scope of treatment, care and support for mothers and their infants. Starting in 2007, FAI assisted the MOHCW to steadily increase coverage of PMTCT activities. By the end of 2010, 1,381 health facilities in 60 districts were offering PMTCT services. These facilities did not all necessarily provide the full comprehensive package of PMTCT services,13 but this activity has led to a situation whereby the majority of districts have nearly 100% of clinical sites offering PMTCT services. Exceptions are facilities serving the uniformed forces.                                                              13  According to FAI project practice,   a comprehensive service has all of the following in place: on‐site prevention services, counseling, rapid HIV testing, ARV prophylaxis and sdNVP.      END-OF-PROJECT EVALUATION OF FAI PROJECT 19 Integration of PMTCT within MNCH Services. The introduction of Family and Child Health units (creating a “one-stop-shop” where mothers, babies and their families access all services under one roof), superseding the MCH service within health facilities, has enhanced the integration of PMTCT services into the continuum of care that includes family planning, immunization, and MCH. The level of PMTCT integration with other services differs according to the size and geographic setting of health facilities. Integration decreases the higher one goes in the health system. By nature of their small size and fewer staff, rural health centers are more integrated than district, provincial or central hospitals, with some exceptions observed during the review. Integration at the Ministry level between reproductive health, ART, PMTCT and nutrition units was less visible as program planning and implementation for these various units are done vertically. However, it is a work in progress and integration is becoming more and more evident. (The issue of integration is discussed more fully elsewhere in this report.) Training, Supervision and Mentorship. A major achievement of FAI is in training of cadres of health workers in different PMTCT topics. EGPAF provided technical assistance to the MOHCW in developing training manuals and tools over the 5-year project period, and in embedding in program structures the concept of nurse mentorship, especially in ART initiation. Ongoing supervision and support has been regularly and jointly provided by DNOs and DFPs. The Integrated Management of Adult and Adolescent Illness/ntegrated Management of Pregnancy and Childbirth (IMAI/IMPAC) training manual is comprehensive. Health staffs expressed confidence in their capacity to provide PMTCT services based on their training. However, there was a need expressed throughout the districts to increase human capacity in ART initiation, as well as in data management, including patient tracking systems. Mentorship for trained nurses was piloted in districts such as Mutare and Mutoko in order to improve the performance and confidence of nurses in service provision. However, DHTs noted difficulties in hosting training programs for mentoring due to inadequate financial resources for meal and accommodation expenses. PMTCT Guidelines and Protocols. Rollout and implementation of the revised 2010 WHO guidelines was critical towards meeting the goal of elimination of pediatric HIV. The FAI program therefore prioritized the training of health professionals from all districts in implementation of the guidelines. Generally, providers are adhering to MOHCW/WHO PMTCT guidelines, protocols and related regimens. Some nurses even expressed knowledge about, and commitment to, the program’s contribution to the Millennium Development Goal (MDG) targets for 2015. The FAI project developed excellent pamphlets and posters that chart the components of PMTCT. These are pasted on walls in consultation rooms for ease of reference for the nurses when they attend to patients (see example in photo at right). Availability of PMTCT Supplies and Commodities. There has been a striking improvement over the past 5 years in terms of the ready availability of medicines and supplies, due to general improvements in the economy and the use of the Delivery Team Top Up Unit (DTTU) by FAI. The    END-OF-PROJECT EVALUATION OF FAI PROJECT 20 DTTU delivers medicines from NATPHARM to rural health centers at regular intervals. No significant gaps or shortages in terms of commodities were observed at any health facility. Of particular note is the fact that cotrimoxazole, the supply of which has been problematic in the past, is now more dependably available. While the general availability of PMTCT commodities and medicines has improved over the program period, there is limited availability of hemoglobin (HB) meters and pregnancy test kits. The use of AZT in the MER 14 regimen requires close monitoring of HB since clinical assessment is not accurate enough as a monitoring tool. Since the MER 14 regimen also requires that women book early, pregnancy test kits should therefore be widely available to confirm early pregnancies. Sphygmomanometers (BP machines) and weighing scales were also reported unavailable or non-functional in some clinics. While the FAI project agreement with USAID did not include purchase of these items, they are a vital component of care for the pregnant woman. J. Psychosocial Support A further development in enhancing comprehensive care has been the initiation of psychosocial support groups for pregnant women and lactating mothers to compliment formal health education sessions in health facilities. Facilities have formed such groups to provide opportunities for sharing and learning among women (and men, where there is interest) on issues such as HIV prevention, care and treatment, family planning, nutrition, breast feeding and other issues related to positive living. The evaluation team was introduced to one such group in Hwange District, and the FAI 2008-2009 Annual Report notes that at least 57 PMTCT support groups had been assisted by FAI during the period. K. Early Infant Diagnosis Using DNA-PCR The FAI project has made it possible for all supported sites visited by the evaluation team to offer Early Infant Diagnosis (EID) using Dried Blood Spot (DBS) testing. (EGPAF reports are not clear as to whether all FAI-supported health facilities do so.) Figure 4 below indicates steady increase between 2009 and 2011 in the number of infants tested using DNA PCR and Rapid HIV test kits in clinics in districts where the Kapnek Trust is the FAI implementing partner. However, using the number of HIV positive pregnant mothers as the denominator, a large disparity still exists in infant testing for HIV. For example, in 2011, only 56% of exposed infants were actually tested for HIV. This is a demonstrable increase from 31% in 2010, but a lot still needs to be done to ensure that the rates of testing of infants reach the 85% target set by the MOHCW. Figure 4: Trends in HIV testing in infants in districts supported by the Kapnek Trust 0 5000 10000 15000 20000 25000 30000 2009 2010 2011 DNA PCR Rapid HIV test HIV pos preg women    END-OF-PROJECT EVALUATION OF FAI PROJECT 21 Also of note is the fact that there has been a decrease in the turnaround time for the receipt of DBS results from NMRL to district and rural health facilities from an average of three months, when DBS testing started, to 3-4 weeks at the time of the evaluation. However, as previously noted, this remains a major challenge, as infant morbidity increases and mothers and infants are lost to follow-up in clinics when test results are delayed. Mother and baby pair follow-up. One of the FAI objectives in ensuring a robust PMTCT program was to strengthen follow-up of HIV-exposed infants and mothers. Health facilities use the ANC and the Exposed Infant Follow-up Register to track those missing scheduled appointments. The use of coding for HIV status of an infant in the recently revised Child Health Card not only assures follow-up of an HIV exposed or infected child within the continuum of care, but is also crucial in reducing loss to follow￾up. Yet, again, loss of clients to follow-up was found to be a challenge across the cascade of PMTCT services. Some form of a follow-up system was observed in most health facilities visited, but clear and systematic tracking and documentation remains a challenge, often exacerbated by the excessive burden of reporting on clinic staff. For further discussion of this issue, see below. Challenges.  Some of the challenges in tracking mother and baby pairs were the mobility of mothers, many of whom migrate to other districts and urban areas after delivery, and the lack of transport to carry out home visits. However, with support from UNICEF and CIFF funding, village health workers are being trained to mobilize communities and encourage mothers to follow treatment protocols. L. Human Resources The human resource situation was found to be one of the most significant constraints impacting PMTCT and other health services. It was mentioned by DNOs, DFPs, and other key informants at all levels of our assessment. Nurses across the country are committed to the success of PMTCT. They showed passion for the intervention and were pleased with successes in reducing the number of HIV infected children and having mothers on ART. But the increasing burden of work on nurses, due to increases in the numbers of vertical programs, removal of user fees, and the hiring freeze, present a daunting mismatch between the supply side and the demand side of health services. Challenges  Due to the GOZ hiring freeze, rural health facilities maintain no more than two nurses per site, despite a steadily increasing demand for PMTCT (and other) services.  A problem universally cited by overstretched staffs is the number of registers into which they are required to enter data, not to mention the number of indicators of which they must keep track. Almost without exception, clinic personnel complain about too many registers, sometimes numbering as many as 14 (see table next page). The lengthy and tedious exercise of ensuring timely completion of registers takes nurses away from patient care and compromises the quality of that care. While data is critical to monitoring program implementation and strategic decision making, this has become an unacceptable burden for nurses.    END-OF-PROJECT EVALUATION OF FAI PROJECT 22 Required registers in facilities providing PMTCT services 1. ANC booking register 8. HIV infant diagnosis clinic register 2. Confidential register (Testing and Counseling) 9. HIV DNA/PCR laboratory request form 3. Laboratory test site register 10. Pre-ART register 4. Maternal PMTCT dispensing register 11. ART register 5. Infant PMTCT dispensing register 12. POC/CD4 register 6. Delivery register 13 PNC register 7. Exposed infant follow-up register 14. EPI register 5.2 SUSTAINABILITY 5.2.1 Background The sustainability of interventions is most often thought of as a financial issue: will a particular program be able to continue, i.e. sustain itself financially, following cessation of external donor support? In fact, while the capacity of a program to continue may be first and foremost a matter of funding, there are usually other considerations to be addressed when determining whether the program can survive. In the view of the evaluation team, sustainability for FAI involves not only the capacity of the program to finance itself in the future, but three other key issues as well: 1. Whether the technical and managerial capacity of the MOHCW has been built to a point where, given adequate resources, it can go it alone. 2. Whether sufficient popular awareness has been created in the population to ensure a permanent demand for PMTCT services; and 3. Whether male involvement in the program and, more broadly, in ensuring the health of the family, has been developed to a point where gender imbalance is no longer an impediment to ensuring good family health. 5.2.2 Findings and Observations A. Technical and Managerial Capacity of the MOHCW As will be noted elsewhere in this report, FAI has supported, and indeed insisted on, MOHCW’s ownership and leadership of the PMTCT program. Building the Ministry’s capacity to lead and sustain the program has involved: developing national PMTCT policies and training manuals; revising the Child Health Card to include PMTCT information; and developing a wide range of training activities, which have established a knowledge base that will remain long after FAI (or subsequent) support is concluded. EGPAF’s secondment of key senior managers to lead the PMTCT program within the MOHCW has ensured that the program has high priority and a seat at the table equal to all programs dealing with family health. Establishment of the PMTCT Partnership Forum has ensured regular review, exchange of views and sharing of experiences among key stakeholders. The addition of DFPs at district level, discussed at length elsewhere in this report, has heightened technical and supervisory capacity to the benefit of health facilities.    END-OF-PROJECT EVALUATION OF FAI PROJECT 23 B. Popular Awareness Until people have been informed and educated about the importance of a particular service to the point where it is routinely expected and valued, it cannot be said to be a permanent part of health care. The evaluation team observed that, because of the efforts described on the part of NGOs, health care workers, Village health Workers (VHWs) and others, PMTCT has evolved from something new to an accepted and expected aspect of maternal and child care, at least on the part of women. With over 90% of the population now not only aware of but well informed about PMTCT, limitations on the utilization of services are caused by staff shortages or problems of access, not lack of demand. A nurse at Mpilo Hospital in Bulawayo asserted to the team that, at this point, 100% of women coming to the hospital for ANC and postpartum care are aware of the PMTCT program and its significance. She spoke of her “passion for PMTCT,” her expectation that a mother-to-child transmission rate of HIV of below 5% “can be achieved,” and only bemoaned the fact that staff turnover and other obstacles to access often made timely provision of services and follow-up difficult. The problem is no longer awareness or demand, and in that sense the sustainability of PMTCT services seems assured. C. Male Involvement Gender imbalance, as reflected in the general lack of male involvement in information gathering and decision making regarding family health in Zimbabwe, can work against the sustainability of the program unless steps to increase such involvement are successful. As discussed elsewhere in this report, male involvement in the PMTCT program, as reflected in the number of men who involve themselves in any significant way in their partners’ pregnancies and visits to clinics for ANC and PMTCT, remains low despite various efforts to increase that number. There are a few cases of districts where as many as 30% of men accompany their wives or partners to clinics for ANC and HIV testing. However, the percentage in most cases is much lower, in the 5% - 10% range at best. In the bustling Edith Opperman Maternity Clinic in Harare, the matron reported that no more than 2% of husbands accompany their wives on clinic visits. This is not to say that men are not aware of the services and their importance but that they are simply not involved. Reasons for male non-involvement may be traditional or cultural, or the result of extended separation. In many parts of the country, men leave for most of the year to seek employment in South Africa or Botswana. Even when they are home, however, taboos against acknowledging a pregnancy in its early stages keep women as well as men away from the clinic until well beyond the critical early months of the pregnancy. Many men do not want to know their own, or their wives’ HIV status. Or they may seek testing apart from their wives and not share the results. Efforts have been made and are being made, through clinic outreach, engagement of community groups, and training of VHWs as advocates, to increase male awareness about the importance of these issues to the health of their families, and to pique their willingness to get involved in a timely manner. The evaluation team suggests that it would be an excellent idea to inventory those approaches that have shown signs of success, so that lessons from those efforts can be applied to achieving greater gender balance in service utilization. Increasing male involvement in issues of family health can only strengthen the demand for high quality services and thus underline the importance of their being sustained. D. Financial Sustainability It is clear that dependence on external financial support for the national application of the PMTCT program will continue for the foreseeable future. Senior MOHCW managers are quite clear in their acceptance of the reality that early reduction or withdrawal of donor funding for structural support, programming and ongoing trainings would place the program in dire straits and compromise the    END-OF-PROJECT EVALUATION OF FAI PROJECT 24 substantial success achieved to date. USAID’s approval of a new, expanded funding package for PMTCT, through largely the same NGO consortium that implemented FAI, further attests to that fact. An example of the impact of financial support withdrawal was observed by the evaluation team in Chitungwiza District. Here, FAI funding was channeled through Zimbabwe AIDS Prevention Project (ZAPP), which was dropped from the new NGO implementing consortium. Since then PMTCT service provision has been significantly challenged in Chitungwiza, notably due to the ending of salary support for the program’s laboratory scientist, a key position in ensuring quality in PMTCT service provision. Funding for Chitungwiza will probably be replaced in some manner in the short run. Nonetheless, it is not too soon to carry on serious dialogue, using the partnership forum (PPF) to do so, about the capacity of the GOZ to assume budgetary support of aspects of the PMTCT program in the future. One place to start could be in developing a strategy for supporting DFPs when their CIFF funding, as presently planned, is ended in 2015. By that time PMTCT will, by all indications, have taken its permanent place as an indispensable element of maternal and child care. Will that status and that importance be justification enough for the Ministry to absorb its costs, perhaps somewhat modified, to ensure access and quality? It is a question worth addressing sooner rather than later. 5.2.3 Strengths and Challenges Strengths  Management and technical capacity to lead and sustain the PMTCT program already evident in the MOHCW, and continuing to grow.  Critical mass of trained cadres of health workers is being achieved.  Awareness of the availability and importance of PMTCT services appears now to be close to universal among women. Challenges  While men may be aware of PMTCT and related services, they are in large measure uninvolved in the ANC, HIV prevention and other maternal and child care services accessed by their wives or partners. Efforts need to be redoubled to learn from the examples of successful male involvement that do exist, as well as to craft a new generation of related messages targeted to men to reduce gender imbalance in utilization of ANC, PMTCT and related services. (The participation of SAFAIDS in the new PMTCT NGO consortium should contribute to this.)  Beginning to strategize about the long-term ability of the MOHCW to assume financial support of the PMTCT program. A one-item-at-a-time approach to this challenge could start with developing plans to absorb the role of DFP into existing district health structures after 2015. EGPAF has pledged to work with the Ministry to develop such a plan.  At a macro level, urging the GOZ to rethink its policy of cancelling clinic user fees (which deprives the Treasury of resources) while at the same time freezing hiring of health personnel (which user fees might have been used to pay for). Staff shortages inevitably affect access and quality in service delivery, just at a time when demand has been created and is increasing. 5.3 COORDINATION AND INTEGRATION 5.3.1 Background The evaluation team looked at how working relationships within the FAI implementing partnership, between partners and the MOHCW, and between the project, the Ministry and other stakeholders, facilitated or hindered the achievement of desired results. It assessed coordination and integration    END-OF-PROJECT EVALUATION OF FAI PROJECT 25 practices of the PMTCT program with different stakeholders before and after FAI intervention, focusing on the following:  Effectiveness of FAI coordination mechanisms throughout the 5-year period from national down to site/health facility level.  Coordination between FAI consortium members.  The role of the DFP, and the coordination of that role with the DHT team at district level.  Integration of PMTCT and pediatric HIV prevention, care and treatment with other services in health facilities, such as family planning (FP), child health, HIV/ AIDS care and support services, TB, and nutrition interventions; and  Integration approaches by FAI in capacity building and training. 5.3.2 Findings and Observations A. MOHCW Ownership and Management of the PMTCT Program The working partnership between FAI and the MOHCW has been strong and transparent. From the evaluation team’s first briefings, USAID and EGPAF asserted that FAI’s role was to build the capacity of the Ministry to implement a national PMTCT program, never to compete with it. Due to good working relations and technical credibility, FAI has been hailed for working within the existing MOHCW structures to influence policy and programming while assuring Ministry ownership thereof. The Ministry had begun implementing the PMTCT program in 1999, before FAI involvement, with Government and donor funding. However, within the MOHCW, the PMTCT and pediatric HIV prevention, treatment and care program is located within the AIDS and TB Unit, which oversees HIV Testing and Counseling, OI services, ARV Treatment, Sexually Transmitted Infections (STI) Management, and TB control. Without a dedicated team, there were challenges of coordination, planning and implementation for the relatively new PMTCT program. With support from FAI, three senior PMTCT staff members were seconded by EGPAF to the Ministry, responsible for overseeing all aspects of implementation, monitoring and evaluation of PMTCT and pediatric HIV prevention, treatment and care services. Coordination was further strengthened by the establishment of the PMTCT PPF by FAI, which has since served as an important mechanism for exchange between different PMTCT stakeholders. It should also be mentioned that the MOHCW gives the USAID/Zimbabwe Mission high marks for its close consultation with Ministry leaders at all points in the development of this and other health-related interventions. While its internal policies do not permit USAID to fund the Ministry directly, the Mission is always extremely careful to share in detail its ideas and strategies for assistance to the health sector, provided through selected NGOs such as EGPAF. B. Implementing the District Focal Person (DFP) Model. In 2010, an infusion of substantial leveraged funding from CIFF led EGPAF to make changes to the original structure of FAI designed to accelerate national adoption of the 2010 WHO PMTCT guidelines and strengthening of the PMTCT program. With the objective of providing a short term infusion of support to overburdened DHTs, EGPAF, in close consultation with the MOHCW, created the role of DFP. Reporting to the DNO, the DFP is responsible solely for supporting the PMTCT program in a given district (whereas the DNO is responsible for all health services), through regular monitoring, supportive supervision and technical assistance to service facilities and providers. Thirty experienced community health nurses were recruited and trained for the DFP role in April 2011, with four more added in early 2012. DFPs sign one-year contracts with the understanding that they will be retained in    END-OF-PROJECT EVALUATION OF FAI PROJECT 26 this role until 2015. Most are responsible for supporting two districts, dividing their time between them in two-week increments. The establishment of the DFP model by EGPAF was carried out in a way that was described by FAI partners (OPHID, ZAPP and Kapnek Trust) as ‘confusing’ to original partner roles, and something on which they were inadequately consulted. DFPs were not only directly ‘seconded’ to the districts by EGPAF, but paid salaries that are higher than Ministry counterparts and provided with vehicles as well. Partners saw the decision as being at odds with the original FAI structure, namely that EGPAF, as prime contractor, was responsible for advocacy, capacity building and coordination with MOHCW at national level, while the partners were responsible for implementing the program at district and site levels. This all led to frayed relationships between partners and EGPAF, as it was seen as if EGPAF was competing with its partners at the field level. To some extent, those tensions remain. However, the evaluation team found that DFPs have been accepted and well-coordinated with DHTs. DNOs and other stakeholders we interviewed at district level expressed appreciation for the contribution of the new cadre to the PMTCT program. DFPs have reduced the burden on DNOs while improving support supervision of service sites, not least because they can provide transport, a major challenge faced by DHTs. In general, the evaluation team observed that DFPs have made themselves indispensable partners to district health teams in achieving scale-up of PMTCT services so as to achieve national coverage and roll- out of the 2010 WHO PMTCT guidelines. DFPs played a key role in the scale-up process, functioning as a catalyst for program expansion and optimization. But, as has already been discussed, it is a role that is not financially sustainable for the long haul unless it can be absorbed by the MOHCW budget, an unlikely prospect for some years to come. C. Achievements in Integration According to PEPFAR guidance on PMTCT, full integration of PMTCT with pediatric HIV and MNCH services has the potential for increased synergy and efficiency across vertical programs aimed at the same population of pregnant women and young children. Integration is defined as the organization, coordination and management of multiple activities and resources to ensure the delivery of more efficient and coherent services in relation to cost, output, impact, and user acceptability.14 The evaluation assessed FAI’s success in achieving service integration in terms of the extent to which, at service points, PMTCT has been successfully linked with other services offered, such as family planning, TB treatment, and nutrition. Early on, the project focused on prongs 3 and 4 of the national PMTCT program approach, but more recently has placed greater emphasis on prongs 1 and 2, especially prevention of unwanted pregnancies in HIV+ women through use of contraception. Mothers are routinely counseled on dual protection, and the team found that a range of contraceptive methods was readily available at all sites. But there is still need to strengthen the FP component of integrated services. Roughly 25% of women accessing PMTCT services reportedly become pregnant within two years of their last pregnancy, despite counseling and ready access to FP services and methods. Integration efforts by FAI were also noted in the promotion of proper infant and young child feeding (IYCF) practices, which are now part of the PMTCT training package. Exclusive breast feeding (EBF) messaging was strong in health facilities visited by the evaluation team. FGDs confirmed that these messages were being promoted, but also that they have been mixed, some promoting EBF for six                                                              14 PEPFAR guidance on PMCTCT Integration into MNCH services.    END-OF-PROJECT EVALUATION OF FAI PROJECT 27 months, others for a longer period. This has caused confusion, for example, over the timing of supplemental feeding. Cultural factors and fear of stigma also contribute to many mothers not practicing EBF at all. Guidance on EBF and IYCF needs to be clarified and simplified. Levels of integration clearly differ according to the size of a health facility. It is generally a given in rural clinics that, where there are usually no more than two nurses to handle all services, providing them as an integrated whole is essential. The higher one goes in the system, however, the less likely services are to be offered in an integrated manner. In hospitals, for example, due to space limitations, different departments are often widely dispersed. Patients must wait in different queues for different services, often for long periods of time, a sure recipe for frustration. Efforts to improve on this situation have employed different models to suit available physical infrastructure and human resources. In Bonda Mission and Mpilo Hospital (Bulawayo), a ‘one stop shop’ approach is utilized, whereby clients are provided with integrated service in one department. In Chitungwiza Hospital, due to shortage of space, the OI, FP and ANC departments are not located in one place, but they have agreed that ANC clients needing, for example, OI services must be prioritized so they do not have to wait for appointments. Another significant contribution of FAI/EGPAF to service integration on a national scale was the support to MOHCW for revising the national Child Heath Card so as to have PMTCT information included with all other patient data. After finalizing revisions, FAI also funded the printing of cards for distribution to health facilities across the country. 500,000 cards were delivered in October 2012. 5.3.3 Strengths and Challenges Strengths  FAI coordination with MOHCW has been comprehensive, including close collaboration on the original design of FAI, which has ensured Ministry ownership of the PMTCT program.  Support for key personnel has given PMTCT a strong “seat at the table” with other services. The PPF has provided a vibrant structure for coordination of PMTCT and pediatric HIV interventions among stakeholders.  FAI partners have built on each-other’s strengths to achieve good coordination and learning from each other to the advantage of the PMTCT program, especially at field level. While each has a slightly different approach (e.g., OPHID works through coordinators at district level, while Kapnek’s supervisors are Harare-based), quarterly review meetings enable regular sharing of lessons learned.  EGPAF technical support to FAI partners has played an important role in developing their capacity in operations research (OR) and documentation of lessons learned. They have made presentations at international and regional meetings, increasing partner visibility and credibility.  At district level, FAI partners established strong partnerships with the District Health Executive (DHE), especially DNOs. Coordination occurs through quarterly review meetings where findings from site supervision are discussed and improvements agreed upon.  Establishment of the role of DFP contributed to giving the PMTCT program lasting visibility at district level by providing an extra cadre to reduce the burden on overworked DNOs. Coordination with the DHT was a challenge at first, due to the manner in which DFPs were brought on board. But results from the field showed that DFPs are much appreciated.  The FAI consortium successfully advocated for, and funded, the revision and production of the new Child Health Card. It was also a contributor to development of the MOHCW’s “Integrated Training Course for HIV and AIDS,” intended to replace the plethora of training courses for    END-OF-PROJECT EVALUATION OF FAI PROJECT 28 different technical areas with one shorter, more focused, training vehicle built on the concept of integration. Challenges  Absorbing costs that are now donor dependent. The most striking is the cost of salaries and other support for DFPs, presently scheduled to run until 2015. Another is the cost of district level coordinating meetings. In this regard one might consider the Tanzanian example of a district￾centered approach where meetings are held without ‘external’ support.  Expanding true service integration from rural health centers, where it occurs out of necessity because staffs are small and patients often must travel long distances, to larger facilities and hospitals where family health services are often separated. To be truly effective, PMTCT needs to be part of an integrated whole.  Seeming imbalance, or lack of coordination, of donor support at district level and below is a concern. Some districts, or portions of districts, are blessed with donor funding for renovation or refurbishment of facilities. Others, often in the same area, have no such advantages. Many sites visited by the evaluation team are sorely in need of such basic support. Example #1: In Tsholotsho District in Matabeleland North, MSF has funded clinic renovations throughout the district. Meanwhile, in nearby Lupane District, the Jotsholo Rural Health Center has been waiting for a new roof and other essential repairs for 7 years, and the community has only recently been able to scrape together enough funding to build an appropriate clinic latrine. Example #2: The Princess Margaret Rose Clinic in Bulawayo, an exceptionally busy urban clinic that is a primary service provider for the municipality, is in desperate need of a new floor and furnishings, with no funding in sight. 5.4 LEVERAGING RESOURCES 5.4.1 Background Since 2001, EGPAF has been supporting “comprehensive prevention of mother-to-child transmission of HIV (PMTCT) services towards the Zimbabwean Ministry of Health and Child Welfare’s goal to promote delivery of comprehensive, high-quality PMTCT services that are integrated and linked to treatment, care, and support.”15 This support has primarily been funded by USAID with funds from the President’s Emergency Plan for AIDS Relief, or PEPFAR. USAID’s award of $22.3 million in PEPFAR funds to EGPAF for the 2007-2012 Family AIDS Initiatives program was critical to scaling up earlier efforts, and to pursuing the goal of making the PMTCT program truly national in scope. What had not been anticipated when FAI was launched was that this funding would be the fulcrum point for another major award to EGPAF for PMTCT - $60 million over five years – from CIFF, of the United Kingdom. This support, which likely would not have been made available without the commitment USAID had already made, more than doubled resources available for the implementation of FAI. 5.4.2 Findings and Observations The CIFF award was a “game changer” in several ways. It enabled EGPAF, in close consultation with the MOHCW, to implement a DFP model to strengthen PMTCT service delivery. As described earlier, this involved recruiting and training experienced community health nurses and assigning them to work closely with DNOs, providing on-going supervision and mentoring staffs of district health facilities to                                                              15 From 2011 Family AIDS Initiative Annual Report    END-OF-PROJECT EVALUATION OF FAI PROJECT 29 specifically make possible the continued strengthening of PMTCT services. In most cases, each DFP has responsibility for two districts, and every indication is that they have been a welcome addition to overworked district health personnel. Collaboration has been strong, and the fact that, under the CIFF grant, they were provided with vehicles has made them doubly welcome. The fact that DFPs are paid slightly more than their DHT counterparts does not seem to have been a major issue. In part, this may be due to the fact that they do not have the same job security. Their employment by EGPAF as DFPs is confirmed only until September 2015, and those that left the Ministry for this job did not retain civil service status that would guarantee rehiring. CIFF funds have also been allocated to the purchase of over 150 POC CD4 count machines, which have been distributed to high volume health facilities to provide support for the training and outreach activities of village health workers and supporting a two-district pilot project (Tsholotsho, in Matebeleland North, and Beitbridge in Mat. South) to test the installation of an electronic database which might one day replace traditional and time consuming methods of clinic record keeping. Tensions caused by the CIFF award speak to the need for close consultation among partners regarding roles and relationships under the new program, for which OPHID is prime contractor. But there is no gainsaying the fact that the CIFF award enabled FAI to take important steps forward faster than previously planned. It is also clear that no steps were taken without the complete agreement and approval of the MOHCW. Other instances where leveraged funds have benefited FAI include DFID’s support for pediatric HIV care and treatment. It also helped with interim funding of FAI at a time when PEPFAR funds were delayed. UNFPA has supported integration of PMTCT in MNCH. The Clinton HIV/AIDS Initiative (CHAI) provided research funds earlier in the life of FAI. OPHID and the Kapnek Trust have both benefited from UNICEF/CIDA grants. At the other end of the spectrum, a local NGO, known as Painted Dog Conservation, has provided support for clinic renovations and construction of expectant mothers’ shelters in Hwange District. But CIFF is the most significant case of leveraged funding impacting the structure and pace of a major health initiative. 5.4.3 Strengths and Challenges Strengths  Leveraged resources have played a major role in FAI’s success. They have demonstrated the timeliness of USAID’s original funding award, the importance of close and continued consultation with the MOHCW, the advantages of diversifying a funding base, and the positive energy new resources can bring to an enterprise. Challenges  One challenge in accepting external funding is to be sure it does not upset collaborative arrangements, or cause resentments, to the detriment of a project. CIFF funding had the potential to do so, but its positive attributes appear to have outweighed any temporary damage done.  The other, of course, is the future. Once external funding has been used to develop a certain level of capacity, can it be maintained unassisted? A specific test case will be the DFPs. EGPAF’s intention, as agreed with the MOHCW, is that when their role is ended in 2015 the work will be absorbed by the Ministry. But how that will work if the GOZ hiring freeze and budgetary limitations are still in effect is hard to know. What is clear is that planning for “DFP phase-out”    END-OF-PROJECT EVALUATION OF FAI PROJECT 30 should begin now. That is EGPAF’s intention, as outlined in its documentation of the DFP model.16 6.0KEY LESSONS LEARNED  A close, responsive, transparent working relationship between donors and the Ministry of Health, at all levels, is the sine qua non for ensuring Ministry leadership and ownership of a health program as important and far reaching as FAI.  Despite high knowledge levels about PMTCT and the benefits of early ANC booking, in practice the majority of women are continuing to book late, in the second or third trimester of pregnancy. (Example of Chitungwiza, where mothers book as late as 36 weeks).  While some nurses in districts surveyed are initiating ART, many are uncomfortable doing so, even when they have had training. Pediatric initiation remains doctor-centered, meaning mothers and babies must be referred to initiating centers, often at long distances, and may be lost to follow-up.  Most rural health centers still average only two nurses per center, despite the increase in volume of work, not to mention frequent absences for trainings.  The workload of nurses, especially in rural clinics, is made extreme by the burden of having to make detailed entries in an enormous number of registers, seriously compromising the time available for quality patient care.  New ideas and approaches, as well as replication of successful models, are needed to increase male involvement in their wives’ or partners’ ANC bookings, pregnancies, testings, post natal care, etc.  User fees are still a barrier to access to services in some districts, especially in municipalities. Yet, ironically, collecting reasonable user fees might have avoided the need to freeze hiring.  Integration of services works in every way to the advantage of patients, health service providers, and health facilities, but there is much left to be done to make it a universal reality.  Young pregnant women (teenagers) have special needs that are not presently addressed by health facilities or providers. 7.0RECOMMENDATIONS  Priority attention should be given to simplifying indicators and substantially reducing the burden of registers that health workers are obliged to fill out, taking precious time away from providing quality health care to patients. It is thought that experiences in solving this problem in neighboring countries (Malawi, Tanzania) might be instructive.  There is a pressing need to increase coverage of ART initiating sites (by adding more sites) and availability of POC CD4 testing (by equipping more health facilities with POC machines), and at the same time to strengthen mother-baby follow-up strategies to reduce loss to follow-up across the PMTCT cascade.  New approaches for training, encouraging, and mentoring nurses in initiation of ART need to be developed. One might be a “roving doctor,” for every 2-3 districts, dedicated to teaching and mentoring nurses in ART initiation.                                                              16 See “Implementing a District Focal Person Model” Program Brief, July 2012    END-OF-PROJECT EVALUATION OF FAI PROJECT 31  Integration of PMTCT with other services, at all service levels, must be a continuing priority, for reasons bearing on both economics and service quality.  With regard to training, PMTCT content in pre-service curricula for nurses needs to be strengthened. Time that health staff are away from their facilities for in-service training should be minimized by combining curricula. A key resource is the “Integrated Training Course for HIV and AIDS”, recently formalized by the MOHCW with WHO assistance.  The latter idea notwithstanding, it is important to note that the PMTCT program in Zimbabwe is nurse driven, and training of MDs in PMTCT should be scaled up to make them more informed supporters. For example, few doctors have been trained in IMAI/IMPAC. Also, both nurses and doctors need training in cultural and psychological factors influencing health seeking behaviors.  Conduct an analysis of models and practices that have succeeded at district and community levels in strengthening male involvement in ANC, PMTCT, etc., and that might be utilized more broadly in achieving greater gender balance in the program.  The DFP model has proven to be very effective in strengthening PMTCT services across the country. Discussion of whether and (if so) how this role will be absorbed into MOHCW structures and budgets when external funding of DFPs ends in 2015 should not be delayed. Indeed, this single decision will, as much as anything, indicate the extent to which FAI will have set the stage, in the words of its long-term goal, for the MOHCW to support continued delivery of PMTCT services beyond the life of the project.  A targeted approach to caring for young, pregnant women (14-18), one that recognizes that their needs and concerns are different from more mature women, should be developed in line with Prong #2 of PMTCT guidelines.  Explore ways to address the seeming lack of coordination of donor support at district level, and the imbalance between those who do and do not receive support for such things as clinic renovations and furnishings (see examples cited above). Hopefully HSS funds in the UNICEF￾managed Health Transition Fund could be used to support poorer health facilities. Another option might be to engage Ward Health Committees and/or community leaders by offering matching funds for repairs and reconstruction if they can generate at least modest resources locally.  As numbers of pregnant women and infants initiated on ART increase, the national PMTCT program must collect and analyze adherence data as part of routine monitoring and evaluation. This data is essential for quantifying long term outcomes of PMTCT interventions.    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEXES    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEX 1: LIST OF PEOPLE INTERVIEWED    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 Key Informants, Field visits Date Name Position Institution/Organisation 22/10/2012 Ms T. Sibanda PMTCT Trainer Mpilo Central Hospital 22/10/2012 Gift Masawi Nurse-in￾charge/PMTCT Mpilo Central Hospital 22/10/2012 Sr. Sibanda Acting CNO Bulawayo Municipality 22/10/2012 Sipiwe Matatu Nurse Margaret Rose Clinic, Bulawayo 22/10/2012 Sithembile Ncube Nurse Margaret Rose Clinic, Bulawayo 22/10/2012 Ms M. Tshuma PNO Mat North Province 22/10/2012 Mrs Chikukwa PNO Manicaland 22/10/2012 A Gwasira DNO Mutasa district 22/10/2012 Ms Mhandu Community Nurse Mutasa District 22/10/2012 Mrs Ushamba DFP Mutasa/Nyanga districts 22/10/2012 Mrs Nyabunze Matron Bonda Mission Hospital 22/10/2012 Dr Chamunyonga Hospital doctor Bonda Mission hospital 22/10/2012 R Chadzingwa Sister-in-Charge Mutasa Clinic 22/10/2012 H Sauriri PCN Mutasa Clinic 23/10/2012 Mrs Guwira DNO Mutare district 23/10/2012 ET Nyamadzawo Community Nurse Mutare district 23/10/2012 D Zenda DFP Mutare/Buhera 23/10/2012 Sister Mupazi Nurse-in-charge – FCH Mutare Provincial Hospital 23/10/2012 B Marowa PCN Munyarari Clinic 23/10/2012 P Garira Nurse Munyarari clinic 23/10/2012 Ms I Ndlovu DNO Mangwe District 23/10/2012 Dr I Sibanda DMO Mangwe District 23/10/2012 Mr Primrose Sibanda Primary Counsellor Ndiweni Clinic/Bulilima District 23/10/2012 1 Female & 1 Male PMTCT Patients Ndiweni Clinic/Bulilima District 24/10/2012 Ms P Khumalo Acting DNO Esigodini District Hospital 24/10/2012 Ms S Dube Nurse-In-Charge FCH Esigodini Clinic 24/10/2012 B Masilela District Focal Person Esigodini & Insiza Districts 24/10/2012 Ms N Nyilika PCN Umzingwane Clinic 24/10/2012 Ms R Dube PCN Umzingwane Clinic 24/10/2012 Ms A Dube Primary Counsellor Umzingwane Clinic 24/10/2012 Dr Chitsarimatanga DMO Chimanimani district 24/10/2012 Sr Mbano Sister-in-Charge Chakohwa clinic 24/10/2012 Sr Mangezi Nurse Chakohwa clinic    END-OF-PROJECT EVALUATION OF FAI PROJECT 2 24/10/2012 Dr M Mhute DMO Chipinge district 24/10/2012 Dr Kuwengwa Doctor Chipinge district Hospital 24/10/2012 F Masango DFP Chimanimani/ Chipinge 24/10/2012 Mr Dube Community Nurse Chipinge district 25/10/2012 Dr. Mudyiradima PMD Masvingo 25/10/2012 Sifiso Ncube DFP Lupane, Umsuza Districts 25/10/2012 Simangele Moyo PCN Jotsholo Rural Health Center 25/10/2012 Agness Sibanda PC Counselor Jotsholo Rural Health Center 25/10/2012 T . Nyazika Sister-in-Charge Jerera Clinic 25/10/2012 Mrs Madyauta DNO Zaka district 26/10/2012 Mr Bhasera DNO Chiredzi District 26/10/2012 Mr Musaka DFP Chiredzi District 26/10/2012 Latelang Ndlovu DFP Hwange, Tsholosho 26/10/2012 2 Females VHW & Care Giver Lupote Clinic – Hwange 26/10/2012 T Jambwaregota Sister-in-Charge Muteyo clinic 26/10/2012 Mr Gwisai Nurse Mkwasine Clinic 29/10/2012 Johnson Mazizanhanga DFP Mutoko, Mashonaland East 29/10/2012 F Marufu PNO Mashonaland West Province 29/10/2012 R T Banda DNO Kadoma -Sanyati district 29/10/2012 A Denhere Community Nurse Kadoma – Sanyati District 29/10/2012 Theresa Ndoro OPHID Coordinator Mashonaland East, Central 30/10/2012 Mukaratikwa PMTCT Nurse Chitungwiza General Hospital 30/10/2012 Sr Katsande Sister In Charge Seke South Clinic 30/10/2012 Sr Paradza Sister in Charge Seke North Clinic 30/10/2012 Sr. Nleya Nurse-in-Charge Edith Opperman Clinic, Harare    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 Focus Group Discussions Date Target Area Female Male 22/10/2012 Prenatal and Postnatal women Mpilo Hospital 10 0 22/10/2012 Prenatal women Mpilo Hospital 7 0 22/10/2012 Prenatal women Princess Margaret Rose Clinic 8 0 22/10/2012 Prenatal women Bonda Mission Hospital 8 0 23/10/2012 Prenatal women Plumtree District Hospital 15 0 23/10/2012 Prenatal women Munyarari Clinic 10 0 23/10/2012 Village Health workers Ndiweni Rural Health Centre 9 1 24/10/2012 Village Health Workers and Community Esigodini Ward 17 7 2 24/10/2012 Village Health workers Chakohwa clinic 3 0 24/10/2012 Village Health Workers Umzingwane Clinic 6 0 24/10/2012 Village Heads Umzingwane Clinic 4 2 24/10/2012 Post Natal Women Umzingwane Clinic 11 0 25/10/2012 Pre natal women St. Lukes Hospital 10 0 26/10/2012 Mixed group- prenatal, post natal and non-pregnant women Hwange Colliery No. 2 Clinic 19 0 26/10/2012 Prenatal women Lupote Clinic –Hwange 7 0 26/10/2012 Pre and Post Natal women and male partners Muteyo Clinic 10 2 30/10/2012 Post Natal Mothers Edith Opperman Clinic, Harare 8 0 30/10/2012 Pre natal Mothers Seke North Clinic 10 0    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 Contacts, Harare Ministry of Health and Child Welfare Dr. Angela Mushavi, National PMTCT & Pediatric HIV Care & Treatment Coordinator Ancikaria Chigumira, Deputy Director, Nutrition Services Dr. Tsitsi Apollo, National ART Program Manager, AIDS and TB Unit Elizabeth Glaser Pediatric AIDS Foundation Dr. Agnes Mahomva, Country Director Dr. Batsirai Makunike-Chikwinya, Technical Director J.F. Kapnek Charitable Trust Caroline Marangwanda, Deputy Director Maria Savanhu, Site Support Coordinator OPHID Barbara Engelsmann, Director Diana Patel, Deputy Director ZAPP Winifred Chandisarewa, Deputy Programme Director Zimbabwe National Family Planning Council Dr. M. Murwira, Executive Director CDC Celia Woodfill, Associate Director for Science, Zimbabwe DFID Peter Taylor, Programme Manager & Head of Corporate Services Barbara Arab, Deputy UNICEF Beula Senzanje, HIV/AIDS Specialist USAID Matthews Maruva, M&E Specialist Dr. Ruth Bulaya-Tembo, Program Management Specialist Peter Halpert, Director, Health Hamfrey Sanhokwe, SI Specialist Nikki Enersen, Program Office    END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEX 2: EVALUATION TOOLS END-OF-PROJECT EVALUATION OF FAI PROJECT 1 FAI EVALUATION INTERVIEW GUIDE Questions are keyed to specific evaluation questions incorporated in the SOW for the end-of-project evaluation of USAID/Zimbabwe’s Family AIDS Initiatives (FAI) program. 1.) Overall program performance (Questions to be asked of implementing partners, Provincial and District health personnel, service providers and (through FGDs) community groups. This will be supplemented by service statistics collected from sites visited by the evaluation team. Quality of services  What services are provided by the PMTCT program? Is the PMTCT program meeting its purpose, namely to provide greater access to quality services for pregnant women and infants? Why do you think it is, or is not, fulfilling this purpose?  How effectively have services and service providers reached out to, and incorporated the concerns of, men?  Has the quality and quantity of medical supplies been adequate to meet the needs of patients for PMTCT?  What are the safe delivery methods that you practice to reduce MTCT at this facility?( Please note: Vaginal Examination, Episiotomy, forceps delivery, Artificial Rupture of membranes, Caesarian section)  What guidelines are available? How have you used them? (ask to see the guidelines)  Quality of supervision (Probe: regularity, who supervises, quality)  What community outreach services do you conduct( community mobilization) Access to services  How well is the availability of PMTCT services known in the community? Are there other services you would have wanted to see provided? Explain.  Is everyone who should have access to PMTCT services and counseling, including men, actually receiving them? If not, why not? What are the challenges?  What services are offered for youth? Are these adequate? Training  What training in PMTCT have you received?  How are gender issues incorporated in training provided?  Is the training sufficient in terms of quality of content, length of training, methodology of training? END-OF-PROJECT EVALUATION OF FAI PROJECT 2  Have you benefited from refresher courses and/or mentoring? How often and on what issues? 2.) Sustainability of PMTCT program innovations (To be asked of MOHCW units, especially PMTCT, MCH, RH; Provincial and District health offices; international NGOs.)  Which components of the national PMTCT program have shown the most improvement or strengthening (at national, provincial or district level) as a result of FAI support?  What is the evidence for this improvement, whether generally or in terms of indicators? Are there significant regional differences, and what are the suggested reasons for such differences?  Which processes and products developed through FAI have been most successfully institutionalized within the MOHCW?  Have there been significant miss-steps, either nationally or regionally, in the institutionalization of such processes and products?  To what extent have initiatives to mainstream issues of gender in program structure and implementation been made permanent?  At this point in the life of the FAI program, can the MOHCW demonstrate ownership and sustainability of any particular activities launched under FAI?  What are the principle challenges to continuation of FAI program innovations, in terms of service provision to intended beneficiaries, after the program is ended?  What training resources have been developed as the result of FAI support? Will they continue to provide training following this support? What are the remaining gaps in terms of PMTCT training and mentoring? 3.) Coordination and integration (To be asked of stakeholders at national level, MOHCW units, Provincial and District health offices; partners and sub-partners.)  What was the nature of relationships (networking, coordinating, collaborating or partnership) between different FAI stakeholders? Please describe.  What were the major challenges noted in terms of coordination, networking, cooperating and partnership during the implementation of FAI?  Has there been any major change of roles of partners during the FAI implementation period? If yes, how have they changed, and what are the effects of these changing roles on coordination, networking and partnership? END-OF-PROJECT EVALUATION OF FAI PROJECT 3  How effective has been the role of District Focal Persons in the coordination and maximization of PMTCT services?  How has learning and sharing of information, best practices and research occurred among partners and relevant actors for wider learning, informing policy and program improvement?  How have FAI program partners and stakeholders responded to the growing emphasis by MOHCW on integration and provision of comprehensive care within primary health care services? What programs are being integrated with PMTCT? (Probe family planning integration)  Which partners do you work closely with? List partner, its role and the value it brings to the partnership. Any suggestions on alternative roles?  How have you involved the community in the PMTCT program( Community leaders, chiefs, village health workers, lay community conselors) 4.) Leveraging resources (To be asked of implementing partners.)  What share of the overall PMTCT budget was funded from leveraged resources?  What linkages does the PMTCT program have with other projects: a.) within the organization; b.) at national, provincial and district levels?  Do you believe your organization adequately leveraged external resources during the PMTCT program?  If you could change the manner in which resources are allocated across programs, how would you do it? END-OF-PROJECT EVALUATION OF FAI PROJECT 4 Guide for Focus Group Discussions with Community Health Workers and others Feeding Practices How do women on PMTCT generally feed their babies...why is it so? Attitudes and stigma associated with no breastfeeding and HIV Male Participation How are men responding to the PMTCT programme? What made the response positive, What made the response nor so good? What can be done to improve problems/challenges identified? Access and quality How do you see the quality of services offered at the clinic? Do women, men and children pay for services? Explain What services are provided in the PMTCT programme? Are there other services you would have wanted to see provided? Explain. Is everyone who should be accessing PMTCT services actually accessing them? If not why not? What are the challenges? What services are offered for youth? Are these adequate? How common are home deliveries? What are some of the reasons? What is the role of traditional birth attendants? END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEX 3: EVALUATION SCHEDULE ◄ Aug 2012 ~ October ~ Oct 2012 ► Sun Mon Tue Wed Thu Fri Sat 26 27 28 29 30 31 1 2 3 4 5 6 7 8 Phone Call w/USAID Review Materials Evaluation Design, Work Plan & Timeline Preparation Home 9 Review Materials Evaluation Design, Work Plan & Timeline Preparation Home 10 Review Materials Evaluation Design, Work Plan & Timeline Preparation Home 11 Review Materials Evaluation Design, Work Plan & Timeline Preparation Home 12 Submit Draft Evaluation Design, Work Plan and Timeline and Conflict of Interest Statement Review Materials Home 13 Team Leader Travels to Zimbabwe 14 Team Leader Arrives in Zimbabwe 15 Oral Presentation In￾Briefing with USAID Harare 16 Meetings & Interviews with Stakeholders Work Plan & Evaluation Design Preparation Harare 17 Meetings & Interviews with Stakeholders Work Plan & Evaluation Design Preparation Harare 18 Meetings & Interviews with Stakeholders and Work Plan & Evaluation Design Preparation Harare 19 Submit Final Work Plan & Evaluation Design Meetings & Interviews with Stakeholders and Final Preparations for Field Visits Harare 20 Meetings & Interviews with Stakeholders Team Meeting for Final Preparation (continued) Harare 21 Travel 22 Meetings & Interviews with Stakeholders Team 1: Mutasa District Team 2: Bulawayo Urban District: 23 Meetings & Interviews with Stakeholders Team 1: to Mature District Team 2: Plumtree District: 24 Meetings & Interviews with Stakeholders Team 1: Chiminge and Chimanimani District Team 2: Umzingwane District 25 Meetings & Interviews with Stakeholders Team 1: Zaka District: Team 2: Lupane District: 26 Meetings & Interviews with Stakeholders Team 1: Chiredzi Team 2: Hwange District 27 Both teams travel back to Harare ◄ Sept 2012 ~ October-November ~ Nov 2012 ► Sun Mon Tue Wed Thu Fri Sat 28 Team 1: Night in Harare Team 2: Travel to Shamva District 29 Meetings & Interviews with Stakeholders Team 1: Mutoko District Team 2: Shamva District 30 Meetings & Interviews with Stakeholders Team 1: Chitungwiza Team 2: Harare District 31 Team reviews findings and prepares for out-briefing with USAID Harare 1 Prepare for out-briefing with USAID Harare 2 Oral Presentation Out-briefing with USAID Team Meeting for Report Preparation Harare 3 Team Leader travels home 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Prepare Draft Report 22 Prepare Draft Report 23 Prepare Draft Report 24 25 26 Prepare Draft Report 27 Prepare Draft Report 28 Prepare Draft Report 29 Prepare Draft Report 30 Submit Draft Evaluation Report to USAID 1 ◄ Oct 2012 ~ December ~ Dec 2012 ► Sun Mon Tue Wed Thu Fri Sat 2 3 4 5 6 7 8 9 10 11 12 13 Integrating Comments 14 Integrating Comments 15 16 17 Integrating Comments 18 Submit Final Report to USAID 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEX 4: BIBLIOGRAPHY 1. EGPAF – Zimbabwe FAI Program, RFA 690-07-005, April 30, 2007 2. EGPAF/USAID Cooperative Agreement #690-A-00-07-00026-00, October 1, 2007 3. Modification of Assistance to this award (undated) 4. EGPAF/FAI Annual Reports, FY 2008; FY 2009; FY 2010 5. FAI Quarterly Progress Reports, Oct.-Dec. 2011; Jan.-March, 2012; April-June, 2012; July-Sept., 2012 (in draft); and earlier such reports 6. Detailed EGPAF/FAI Work Plans, Project Year 3; Year 4; Year 5 7. Zimbabwe PMTCT and Pediatric HIV Prevention National Plan, 2006-2010 8. Zimbabwe National HIV & AIDS Strategic Plan, 2011-2015 9. Zimbabwe: PEPFAR PMTCT Acceleration Plan/USAID, FY 2012 10. MOHCW-Prevention of Mother-to-Child Transmission of HIV Annual Report, 2009 11. USAID-Zimbabwe, Health System Assessment, 2010 12. Health Transition Fund, A Multi-donor Pooled Fund for Health in Zimbabwe, Oct./2011 13. EGPAF Program Brief: Implementing a District Focal Person Model to Strengthen PMTCT Service Delivery: Early Lessons, July, 2012 14. Mid-term Evaluation, Family AIDS Initiative, June 2010 END-OF-PROJECT EVALUATION OF FAI PROJECT 1 ANNEX 5: SCOPE OF WORK USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 4 of 34 SECTION B – SUPPLIES OR SERVICES AND PRICE/COSTS B.1 PURPOSE The purpose of this contract is to conduct an End of Project Evaluation of USAID’s Prevention of Mother-to-Chid Trainsmission of HIV project in Zimbabwe as described in detail in Section C of this Request for Task Order Proposal (RFTOP). B.2 CONTRACT TYPE This is a Firm Fixed Price (FFP) task order. For the consideration set forth in the task order, the Contractor shall provide the deliverables described in Section F and comply with all contract requirements. B.3 CONTRACT PRICE The firm, fixed price is $_________. B.4 PAYMENT The paying office is USAID/Zimbabwe as indicated in section G.4; payment processing shall be in accordance with the payment schedule in section G.4. B.5 OTHER RFTOP INFORMATION The final statement of work for the task order that will result from this RFTOP will be incorporated at the time of award and shall be based on the proposal by the successful offeror. B.6 PAYMENT SCHEDULE Payment of the fixed price shall be made on the schedule specified below: Deliverable Percent of Fixed Price 1 through 2 30% 3 through 5 30% 6 through 8 40% Payment of the amounts specified above are subject to approval by and acceptance of the deliverables by the COR. END OF SECTION B USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 5 of 34 SECTION C – DESCRIPTION/STATEMENT OF WORK C.1 OBJECTIVE AND GENERAL DESCRIPTION This evaluation is intended for both accountability and learning purposes and will generate knowledge about the magnitude and determinants of project performance. This evaluation will inform future USAID efforts in design and implementation of more effective, efficient and sustainable country owned programs through an informed application of GHI principles. The evaluation will attempt to ascertain the extent to which the project managed to develop capacity within MOHCW and non-governmental organisations to support continued delivery of PMTCT services beyond the life of the project and how these efforts be improved upon. The evaluation will provide important lessons on how to improve sustainability, effectiveness and promote country ownership through; institutionalizing project activities within the MOHCW; improved networking, coordination, cooperation, collaboration and partnerships; leveraging external resources; and addressing critical gender issues. The evaluation findings will be shared with the implementing organisation and its sub-partners, the MOHCW and other relevant national stakeholders. Within USAID, the evaluation will be shared Mission wide and with USAID Washington. The final evaluation report will be posted onto the USAID Development Exchange Clearing House website. The evaluation should conform to the new USAID Evaluation policy ensuring sound methodological design, independence and objectivity of evaluators, and high quality documentation of findings. The Family AIDS Initiatives (FAI) Project, Cooperative Agreement Number 690-A-00-07- 00026-00 is implemented by EGPAF and its three implementing partners – the J F Kapnek Trust (Kapnek), Organization for Public Health Interventions and Development (OPHID) and Zimbabwe AIDS Prevention Project (ZAPP). The bilateral cooperative agreement was initiated in October of 2007 as follow-on to the Call-to-Action project, a centrally funded PMTCT mechanism that was implemented by the same consortium of organizations in-country. The program goal is to ensure delivery of comprehensive, high quality PMTCT services that are integrated with treatment, care and support of families including children living with HIV and AIDS. The program strategic objectives are to:  Support the expansion and provision of quality PMTCT care and treatment services for children and families affected by HIV/AIDS.  Advance research that increases access to and uptake of high quality integrated services of prevention, care and treatment for HIV/AIDS in Zimbabwe.  Advance the FAI consortium leadership role in influencing public health policy and serve as a national advocate to seek the eradication of pediatric HIV/AIDS.  Enhance the FAI partnership’s capacity to operate in an effective, efficient, accountable and responsive manner. The program provides technical support to the national PMTCT program for management, coordination and development of policy and guidelines, and implementation support for service delivery at district and facility level. PMTCT service delivery support under the project is now USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 6 of 34 covering all districts of Zimbabwe. Additional support includes operational research aimed at improving program delivery and informing national policy direction. Advocacy activities are aimed at ensuring that national policies continue to support program development in line with international standards and that the program reaches intended beneficiaries. Available background documents for the evaluator’s use include:  The cooperative agreement document between USAID and EGPAF  FAI project quarterly progress reports  PEPFAR semi-annual reports  USAID portfolio review documents  FAI project annual work plans and reports  The FAI Project/EGPAF strategic plan  PEPFAR annual country operational planning documents  PEPFAR annual reports  The FAI mid-term evaluation report  Zimbabwe national PMTCT program annual reports  The National PMTCT and Pediatric ART Strategic Plan (2006-2010)  The Zimbabwe National HIV and AIDS Strategic Plan (I&II)  The PEPFAR Zimbabwe PMTCT Acceleration Plan C.2 STATEMENT OF WORK USAID Zimbabwe is looking for a suitably qualified contractor to perform an end of project evaluation of a USAID supported Prevention of Mother to Child Transmission (PMTCT) project in Zimbabwe over a period of four weeks. The five-year project was initiated in September 2007 and will end in September 2012. USAID anticipates that the contractor will conduct the performance evaluation in September 2012. C.3 OVERALL RESULTS AND INDICATORS This section sets forth results (outcomes of contactor’s performance) requirements, and performance standards (minimum standards that the contractor must meet) that must be met to USAID’s satisfaction. The Final Evaluation Report shall be evidence-based and respond to the Key questions and evaluation areas outlined below. The key questions listed below are not exhaustive. Offerors are strongly encouraged to propose additional or alternate questions, but the study should at a minimum answer the following: Key evaluation question: Has the project managed to develop capacity within MOHCW and local non-governmental organisations to support continued delivery of PMTCT services beyond the life of the project and how can these efforts be improved upon? Specific Evaluation Questions: 1. 1. Overall Program Performance o What was the quantity, quality and timeliness of project results? 2. Sustainability: Assess project sustainability factors as defined by: USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 7 of 34 o The extent to which processes and products developed through the project were institutionalized within the MOHCW, o Demonstrated ownership of project activities by the MOHCW, and o The ability of the MOHCW to continue service provision to intended beneficiaries after the project has ended. 3. Coordination and Integration: o What nature of relationships (networking, coordinating, cooperating, collaborating or partnership) with different stakeholders facilitated achievement of results? Focus on relations between: - The project and relevant MOHCW departments, - The prime partner and sub-grants - The project and other key national PMTCT stakeholders o What were the enabling and disabling factors in these relationships? o What benefits were derived from these relationships? 4. Leveraging resources: o What was the significance of leveraged resources to the overall project results? o What key factors supported or hindered efforts to leverage external resources? C.4 REPORTS AND DELIVERABLES In addition to the requirements set forth for submission of reports in Sections I and J, and in accordance with AIDAR clause 752.242-70, Periodic Progress Reports, the Contractor shall submit reports, deliverables or outputs subject to the deadlines specified in Section F.4 of this RFTOP, as further described below to the COR (referenced in Section G.2). The contractor will also be responsible for submitting the following deliverables: The following deliverables and reports are required under the Task Order. All deliverables and reports will be in English unless otherwise noted. The Contractor and the Contracting Officer’s Representative (COR) have the authority to make small changes to the deliverables and reports specified below. Any such alteration must not change the basic substance of the deliverable, require funds beyond the amount obligated or exceed the firm fixed price or any budgetary limitation. Each deliverable shall conform to the performance standards as described in the Statement of Work, Section C. 1. Signed statements attesting to a lack of conflict of interest or describing an existing or potential conflict of interest relative to the project being evaluated by each evaluation team member. 2. Final evaluation design, work plan and timeline presented to USAID by the lead evaluator within two weeks of the award of the contract. The evaluation design will include a detailed evaluation design matrix (including the key questions, the methods and data sources used to address each question), draft questionnaires and other data collection instruments, and known limitations to the evaluation design. The final design requires USAID approval. The work plan will include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 8 of 34 3. The evaluation team will meet with USAID upon arrival in Zimbabwe and go through the evaluation work plan and timeline. The team will also provide an oral presentation of preliminary findings in PowerPoint format to USAID and other key stakeholders in separate meetings prior to the evaluation team’s completing its evaluation activities in Zimbabwe and departing Zimbabwe. 4. Draft evaluation report (meeting all the criteria below) delivered to USAID for review within 10 business days from the time of departing Zimbabwe and returning to the Offeror’s base offices. USAID will provide comments within 2 weeks of receipt of draft evaluation report. 5. The final report will be provided to the USAID/Zimbabwe in electronic form within 15 business days following receipt of comments from USAID. The evaluation report must address all evaluation questions included in the statement of work. It must represent a thoughtful, well-researched and well organized effort to address the evaluation purpose. Readers must have sufficient information about the body of evidence and how information was gathered to make a judgment as to its reliability, validity and generalizability. The final report should not exceed 30 pages (excluding appendices) and must include the following sections: o An executive summary: 3-5 page that summarizes the key points (project purpose and background, key evaluation questions, methods, findings, and recommendations) o Background information on the project o Purpose of evaluation o Evaluation team: must be described with particular reference to the existence or lack thereof real or potential conflicts of interest relative to the project being evaluated o Evaluation methods: must be explained in detail and limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.) must be disclosed in the report o Evaluation findings: must be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings must be specific, concise and supported by strong quantitative or qualitative evidence. When applicable, include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team. o Recommendations: need to be supported by a specific set of findings and must be action-oriented, practical and specific, with defined responsibility for the action o The final scope of work, evaluation tools and sources of information must be properly identified and listed in annex 6. All data and records from the evaluation must be submitted to USAID in an easily readable and organized electronic format along with the final report C.5 OVERARCHING ELEMENTS AND IMPLEMENTATION MODALITIES USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 9 of 34 C.5(a) Building Local Capacity The Offeror shall, to the maximum extent possible, use Zimbabwean staff, technical experts, and institutions in carrying out the evaluation of the PMTCT project under the resulting Task Order. C.5(b) Geographical Coverage The USAID funded PMTCT project provides technical assistance to the MOHCW national AIDS and TB Unit for the management, coordination and development of national PMTCT program including development of national policy and guidelines on PMTCT. The project also supports service delivery at district and facility level in all 62 districts of Zimbabwe except Shurugwi and Chirumhanzi districts which are exclusively supported by another organisation. C.5(c) Gender Considerations Equity should be addressed with a focus on gender. The evaluation should provide more details on the effect and results of the project interventions on men, women, girls and boys. C.5(d) Audience The primary audiences for the evaluation report shall be USAID, Government of Zimbabwe, Development Partners, Implementing Partners and key stakeholders especially at sub-national levels. END OF SECTION C USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 10 of 34 SECTION D: PACKAGING AND MARKING D.1 752.7009 MARKING (JAN 1993) a) It is USAID policy that USAID-financed commodities and shipping containers, and project construction sites and other project locations be suitably marked with the USAID emblem. Shipping containers are also to be marked with the last five digits of the USAID financing document number. As a general rule, marking is not required for raw materials shipped in bulk (such as coal, grain, etc.), or for semi-finished products which are not packaged. b) Specific guidance on marking requirements should be obtained prior to procurement of commodities to be shipped, and as early as possible for project construction sites and other project locations. This guidance will be provided through the cognizant technical office indicated on the cover page of this contract, or by the Mission Director in the Cooperating Country to which commodities are being shipped, or in which the project site is located. c) Authority to waive marking requirements is vested with the Regional Assistant Administrators, and with Mission Directors. d) A copy of any specific marking instructions or waivers from marking requirements is to be sent to the Contracting Officer; the original should be retained by the Contractor. D.2 BRANDING STRATEGY FOR EVALUATION SERVICES The contractor shall comply with the requirements of the USAID Graphics Standards Manual available at http://www.usaid.gov/branding, or any successor branding policy. The contractor must also comply with the Branding Strategy in Section D.3 of the IQC. END OF SECTION D USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 11 of 34 SECTION E - INSPECTION AND ACCEPTANCE USAID/Zimbabwe inspection and acceptance of services, reports and other required deliverables or outputs shall be done by the Task Order Contracting Officer’s Representative (COR), designated under separate letter. END OF SECTION E USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 12 of 34 SECTION F - DELIVERIES OR PERFORMANCE F.1 PERIOD OF PERFORMANCE The evaluation is to begin as soon as possible after award of the task order. F.2 PLACE OF PERFORMANCE Performance of this contract will be in Zimbabwe, as specified in Section C. F.3 KEY PERSONNEL (a) The key personnel that the Contractor shall furnish for the performance of this contract are as follows: Name Title (Individuals to be proposed by the Offeror in the proposal to be incorporated into the contract at time of award) Key Personnel specified in the task order are considered to be essential to the work being performed there under. Prior to replacing any of the specified individuals, the Contractor shall immediately notify both the Contracting Officer and USAID/Zimbabwe Contracting Officer’s Representative reasonably in advance and shall submit in writing a justification (including proposed substitutions) in sufficient detail to permit evaluation of the impact on the program. No replacement of key personnel shall be made by the contractor without written consent of the Contracting Officer. F.4 REPORTS AND DELIVERABLES The following deliverables and reports are required under the Task Order. All deliverables and reports will be in English unless otherwise noted. The Contractor and the Contracting Officer’s Representative (COR) have the authority to make within scope changes to the deliverables and reports specified below. Any such change must not change the basic substance of the deliverable, require funds beyond the amount obligated or exceed the total ceiling price or any budgetary limitation. Each deliverable shall conform to the performance standards as described in the Statement of Work, Section C. Schedule of Deliverables: Task Outputs/Deliverables Time-frame 1. Signed Statements Signed statements attesting to a lack of conflict of interest or describing an existing or potential conflict of interest Within two weeks of USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 13 of 34 relative to the project being evaluated by each evaluation team member. contract award 2. Work plan Evaluation design, work plan and timeline presented to USAID by contractor within two weeks of the award of the contract. The evaluation design will include a detailed evaluation design matrix (including the key questions, the methods and data sources used to address each question), draft questionnaires and other data collection instruments, and known limitations to the evaluation design. The final design requires USAID approval. The work plan will include the anticipated schedule and logistical arrangements and delineate the roles and responsibilities of members of the evaluation team. Within two weeks of contract award 3. In brief The evaluation team will meet with USAID upon arrival in Zimbabwe and go through the evaluation work plan and timeline. Upon arrival in Zimbabwe 4. Periodic briefings and reports The Contractor shall provide progress briefings and reports to USAID Mission on a weekly basis. The Contractor shall use e-mail, phones and hard copies in meeting this requirement. Minutes of the key reporting meetings will be recorded by the Contractor. USAID will concur on key issues after these meetings. Weekly 5. Oral presentation of preliminary findings The Contractor shall provide an oral presentation of preliminary findings in PowerPoint format to USAID and key stakeholders identified during the evaluation in separate meetings upon completion of evaluation activities in Zimbabwe and prior to departure from Zimbabwe. Prior to departure from Zimbabwe 6. Draft Report Draft evaluation report (meeting all the criteria below) delivered to USAID for review within 10 business days from the time of return to their base offices. USAID/Zimbabwe will provide final comments within 2 weeks of receipt of the draft report. within 10 business days from the time of return to base office 7. Final report The final report should not exceed 30 pages (excluding appendices) and must include the following sections: 1. An executive summary: 3-5 page that summarizes the key points (project purpose and background, key evaluation questions, methods, findings, and recommendations) 2. Background information on the project 3. Purpose of evaluation 4. Evaluation team: must be described with particular reference to the existence or lack thereof real or potential conflicts of interest relative to the project being evaluated 5. Evaluation methods: must be explained in detail Within 15 business days of receipt of USAID comments to the draft report. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 14 of 34 and limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.) must be disclosed in the report 6. Evaluation findings: must be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings must be specific, concise and supported by strong quantitative or qualitative evidence. When applicable, include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team. 7. Recommendations: need to be supported by a specific set of findings and must be action-oriented, practical and specific, with defined responsibility for the action 8. The final scope of work, evaluation tools and sources of information must be properly identified and listed in annex 8. Data and Records All data and records from the evaluation must be submitted to USAID in an easily readable and organized electronic format along with the final report Along with the final report F.5 PERFORMANCE STANDARDS Evaluation of the Contractor’s overall performance shall be conducted jointly by the COR and the Contracting Officer, and shall form the basis of the Contractor's permanent performance record with regard to this contract. F.6 TECHNICAL APPROVALS The Contractor is required to obtain approvals from the COR for the following:  Approval of a work plan and modifications including activities, and agreed deliverables as described above;  Approval and acceptance of all reports and completion of deliverables;  Approval of international travel;  Approval of extension of deadlines for submission of reports and deliverables as specified in F.4 above due to unavoidable delays. However, such delays cannot exceed the performance period of the contract as stated in section F.2.  Approval of key personnel and any changes thereof. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 15 of 34 F.7 752.7005 SUBMISSION REQUIREMENTS FOR DEVELOPMENT EXPERIENCE DOCUMENTS (JAN 2004) (a) Contractor Reports and information/Intellectual Products. 1. The Contractor shall submit to USAID’s Development Experience Clearinghouse (DEC) copies of reports and information products which describe, communicate or organize program/project development assistance activities, methods, technologies, management, research, results and experience as outlined in the Agency’s ADS Chapter 540. Information may be obtained from the COR. These reports include: assessments, evaluations, studies, development experience documents, technical reports and annual reports. The Contractor shall also submit two copies of information products including training materials, publications, databases, computer software programs, video and other intellectual deliverable materials required under the Contract Schedule. Time-sensitive materials such as newsletters, brochures, bulletins or periodic reports covering periods of less than a year are not to be submitted. 2. Upon contract completion, the contractor shall submit to DEC an index of all reports an information/intellectual products referenced in paragraph (a) (1) of this clause. (b) Submission requirements. 1. Distribution. (i) At the same time submission is made to the CTO, the contractor shall submit, one copy each, of contract reports and information/intellectual products (referenced in paragraph (a) (1) of this clause) in either electronic (preferred) or paper form to one of the following: Online: http://dec.usaid.gov Through email: docsubmit@usaid.gov By mail: USAID Development Experience Clearinghouse M/CIO/ITSD/KM Ronald Reagan Building M.01 Washington, DC 20523-6100 (ii) The contractor shall submit the reports index referenced in paragraph (a) (2) of this clause and any reports referenced in paragraph (a) (1) of this clause that have not been previously submitted to DEC, within 30 days after completion of the contract to one of the address cited in paragraph (b)(1)(i) of this clause. 2. Format. 1) Descriptive information is required for all Contractor products submitted. The title page of all reports and information products shall include the contract number(s), contractor name(s), name of the USAID cognizant technical office, the publication or USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 16 of 34 issuance date of the document, document title, author name(s), and strategic objective or activity title and associated number. In addition, all materials submitted in accordance with this clause shall have attached on a separate cover sheet the name, organization, address, telephone number, fax number, and Internet address of the submitting party. 2) The report in paper form shall be prepared using non-glossy paper (preferably recycled and white or off white) using black ink. Elaborate art work, multicolor printing and expensive bindings are not to be used. Whenever possible, pages shall be printed on both sides. 3) The electronic document submitted shall consist of only one electronic file which comprises the complete and final equivalent of the paper copy. 4) Acceptable software formats for electronic documents include WordPerfect, Microsoft Word, and Portable Document Format (PDF). Submission in PDF is encouraged. 5) The electronic document submission shall include the following descriptive information: A. Name and version of the application software used to create the file, e.g., WordPerfect Version 9.0 or Acrobat Version 5.0. B. The format for any graphic and/or image file submitted, e.g., TIFF-compatible. C. Any other necessary information, e.g. special backup or date compression routines, software used for storing/retrieving submitted data, or program installation instructions. END OF SECTION F USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 17 of 34 SECTION G - CONTRACT ADMINISTRATION DATA G.1 ADMINISTRATIVE CONTRACTING OFFICE (a) The Contracting Officer with authority to administer the contract is based at: U.S. Agency for International Development –South Africa USAID/Southern Africa 100 Totius Street P. O. Box 43 Groenkloof 0027 Pretoria, South Africa (b) Contracting Officer's Authority: The Contracting Officer is the only person authorized to make or approve any changes in the requirements of this task order and notwithstanding any provisions contained elsewhere in this task order, the said authority remains solely in the Contracting Officer. In the event the Contractor makes any changes at the direction of any person other than the Contracting Officer, the change shall be considered to have been made without authority and no adjustment shall be made in the contract terms and conditions, including price. G.2 CONTRACTING OFFICER’S REPRESENTATIVE (COR) The COR will be designated by a separate Administrative letter issued by the Contracting Officer at the time of contract award. A copy of the COR designation letter will be given to the Contractor. G.3 ACCEPTANCE AND APPROVAL In order to receive payment, all deliverables must be accepted and approved by the COR. G.4 INVOICES The contractor shall submit invoices as follows: One (1) original of each invoice shall be submitted on an SF-1034 Public Voucher for Purchases and Services Other Than Personal to the Office of Financial Management, USAID/Zimbabwe; One copy of the voucher and the invoice shall also be submitted to the COR. Electronic submission of invoices is encouraged. The SF￾1034 must be signed, and it must be submitted along with the invoice and any other documentation in Adobe. Paper Invoices shall be sent to the following address: Office of Financial Management P.O. Box 6988 1 Pascoe Ave Belgravia USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 18 of 34 Harare, Zimbabwe Electronic invoices shall be sent to the Office of the Chief Financial Officer to this address: bills@usaid.gov. If submitting invoices electronically, do not send a paper copy. G.5 ACCOUNTING AND APPROPRIATION DATA [To be filled in at time of award] END OF SECTION G USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 19 of 34 SECTION H - SPECIAL CONTRACT REQUIREMENTS H.1 LANGUAGE REQUIREMENTS All deliverables shall be produced in English. H.2 LOGISTIC SUPPORT The task order shall be managed in Zimbabwe. The prime Contractor's home office shall provide managerial oversight and administrative backstopping, and technical assistance as needed. The Contractor shall be responsible for all administrative support and logistics required to fulfill the requirements of this task order. These shall include all travel arrangements, appointment scheduling, secretarial services, report preparations services, printing, and duplicating. H.3 GOVERNMENT FURNISHED FACILITIES OR PROPERTY The Contractor and any employee or consultant of the Contractor is prohibited from using U.S. Government facilities (such as office space or equipment) or U.S. Government clerical or technical personnel in the performance of the services specified in the Task Order unless the use of Government facilities or personnel is specifically authorized in the Task Order or is authorized in advance, in writing, by the COR. H.4 CONFIDENTIALITY AND OWNERSHIP OF INTELLECTUAL PROPERTY All reports generated and data collected during this project shall be considered the property of USAID and shall not be reproduced, disseminated or discussed in open forum, other than for the purposes of completing the tasks described in this document, without the express written approval of a duly-authorized representative of USAID. All findings, conclusions and recommendations shall be considered confidential and proprietary. END OF SECTION H USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 20 of 34 PART II - CONTRACT CLAUSES SECTION I - CONTRACT CLAUSES All applicable clauses as outlined in the basic Evaluation Services Indefinite Quantity Contract (IQC) and its subsequent amendments (if any) are incorporated by reference. END OF SECTION I USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 21 of 34 PART III - LIST OF DOCUMENTS, EXHIBITS AND OTHER ATTACHMENTS SECTION J - LIST OF ATTACHMENTS None. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 22 of 34 PART IV - REPRESENTATIONS AND INSTRUCTIONS SECTION K - REPRESENTATIONS, CERTIFICATIONS AND OTHER STATEMENTS OF OFFERORS Not required. END OF SECTION K USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 23 of 34 SECTION L – INSTRUCTIONS L.1 GENERAL INSTRUCTIONS TO OFFERORS (a) The Government anticipates the award of one (1) task order as a result of this RFTOP; however, it reserves the right to make multiple awards or no award. (b) Separate Technical and Cost Proposals: Regardless of the method used to submit proposals, the Technical Proposal and Cost Proposal must be kept separate from each other. Technical Proposals must not make reference to pricing or cost data in order that the technical evaluation may be made strictly on the basis of technical merit. (c) Government Obligation: The US Government is not obligated to make an award or to pay for any costs incurred by the Offeror in preparation of a proposal in response hereto. (d) RFTOP Instructions: If an offeror does not follow the instructions set forth herein, that Offeror's proposal may be eliminated from further consideration or the proposal may be downgraded and not receive full credit under the applicable evaluation factor. The Government may determine an offer to be unacceptable if the offer does not comply with all of the terms and conditions of the RFTOP and prospective task order. (e) Accurate and Complete Information. Offerors must set forth full, accurate and complete information as required by this RFTOP. The penalty for making false statements to the Government is prescribed in 18 U.S.C. 1001. (f) Potential Offerors may submit questions in writing, by email to Tracy Swift at tswift@usaid.gov. The deadline for receipt of questions and/or requests for clarifications is indicated on the cover page of this RFTOP. No questions will be accepted after this date. If substantive questions are received which affect the response to the solicitation, or if changes are made to the closing date and time, as well as other aspects of the RFTOP, this solicitation will be amended. Oral instructions or explanations given before the award of the contract resulting from this solicitation shall not be binding. L.2 DELIVERY INSTRUCTIONS (a) Submission, Marking and Copies The Offeror should submit the proposal through one of the two following methods: i. Electronically (preferred)- email with up to 10 attachments (5MB limit) per email compatible with Microsoft Office 2003 (MS Word or Excel). Multiple emails may be sent to accommodate the proposal size and content, but each must contain very clear identification of the attachment and instructions for assembling the proposal. Offerors may also send an Adobe Acrobat portable document format (.pdf) for electronic submission; however, zipped files attachments are not allowed. The subject line for every USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 24 of 34 such email must include the following: “PMTCT End of Project Evaluation RFTOP.” (Facsimile of the entire proposal is not authorized); or ii. Hand delivery (including commercial courier) – sending an original and three (3) copies of the technical proposal and one original and two (2) copies of the cost proposal. A standard diskette(s) or CD(s) containing one complete copy of the entire proposal compatible in a MS Windows environment must also be submitted to the issuing office. The information requested below must be placed in sealed envelopes clearly marked on the outside with the following information: RFTOP No.: SOL-613-12-000008 (Title) PMTCT End of Project Evaluation (b) Closing Date and Time All proposals in response to this RFTOP shall be due at the below address, not later than the date and time, as indicated on the cover page of this RFTOP. (c) Addresses Proposals shall be delivered to the following addresses: (i) Electronically – to: proposals@usaid.gov (ii) If Hand-Carried, or via Courier Service: U.S. Agency for International Development Office: USAID/Zimbabwe 1, Pascoe Avenue P.O. Box 6988, Belgravia Harare, Zimbabwe L.3 INSTRUCTIONS FOR THE PREPARATION OF THE TECHNICAL PROPOSAL (a) General The Technical Proposal in response to this solicitation should address and follow the technical evaluation factors listed in Section M. The technical proposal shall consist of the technical proposal itself and an annex. These two parts together will constitute the Offeror’s technical proposal (and be physically bound together if possible for any hard copies). The proposal must be well organized, complete, clear, and succinctly presented. (b) Page Limitation USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 25 of 34 Proposals will be concise, specific and complete and detailed information should be presented only when required by specific RFTOP instructions. The technical proposal itself shall not exceed 10 pages, excluding the Annexes. Any additional pages will not be evaluated. Proposals shall be written in English and typed on standard 8 1/2" x 11" paper (210mm by 297mm paper) or A4 paper, single-spaced, 12 characters per inch with each page numbered consecutively. Items such as cover pages, dividers and the table of contents are not included in the 10-page limitation. (c) Technical Proposal Annex The technical proposal annex shall contain resumes and letters of commitment from Key Personnel, teaming agreements (if any), past performance references and any other supporting documentation requested by the RFTOP. The technical proposal annexes shall not exceed a total of thirty (30) pages. (d) Organization The technical proposal should be organized into the following sections as follows: Table of Contents 1. Technical Approach a. Overall Technical Approach b. Evaluation Plan c. Institutional Capacity 2. Key Personnel 3. Contractor Past Performance Information 4. Annexes These sections, including the annex where relevant, should include all information required to fairly evaluate the Offeror under the applicable evaluation factor. Specific guidance on the content of each of these sections is set forth below in Sections L.4(a) through L.4(d). L.3(a) TECHNICAL APPROACH - [see Section M.2(a)] The purpose of the Technical Approach Section of the Technical Proposal is to provide enough information to permit thorough evaluation of the proposal pursuant to the criteria described in Section M.3. This section should include the following subsections: (a) Overall Technical Approach; (b) Evaluation Plan; and (c) Institutional Capacity. Specific guidance on each sub￾section is provided below. (i) Overall Technical Approach In this sub-section, the Offeror should provide information sufficient to address the evaluation criteria set forth in Section M.2(a)1. The Offeror should describe its overall technical approach (including providing any context or background that is useful in justifying the approach chosen). USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 26 of 34 The approach should be presented as methodologically sound and based on clear analysis of the challenges to be tackled. Offerors are encouraged to present creative and innovative approaches, with the potential for replication, to deal with these complex issues. The evaluation methodology to be used must be explained in detail and be appropriate and of sufficient rigor to produce valid results. Limitations to the evaluation with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.) must be disclosed in the evaluation report. The evaluation team will be expected to:  Employ study techniques that ensure internal validity of study results  Utilize social science methods and tools that reduce the need for evaluator-specific judgments.  Employ standardized recording and maintenance of records from the evaluation (e.g. focus group transcripts).  Collect data on variables corresponding to inputs, outputs, outcomes and impacts, as well as financial data that permit computation of unit costs and analysis of cost structure.  Produce evaluation findings that are based on facts, evidence and data. This precludes relying exclusively upon anecdotes, hearsay and unverified opinions. Findings should be specific, concise and supported by quantitative and/or qualitative information that is reliable, valid and generalizable. (ii) Evaluation Plan In this sub-section, the Offeror should provide information sufficient to address the evaluation criteria set forth in Section M.2(a)(2). The evaluation plan should describe the methodology to be used, including sample size and coverage (relevant to activity geographic and demographic coverage), techniques (including comparisons of project and non-project areas and stakeholders, where possible), site selection, information sources, interview protocols, etc. The methodology should clearly describe how the offeror will work with the mission staff, partner NGOs, and government and school/community stakeholders during the evaluation. Equity should be addressed with a focus on gender and orphans and other vulnerable children. The evaluation should provide more details on the effect and results of the project interventions on men, women, girls and boys. (iii) Institutional Capacity In this sub-section, the Offeror should provide information sufficient to address the evaluation criteria set forth in Section M.2(a)(3). In this section, the Offeror should provide information on the existing capacity and capabilities of the Offeror and its actual experience in providing similar programs to those required under Section C of this RFTOP. This should include demonstrated capacity of Offeror to manage (technically, administratively and financially) a project of similar type and complexity and to deliver the required results and proof of success in collaborating with partners and various stakeholders. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 27 of 34 L.3 (b) KEY PERSONNEL: [see Section M.2(b)] In this sub-section, the Offeror should provide information sufficient to address the evaluation criteria set forth in Section M.2(b). The Contractor shall describe the proposed staffing plan for implementing the SOW described in Section C. The contractor shall demonstrate the commitment of personnel proposed through letters of intent. The evaluation effort will be led by the contractor. One USAID/Zimbabwe employee will form part of the evaluation team. The contracted key personnel should possess either individually or as a team both evaluation methodology and PMTCT technical area expertise. It is preferred that at least one of the key personnel possesses the following qualifications and experience:  An advanced degree in public health, epidemiology, social sciences, health care management or other relevant course of study.  Expertise in conducting public health evaluations in developing countries (particularly in Africa) using strong evaluation methodology  At least 5 years’ experience in international public health in the field of HIV / AIDS. Additional experience in PMTCT programming would be beneficial  At least 10 years’ experience in project management and implementation (preferable in an international development context)  Demonstrated leadership experience and excellent team working skills  Demonstrated understanding of the Zimbabwe country context and public health sector  Demonstrated knowledge of USAID policies and procedures  Excellent English communication skills, both oral and in written, are essential. Candidate must have the ability to present information, analysis, and recommendations in clear written and oral formats Evaluation team members are to provide a signed statement attesting to a lack of conflict of interest or describing an existing or potential conflict of interest relative to the project being evaluated that could lead reasonable third parties to conclude that the evaluator or evaluation team member is not able to maintain independence and, thus, is not capable of exercising objective and impartial judgment on all issues associated with conducting and reporting the work. Real or potential conflicts of interest may include, but are not limited to: 1. Immediate family or close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant/ material though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant/material though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 28 of 34 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. L.3 (d) CONTRACTOR PAST PERFORMANCE INFORMATION [see Section M.2(d) The Offeror’s proposal must include copies of PPIRS Contract Performance Reports (CPRs) from at least five (5) but no more than 10 of their most recent contracts (performed within the last three years) of similar size and scope. USAID reserves the right, however, to use past performance information obtained from sources other than those provided by the Offeror. (Note: Firms lacking relevant past performance history shall be given a “neutral” Past Performance rating that neither rewards nor penalizes those Offerors). Past Performance documentation shall not count against the 10 page limit of the technical proposal. For each contract, order or program listed, please provide the following information:  Contract, Order or other identifying number;  Agency or entity providing the contract or funding;  Description of the scope of work, including, but not limited to a brief discussion of the complexity/diversity of tasks;  Primary location(s) of work;  Term of performance;  Skills/expertise required;  Dollar value;  Contract or order type, e.g. fixed-price or cost; and  Contact information for two persons, including name, job title, mailing address, phone numbers and e-mail addresses. (USAID recommends that you alert the contacts that their names have been submitted and that they are authorized to provide performance information concerning the listed contracts if and when USAID requests it). If extraordinary problems impact any of the referenced contracts, provide a short explanation and the corrective action taken in this section of the Technical Proposal. Describe any quality awards or certifications that indicate exceptional capacity to provide the service or product described in the statement of work. This information is not included in the page limitation. L.4 PRICE PROPOSAL USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 29 of 34 Offerors must complete Section B.3 of this RFTOP, entitled “Firm, Fixed Price” and submit (as a separate section) with its Technical Proposal. L. 5 INSTRUCTIONS FOR THE PREPARATION OF BRANDING AND MARKING PLANS In accordance with Section D of this RFTOP, the apparent successful Offeror, upon request of the Contracting Officer, shall submit and negotiate a Branding Implementation Plan (BIP) and a Marking Plan (MP). The BIP will be included in and made a part of the resultant contract. The BIP will be negotiated within the time that the Contracting Officer specifies. Failure to submit and negotiate a BIP will make the offeror ineligible for award of a contract. Offerors should submit a preliminary BIP and MP (not to exceed two pages) as a separate annex to the cost proposal. The BIP and MP will not be a part of the technical evaluation. The successful Offeror will be required to submit formal Branding Implementation Plan and Marking plans for final review before award, or if a competitive range is established, those offerors in the competitive range will be required to submit formal and detailed Branding Implementation Plan and Marking plans. END OF SECTION L USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 30 of 34 SECTION M - EVALUATION FACTORS FOR AWARD M.1 GENERAL INFORMATION FOR OFFERORS (a) The Government may award a task order without discussions with offerors in accordance with FAR 52.215-1. (b) The Government intends to evaluate task order proposals in accordance with Section M of this RFTOP and award to the responsible contractor(s) whose task order proposal(s) represents the best value to the U.S. Government. “Best value” is defined as the offer that results in the most advantageous solution for the Government, in consideration of technical, cost, and other factors. (c) The submitted technical information will be scored by a technical evaluation committee using the technical factors shown below. The evaluation committee may include experts who are not employees of the Federal Government. When evaluating the competing offers, the Government will consider the written qualifications/capability information provided by the Offerors, and any other information obtained by the Government through its own research. (d) For overall evaluation purposes of this RFTOP, technical factors are considered significantly more important than cost/price factors. M.2 EVALUATION FACTORS (a) The factors below are presented by major category, with relative weights identified, so that Offerors will know which areas require emphasis in the preparation of proposals. The factors below reflect the requirements of this particular solicitation. Offerors must note that these factors: (1) serve as the standard against which all proposals will be evaluated, and (2) serve to identify the significant matters which Offerors must address in their proposals. (b) Each technical proposal will be evaluated and scored by the Technical Evaluation Committee in accordance with the evaluation factors set forth in this solicitation. (c) Offerors will be evaluated on the basis of the following four technical evaluation factors and sub-factors with the following weights (total of 100 points): 1. Technical Approach 60 points a. Overall Technical Approach 40 points b. Evaluation Plan 10 points c. Institutional Capacity 10 points 2. Personnel 30 points 3. Contractor Past Performance Information 10 points Specific information on each evaluation factor and sub-factor is provided below. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 31 of 34 M.2(a) TECHNICAL APPROACH (60 points) - [(Section L.3(a)] The Offeror's technical approach to the program and the results proposed shall be evaluated. The Technical Approach factor shall be scored based on the following considerations: 1) Overall Technical Approach - 40 points Evaluation under this sub-factor will focus on the soundness, realism and innovativeness of the overall technical approach presented in the technical proposal. The following considerations will be evaluated under this sub-factor. These considerations will not be scored separately as individual elements. They are included to provide potential offerors with additional information regarding this evaluation criterion and to help offerors prepare their proposals:  The overall technical approach to achieving the objectives outlined in Section C demonstrates a clear and complete understanding of the PMTCT activity in Zimbabwe.  Innovativeness, pragmatism and creativity in the overall approach to attain the planned outputs and results during the timeframe of the contract  Execution of required tasks is clearly defined, feasible, and technically sound  Additionally, the technical approach demonstrates an understanding of the relevant stakeholders, involvement, roles and responsibilities and reflects best practices and lessons learned from past experiences.  The technical approach considers the effect and results of the project on women, men, boys and girls separately. 2) Evaluation Plan - 10 points The Evaluation Plan shall be evaluated under this sub-factor. The evaluation plan sub-factor relates to the Offeror’s choice of methodology to be used, including sample size and coverage, techniques, site selection, information sources, and interview protocols. 3) Institutional Capacity - 10 points The institutional capacity sub-factor evaluates the existing capacity and capabilities of the offeror and its actual experience in providing similar programs to those required under Section C of this RFTOP. The following considerations will be evaluated under this sub-factor. These considerations will not be scored separately as individual elements. They are included to provide potential offerors with additional information regarding this evaluation criterion and to help offerors prepare their proposals:  Demonstrated capacity of Offeror to manage (technically, administratively and financially) a project of similar type and complexity and to deliver the required results.  Proven success in collaborating with partners and various stakeholders. M.2(b) Personnel (30 points) - [(Section L.3(b)] USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 32 of 34 The personnel factor evaluates the extent to which the qualifications, skills and experience of proposed Key Personnel and other proposed staff meet or exceed those required in Section L.4(b). Evaluation includes the overall staffing plan and approach, as well as the capabilities of specific key personnel. The following considerations will be evaluated under this factor. These considerations will not be scored separately as individual elements. They are included to provide potential offerors with additional information regarding this evaluation criterion and to help offerors prepare their proposals:  The appropriateness of the persons proposed for the key personnel positions, including a review of their experience in areas relevant to the successful implementation of the proposed activity, education, other skills and performance history as shown through references and/or other sources. Whether the key personnel will be able to devote adequate time, over the life of the activity, to the management of the activity proposed shall also be considered.  Appropriateness of other proposed staff or consultants experience, education and skills in relation to their functions and responsibilities in the project. This includes experience, education and skills of staff related to the following areas:  Realism and effectiveness of the staffing plan.  Effectiveness of plan and the ability to maximize resources, for purposes of cost effectiveness,  Use of Zimbabweans in staffing some of the proposed positions for purposes of indigenous knowledge and building of host country capacities. M.2(d) CONTRACTOR PERFORMANCE INFORMATION (10 Points) – [(Section L.3(d)] (a) Performance information will be used for both the responsibility determination and best value decision. USAID may use performance information obtained from other than the sources identified by the Offeror/Subcontractor. USAID will utilize existing databases of Contractor performance information and solicit additional information from the references provided in Section L.10 of this RFTOP and from other sources if and when the Contracting Officer finds the existing databases to be insufficient for evaluating an Offeror’s performance. (b) If the performance information contains adverse past performance information to which the Offeror has not had a prior opportunity to respond, USAID will provide the Offeror an opportunity to comment on it prior to its consideration in the evaluation, and any Offeror comment will be considered with the adverse performance information. (c) USAID will initially determine the relevance of similar performance information as a predictor of probable performance under the subject requirement. USAID may give more weight to performance information that is considered more relevant and/or more current. Performance in Zimbabwe is more relevant than performance in other countries. USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 33 of 34 (d) The Contractor’s performance information determined to be relevant will be evaluated in accordance with the elements below: 1) Overall Past Performance 10 points The following considerations will be evaluated under this sub-factor. These considerations will not be scored separately as individual elements. They are included to provide potential Offerors with additional information regarding this evaluation criterion:  Quality of product or service, including consistency in meeting goals and targets;  Timeliness of performance, including adherence to contract schedules and other time￾sensitive project conditions, and effectiveness of home and field office management to make prompt decisions and ensure efficient completion of tasks;  Business relations, addressing the history of professional behavior and overall business-like concern for the interests of the customer, including coordination among subcontractors and developing country partners, cooperative attitude in remedying problems, and timely completion of all administrative requirements;  Customer satisfaction with performance, including end user or beneficiary wherever possible;  Effectiveness of key personnel, including appropriateness of personnel for the job and prompt and satisfactory changes in personnel when problems with clients were identified; and  Cost control, including forecasting costs as well as accuracy in financial reporting, ensuring that unnecessarily expensive technical assistance is not used when lower cost advisors are adequate, and pacing the expenditure of level of effort such that contract deliverables and outputs can be produced within budget. M.3 PRICE EVALUATION The contractor’s firm, fixed price will be used as part of the trade-off analysis in determining source selection. M.4 DETERMINATION OF THE COMPETITIVE RANGE AND CONTRACT AWARD (a) Competitive Range: If the Contracting Officer determines that discussions are necessary, he/she will establish a Competitive Range composed of only the most highly rated proposals. In certain circumstances, the Contracting Officer may determine that the number of most highly rated proposals that might otherwise be included in the competitive range exceeds the number at which an efficient competition can be conducted. Should that be the case, the Contracting Officer may then limit offers in the competitive range to the greatest number that will permit an efficient competition among the most highly rated offers. (b) Award: In accordance with FAR 52.215-1(f), the Government intends to award a Contract from this solicitation to the responsible Offeror(s) whose proposal(s) represent the best USAID/Zimbabwe | PMTCT End of Project Evaluation SOL-613-12-000008 Page 34 of 34 value after evaluation in accordance with the factors and sub-factors as set forth in this solicitation. This procurement also utilizes the tradeoff process set forth in FAR 15.101-1. If the Contracting Officer determines that competing technical proposals are essentially equal, cost/price factors may become the determining factor in source selection. Conversely, if the Contracting Officer determines that competing cost/price proposals are essentially equal, technical factors may become the determining factor in source selection. Further, the Contracting Officer may award to a higher priced Offeror if a determination is made that the higher technical evaluation of that Offeror merits the additional cost/price. END OF SECTION M