MALAWI COMMUNITY-BASED FAMILY PLANNING AND HIV & AIDS SERVICES PROJECT MID-TERM EVALUATION MAY 2010 This publication was produced for review by the United States Agency for International Development. It was prepared by Carolyn Barnes, Dan Blumhagen, and Douglas Huber through the Global Health Technical Assistance Project. MALAWI COMMUNITY-BASED FAMILY PLANNING AND HIV & AIDS SERVICES PROJECT MID-TERM EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 10-01-334) is available in printed or online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources. Documents are also made available through the Development Experience Clearing House (www.dec.org). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. MALAWI CFPHS PROJECT: MID-TERM EVALUATION i ACKNOWLEDGMENTS The mid-term evaluation team would like to acknowledge the generous support and assistance it received. The team particularly wishes to thank the U.S. Ambassador Peter W. Bodde and United States Agency for International Development (USAID) Mission Director Curt Reintsma who gave of their limited time to discuss progress being made to advance the family planning (FP) agenda in Malawi. The Community-Based Family Planning and HIV&AIDS Services (CFPHS) Project's officers and staff gave more time than they had available. In particular, Mexon Nyirongo, Chief of Party, Chimwemwe Msukwa, Monitoring and Evaluation (M&E) Advisor, Caroline Bakasa, Population Services International (PSI) Program Officer, Ricky Nyaleye, PSI Communications and Listening Club Coordinator, and the project's district coordinators (DCs) gave generously of their time to plan the team‘s visits to seven project districts. Moreover, they and Njuru Nganga, Deputy COP, Flora Khomani, HIV/AIDS Advisor, Olive Mtema, Policy Specialist, Rudi Thetard, Country Director for Management Sciences for Health (MSH), and Ricki Orford, Country Director for PSI, briefed us and answered numerous questions that provided us with information and insight on the Project's experience. Key individuals in the Ministry of Health (MoH) at the national level kindly gave of their time to meet with the evaluation team, including Mr. Chris Kang‘ombe, the Principal Secretary, Dr. Chisale Mhango, Director, Reproductive Health Unit (RHU), and Sheila N. Bandazi, Director of Nursing Services. This permitted the team to better understand the MoH policies, directions, and concerns about family planning and HIV/AIDS in Malawi. On the team‘s visits to project districts, the MoH district health management teams (DHMTs) briefed us on the project and answered questions, and at times accompanied us when visiting facilities within the district. Special thanks goes to Lilly Banda, Deputy Team Leader, Office of Health, Population and Nutrition (HPN) for all of her support and assistance, which included providing the evaluation team with key documents to study during their preparatory stage and their stay in Malawi, a thorough briefing on the project, and accompanying the team on its visits to project sites. Ms. Banda, Martha Nanthoka, and Martin Mtika, of the HPN Office, provided valuable translation services which enabled us to hold discussions with community leaders and members. Alisa Cameron, Team Leader, Office of HPN, and others at USAID kindly gave their time to meet with us. The team would also like to thank the numerous people from USAID cooperating agencies, other parts of the Malawi Government, faith-based organizations, nongovernmental organizations (NGOs) and community-based organizations (CBOs) that briefed the team on their related programs (listed in Appendix 2). The team gained much from its meeting with community-based distribution agents (CBDAs), health surveillance assistants (HSAs) and local traditional authorities (TAs) who shared their experiences and opinions with us. The team appreciates the willingness of Susan Watkins, Michelle Poulin and Nicole Angotti to provide us copies of numerous published and unpublished papers from Malawi research projects of the Population Studies Center, University of Pennsylvania. Reviewing these papers before the team‘s arrival in Malawi enabled us to understand community perspectives on HIV/AIDS and family planning. ii MALAWI CFPHS PROJECT: MID-TERM EVALUATION Finally the team thanks Maggie Rajala and other members of the GH Tech team for all their assistance with the team's organization, assignment preparation, logistics and final organization of the report. In Malawi, Fanaseyi Huwa, the team‘s logistics coordinator, ensured that the team had appointments to meet the right people and accommodations along the way, and particularly Aubrey Nkhoma, who drove us safely over 7200 kilometers of Malawian roads. Despite the best efforts of the CFPHS Project and USAID/Malawi, factual errors may be present in the report. They must be considered the responsibility of the principal authors who tried to grasp the complexity of the project and its environment within a six-week period. MALAWI CFPHS PROJECT: MID-TERM EVALUATION iii CONTENTS ACRONYMS ..................................................................................................................................... v EXECUTIVE SUMMARY .................................................................................................................ix Recommendations .................................................................................................................. xii I. INTRODUCTION ..................................................................................................................... 1 II. INCREASE ACCESS TO COMMUNITY FP/RH AND HIV/AIDS SERVICES ......................... 3 Health Surveillance Assistants ................................................................................................ 3 Community-Based Distribution Agents (CBDAs) .................................................................... 4 Community-Based Work ......................................................................................................... 5 Retention of CBDAs ................................................................................................................ 7 III. STRENGTHEN CONTRACEPTIVE DELIVERY SYSTEMS, WHERE VOLUNTARISM AND INFORMED CHOICE ARE ASSURED AND WHERE QUALITY OF SERVICES IS IMPROVED ......................................................................................................................... 9 Support for Facility-Based FP Services ................................................................................... 9 Repositioning Family Planning at the District Level .............................................................. 10 Quality Assurance, Including Infection Prevention ................................................................ 11 Environmental Quality ........................................................................................................... 11 Involving the Faith-Based Community ................................................................................... 11 Informed Choice .................................................................................................................... 12 IV. BEHAVIOR CHANGE COMMUNICATION AND CHANGED BEHAVIORS ......................... 13 Behavior Change Communication Strategy .......................................................................... 13 Training .................................................................................................................................. 13 Open Days ............................................................................................................................. 14 Radio Drama ......................................................................................................................... 15 Radio Listening Clubs ........................................................................................................... 16 Community Drama ................................................................................................................. 18 Print Materials ........................................................................................................................ 19 Family Planning Concerns and Misconceptions ................................................................... 19 Changed Behaviors ............................................................................................................... 21 V. PROJECT IMPACT ............................................................................................................... 25 VI. HEALTH SECTOR CAPACITY STRENGTHENED .............................................................. 27 Policy Dialogue ...................................................................................................................... 27 Central Medical Stores & Pharmaceutical Logistics .............................................................. 28 Ministry of Health ................................................................................................................... 31 Data Collection Systems and Links with the HC and District HMIS ...................................... 32 iv MALAWI CFPHS PROJECT: MID-TERM EVALUATION VII. MANAGEMENT: KEY ASPECTS.......................................................................................... 33 Data Issues ............................................................................................................................ 33 Data Quality ........................................................................................................................... 35 Overall Project Management ................................................................................................. 36 Project Oversight ................................................................................................................... 36 VIII. LOOKING TO THE FUTURE ................................................................................................ 37 Youth and Adolescents ......................................................................................................... 37 Social Marketing .................................................................................................................... 37 Linkages in the HIV/AIDS Services and Support Chain ........................................................ 38 Conclusions ........................................................................................................................... 38 APPENDICES APPENDIX 1: SCOPE OF WORK ................................................................................................ 41 APPENDIX 2: PERSONS CONTACTED ..................................................................................... 51 APPENDIX 3: REFERENCES AND SOURCES CONSULTED ................................................... 59 APPENDIX 4: TABLES ................................................................................................................. 65 APPENDIX 5: EXPANDED RECOMMENDATIONS AND COMMENTS FOR STRENGTHENING SERVICE DELIVERY .................................................................................... 69 FIGURES Figure 1: Pharmaceutical Logistics Flow ....................................................................................... 29 Figure 2: Nkotakota 2007-2009: Contraceptive and Determine® (HIV) Distribution (units).......... 35 MALAWI CFPHS PROJECT: MID-TERM EVALUATION v ACRONYMS ADS Automated Directives System: USAID Policy Manual AHS Adventist Health Services ANC Antenatal care ARVs Antiretroviral drugs BASICS Basic Support for Institutionalizing Child Survival BCC Behavior change communication BLM Banja la Mtsogolo: The Marie Stopes affiliate in Malawi BTL Bilateral tubal ligation CFPHS Community-based family planning and HIV&AIDS services CBD Community-based distribution CBDA Community-based distribution agents CBO Community-based organization CHAM Christian Health Association of Malawi CHW Community health worker CMS Central medical stores CO Clinical officer COTR Contracting Officer‘s Technical Representative CPR Contraceptive prevalence rate CYP Couple-years of protection D & C Dilatation and curettage DC District coordinator DHO District health officer DHS Demographic and health survey DHMT District health management team DIP District Implementation Plan DMPA Depo-medroxyprogesterone acetate (Depo-Provera) FAM Fertility awareness-based method FGD Focus group discussion FHI Family Health International FIGO International Federation of Gynecologists and Obstetricians FP Family planning FP TR Family planning training GBV Gender-based violence vi MALAWI CFPHS PROJECT: MID-TERM EVALUATION GH Tech Global Health Technical Assistance Project GoM Government of Malawi HC Health center HF Health facility HIV Human immunodeficiency virus HIV+ HIV positive HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome HMIS Health management information system HPN Health, population and nutrition HSA Health surveillance assistant HTC HIV testing and counseling HTSP Health timing and spacing of pregnancy HTSS Health technical and support services ICM International Confederation of Midwives ICN International Council of Nurses IEC Information, education and communication IP Infection prevention IRH Institute for Reproductive Health IUCD Intra-uterine contraceptive device LTM Long-term method LTPM Long-term and permanent method MACRO Malawi Counseling and Resource Mobilization M&E Monitoring and evaluation MAM Muslim Association of Malawi MCHIP Maternal and Child Health Integrated Program MDHS Malawi Demographic and Health Survey MMR Maternal mortality ratio MoH Ministry of Health MoPD Ministry of Planning and Development MSH Management Sciences for Health MVA Manual vacuum aspiration NGO Nongovernmental organization ORS Oral rehydration solution PAC Post-abortion care PEPFAR President‘s Emergency Plan for AIDS Relief MALAWI CFPHS PROJECT: MID-TERM EVALUATION vii PLWHA People living with HIV/AIDS PMP Performance Monitoring Plan PMTCT Prevention of mother-to-child transmission PSC Personal Services Contractor PSI Population Services International QMAM Qadria Muslim Association of Malawi RH Reproductive Health RHU Reproductive Health Unit RMS Regional medical stores SDM Standard days method SRH Sexual and reproductive health STI Sexually transmitted infection SWAp Sector-wide approach TA Traditional authority TFR Total fertility rate UNFPA United Nations Fund for Population Activities USAID United States Agency for International Development WCBA Women of child-bearing age viii MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION ix EXECUTIVE SUMMARY USAID contracted with the Global Health Technical Assistance Project (GH Tech) to conduct a mid-term evaluation of the Malawi Community-Based Family Planning and HIV&AIDS Services Project (CFPHS). A team of three experts—Carolyn Barnes, Douglas Huber, and Dan Blumhagen (team leader)—worked in Malawi from April 7 to May 14, 2010, to conduct the evaluation. During this time, the team logged over 4,500 miles by road visiting seven of the eight project districts, which spread from the Tanzanian border in the north, to the furthest south of the country. The team spoke with dozens of people, and reviewed many documents to come to the conclusions it reached. The objectives of the evaluation are to: Assess the extent to which program outputs and outcomes are being achieved; Program outputs and outcomes are generally being achieved and exceeded: representative results include the performance to date compared to the end-of-project targets. Taking the simplest example, the project has delivered 247,000 couple-years of protection (CYP) in the first two and a half years, in contrast to the target for the life of project of 225,000. Some illustrative accomplishments in the targeted districts are: Couple-Years of Protection 247,000 FP/RH Counseling Visits 730,000 Service Delivery Points 1,514 People Counseled & Tested for HIV 93,000 All in all, the project has already or will soon surpass all planned program outputs and outcomes. It has tapped a phenomenal pent-up demand for family planning methods that is rarely seen in Africa. Assess and document the progress/extent to which the program outputs and outcomes are being achieved under the Management Sciences for Health (MSH) program to support integrated community-based family planning and HIV/AIDS services; Integration is difficult to assess, since there are many interpretations of what ―integrated services‖ actually means. A minimal definition is that multiple basic health interventions are provided simultaneously at the point-of-service delivery. Services are integrated at the community level, where a single individual—typically a community-based distribution agent (CBDA), family planning provider or health surveillance assistant (HSA)—provides FP and HIV testing and counseling (HTC) services. However, the numbers trained to date, especially in HTC, are small, and a lack of HTC test kits has compromised performance. To the extent that services have been provided, the integration through CBDAs is well-received. The services are often integrated in the health centers (HCs) where there are often only a few staff. Several USAID supported projects, such as the MSH Basic Support for Institutionalizing Child Survival (BASICS) project, also promote integrated care. At the HC level, due to the shortage of staff, antenatal care (ANC) and FP services may be provided on separate days though the under-five services are available on a daily basis. At the district hospital level, services are integrated and offered on a daily basis in MCH clinics, though in different rooms, so a woman needs to move from room to room for ANC, under-five care, and family planning. x MALAWI CFPHS PROJECT: MID-TERM EVALUATION Assess and document the progress made through MSH and its sub-partners in improving access and quality of family planning and HIV services at the community level; In areas targeted by the program—i.e., the eight districts and the communities served by CBDAs and HSAs, there has been an enormous increase in access to FP services. In discussions with the CBDAs, the team felt that they were providing quality services, but the team could not observe direct client interactions. However, even in the targeted districts, the number of trained CBDAs and HSAs is relatively small in relation to the population over age 14. Reaching all communities in the district would mean expanding the number of providers approximately four fold. Assess progress made in health promotion and behavior change at the community level; CFPHS has led to significant increases in the use of modern family planning and HTC. It has created demand and permitted those who want it to use FP and have HTC accessible to them. CBDAs are respected members of their communities, who are recognized as experts and whose advice is sought out. The mere figures presented above showing increases in CYPs and HTC indicate the incredible amount of behavior change that they have been able to introduce to their communities. Behavior change activities have also included radio dramas and associated listening clubs, open days, and drama troupes. While open days and local drama troupes are the most popular form of outreach, the members of the radio listening clubs have become effective community promoters in their villages. Assess the effectiveness and efficiency of MSH’s organizational structure in achieving program objectives; Most of the management issues stemmed directly or indirectly from communications and budgeting issues between USAID/Malawi and MSH. There was some confusion about the numbers of CBDAs to be trained, but MSH successfully transferred funds from other lines to be able to meet the Mission‘s training goals. Although the result of MSH‘s budget realignment was that the project turned out to be very successful, the evaluation team is concerned about supervision from central to district levels, refresher training, and that effective M&E could not be fully funded until the third year of the project. Determine progress towards achieving M&E goals and targets (review M&E plans to guide program direction, monitor/evaluate project activities and contribute to USAID/Malawi/HPN results); and MSH limited itself to the 14—later 19—indicators included in the project award. The most serious issue is that baseline levels were not established for any of these indicators. The project relied on common USAID/President‘s Emergency Plan for AIDS Relief (PEPFAR) indicators, and no project specific indicators were used. This issue weakened project management. There are many problems with data quality, but this is common with community-level service statistics that are aggregated through a decentralized system. As discussed below, the team does not feel that these deficits are significant. However, there were also many types of analysis that could have been done with data already collected. An example of this would have been to disaggregate depo-medroxyprogesterone acetate (DMPA) delivery by district (perhaps corrected for number of CBDAs) to find out where the more and less successful activities were. They could have then intervened in the one and learned from the other. The team feels this could have led to better management and targeted supervision. In response to this issue, MSH reported having difficulty in identifying an appropriate M&E officer. This was compounded by a lack of resources and transportation constraints that MALAWI CFPHS PROJECT: MID-TERM EVALUATION xi hindered analysis for project management at the district level and adequate on-site mentoring to improve the quality of data reported and data use. Make recommendations on strengthening successful efforts and adjustments/remedial actions that need to be considered to ensure sustainability of program outcomes, and institutionalization of policies and lessons learned. The HSA component of the project is sustainable, given that this cadre of workers has an existing role in the Ministry of Health (MoH) hierarchy. This assumes that funding for 11,000 HSAs (of which 6,000 are still donor funded) continues, and that adequate commodities can be procured. The CBDA component is not currently sustainable, since MSH has not been able to build appropriate supervisory and other organizational support structures within the MoH. While many CBDAs will continue their work in the absence of donor funding—as happened in the late 1990s and early 2000s—the team expects their effectiveness will fade over time. The team has been told that the MoH has little capacity to continue support for CBDAs after donor funding shifts, although the MoH is having discussions about CBDAs receiving support through other funding mechanisms, including the NGOs and international donors. The single largest issue for sustainability is the supply of commodities. While the logistics issue is very complicated, the team feels that about half of the difficulties stem from districts not submitting appropriate requisitions—or not making monthly drug requisitions at all—and the other half comes from larger issues at the regional medical stores (RMS) and central medical stores (CMS). Districts that made a commitment to procure according to guidelines had little problem with stock-outs. Both levels—and the donors supporting them—should make a commitment to procure sufficient test kits before any additional providers are trained. If district pharmacies have the recommended three months supply on hand, intermittent stock-outs at the CMS will make relatively little difference. Obviously, the MoH and donors have to make a similar commitment. Given the current environment of free access to a Basic Essential Health Services, the team is not sure that a social marketing program will have much success until the family planning/reproductive health (FP/RH)/HIV demand situation is better understood. Some aspects of current social marketing programs have had difficulty establishing their products in face of free competing products from the public sector. If specific population segments can be targeted, the social marketing programs will have better chance of succeeding. The Mission should carefully assess the feasibility of such a program through the Maternal and Child Health Integrated Program (MCHIP) (PSI) pilot program. Given current and future HPN office management skills, the follow-on activities should be designed to simplify management. The Mission should consider an experienced management personal services contractor (PSC) to provide additional support to the office at critical points. Conclusion: It is sometimes easy to forget the overall accomplishments. The CFPHS Project has been extremely successful: – All Project Targets will be met or exceeded by the end of the Project Period, September, 2011. – CFPHS has been extremely successful in extending FP/HIV-AIDS Services to Rural Communities in eight Districts. xii MALAWI CFPHS PROJECT: MID-TERM EVALUATION Nonetheless, there is a great deal to do. For the team, this project resembles a gleaming building that shows how it will enhance the skyline of a great city. However, this building is still supported by scaffolding and would collapse of its own weight were that scaffolding taken away. The remaining months of this project, and any other follow-on project, need to focus on strengthening the structure so it can stand more on its own. These challenges must be met, particularly if there are plans to expand services in the near future. RECOMMENDATIONS The recommendations are presented at the beginning of the report for reference purposes. The observations they are based on are discussed more thoroughly in the text. These are in order of where they occur in the text. 1. Supervisors should be retrained to provide supportive supervision rather than merely collecting information and passing it on. Supportive supervision should focus on performance of CBDAs, including problem-solving for difficult clients, community mobilization, counseling, infection prevention (IP), improving service delivery coverage, and drug logistics/supplies. Supervisors should also provide upgrading of skills and knowledge on a regular basis, particularly at monthly meetings. 2. Train HSAs to provide routine oral contraceptives and HTC. This will enhance effective supervision of CBDAs providing these services and will improve access for pill clients closer to the HSA than the CBDA. Several HSAs requested this training and authorization to provide the services. 3. Distribute (1) simple written client information for hormonal contraceptive methods to inform recipients about what they may experience, and (2) similar job aids for HSAs, including one￾page protocols for temporary management of side effects. Also, (3) give providers permission to give one cycle of pills or Brufen, when appropriate, for bleeding side effects as per MoH DMPA standards. (ref. MoH RH service delivery guidelines). 4. All HTC provider courses should include a significant amount of formal role-play under the observation of trained staff. 5. CFPHS should incorporate elements of the Adventist Health Services (AHS) support for CBDAs to improve retention and performance. 6. Continue to support post-abortion care (PAC), including the regular provision of immediate post-abortion family planning. Ensure family planning is also provided following dilation and curettage (D&C) procedures. 7. Introduce the health timing and spacing of pregnancy (HTSP) concept and its important implications for child and maternal health at the community and health facility (HF) levels. 8. Urge the MoH to provide incinerators where feasible and to monitor disposal of contaminated waste, particularly where incinerators are not yet installed. 9. Greater attention should be given to increasing CBDAs' skills and confidence in advising and counseling on gender-based violence (GBV) related to FP and HIV/AIDS. This could be done in CBDAs' monthly meetings with the special attendance of the District FP Coordinator, through CBDAs sharing of their experiences in monthly meetings, and in the MSH-planned CBDA refresher training course. 10. To help keep and increase the audience, the radio drama program ought to include additional time, e.g., 15 minutes, for: a) an engaging health professional to respond to questions sent by MALAWI CFPHS PROJECT: MID-TERM EVALUATION xiii listeners, b) reading letters from listeners, and c) airing recordings of songs created and performed by drama troupes and listening clubs. 11. For the re-airing of the radio drama, the listening club component should be scaled up to include new TAs and some new sub-areas in the current behavior change communication (BCC) focused TAs. PSI should continue to maintain contact with the current clubs, and provide members who are active in community outreach with identification badges. 12. To expand access to the PSI-scripted drama performances, CFPHS should support the scaling up of this successful BCC approach by training and funding more drama troupes in new TAs and some new sub-areas in the current BCC focused TAs. The drama troupes should be provided with Zina Umanena T-shirts, or some other form of identification. 13. To reduce concerns and misconceptions about FP, a multi-pronged approach should be adopted. (1) Funds should be made available for printing more copies of existing brochures, with targeted distribution. (2) During their monthly meetings, and in other group settings and individual talks, CBDAs should be encouraged to share experiences about addressing topics in the brochures as well as how they decide to whom they distribute the additional pamphlets they receive. (3) CBDAs should receive a steady supply of the insert on oral contraceptives to distribute to potential and new users. (4) CBDAs, as well as listening clubs and CBOs active in FP outreach activities, should receive an ample supply of the brochure on FP methods. 14. Funds should be allocated for the development and dissemination of leaflets on the range of popular FP methods, especially injectables, which include information about possible temporary side effects and actions to take. These might be patterned after the ―insert‖ on oral contraceptives. 15. The Mission, in consultation with the MoH, should determine the importance of integrating STI services with integrated health care and provide additional resources for these services if it is appropriate. 16. CFPHS should use some of the additional training funds made available under the 2010 budget expansion to train HC and DHMT staff methods to effectively manage their drug supply chain. 17. The USAID Mission, in collaboration with other donors, should consider a major effort to reform the medical procurement system. This will require starting at the very top of the chain, with the Government of Malawi (GoM) procurement procedures and regulations, and making sure that all who are responsible are appropriately trained and supervised to carry out their duties. Some form of cash transfer and project assistance may be needed to accomplish the goals. 18. Training of additional providers of DMPA and HTC should be undertaken on a district basis after assessing the district‘s ability to support services and ensure that there are sufficient supplies in its logistics chain that would enable it to deliver services over an extended period. ―No product no program.‖ 19. MSH should examine its data sets and find ways of disaggregating data that are useful for management use. xiv MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 1 I. INTRODUCTION Malawi has a population of 13 million people and is one of the ten poorest nations in the world. Over half of the population lives on $1 per day or less. Food insecurity affects 25% of the population and 45% of its children are stunted from malnutrition. The functional literacy rate is 63% overall and lower for women. The 2004 Malawi Demographic and Health Survey (MDHS) reported a maternal mortality ratio (MMR) of 984/100,000, a total fertility rate (TFR) of 6.0, and a contraceptive prevalence rate (CPR) of 28% for modern methods. While this CPR reflects substantial progress since the 1992 CPR of 7%, the unmet need in 2004 was also 28%, equal to the CPR. Moreover, the CPR has recently stagnated, increasing by only two points between the 2000 and 2004 surveys. In 2004, four in five women reported wanting to delay or stop child bearing, and 40% of pregnancies were unplanned and unwanted. The desired family size in 2004 was 4.1 compared to the actual family size of 6.0. However, translating these desires into increased CPR could be challenging, given client ambivalence about family planning methods and poor availability and quality of family planning services. In addition, couples and health workers are often misinformed about the safety and effectiveness of modern contraceptives. Only 54% of the health facilities provided family planning services, according to a 2006 sector￾wide approach (SWAp) report. A lack of health staff, particularly nurses, limited women‘s access to family planning. Rural women work long hours, making it difficult to access a clinic for family planning that is many kilometers away. The 2004 MDHS found an HIV prevalence rate of 12% among the 15–49 age group. HIV prevalence was 17% in urban areas compared with 11% in rural areas. HIV prevalence appears to be declining in urban areas, though it continues to increase in rural areas. Poverty, inaccessible local health services and low literacy appear to be factors keeping prevalence rates low. Gender inequalities, cultural practices, and male dominance in decisions about sex make women particularly vulnerable. Cross-generational sex is common and is exacerbated by food shortages and poverty, sometimes pushing young women into sex with older men who have resources. The number of HIV counseling and testing sites has steadily increased. By mid-2005, almost 40% of the 620 health facilities in Malawi were providing HTC services. However, as of June 2005, only 70 (48%) were located in rural areas. The 2004 MDHS estimated that only 14% of Malawians had ever been tested for HIV (13% of women, 15% of men). However, the number of persons going for testing is increasing every year. The Malawi MoH decentralized health service delivery to the district level. Therefore, district assemblies, in collaboration with the DHMT, have control over costs and the budget and are better positioned to establish their own priorities. However, health workers continue to leave the public health sector, making it more difficult to fill vacancies. In 2006, the average vacancy rate for nurses was 61% in the public health sector and 77% at Christian Health Association of Malawi (CHAM) facilities. Given the many causes for the human resource crisis, it is unlikely that staff shortages will be resolved in the foreseeable future. This is the situation that USAID found when it re-established a major family planning program, the CFPHS project. At its heart, the project is very simple. Implementation has proven to be different. There are basically three thrusts to the program. (1) There is the establishment of a 2 MALAWI CFPHS PROJECT: MID-TERM EVALUATION network of community-based workers in eight underserved districts.1 This included the support network required, including training, supervision, and commodities. (2) There is a component seeking to effect behavior change at all levels using the various forms of the media and local level activities such as ―open days‖ and drama troupes. (3) There is a policy reform activity that sought to change dysfunctional policies and guidelines at the national level. The team's Scope of Work is presented in Appendix 1. The project districts were Karonga in the North close to the Tanzanian border, Kasungu, Salima, Mangochi and Nkotakota in the center of the country, and Balaka, Phalombe and Chikwawa in the South. The team visited all districts except Balaka, spending typically one day in each district. Since the team split up into three sub-teams in each site visit, it was able to talk to a wide variety of people despite the limited time. Appendix 2 contains the list of people met in the districts, such as members of the district health teams, HC staff, community based service providers, religious leaders, and nearly 200 community members. MSH and PSI helped organize the visits, with guidance from the evaluation team. The team also held meetings with the USAID HPN office, members of the CFPHS implementation team and managers, key persons in the MoH, and officers of related projects. Prior to arrival and during the evaluation work in Malawi, the team reviewed numerous documents, some provided by the Mission and many from the academic literature. These are listed in Appendix 3. The team consisted of Carolyn Barnes, Douglas Huber, and Dan Blumhagen,—who acted as team leader. Together, the team brought nearly a century of international experience to bear on the CFPHS Project. 1 The district is the basic unit of official local government. There are 28 districts in Malawi. Below the district level, governance is by TAs, including village headmen and group village headmen. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 3 II. INCREASE ACCESS TO COMMUNITY FP/RH AND HIV/AIDS SERVICES HEALTH SURVEILLANCE ASSISTANTS Given the setting described in the introduction, a major challenge is how Malawi will actually provide FP and HIV/AIDS services to the 85% of its people who live in rural areas. For most of them, getting to a fixed health facility involves travel over a very rough road, typically on the back of a bicycle. Many would have to pay the equivalent of six U.S. dollars to get someone to take them round trip, in a country where more than half of the people live on less than a dollar a day. The CFPHS project was designed to bring family planning and HIV/AIDS close to where people live so they could receive services without facing insuperable barriers. As in many countries, Malawi has trained a variety of community-based workers with a set of basic skills that will be available to people in their village cluster. There are many different names for these workers, but this project only dealt with two types, HSAs and CBDAs. Both types of workers have a long history in Malawi. The HSAs were originally used decades ago as part of epidemic disease control (hence the name), and continue to be under the direct supervision of the District Environmental Health Officer. Over the years, HSAs‘ roles have expanded, as shown in the attached box. Each of these bullets has a long list of specific activities. Family planning, for example, is included in the ―Home Visits‖ section. HSAs are the lowest ranking and most numerous employees of the MoH, comprising 11,000 out of the 33,000 workforce. The Ministry is only able to support 5,000 of these, with the salaries of the remaining being paid by the Global Fund. This has serious implications for long-term sustainability of the overall health care system. HSAs have a standard basic 12-year education, followed by 12-week training in preventive health, family health, and basic management and administration. They were trained to provide injections. Because HSA job responsibilities have been gradually added over the years, there is no policy or guidelines on what they are and what they should be doing. Efforts to establish such a policy are ongoing, and are discussed below in the policy section of the paper. The CFPHS project conducted an additional six days of training in providing DMPA for 361 HSAs and additional training in HIV/AIDS Testing and Counseling for 14. Provision of DMPA by HSAs is welcomed by clients, the community, and fellow health workers at the HC and district health facilities. Services closer to the community are important to improve access. Some women appreciate the confidentiality of DMPA so that neither their husbands nor the community are aware that they are using contraception. Experience to date with the relatively small number of HSAs trained shows they can deliver DMPA safely and with good client satisfaction. HSA Job Description: Inspect facilities within the community using a checklist. Plan and conduct village clinics. Plan and conduct outreach clinics. Plan and conduct home visits using a half page list of duties to be completed at the household level where possible. Maintain a village register and fill in community monitoring tool. 4 MALAWI CFPHS PROJECT: MID-TERM EVALUATION COMMUNITY-BASED DISTRIBUTION AGENTS (CBDAS) CBDAs have been used in a variety of projects over the past twenty years, although there was a hiatus when donor attention was focused on other issues. The AHS, in particular, has continued to support them as a part of their community outreach system. Some of the CBDAs recruited by CFPHS had been active a decade before, and some special individuals had continued to do their work without support. CBDAs typically also have a basic twelve-year education.2 Under the CFPHS, CBDAs were newly selected from the communities they would serve in the traditional authority (TA) areas. Selection was typically done by the communities with leadership from the village headman (or group village headman) who requested appropriate candidates to be recommended by the community. The community then voted or developed a consensus for the one to serve them. CBDAs received two weeks (10 days) of training, except for those previously trained in 1995– 2000, who received only five days. Primary and secondary supervisors received the same two weeks plus a third week to cover their supervision responsibilities. Those CBDAs selected for HTC provision received an additional three weeks (21 days) training, and their supervisors received only one week, limited to their role as supervisors. Some HSA supervisors did not feel that they had adequate training in providing some services, particularly oral contraceptives. CBDA/HTCs also serve a three-month internship in a district hospital to ensure that they have adequate laboratory testing skills. CFPHS uses CBDAs to provide male and female condoms, oral contraceptives, information, education and communication/behavior change communication (IEC/BCC), and to provide HTC services. The CFPHS used two different training contractors for the HTC curriculum. When the MoH shortened the HTC curriculum for budgetary reasons, one contractor eliminated the modules dealing with counseling couples together, and counseling youth on contraceptive options, in favor of retaining role-playing in the curriculum. Not surprisingly, recent follow-up visits to CBDAs show that they are uncomfortable in these areas. The other contractor eliminated role-playing and compressed the five-week curriculum into three weeks. Role-playing is crucial to learning, and needs to be returned to the curriculum where it has been deleted. Another way needs to be found to add the other two modules. CBDAs also recruit clients and distribute contraceptives to family planning acceptors (ref. MoH, Guidelines for CBDAs in Malawi, April 2004). Each CBDA typically serves a population of 1,000 to 1,200 [about 150–200 women of child-bearing age (WCBA), although HSAs that provide DMPA may serve a larger clientele. The primary supervisors of CBDAs are HSAs who meet with them monthly. Their secondary supervisor is often a nurse who is also a FP provider, and who has less frequent contact with CBDAs. Both categories of supervisors have taken the CBDA course. Within the context of this project, 76 CBDAs were trained and approved in HTC. Some of them have had difficulties in completing the ―internship‖ part of the course in laboratory techniques. It is difficult to generalize about the performance of those who successfully completed training because the test kits have been erratically available over the past six months. As noted, a variety of agencies use CBDAs, and there is no standard job description, education, reimbursement, or responsibilities. They are not employees of the MoH, although some have moved on to become HSAs. There are current efforts to regularize the role of the CBDA, but these have a long way to go. 2 In some cases, CBDAs with only 10 years of education were tried in the past, and some of these were recruited under the CFPHS, but they generally have had more difficulties with the curriculum. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 5 Fifty-six percent of the CBDAs are men, indicating a concerted effort at a gender balance. In contrast, a much higher proportion of the HSAs are men. Part of this is due to the fact that more men than women have completed twelve years of education and are available to stay in the hard to reach areas. The supervision for CBDAs by primary and secondary supervisors focuses mainly on the accuracy and completeness of monthly reports. There is little discussion or guidance on providing services or problem-solving. A simple way that this could be done is that the CBDAs who are gathered could be encouraged to share their experiences and problems that they have encountered over the previous month with the other CBDAs. This would not take any more supervisor time, and would allow the supervisor to facilitate, assist, and interact to improve performance in all aspects of their work and to provide encouragement. These elements have been shown to be especially important for maintaining the productivity of volunteer community health workers (CHWs). The supervisors should also be giving updated knowledge and guidance on relevant issues. Refresher training for HSAs should give priority to the short updates HSAs can provide to CBDAs at monthly or quarterly meetings. This is a major missed opportunity. MSH can look to examples from other African countries where MSH colleagues have demonstrated good results. Recommendation 1: Supervisors should be retrained to provide supportive supervision rather than merely collecting information and passing it on. Supportive supervision should focus on performance of CBDAs, including problem-solving for difficult clients, community mobilization, counseling, IP, improving service delivery coverage, and drug logistics/supplies. Supervisors should also provide upgrading of skills and knowledge on a regular basis, particularly at monthly meetings. In addition, some supervisors feel that they do not have sufficient training on the services their CBDAs are providing, such as oral contraception and HTC. This should be addressed through retraining or refresher courses. Recommendation 2: Train HSAs in routine provision of oral contraceptives and HTC. This will enable proper supervision of CBDAs providing these services and will improve access for pill clients closer to the HSA than the CBDA. COMMUNITY-BASED WORK The evaluation team met jointly or separately with HSAs and CBDAs in all seven of the districts visited. The team was very impressed with the seriousness with which HSAs and unpaid CBDA volunteers took on arduous tasks. It found the dirt roads, the 10, 15, 20 kilometers from the CBDA‘s place of work to the HC difficult in the team‘s 4x4 SUVs, and yet they willingly travel such distances to collect their supplies so they can help their communities. The enormous success of this project is solely due to their efforts. They were proud, and justifiably so, of their efforts. Both CBDAs and HSAs live in their communities, and are available to community members throughout the week. What was once considered to be a three-day a week commitment by CBDAs has often become a full-time job for the dedicated and conscientious volunteers. For the methods they provide, these HSAs and CBDAs inform clients about the advantages and disadvantages of the method as trained and use the screening checklist. However, they are not well versed in counseling clients about the most common non-harmful bleeding changes (irregular or prolonged bleeding, or spotting) and amenorrhea that typically are the most important reasons for discontinuing contraception. This is especially true for DMPA. For women with initial bleeding changes, community providers could provide reassurance that these are normal, expected, and will subside over time. Since they lack training in managing side effects, community providers frequently refer women to the HC or hospital for examination for any side effect. Many women cannot make the trek to the fixed facility, and either endure the side effect or 6 MALAWI CFPHS PROJECT: MID-TERM EVALUATION stop using the method. Rather than referral, women typically need reassurance that the side effect is normal and will subside with a few tablets of ibuprofen or a cycle of oral contraceptives for temporary bleeding.3 Adding these interventions to providers‘ therapeutic armamentarium could markedly improve management of simple side effects. In addition, written guidance for pill and DMPA clients, similar to the pill insert that PSI has developed, could be provided as part of counseling to focus on common non-harmful side effects and to reassure the woman and her family. Some of these have already been developed by PSI as package inserts—and about 13,600 have already been distributed by CBDAs—but a full set of client information sheets for all methods should be developed and distributed to acceptors, as well as to those who are interested in a particular method. Recommendation 3: (1) Distribute simple written client information for hormonal contraceptive methods—for both pills and injectable contraceptives—to inform recipients about what they may experience; (2) Develop concise job aids for HSAs, including one-page protocols for temporary management of side effects; and (3) Encourage providers to give reassurance or one cycle of pills or ibuprofen, when appropriate, for bleeding side effects as per MoH DMPA guidelines. (Ref. MoH RH service delivery guidelines.) The team‘s meetings with CBDAs and HSAs, both individually and in groups, show them to be well motivated. When questioned about technical or social aspects of their work, they were able to respond with ease, showing that they had mastered the knowledge base that they were given. The team was unable to view any ―real‖ interactions with clients during the team‘s visits. Obviously, the entourage the team traveled with would have made this difficult. One of the team members did some ―role plays‖ 4 with HTC providers where he played a client who refused to believe that his test was positive. Most of the dozen providers were unable to cope with it, and rapidly turned from sexual history to other explanations, such as blood transfusions and prevention of mother-to-child transmission (PMTCT). Two or three drilled down to get the accurate information. Most supervisors who were presented with the scenario were also unable to provide appropriate advice. The team concludes from this that most HTC providers do not have sufficient experience dealing with people who do not respond in the ―normal way.‖ When the team looked at the training programs, it discovered that one of the main trainers, Malawi Counseling and Resource Mobilization (MACRO), had reduced role-plays in their curriculum, due to shorter courses as directed by the MoH. The second training organization, Lighthouse, cut two modules off the five￾week curriculum (Youth and Couples Counseling) to be able to retain the necessary role-play in the training. Recommendation 4: All HTC provider courses should include a significant amount of formal role-play under the observation of trained staff. The single most important thing that is slowing project implementation is the supply chain for DMPA and for the DETERMINE-HIV® test kits. The demand for both has been higher than the project anticipated, and the supply chain has not always kept up. Some periods of stock￾outs reduced performance, especially for HTC. This is discussed below in the logistics section of this paper. 3 The team recognizes that some side effects can be harbingers of more serious conditions, such as cervical cancer. Nonetheless, if the symptom subsides after the short-term therapy the team is suggesting, it is unlikely to delay treatment significantly. 4 There were cultural and linguistic barriers to carrying this out successfully. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 7 RETENTION OF CBDAS Sustainability and retention of CBDAs as volunteers is a challenge that has been noted in several previous programs. CBDAs for other dedicated services in Malawi have been recruited and trained, only to have dropped out or discontinued working after the project ends. Sustainability of the CBDA program remains a big question for providers in the field and for MSH. Despite hardships, retention has been very good for CBDAs over the life of the project. All districts have an attrition rate of approximately 10%, which is good for a volunteer activity. The sole exception was Kasungu district, where about half the CBDAs have left. While there are a number of purported reasons for this—―he got a better job‖—this needs further exploration to determine the reason for the cluster in one district. MSH reports that it is actively pursuing such an examination. AHS has been successful in retaining CHWs over an extended period, from 1994 to present. Many of the ―non-negotiable requirements for making community-based programs and workers better‖ (refer to the June 2008 Operations and Policy Issues affecting Community-based Distribution of Injectable Contraceptives) are incorporated into the AHS approach to CHWs: Improved supportive supervision and quality of care, Refresher training, Opportunity for advancement, Provision of substantial and consistent resources for performing their work (carry bags for materials, replacement of bicycle every three years—CBDAs provide maintenance themselves), Adequate and continuous supplies, Policy/guidelines/standards/regulation, Community support and involvement (communities determine who will be the CHW in AHS programs and have the option to replace them), and Community respect and recognition—an important element in AHS programs. AHS found that female CHWs had lower drop-out rates than men, in part because men transitioned to outside paid work. AHS replaced the bicycle every three years, during which the CHWs successfully maintained their own bicycles (the old, usually worn-out bicycle was retained by the CHW). This avoided the headaches of managing a large-scale bicycle repair plan. (See Appendix 5). In the high-performing AHS community-based DMPA program in Zomba district, of the initial 178 community DMPA providers, 29 were selected as paid primary supervisors (at 10,000– 12,000 Malawian kwacha/month). Community DMPA in the months March 2009 to February 2010 increased from 250 to 3,500–4,000 clients (injections) per month (total of new, subsequent, and restarting clients). During the same period, DMPA provided through fixed facilities rose modestly from roughly 1,200 to 1,700 clients per month. The overall one-year increase was from about 1,500/month to 5,500/month, approaching a four-fold increase in CYPs. (AHS FP & RH Project documents.) 8 MALAWI CFPHS PROJECT: MID-TERM EVALUATION In the CFPHS project, common requests by CBDAs to improve their performance include: (1) bicycles and spare parts, (2) gum boots, (3) soap, (4) umbrellas, (5) replacement shirts with the CBDA logo, and (6) financial payments. They also request refresher training and the opportunity for career advancement, such as access to HSA training, and HSA positions (and a few have been recruited by the MoH as HSAs). Recommendation 5: CFPHS should incorporate elements of the Adventist Health Services support for CBDAs to improve retention and performance. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 9 III. STRENGTHEN CONTRACEPTIVE DELIVERY SYSTEMS, WHERE VOLUNTARISM AND INFORMED CHOICE ARE ASSURED AND WHERE QUALITY OF SERVICES IS IMPROVED Several components of the CFPHS project are designed to improve FP delivery at the district level and support the work of community services. Examples include data collection systems and the links with the district and health management information systems (HMISs). Strengthening services at the HC and district hospitals focuses on long-term and permanent methods (LTPMs) in partnership with the district health officer (DHO) and the district family planning coordinator. The project works in collaboration with the MoH to implement quality assurance programs and reposition family planning at the district level. SUPPORT FOR FACILITY-BASED FP SERVICES Referrals from CBDAs or HSAs for LTPMs are done using the MoH issued ―health passports‖ that all Malawians carry, and which are used to refer people from one level of the health system to the next. This appears to work reasonably well for clients if they can make the trip to the center. However, like health referral systems worldwide, the community providers infrequently receive information back from the HC or district hospital about the results of the referral. MSH has provided training through the MoH for Jadelle and female sterilization. Some intra￾uterine contraceptive device (IUCD) training was included, but without an adequate case load and experience for providing the method, most providers are no longer confident that they can perform the procedure. The HC providers and supervisors are well trained in the basics of providing Jadelle, though many do not have the confidence or enough refresher experience to insert IUCDs. Jadelle training includes observation and practice with a few insertions, but often not hands-on experience with removal (in large part due to the small case load for removals). Some nurses and clinical officers (COs) reported removing Jadelle and a few Norplant (6 rods). Jadelle removals were apparently accomplished without too much difficulty, though with an increased case load for insertions, removals will also go up in the near and distant (4–5 year) future. Jadelle services suffer from the cumbersome sterilization procedures for trocars. Sterilizers are not available at many HCs, and in some cases gloves were in short supply. Used trocars must be taken to the district hospital for sterilization, often requiring nurses to use their own funds for transport. Ordering disposable trocars (perhaps from the United Nations Fund for Population Activities [UNFPA]) or replacing Jadelle with Implanon, which has its own insertion device, may alleviate some of these problems. Despite the difficulties, the demand for Jadelle is growing rapidly. See the table below, which shows that in Nkhotakota, more Jadelle were inserted in the first quarter of 2010 than in the previous two years combined. The Chinese two rod system, Sino-Plant (Zairn) was recently approved (week of 26–30 April), which may reduce the cost for two-rod systems from $20 to $7. Other donors may use this to meet the burgeoning need, since USAID is prohibited from procuring from this source. Female sterilization, bilateral tubal ligation (BTL), is widely—but not universally—available in district hospitals, where physicians and COs provide the service based on demand and on the availability of time to perform the 20 minute procedure. One team member observed a BTL 10 MALAWI CFPHS PROJECT: MID-TERM EVALUATION procedure in Kasungu District Hospital. It was performed under local anesthesia with the client not sedated, using good sterile technique. The woman walked out of the hospital about 30 minutes after the procedure was performed. Demand is high, and on that particular day, seven women showed up when only two were expected. All women stated that they had their husband‘s permission, which, in some cases, had required extended negotiation. Banja la Mtsogolo (BLM) facilities provide a range of FP/RH services that supplement public services through a network of clinics in Malawi. BLM is part of the Marie Stopes network, and provides a wide variety of reproductive health services, including, IUCDs, Jadelle, tubal ligations, and vasectomies. One BLM facility reported four ―free days‖ in April for public clients who were not charged. They performed 16–21 female sterilization procedures per day. At the HC level, clients for tubal ligation are often referred to BLM for services. In addition to fixed sites, BLM also has mobile teams that will go out to HCs and perform the procedures in the local facilities. BLM has also succeeded in providing IUCD services in outreach sites, though services in public facilities are very low. Some public facilities do not insert a single IUCD in a year or more. BLM sensitizes the community and has well-trained staff to provide the service, which appears to be well-received. The team did not have the opportunity to fully review how and why BLM is achieving success with IUCDs. PAC, including the family planning component, is provided where Jhpiego has trained service providers in manual vacuum aspiration (MVA) and post-abortion FP. Almost all women leave the facility with a method of family planning, usually DMPA, followed by pills or condoms (male or female). This is a good service, though there is a lack of MVA equipment. When the woman is sent to the operating room for D&C, post-abortion family planning is less common. This deserves attention to ensure that these women also receive family planning counseling and services. (ref. International Federation of Gynecologists and Obstetricians (FIGO)/International Confederation of Midwives (ICM)/International Council of Nurses (ICN)/USAID Consensus statement on post￾abortion FP, September 2009). Recommendation 6: Continue to support PAC services including the regular provision of immediate post-abortion family planning. Ensure post-abortion family planning is also provided following D&C. REPOSITIONING FAMILY PLANNING AT THE DISTRICT LEVEL In many countries, FP has been separated from standard maternal and child health programs. Nonetheless, FP remains one of the most important things that can be done to ensure healthy mothers and healthy children. Repositioning family planning involves a combination of interventions to strengthen the role of FP in reducing unintended pregnancies, improving maternal and child health, and promoting family well-being. Repositioning for Malawi should include the well-established concept of HTSP along with the evidence supporting the importance of 3–5 years of birth spacing for improved child and maternal health. This should incorporate concepts of integrating family planning with child and maternal health care and should be introduced to BCC planners, CBDAs and HSAs for community and couple education. At present, HTSP is little emphasized in the CFPHS project—except that the PSI radio drama and some of the pamphlets touch on spacing for improved child and maternal health. In addition, some women receive counseling in health birth spacing during antenatal and post-natal visits at pre- and post-natal HC visits. Recommendation 7: Introduce the HTSP concept and its important implications for child and maternal health at the community and HF level. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 11 QUALITY ASSURANCE, INCLUDING INFECTION PREVENTION MSH quality of care monitoring and supervision from the central offices covers the basics, such as IP, record-keeping, counseling, privacy, return dates, management of side effects, and referrals. However, as described above, there could be better emphasis on the most common side effects and misconceptions that hamper FP acceptance and continuation. MSH DCs need updated training on the elements of quality mentioned for CBDAs and HSAs, in addition to ensuring accuracy of reporting. ENVIRONMENTAL QUALITY To begin with, the team would like to report that the Malawi hospitals are cleaner than almost any other hospitals that they have seen in Africa. There are no piles of discarded waste in the corridors, and men swinging mops are seen constantly. Hospital staff should be congratulated on the cleanliness of their facilities. However, there is always a great deal of concern that medical waste—particularly things such as needles and lancets—are disposed of adequately, and specifically that they are not available to be reused by unscrupulous individuals. The team examined waste disposal systems in all hospitals and HCs it visited. Contaminated waste disposal is generally satisfactory, but not ideal. Burning in a pit or enclosure isolated from the HC or hospital is common. Incinerators were available in a few locations. One new incinerator was not being used in favor of an enclosed area for burning. In two locations in one district, an incinerator was surrounded by a pile of unburned medical waste, and in the other, some unburned needles and syringes were seen. The incinerator in Karonga had been used until the bricks and iron doors and grills had been burned away. The December 2009 earthquake opened cracks in its structure. Fortunately, a new incinerator is already under construction. Recommendation 8: Urge the MoH to provide incinerators where feasible and to monitor disposal of contaminated waste, particularly where incinerators are not yet installed. INVOLVING THE FAITH-BASED COMMUNITY Faith-based organizations need to be better engaged in advocating for family planning and HTSP. Leadership from Qadria Muslim Association of Malawi (QMAM), the Muslim Association of Malawi (MAM) and CHAM, have all expressed disappointment that the CFPHS project has not engaged with them to help disseminate positive statements and positions on family planning. This applies particularly to QMAM and MAM which would gladly have more on Islamic positions on FP published and these materials disseminated to sheiks and imams to be used in educating Muslim communities. The project supported one pamphlet approved by MAM, though both organizations are ready to do much more. As the evaluation team traveled the countryside and read the newspapers, it was struck by the numbers of new Islamic HCs and hospitals. During the team‘s visit, for example, Pakistan announced that they would build a new hospital. While some dialogue has begun, very much more work needs to be done to ensure that this large community—which is largely open to birth spacing— becomes involved. Catholic leadership has not been engaged, though Catechists have been trained at the district level as trainers for the standard days method (SDM) using CycleBeads as a method of natural family planning. This is also called a fertility awareness-based method (FAM). CFPHS and the SDM trainers should inform the bishops in districts where SDM has been introduced. Based on the team‘s discussions with several priests, a bishop, trainers, providers and couples, there appears to be good receptivity for SDM. However the lack of CycleBeads 12 MALAWI CFPHS PROJECT: MID-TERM EVALUATION constrains expansion. Even with the additional 18,000 sets of CycleBeads on order (above the initial 1,000) there is likely to be unmet need. (See Appendix 5). INFORMED CHOICE While, as noted above, the team did not actually observe provider-client interactions, it believes from its conversations with clients and providers that full information is provided on different methods, and that there is no coercion or remuneration for increasing client load. The team saw the ―Tiahrt Charts,‖ both in Chichewa and English, in the health facilities, and even in the CBDA‘s humble dwellings. In the current situation, where demand is high and supply is low, there is no need to resort to such methods to gain valued and satisfied clients. The only cadre of providers not providing full information on different methods is those providing SDM. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 13 IV. BEHAVIOR CHANGE COMMUNICATION AND CHANGED BEHAVIORS This section presents information on the BCC component of the project, beginning with the strategy and core messages. Next is a discussion of each of the BCC approaches used, with the team‘s observations, conclusions, and recommendations. Thereafter, the concerns and misconceptions about FP in the wider community are presented. The impact of the CFPHS in increasing access to FP services and HCT is discussed together with examples of the team‘s observations, which include integration of FP and HIV prevention at the community level. It should be noted that in Malawi, IEC is considered a sub-component of BCC when the IEC is aimed at changing behaviors. BEHAVIOR CHANGE COMMUNICATION STRATEGY The CFPHS Implementation Plan, developed in the initial phase of the project, sets out a BCC strategy. The strategy targets women of reproductive age and their partners, as a couple. The secondary target was health providers, village headmen, and other community gatekeepers, that is, influential persons. The strategy involved addressing the following: Correct use of modern contraceptive methods, Decrease discontinuation of modern FP methods, Increase discussion of FP methods in a family setting, and Reduce acceptability of GBV. A multi-dimensional approach to demand creation leading to behavior change was planned to create a positive social context that promoted and supported individual behavior change related to FP and HIV prevention, and reduced acceptability of GBV. Core messages were the benefits from FP, HIV/AIDS testing and counseling, and inter-spousal communication. In addition, information on modern FP methods was provided to increase awareness about options. The communications strategy was refined by PSI in August 2008. Couples were defined as those who are married or in a stable relationship. The messages for couples, men, and women centered on stressing that ―Family is two‖ and that the couple should talk to each other and decide together about which FP method is appropriate for them. They should also know their HIV status so they can plan for the future, and use condoms for the health of the whole family. Promotion of spousal communication aims at encouraging partners, fearful of raising the subject with their spouse, to begin such a conversation, and reducing the risk of domestic violence when partners act on their own. TRAINING The project training of CBDAs included GBV. CFPHS also provided limited support to a PSI social marketing project. These activities were aimed at influencing positive changes in the social landscape and individual behavior. During the first six months of CFPHS, a CBDA trainers guide, which included interactive exercises, and a module to be used by CBDAs in community work were developed on GBV. Thereafter, the module and guide underwent pre-tests and reviews prior to finalization. The trainers guide assumed that more time would be available in the initial training of CBDAs than was actually possible during the training period. The module for CBDAs to use in 14 MALAWI CFPHS PROJECT: MID-TERM EVALUATION community sensitization included a definition of GBV, identification of common forms and causes of GBV, and the linkage between GBV and HIV/AIDS, as well as between GBV and FP. The module is deficient, since it does not include suggestions on ways to discuss such topics in an interactive manner with adults and youth. The team found that CBDAs feel comfortable stressing the importance of couples discussing family planning and HTC as a way to reduce the potential for domestic violence. In general, however, CBDAs appear to be uncomfortable talking about GBV and providing advice to persons involved in a domestic violence situation. CFPHS provided limited support for PSI to provide a series of one-day training sessions for private practitioners on different aspects of family planning. Since relatively few private sector outlets exist in the project districts, the trainings were offered to all districts with private clinics, pharmacies, and drug stores. Fifteen private providers from six of the project's eight districts attended one or more training sessions. The number of provider contacts (double-counting across the sessions) totaled 292. Related to the training, CFPHS funded two full-time staff for the PSI￾funded social marketing project focused on female condoms, oral contraceptives, and injectables. Recommendation 9: Greater attention should be given to increasing CBDAs‘ skills and confidence in advising and counseling on GBV related to FP and HIV/AIDS. This could be done in CBDAs‘ monthly meetings with the special attendance of the District FP Coordinator, through CBDAs sharing of their experiences in monthly meetings, and in the MSH-planned CBDA refresher training course. OPEN DAYS Open Days are organized events held in communities to create awareness, stimulate contemplation, provide information, improve the social context for individual behavior change, and create demand for FP and HTC services. Beyond MoH Open Days supported by the project, the evaluation team found that the BCC activities had stimulated community-initiated Open Days. From 2008 until August 2009, MSH facilitated Open Days in all eight districts. These gatherings involved speakers, drama skits, an interactive element, and availability of HTC services. An estimated 8,100 persons attended the 10 Open Days coordinated by MSH. Thereafter, PSI took the lead in coordinating these events that require lots of organizational and logistical preparation. Since then, the plan has been to hold an Open Day in each district every two months. Planning is done jointly by PSI, the District Health Information/BCC officer, and other key stakeholders. The objective continues to be demand creation through emphasizing the benefits from use of modern FP methods and acquainting attendees with the services provided by CBDAs and where different FP methods are offered in their area. These topics are covered in talks by MoH officers, radio￾listening club members, and CBDAs. Their messages are reinforced through songs, the PSI￾scripted drama, and display of IEC materials by CBDAs and HSAs who respond to questions. HTC services are also offered at these gatherings. The Open Day interactive segment involves asking attendees to respond to six to eight questions and the attendees then deciding who responded best and should be rewarded a prize (T-shirt or ballpoint pen). This approach provides an incentive to listen to the talks and drama presentation. PSI records show that from September 2009 through March 2010, an estimated 40,186 people attended the Open Days. Approximately 46% of the attendees were males, indicating these events have been successful in reaching males as well as females. The data for each Open Day are collected by listening club members. According to an established pattern, a crowd would be divided into sections and each member would be given a section to count. The counters rotate and MALAWI CFPHS PROJECT: MID-TERM EVALUATION 15 recount another person‘s section and then the numbers are compared before compiling the final figures to ensure that the crowd count is accurate. Of course, they are unable to determine whether people present at any one event had previously attended a different Open Day or other event. During Open Days, 1072 people took advantage of HTC services, even though there is a general preference for door-to-door services. Gender was only provided for 974 of these and 42% were men. Overall, 8.8% tested positive. In the team‘s focus group discussions (FGDs), one group of women said that they liked the MoH Open Days more than the separate drama performances and listening to the radio drama. They felt that they learned the latest information, and that this came from trusted sources. At the same time, the women acknowledged the work of their CBDA, but said they want to be taught more. The second type of Open Day emanates from community initiative. A village headman organizes this type of event, sometimes with the involvement of members of a radio listening club. The event normally involves a speaker from a local HC and the local CBDA, together with a drama skit and songs. This type of Open Day exemplifies traditional community leaders and members taking ownership of promotion of FP, reduced GBV, and HIV/AIDS prevention. RADIO DRAMA A specially-created radio drama about FP, GBV, and, to some extent, HIV/AIDS was a major element in the BCC project component. Efforts began in late 2008 with a workshop to develop the storyline for the radio script and selection of two scriptwriters. Upon finalization of four radio scripts, pre-tests of the episodes were held in two districts. Thereafter, participants in a three-day workshop reviewed the four scripts and results from the pre-tests from a technical perspective. The participants included the Family Planning Association of Malawi, the MoH Health Education Unit, a GBV consultant, a DC, and the script writers. The feedback informed modifications and the development of the other episodes. The launch of the radio drama involved promotional materials (discussed below) delivered to the eight project districts and radio promotional advertisements. Entitled Zina Umanena (―Some Things Are Worth Talking About‖ or ―Let‘s Communicate‖) the weekly radio drama program consisted of a series of 26 episodes. A 27th episode was added to address comments and questions about the topics covered by the drama. The program was aired weekly on the main radio station at 3 p.m., a time convenient for listening club attendance. It was also aired during prime time on Saturday evenings on a local station and at prime time on Sunday evenings on a national radio station. Due to unforeseen delays in finalizing Zina Umanena, it began airing in mid-August 2009 and continued through mid-February 2010. Re-airing is temporarily halted since PSI wants to modify the program to incorporate more on HIV/AIDS and male involvement. PSI expects to receive funds for carrying out the modifications through the USAID MCHIP. It anticipates that airing will begin in August 2010. Since the program was broadcast on popular radio stations, an unknown number of persons, in addition to participants in the PSI-trained listening clubs (discussed below), heard one or more radio drama episodes. However, they did not have the benefit of a more interactive approach. Financial constraints precluded any effort to assess the extent to which people beyond the listening clubs heard the program. Most of the community members with whom the team spoke are aware of the Zina Umanena radio program, and some had listened to it. Lack of a radio at home and lack of batteries for a 16 MALAWI CFPHS PROJECT: MID-TERM EVALUATION radio were given as reasons for not listening. Those who had listened felt that the drama characters and script were realistic. Recommendation 10: To help keep and increase the audience, the radio program ought to include additional time, e.g., 15 minutes, for (a) an engaging health professional to respond to questions sent by listeners, (b) reading letters from listeners, and (c) airing recordings of songs related to FP, GBV and HIV/AIDS created and performed by drama troupes and listening clubs. RADIO LISTENING CLUBS Organization and Attendance The project's four BCC-experienced listening club officers are each responsible for two districts. On average, each listening club officer's activities have centered in 2.25 TAs (sub-district administrative areas) per district, with a range of one to three TAs per officer. This was done for ease of supervision and reporting, but also to concentrate both the listening club activities and drama activities in the same TA in order to have an effect on behavior change. The intent was to scale up to other TAs in future project years. A collaborative process involving knowledgeable persons led to the identification of existing CBOs, particularly radio listening groups, to approach about being one of the 25 CFPHS radio listening clubs per district. The team found slight variations among the districts in the core purpose of the CBO selected. Some organizations were already engaged in health-related matters, others were organized around listening to radio programs (such as on agriculture), and still others had formed for another purpose. The CBO leader and a literate person specifically chosen to be the CFPHS radio listening club leader, together with their respective CBDA, received special training. Three training sessions of three days each were held in each project district. District FP Coordinators were involved in the training that included use of the Discussion Guide, which was developed by PSI for the listening clubs and pre-tested. For each episode, the Guide has specific questions and suggested activities, such as role-playing and sharing what they learned. The training also covered skills in leading discussions, guidance on use of the wind-up/solar power radios each group received, and instruction on the completion of reports on each listening session. The listening club reporting formats on attendance at each episode track new and continuing participants. In total, 23,540 individuals attended one or more times, and two-thirds of them were women. The team commends the Mission and PSI on this careful reporting that avoids double￾counting. Based on the monthly attendance records by district, the team estimates that each of the 200 listening clubs averaged 25 persons per episode for most months, but averaged 20 during the four low-attendance months. Observations The team‘s FGDs with members of listening clubs elicited a range of responses about the most important information gained from the radio program and their subsequent discussions. The responses given reflect the information that resonated the most at a personal level. The following provide examples of the responses of both men and women, unless otherwise noted. Spacing of children is important for the health of the woman and her children. If a man does not allow his wife to practice family planning, that is domestic violence. ―We are enlightened about domestic violence‖ (male respondent). MALAWI CFPHS PROJECT: MID-TERM EVALUATION 17 ―I initially thought that FP was a subject for women, but now I realize it‘s an issue for both partners‖ (male respondent). Don't keep having children because you want a male child or because of peer pressure; female children are as important as male children. ―Before, I was afraid to talk to my wife about use of modern FP method, but now I feel free to approach her and agree about use of a modern method‖ (Muslim man). ―We have learned the benefits of FP and right number of well-spaced children‖ (Muslim man). A significant aspect of the listening clubs is that participants are encouraged to disseminate information within their communities. They share what they have learned, thereby serving as peer motivators. Listening clubs based on CBOs formed around HIV/AIDS incorporate FP into their HIV prevention activities. Also, as the radio drama concluded, the listening club officers encouraged club members to be active in locally organized Open Days (the second type discussed above). At these gatherings, the clubs may perform special FP songs and skits that they have created. The following are examples of CBOs involved in listening clubs and their club activities. A Muslim social affairs CBO‘s listening club members divide into small groups to visit and talk with members of the community at their homes and where men and women gather. Working in collaboration with their local mosque leader, these members hold open-air functions, putting up posters and distributing pamphlets on FP, which they obtain from the nearest health facility. A religious leader speaks on these occasions about the relationship of family planning and the Koran. In addition, club members work ―hand-in-hand‖ with the marriage counselor at the mosque to encourage people to go for HTC. A CBO was formed in 2004 around HIV/AIDS work involving out-of-school youth, and widows and orphans affected by HIV. It includes some people living with HIV/AIDS (PLWHA). The club integrates messages about HIV and FP, and an HIV+ man provides a ―testimony‖ at local open days and meetings organized through the respective village headman. Another CBO participating in the listening club program was organized to help PLWHA and currently consists of 80 members of whom 75% are on antiretroviral drugs (ARVs). The listening club members talk to individuals on a one-to-one basis and hold meetings in their community (a large peri-urban market center) and other near-by sites. They encourage their PLWHA members to use condoms to avoid spreading the virus, use a modern FP method, and carefully consider issues if they want to have more children. Beyond forming the basis of a radio-listening group, this CBO also benefited from the CFPHS￾provided opportunity to send two to three members for special training on FP. The purpose of this activity was to better integrate FP into HIV/AIDS work in communities. Ten CBOs sent two-to￾three members each for the training conducted by an FP specialist in their district. As a result, the two men and one woman trained report that during small group sessions in their community, they are better able to talk about FP methods, including emphasizing that condoms are the only method that protects against HIV transmission. The group regrets, however, that it does not have condoms to distribute. This group and others mentioned that they have no T-shirts or other means of identification when they do outreach work in their communities. In sum, the team notes that the approach used towards listening clubs has been successful in educating the participants about FP, HTC, and GBV. Through encouraging them to share what they have learned with others beyond their immediate households, they served as peer motivators. In addition, the team found that by selecting CBOs that were formed around HIV/AIDS, the project has integrated FP and HIV/AIDS prevention promotion at the community level. Also, by 18 MALAWI CFPHS PROJECT: MID-TERM EVALUATION selecting a Muslim-based social organization, the project has led to the group's active engagement in promoting HTC and FP among married couples. Hence, the team concludes that project investment in listening clubs has a positive, planned spill-over effect in communities. Recommendation 11: For the re-airing of the radio drama, the listening club component should be scaled up to include new TAs and some new sub-areas in the current BCC-focused TAs. PSI should continue to maintain contact with the current clubs, and to provide members who are active in community outreach with identification badges. COMMUNITY DRAMA PSI was responsible for the development of a 20-minute drama script based on the storyline of the radio drama, which used some of the same characters. The listening club officers, who are also responsible for community drama, identified existing drama groups working in the BCC focal TAs. They were assisted by the MSH DC and official representatives from the MoH in each district. Three drama troupes in each district were recruited, totaling 24 troupes. Each troupe, consisting of eight-to-twelve members—together with its respective HSAs—received two days of training by members of a well-known drama troupe. The training covered the script and acting, as well as the interactive portion that involves the audience asking questions. It also included instructions on completion of their monthly report on attendance at their performances. HSAs in the catchment area of the drama troupes attended, since they were expected to attend the performances and respond to questions that the troupe could not answer. The memorandum of understanding between PSI and each troupe specified that it was to perform two shows a month from October 2009 through March 2010. Each troupe was paid 5,000 kwacha a month to cover transport costs. The team found that the decision-making process related to sites for the two performances varied. In some districts, site selection involved a dialogue between the troupe, the listening club officer, and health officers in their area. In other districts, the troupes were more active in site selection. Performances were held on the weekend, when more people are likely to attend. For a performance in Karonga, the chief invited 20 headmen to attend so that they could learn more. Each performance involves an interactive element, normally at the end of the drama, where participants are encouraged to make comments and ask questions. In most districts, especially during the initial months, an HSA or CBDA was present to respond to any questions that the troupe could not answer. During this process, the troupe gained knowledge and hence increased its capacity to respond. In one district, a CBDA became a member of each troupe, often serving as the drama narrator. One drama troupe the team met uses song, dance, and drums to energize its audience and to attract more people to attend. This troupe appears to be exceptionally good, and it was the only group met that had uniforms procured through their own efforts. Members of other troupes the team met raised the issue of not having T-shirts or uniforms by which they can be identified. Geographic mobility is an issue. Among the drama groups the team met, the furthest distance two of them had traveled ranged from 4–8 kilometers. One group, however, reported walking a day to reach a site and then having to find lodging that evening and return home the next day. Both lack of access to transport and the cost of bicycle taxis limit the geographic range covered by the drama troupes. Although they receive a small traveling allowance, the troupes vary in the way in which they use it: transport, food on performance days, and distribution among members. The monthly reports of the 24 drama troupes indicate that they reached 52,662 individuals—66% of them female—during their six-month agreement with PSI. The data cover the two performances per month, in addition to some troupes reporting on additional performances. The MALAWI CFPHS PROJECT: MID-TERM EVALUATION 19 troupes were instructed to exclude Open Day performances to avoid double-counting individuals who had seen the performance in their own community. Drama performances are extremely popular. The opportunity to ask questions and receive answers was particularly appreciated. The team‘s finding is the same as the one reached in a PSI internal review. The team noted, however, that due to transportation challenges, the geographic range that a troupe can reasonably be expected to travel is rather limited, which limits greater access of communities to this successful behavior-change mode of communication. Recommendation 12: To expand access to the PSI-scripted drama performances, CFPHS should support the scaling up of this successful BCC approach by training and funding more drama troupes in new TAs and some new sub-areas in the current BCC-focused TAs. The drama troupes should be provided with Zina Umanena T-shirts or with some other form of identification. PRINT MATERIALS PSI developed high-quality brochures, posters, and informational sheets to accompany the launch of the radio drama. Designed to inform readers as well as advertise the radio drama, six brochures were produced. Each contains six questions, posing some common concerns and misconceptions, followed by answers. The themes are: Planning for a Family and the Risks of HIV, Gender-Based Violence and Family Planning, Men‘s Involvement in Family Planning, Couple Communication on Family Planning, Church and Family Planning (with a statement thanking the Malawian Council of Churches for the endorsement of information in the brochure), and Family Planning in Respect of Islam (with a statement that the Muslim Association of Malawi endorsed the information in the brochure). Using the same graphics as the first four items above, posters were produced on these themes, as well as one specifically advertising Zina Umanena. In addition, a pamphlet on FP methods was developed based on the family planning training (FP TR) chart. Also, an ―insert‖ with instructions about taking oral contraceptives was developed. The number of each of the leaflets and the inserts distributed in each project-targeted district was based on its number of CBDAs. In addition, each district received 175 copies of each type of poster. When visiting seven of the eight targeted districts, the team saw posters displayed at shops near health facilities, at venues of some listening clubs, and at the homes of some CBDAs. FGD participants were often familiar with some of the brochures, and a few persons carried them to the FGD. In most cases, the CBDAs no longer had brochures to distribute. The oral contraceptive ―insert‖ was found to be used by CBDAs in one area. The secondary supervisor, who was also a CBDA trainer, requested that the insert be made available to every couple considering the pill. FAMILY PLANNING CONCERNS AND MISCONCEPTIONS In its FGDs, the team frequently asked the participants about the comments and questions they receive from community members. The team often heard that husbands don‘t like using condoms nor do they like their wives to use female condoms because they do not permit body contact. Also, men sometimes say that a woman is ―not sweet‖ if she uses a contraceptive method. 20 MALAWI CFPHS PROJECT: MID-TERM EVALUATION The responses the team received were similar across the eight districts, and indicate a lack of information and misconceptions. Some of these are presented below. Questions: How can a woman who is HIV+ deliver an infant who is HIV-negative? How can one spouse be HIV+ and the other HIV-negative? If a woman uses a long-term method (LTM), if she remarries can she have children? Is FP for everybody? If a modern FP method is used, can one have intercourse in the usual way? Misconceptions: Condoms Contain micro-organisms that are harmful. Oral Contraceptives Accumulate in the stomach and cause cancer, Accumulate in the stomach and cause frequent bleeding, and Cause irregular bleeding or lighter periods, which is harmful. Injectables Cause epilepsy and heavy bleeding, Lead to aching and pain on that side of the body, Cause cancer of the uterus, and Cause infertility, especially related to amenorrhea. General Modern FP methods lead to a woman becoming sterile, Can cause early death, Modern FP methods cause a man to become weak, and Women will be promiscuous if they use FP; without FP if a promiscuous women has a child, one can tell from the features of the child if the child‘s father is the woman's husband. Recommendation 13: Adopt a multi-prong approach to reduce concerns and misconceptions about FP: (1) funds should be made available for printing more copies of the brochures, with targeted distribution; (2) during their monthly meetings, CBDAs should be encouraged to share experiences about addressing topics in the brochures in group settings and individual talks, and how they decide to whom they distribute the additional pamphlets they receive; and (3) CBDAs should receive a steady supply of the insert on oral contraceptives to distribute to potential and new users. In addition, CBDAs—as well as listening clubs and CBOs active in FP outreach activities—should receive an ample supply of the brochure on FP methods. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 21 CHANGED BEHAVIORS Family Planning In spite of concerns and misconceptions in the wider community, CFPHS has led to significant increases in the use of modern FP methods. BCC combined with CBDA group meetings and home visits, and increased access of communities, have contributed to 246,813 CYPs. The team‘s FGDs with listening club members, other women users, and men revealed reasons for individual behavior change, the process, and choice of contraceptive method. Men in particular are concerned about the economic and financial implications of supporting a large number of children. Both men and women have a better understanding of the importance of spacing children, for the health of the woman and child. When the spouse is not home during a CBDA door-to-door visit, the women normally share the information with their husbands afterwards. For some couples, the conversation starts with whether they should use a modern method. Then it proceeds into a discussion about what method should be used. In a community 23 kilometers from a health facility and without access to a HSA provider of DMPA, the team found that the decision to use an oral contraceptive was made because they are accessible through the CBDA who resides in their village. In a couple of cases, the women said that their husbands wanted them to use DMPA due to concern that the woman might forget to take an oral contraceptive pill each day. The information the team gathered from women reveals no clear pattern on the method chosen and the number of children they have. The data do indicate that women start having children at an early age and/or the children are not adequately spaced. In the vast majority of cases, the team learned that the women started using a modern FP method as a result of the project. It also found that a number of women switched methods because they were experiencing heavy bleeding from DMPA. Appendix 4, Table 3 provides examples of the data collected on women‘s age, number of children, and current FP method, together with comments. HIV/AIDS Prevention and HTC HIV prevention messages and HTC have been integrated into FP BCC at the community level. To date, the project has reached 808,606 individuals through community outreach that promotes HIV/AIDS prevention through behavior change beyond abstinence and/or being faithful. The CFPHS project has also resulted in 93,533 persons counseled and tested for HIV who received their test results. As discussed previously, Open Day events serve as a venue for HTC and listening club members serve as peer motivators of FP, HIV prevention, and HTC. Below are examples of ways the project has had a positive influence on the willingness of people to be tested for HIV. Most of the community members the team met reported that they have received HTC. In the northern zone, they were aware that they should be tested annually. Particularly in the southern zone the team heard from some that they had been tested many times. In areas without a CBDA trained in HTC, the listening club members and some women FP users reported going as a group and their spouses going independently. A smaller number reported going with their spouses or having been tested as part of their antenatal care. A Muslim sheik wore his ―Know Your Status‖ T-shirt to the team‘s FGD, where most of the married men reported that they and their spouses 22 MALAWI CFPHS PROJECT: MID-TERM EVALUATION received door-to-door HTC from their CBDA. CBDAs not yet trained in HTC told the team of their wish to be trained to provide the service to meet the demand in their catchment area.5 The team learned from those who know someone who is HIV+, that the person is accepted in the community. In the above FGD, the Muslim religious leader said that he reassures the person and encourages him to keep healthy and continue to attend prayer meetings. A village headman told of his Roman Catholic Church welcoming two HIV+ individuals into the church and these individuals are sometimes used as models to encourage ―Know Your Status.‖ However, in many cases the people the team met did not know an HIV+ person, suggesting that some people living with AIDS may fear stigma if they make their status public. Changed Social Context Progress is being made in changing the social context which influences personal behaviors related to modern FP use and HIV prevention. ―We had heard about such methods before, but now with people in our community talking about it, it becomes more concrete and real to us,‖ explained one woman. ―These positive behaviors should become the norm,‖ remarked a traditional leader. Male members of a listening group composed largely of Muslims told us: Now we are able to talk about HTC in our Muslim community. When a woman gets pregnant, nowadays she goes for HTC. We have more information about FP, so we can talk with others about it. This is a significant finding since research at the congregational level in Malawi found that although nearly all of the Muslim religious leaders interviewed approved of modern FP, use of a method was low among Muslim women (Yeatman and Trinitapoli, 2007). Another example of changing social norms is that some traditional leaders incorporate talks by CBDAs into their meetings with community members on other issues, and host local Open Days and drama troupes. In addition, men and women in the team‘s FGDs were open to discussion of ―bedroom‖ issues. In contrast, a study conducted for the project in the first half of 2008 found that ―people feel most comfortable to discuss FP with health workers because they are professional (trusted) and keep secrets‖ (Mkandawire, 2008). In conclusion, the team found that the project has progressed well in its efforts to change social norms and individual behaviors. At the community level, there have been advances in the integration of FP and HIV/AIDS messages. The information the team gathered on use of FP indicates that greater attention should be given to help ensure that potential users of a particular method are aware of some of the side effects that may temporarily occur. Recommendation 14: Funds should be allocated for the development and dissemination of leaflets on the range of popular FP methods, especially injectables, which include information about possible temporary side effects and actions to take. These might be patterned after the ―insert‖ on oral contraceptives. The evaluation team was asked to look at integration of sexually transmitted infections (STIs) into other basic health services. Originally, the project intended to integrate HIV/AIDS and STI prevention into FP/RH services at the district and community level. As STI prevention is part of the basic health package, coursework was prepared, and it was part of the CBDA training 5 Door-to-door HTC tends to be more broadly accepted in Malawi than other testing options because it meets three criteria: it is convenient, confidential, and rapid blood tests are credible (Angotti, N.et al. 2009). MALAWI CFPHS PROJECT: MID-TERM EVALUATION 23 curriculum. The team held many discussions with CBDAs and HSAs on their daily work and the information that they passed on to their clients, but none of these reported working or communicating STI issues. The team did not specifically investigate whether it was integrated at the HC and district hospital level, but, in the rare cases when it was discussed, services were provided as part of the HIV/AIDS services, separate from integrated health care. The Mission informed the team that this element of the project had never been fully implemented. Recommendation 15: The Mission, in consultation with the MoH, should determine the importance of integrating STI services with integrated health care and provide additional resources for these services if it is appropriate. 24 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 25 V. PROJECT IMPACT The project impact is shown on tables one and two in Appendix 4. In brief, the project either has already met or surpassed all output objectives, or is likely to do so in the remaining 18 months. The outcome measures: CPR, TFR, Reduced Unmet Need, etc., will be measured in the 2010 MDHS, which has already begun work. Illustrative accomplishments in the targeted districts are: CYPs 247,000 FP/RH Counseling Visits 730,000 Service Delivery Points 1,514 People Counseled & Tested for HIV 93,000 The team went back and examined DHO pharmacy records to see whether there was a broader effect on the distribution of contraceptives than before. In general, the team found that two to three times as many contraceptives were being distributed in 2009, particularly DMPA, than in 2007. (See the table below from Nkotakota). The increase of Determine-HIV ® use was more on the order of three to four times what it had been before. The team feels that this shows an incredible spread effect of the project. That is, not only were the trained CBDAs and HSAs providing more goods and services, but people were demanding far more from the pre-existing fixed facilities. The team believes that the CFPHS impact will be far greater than is illustrated here, primarily because the project managers did not go beyond the USAID-mandated indicators, and did not disaggregate their data by district. These differences will become clear in the MDHS if it presents disaggregated district-level data. 26 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 27 VI. HEALTH SECTOR CAPACITY STRENGTHENED One of the main goals of the USAID portfolio—going beyond the scope of the CFPHS to include all other elements of their health activities—is to strengthen the MoH. One example of this is the Logistics program, with the DELIVER project working closely with CFPHS to improve pharmaceutical availability at the district, HC, and community level. A second activity is to integrate basic health services, with the MSH BASICS and CFPHS projects working together to establish integrated services, particularly at the district level. The policy section of CFPHS worked with the relevant bodies to issue policies and guidelines that would enable the expansion of community level services. These issues and their impact on providing family planning and HIV/AIDS services to the people of Malawi are discussed below. POLICY DIALOGUE Policy Decision to Allow Non-professionals to Administer DMPA at the Community Level in March 2008 A debate regarding whether to allow lower-level health workers—specifically HSAs—to administer DMPA has been ongoing for years in Malawi. As many as 10 years ago, DHOs in several districts began allowing HSAs to administer DMPA outside of the HC context and directly in the community. Many of these districts had a large number of Catholic facilities that sometimes were the only HC in a given area and obviously did not provide FP services. The provision of DMPA by HSAs in these districts was not openly discussed at the national level but was openly acknowledged in districts like Zomba that had passionately activist DHOs and Zonal Officers promoting family planning access for rural women. Nurses and doctors (nurses in particular) were very reluctant to allow lower-level non-medical professionals with only a 12- week training program to administer injections. Yet, many advocates of HSA provision of DMPA argued that HSAs already administer immunizations to children under five and tetanus shots to pregnant women at the community level. On March 14, 2008, the MoH Senior Management Committee agreed by consensus to allow HSAs to administer DMPA at the community level, with the understanding that the Ministry would first pilot the approach in several districts. Dr. Chisale Mhango, a USAID-supported technical advisor to the MoH Reproductive Health Unit played a key role in helping the Ministry make its policy decision. In early 2008, several months after the CFPHS project started, Dr. Mhango met with key members of the CFPHS project team to gather evidence about the efficacy and safety of providing DMPA at the community level, based on evidence and other country experience. Some Senior Management Committee members strongly opposed allowing HSAs to provide DMPA. Rebuttals to these arguments cited the good work that HSAs are currently doing, particularly in providing immunizations without incident. Regarding the safety of DMPA, proponents stated that the Malawian government had already approved DMPA and therefore the MoH Senior Management Committee did not need to consider this issue. At the end of the March meeting, the group reached a consensus in favor of allowing HSAs to start administering injectable contraceptives, with strong guidelines and provisions for training and supervision. The group agreed that implementation should not happen all at once but rather begin with eight pilot MSH project districts. 28 MALAWI CFPHS PROJECT: MID-TERM EVALUATION As of this meeting, Malawi was the first country in Sub-Saharan Africa to make the policy decision to implement community-based distribution/provision of DMPA on such a wide scale: in eight districts (nine, including Zomba) that would cover approximately one-third of the country. Other countries that had piloted community-based distribution (CBD) of DMPA at that time had only attempted implementation in 2–3 districts. Development of CBD Provision of DMPA Guidelines The next critical step that the GoM took was to develop national guidelines for CBD of DMPA. The CFPHS project used findings from the June 2008 study tour to Madagascar (financed by the CFPHS project and hosted by Family Health International [FHI]) and from previous USAID￾funded analysis in Malawi to facilitate a working group to develop national guidelines. The guidelines describe program provisions for training, integration of FP and HIV services, service delivery, monitoring and supervision, quality assurance, and logistics management. These guidelines were approved by the Ministry in August 2008 and have been disseminated formally to all nine6 districts currently implementing the CBD of DMPA program using HSAs. Malawi was also the first country in sub-Saharan Africa to develop national guidelines for CBD of DMPA. Rwanda developed national guidelines for implementing CBD of DMPA in September 2009, and heavily relied on Malawi‘s guidelines to develop its own policy documents. Inclusion of CBD of DMPA and Social Marketing Policy Language in Malawi’s Sexual and Reproductive Health (SRH) Policy in March 2009 The CFPHS team worked very closely with Dr. Mhango and his colleagues at the Reproductive Health Unit to provide evidence in support of including language allowing for community-based provision of DMPA by non-medical professionals and social marketing of contraceptives. This effort was a more lengthy process than might have been expected. The group finally agreed to leave in the added language allowing CBD of DMPA and social marketing to bolster the family planning program and increase access to services. Inclusion of these two items in the national SRH policy also assures sustainability and scale-up of the CBD of DMPA program and will make implementation of a national social marketing program more feasible. CENTRAL MEDICAL STORES & PHARMACEUTICAL LOGISTICS In principle, all pharmaceuticals used in the public sector—which, in this case, relates to all project elements except the social marketing component managed by PSI—pass through the CMS, and are distributed through a complicated chain that looks something like that in the graph on the next page. At any step along the way, supplies can be interrupted, and stock-outs may occur at each subsequently lower level. 6 As noted elsewhere, Zomba District also has an HSA based DMPA program. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 29 Figure 1: Pharmaceutical Logistics Flow RMS requisition pharmaceuticals according to a Pharmaceutical Logistics Management System software. This software—also in use at district hospitals—is intended to ensure that there is a three months‘ supply of drugs available at each level so that, even if demand tripled, there would be sufficient stock on hand. CMS procurement follows GoM Procurement Regulations, which requires a lengthy process, made even slower by SWAp procedures and a lack of funding to procure full needs. While procurement is supposed to occur annually, with a two-year award to drug suppliers, in fact it does not happen that way. The 2007 tender was first analyzed in October 2009 and the first partial awards were made in April 2010. In general, SWAp funds, which come from a pool established by most donors, are used for the procurement,7 although some GoM funds are also used. Any 7 ―An unintended consequence of a SWAp is drug stockouts. The stockouts can occur as a result of lengthy, structured procurement procedures and process reviews required under the SWAp agreement, and government personnel‘s unfamiliarity with the new procedures.‖ … While family planning is a priority within SWAp, so are numerous other programs (related to vaccines, malaria, tuberculosis, HIV/AIDS, etc.) … While there is a method to ringfence—i.e., earmark—the budget for drugs, there is no mechanism to further safeguard FP products in particular. (Robertson et al., 2008). Central Medical Stores Procurement Government of Malawi funds SWAp Funds Donor Provided Commodities Central Medical Stores Warehouses Regional Medical Stores District Hospital Pharmacies District Health Centers Supervisors CBDAs & HSAs 30 MALAWI CFPHS PROJECT: MID-TERM EVALUATION drug deliveries are stored in 11 CMS warehouses, where they may or may not be tracked. For example, in a recent case, the USAID Mission Director and Principal Secretary made an unannounced visit to the warehouses, and found ten million condoms that were not recorded in the system. The DHMT uses the supply chain manager software to generate the first version of their monthly requisition. The team‘s first priority is not necessarily preventive medicine, particularly since both DMPA and Jadelle are among the highest cost items—even when subsidized.8 As Robertson et al. (2008) predicted: ―208 [was] the first year that districts need[ed] to purchase injectable contraceptives directly from the CMS. The primary concern is that—faced with stockouts of other essential drugs and commodities—DHOs might decide not to purchase injectable contraceptives.‖ Despite the fact that most contraceptives are provided by donors, and thus only carry a 5% handling charge, typically contraceptive requisitions are less than would be needed. Some districts are better than others. District hospitals are notorious for maintaining poor stock records, and the evaluation team found evidence of this: in two of the five district pharmacies visited, some, if not all (Salima), of the stock cards could not be located immediately. When stock cards were present, the physical inventory matched the numbers on the cards. One of the additional problems that the district pharmacies have is that their computers are aging, and many are rendered useless by computer viruses. HCs are dependent on getting their stocks from the District Hospitals.9 These centers do not have access to the logistics software, and their staff is not trained in how to estimate needs. As a result, the stock cards are even more erratically maintained than at some of the district hospitals. Since clinical responsibilities (‗there are three women in labor!‘) trump preparing medication requisitions, the task is often passed down the chain. The team heard from two sources that the clinic sweeper would often end up with the task. The final step comes when a CBDA or HSA supervisor collects supplies to provide to their community-based distributers. The HC clinician will want to retain sufficient stock for their own clinical needs, and will then allow what is in excess to be provided to communities. As the result of the team‘s work, meeting with pharmacists in six of the eight districts, reviewing their records, and then meeting with the RMS in Mzuzu and Blantyre, as well as holding discussions with the CMS chief logistics officer, the team is convinced that about half of the problems lie with CMS, but the other half stem from erratic and inappropriate ordering from DHMTs and HCs. Taking the case of Determine-HIV® in 2009, the team found that Nkotakota had no stock-outs, Chikwawa had three brief stock-outs, and Phalombe was without test kits for nearly half the year. Phalombe complained that they‘d never received any stock: RMS/Blantyre reported that they‘d never asked for any. The two districts that followed appropriate procedures tell the team that district ordering practices can overcome erratic supply from the central level. CFPHS is absolutely dependent on the supply of commodities. Without test kits and contraceptives, there is no service delivery. While addressing the problems of the CMS may be beyond the scope of the project, the team feels that some of the additional funds made available in 2010 and 2011 should be used to try to ameliorate this situation. Recommendation 16: CFPHS should use some of the additional training funds made available under its 2010 budget expansion to train health center and DHMT staff methods to effectively manage their drug supply chain. 8 Unfortunately, for the most part, the district hospitals have huge debts to the CMS, totaling some two billion kwacha (over $12 million) which must be serviced and therefore compete with further contraceptives. 9 The Regional Medical Stores will occasionally deliver drugs directly to health centers. These are supposed to be recorded by the DHMT, but this seldom happens. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 31 This is a very simplified discussion of the problems of pharmaceutical logistics in Malawi. Other issues—such as procurement of supplies outside the official chain—can easily be imagined. Despite the best efforts of donors to airlift essential stocks, and provide ever-increasing quantities to meet increasing demand, the problems will continue for the foreseeable future. The team would like to point out that the MSH/BASICS project, which was recently evaluated, also reported difficulty with obtaining needed commodities such as anti-malarials and HIV test kits. Recommendation 17: The USAID Mission, in collaboration with other donors, should consider a major effort to reform the medical procurement system. This will require starting at the very top of the chain, with GoM procurement procedures and regulations, and making sure that all who are responsible are appropriately trained and supervised to carry out their duties. Some form of cash transfer and project assistance may be needed to accomplish the goals. One of the major issues is that there are two sources for drugs in the CMS: those purchased by the CMS directly, and those donated by external donors. Districts are not charged for the donated drugs, but they are assessed a 5% handling fee. To obtain CMS procured drugs, districts must pay the cost of the drug plus a 12.5% handling fee. CMS-procured DMPA is actually the most expensive drug in the inventory, and districts with limited budgets are reluctant to purchase it. Currently there are reportedly one million doses of DMPA sitting in the CMS warehouse that no district will buy. It is not clear how this standoff will be resolved. Perhaps a donor will pay CMS the total amount and make the DMPA available at no costs to the districts. Even with this stock, the issue remains. The system cannot support two or three times as many contraceptive providers as currently exist. Attempting to train more providers without fixing the logistics first is likely to fail, squandering a great deal of money in the process. A better use for these funds would be to establish training programs to help those at the district and community levels to be able to predict and order their own supplies effectively. Recommendation 18: Training of additional providers in DMPA and HTC should be undertaken on a district-by-district basis after assessing both the district‘s ability to support services and ensure that there are sufficient supplies in the CMS system and the district's logistics chain that would enable them to deliver services over an extended period. ―No product, no program.‖ MINISTRY OF HEALTH Lines of Authority: Most issues with the MoH go beyond the scope of this evaluation. One of the issues that the team has noted that may cause difficulties in the future is that lines of authority and supervision are very confusing. To start at the top, for example, the FP authority is split between the MoH and the Ministry of Planning and Development (MoPD). The MoPD, however, does not have any real power over FP but covers population as it relates to development, since the Department of Reproductive Health in the MoH has the funds to support activities. To an outsider‘s eye, this seems to be an unproductive organizational chart. There are three other specific examples that affect project implementation: Pharmacy and Logistics: DHMT pharmacy management is under the Health Technical and Support Services (HTSS) Department in the MoH. When a separate USAID activity wished to place logistics specialists in the CMS and RMS, creating a position in the HTSS took too long, so they were placed under the CMS. Recently, the director of CMS directed these specialists that, since they were not in the HTSS, they could not support the DHMTs, reducing most of their potential impact. It remains to be seen how this will play itself out. Integrated Health Clinics: As described above, there are special buildings where a complete array of services is offered, but each of the services report up their own vertical chain of command and not to the individual in charge of the clinic. These facilities are thus integrated 32 MALAWI CFPHS PROJECT: MID-TERM EVALUATION geographically but not functionally. Each specific function, for example, HTC, has duties outside the clinic which takes staff away during clinic hours. There is no, or little, cross-training of staff to cover the existing functions if a person is out. Ultimately, this will have to be resolved. Health Surveillance Assistants: These staff, the most numerous of all MoH employees, were initially created in the era of vertical malaria programs, and were—and continue to be— supervised by the environmental health officer. This is despite the fact that their duties are more clinical than environmental. They currently do not have any established policy or guidance that creates their position and their place in the hierarchy. While this may be rationalized as policies are developed with the support of the policy section of CFPHS, it is unclear whether they will be moved to a more appropriate position, such as the division of Community Nursing. While the team has no recommendations to make, these issues should be followed since they may create problems in the future. DATA COLLECTION SYSTEMS AND LINKS WITH THE HC AND DISTRICT HMIS Tally sheets and client registers themselves are generally kept accurately, based on the samples the team saw. Some CBDAs do not report or report infrequently. Unfortunately, while these may be good at the individual level, by the time they are passed up the chain to the district Health Management Information Officer, the data has become much less reliable. Causes range from delayed reporting—with the rainy season or difficult terrain being one reason—to transcription and other errors when aggregating and forwarding through each reporting cycle. Every one of the six HMIS officers interviewed was skeptical of the quality of the data. This is discussed further below. The contractor is experimenting with an innovative approach to submitting data from the field using cell phones. The team has been told that this has entered the pilot phase, but it did not actually see the system working. If the difficulties can be worked out, this could be extremely advantageous. While some areas may not have cell coverage, CBDAs could be helped to find an area nearby where there is transmission. This would greatly save on travel to deliver their tally sheets. On the other hand, the ease of submitting reports may reduce or eliminate the regular supervisory visits. Another issue that will have to be addressed is how to pay for and distribute cell phone cards, and how to restrict cell phone use so that there are minutes remaining to transmit the data. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 33 VII. MANAGEMENT: KEY ASPECTS DATA ISSUES The project reports stunning successes, which is extremely unusual for FP programs in most of Africa. Since the evaluation team is specifically asked to ―Determine progress towards achieving M&E goals and targets review M&E plans to guide program direction, monitor/evaluate project activities and contribute to USAID/Malawi/HPN results‖ (see Scope of Work, Appendix 1), the team felt that it should focus some time on investigating the data collection and analysis systems to see: (1) whether appropriate data were being collected for management purposes, and (2) whether the reports were accurate or not. Standard Indicators The project relied almost exclusively on the 14 (expanded to 19) indicators that come from the FP/RH and the PEPFAR indicator lists of the USG. It is extremely important to remember that the ―standard‖ indicators are for USAID/PEPFAR global reporting to Congress, and are NOT designed to be the sole source of information for management decisions. Other data—including data that the project was already collecting—could have also been used for better management. As the PEPFAR 2007 indicator guide says: ―They are not designed to provide information on all dimensions of a program in country-specific settings.‖ The Department of State, Office of the Director of U.S. Foreign Assistance and PEPFAR standard indicators are a good place to start, but good management requires more. The CFPHS would have been managed more effectively had the contractor and Mission followed Automated Directives Systems (ADS) 203.3.4 and developed a custom set of indicators in addition to the standard indicators. The Performance Monitoring Plan (PMP) would have been a stronger document if the Mission and contractor had thought through additional information that would help them understand implementation better. Milestone indicators would have been particularly useful in tracking project progress. MSH International has a great deal of experience in identifying and using a wide variety of indicators, and it is unclear why this experience was not tapped in Malawi. Other Project Data Use The CFPHS collected a very large amount of data that they did not use effectively. See the section on data quality for a review of the process by which service statistics were gathered and collated. What is important is that, with the exception of distinguishing male and female beneficiaries, disaggregated data were not used by any level of the project. That is to say, when all doses of contraceptives reached each higher level, they were aggregated, and moved to the top. Even the Lilongwe office did not compare, for example, how each district was performing in comparison with the other districts. DCs were not taught, for example, to look at the data that were passed to them, and see which areas of their districts—which CBDAs and which HSAs— were performing better or worse than the others. This use of performance data would have helped them target supervision and support activities to ensure that each person was performing to his/her potential. It is a missed opportunity. In a few cases, often spurred by comments made by visiting consultants, DCs actually took the trouble to begin to disaggregate the data that came to them. In one particular case, the DC compared the performance of CBDA with HSAs. It is unclear how well the DC was able to use the findings, but it is an example of what can be done with few additional resources. Similarly, the Lilongwe office did not compare, say, monthly statistics on DMPA disaggregated by district. This would have allowed them to compare, say, Chikwawa with Phalombe, and see where there were problems that needed to be addressed. The team is not clear why existing data 34 MALAWI CFPHS PROJECT: MID-TERM EVALUATION were not used to help manage the project, particularly since it could have allowed targeted supervision, and reduced overall supervision costs. Part of the limited use of data stemmed from a lack of funding, and part came from an inability to identify an effective M&E manager (one has been hired recently), but the team feels that a major reason for the omission is that the project managers did not think creatively about how to use indicators to manage activities. Recommendation 19: MSH should examine its data sets and find ways of collecting new data and disaggregating existing data to improve project management. Baseline Data The team was glad to see that a baseline data survey was done. It contained information drawn from one hospital and three HCs in each district, 800 youth and adult community members (with nearly half living near the towns), and 441 FP/HIV/STI clients in exit interviews. Unfortunately, the survey did not establish a baseline for a single one of the 19 indicators in the PMP. In addition, since a major thrust of the project was rural communities, it would have been more appropriate to have focused on rural community members. The survey may have had useful information but it failed in its primary responsibility of establishing a baseline for each of the performance indicators that would be used to track project accomplishments. The contractor and USAID are fully aware of the deficiencies of this study, which are in part due to the selection of a subcontractor who struggled to produce even this report. Since there are no baseline data, CFPHS can—and should—establish proxy indicators for the project with 2007 baselines and 2008 through 2011 data, using contraceptive and HIV test kit data from district hospital pharmacies. Pharmacy stock cards typically go back to at least 2006 and frequently to 2003 and earlier. A review of the stock cards for each of the relevant commodities can quickly show how many commodities were issued for each district for each year. RMS will also need to be consulted to get a complete district profile, since they frequently ship directly to HCs. These data will give an idea of the total contraceptive distribution by district per year. The team attempted to do this, but was stymied by a lack of time. Data for Nkhotakota from 2007–2009 are shown below. In this case, 2007 serves as the baseline, and 2008 and 2009 as evidence of project impact. While not presented here, the first quarter of 2010 for Jadelle was 572, or more than the two previous years combined. Most importantly, this type of analysis shows, from a source independent of the project, the overall impact of what CFPHS has been doing. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 35 Figure 2: Nkotakota 2007-2009: Contraceptive and Determine® (HIV) distribution (units)10 DATA QUALITY There were fundamentally two types of data collected by the project: data such as number of people trained, which is easy for a central office to track, and service statistics, such as number of pills delivered to clients, which may be subject to more error. The team has no question about the reliability of the first type of data. Service statistics take a long time and many hands to reach Lilongwe. The CBDAs record the service data, and pass it on to their first-level supervisor, who aggregates the data from all CBDAs and passes it on to the second level supervisor, who aggregates the data and combines it with HC data, and forwards the forms—all hand written—on to the District Hospital. At this point, it is entered into the Ministry Health Information System—if it is working. The team spoke to many people in the data collection and processing chain, and all reported that they felt that community-level data were inaccurate. The primary problem may be that some people may only occasionally fill out their forms, some may not have understood the process, or some may submit them late so that they are not included. Both the Mission and the contractor knew about the problems with the data, and worked as best they could to improve the information collection system. In this setting, it is almost impossible to get ―good‖ service statistics. The team‘s extensive experience with analyzing service statistics in other settings suggests that this type of error is more likely to underestimate the services delivered, so the project probably provided more services than claimed. Since they are far ahead of their targets, and are aware of the problems, this data inaccuracy is not an issue that necessarily needs to be addressed. ―Fixing‖ this problem is extremely costly, and is not worth the time and effort to get ―accurate‖ data. The CFPHS should continue to work at all levels to teach people how to manage data better. It might be interesting to try to get records from all project levels and get a better estimate of the amount of undercounting, but this is clearly beyond the team‘s scope of work and time allotment. The team also does not think that this would be a useful exercise for the Mission or contractor. On the other hand, a second type of service statistic has to do with the number of people who heard messages through drama performances and ―open days‖ since these attracted large crowds. In the current situation, where a project goal is to have an ―open day‖ every two months in each district, counting the crowd at any one event can be quite accurate. However, attendees who enjoy the event are likely to attend more than one. Because of this, the total number of people attending each of the events double-counts the total number of people that the type of event reaches. The ―open days‖ were one of the most popular events. Because of this, the team believes that many 10 This table is somewhat misleading since there is an unexplained drop in LoFemenal. In Chikwawa, for example, demand for LoFemenol rose from 12.500 in 2007 to 23,000 in 2009. The team warns, however, that these figures have not been corrected for any shipments from the CMS directly to Health Centers, which can be substantial, and may explain the fall for Ovrette and LoFemenal in 2009 in Nkotakota— although both of these may also be due to the marked rise in use of Jadelle and DMPA. 2007 2008 2009 Jadelle n/a 90 458 Ovrette 600 3100 1100 LoFemenal 8100 14800 7600 DMPA 10400 12100 16400 Determine 23300 53500 71500 36 MALAWI CFPHS PROJECT: MID-TERM EVALUATION people heard the messages, and, by whatever means, many people adopted family planning and received HTC. This is as far as one can go. Since the totals of people attending one or more event will overestimate the number of people reached, the team feels that the data should be used judiciously for management and internal reporting purposes. The Mission and contractor should be careful to report these figures as being ―total attendance‖ at the events and not ―individuals attending the events.‖ If funds are available, it might be useful to do a survey at a sample of ―open days‖ to see what proportion of people were attending for the first time, and how many had attended one or more previous events. OVERALL PROJECT MANAGEMENT The evaluation team was specifically charged with evaluating the management of the CFPHS. The team has made many comments above about deficiencies found while doing its field visits. These include, among others, inadequate supervision, inadequate training, inadequate refresher training, inadequate IEC materials, inadequate performance data collection and use, and inadequate arrangements for transportation. During the last two weeks in country, the team made a special effort to understand why the difficulties occurred. In every case, MSH reported that there were insufficient funds to carry out all the desired activities. The team investigated the funds flow for the project, and found the core of the issue is that there was confusion between USAID and the contractor on how many CBDAs were to be trained during the first year. When the Mission set a higher goal than the contractor had anticipated, MSH did a good job in shuffling funds between line items to accomplish the high target. It would have been better had project funding and project targets been better matched. Lacking this, the contractor successfully shifted funds from one priority to another—specifically funds moved from IEC to training--and slowed some activities—specifically additional HTC training—because of a lack of resources. There is no recommendation because any disagreements between the contractor and the Mission have been resolved. PROJECT OVERSIGHT The evaluation team was pleased to find that the contracting officer‘s technical representative (COTR), Ms. Lilly Banda-Maliro, was able to make frequent trips to the field, and met even more frequently with her counterparts at MoH, MSH, Consella-Futures, and PSI. Her field visit reports were useful in tracking the progress of the overall project. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 37 VIII. LOOKING TO THE FUTURE In this section, drawing from the team‘s interviews, FGDs and study of documents, the team looks beyond the CFPHS Project to highlight key issues that should be considered in future donor and agency projects in support of MoH policies and strategies. YOUTH AND ADOLESCENTS A number of documents point to the need for greater access of youth and adolescents to FP and HTC services, as well as information and education on RH, especially FP and HIV. For example, the MDHS 2004 shows that 50% of all girls had begun childbearing by the age of 18 and 68% by age 19, with a high unmet need for birth spacing. In addition, the MDHS showed that young women age 15–19 have low levels of awareness of contraceptive methods. This contributes to unintended pregnancies, leading to girls (including those less than 15 years old) dropping out of school, unsafe abortions, and early and often undesired marriages. Researchers who studied the marital process in Malawi concluded that the process leading to marriage, with its concurrent rapid changes in sexual partnerships and sexual behaviors, is integrally related to HIV/AIDS risks (Clark et al., 2009). Sexual intercourse occurs in casual partnerships and with steady partners, but the frequency of times per week is greatest among those who have promised to be married. Condoms tend to be used most often among couples who have promised to be married. As a result, those in less stable sexual relationships are more at risk for HIV infections and unintended pregnancies. In support of the National Reproductive Health Strategy 2006–2010 of the MoH, a pilot of Youth Friendly Health Services was implemented in six selected districts to address the sexual and reproductive health care needs of adolescents. There were varying degrees of success across districts and facilities, ranging from the program not being implemented to very active implementation (MoH, RHU 2009). The lessons learned from this pilot program and implications of the data from the 2010 MDHS should be used to pilot a more systematic approach, which extends beyond health facilities and includes outreach through peer educators to adolescents and youth in rural communities. The team suggests that international donors and agencies, either individually or collaboratively, support the adolescent and youth sub-strategy in the forthcoming Reproductive Health Strategy for 2011–2015. SOCIAL MARKETING Social marketing through the private sector may appear to be an attractive strategy for increasing access to contraceptives on a sustainable basis. However, three key factors in the current context present challenges to successful implementation of this strategy: (1) FP services are provided free to all citizens of Malawi as part of the essential health package: (2) pharmacies, drug stores, and private providers are relatively scarce in rural districts: and (3) current efforts have had limited success. Thus, it appears prudent to approach social marketing through a phased funding approach, focused on urban areas and then large towns (where ability to pay is higher than other areas), with quantitative benchmarks for each two-year phase. Also, suggestions have been made about permitting CBDAs to social market contraceptives, oral rehydration solution (ORS), and water purification products as a means to provide these volunteers with a source of income and hence decrease attrition in their ranks. Authority is required by the Malawian Government for sale of these products in the community. The feasibility of such an approach is related to the current free access of persons to condoms and the availability of a vast supply of male condoms within the CMS to meet demand. Unless there are powerful champions of this approach within the MoH to advance a dialogue process that will lead to the requisite authority or permission to pilot test the approach, it does not appear to be the right 38 MALAWI CFPHS PROJECT: MID-TERM EVALUATION time to pursue this strategy, except as it relates to policy dialogue. When approved, a pilot program may be warranted. LINKAGES IN THE HIV/AIDS SERVICES AND SUPPORT CHAIN In principle, the linkages between key intervention points in the chain between community-based HTC diagnosis of a person as seropositive to appropriate facility-based health care and to community-based support and care groups is straightforward and seamless. However, the team found evidence of weaknesses in the chain. When a person is diagnosed as positive, through a three-stage test procedure, the provider of the service refers the person to a health facility and encourages the person to seek treatment. The team learned from a CBDA of some HIV+ individuals in his area not acting on his advice because of lack of funds for transport and lack of motivation. Currently, those who do regularly report to health facilities for ARVs may incur relatively high transport costs since districts are in the process of rolling out the program to all HCs. Also, at this stage, normally the district hospital is the only or main facility for carrying out CD4 counts. In one district, the PLWHA told us that the machine was frequently out of order when they went for their CD4 count. Lists exist of local CBOs of support groups composed largely or exclusively of PLWHA and of small organizations formed to provide care and support to those living with HIV/AIDS and their families. The information the team gathered suggest that greater attention should be given to understanding the major weaknesses in the linkage chain and viable ways to address these weaknesses. Although the team met with only three AIDs-related CBOs, the information acquired highlights that they are largely isolated from support beyond their communities. None had received any training related to care and support, such as psycho-social counseling. However, two had received some key inputs: one had been allocated about four hectares of land for group cultivation and the other had successfully applied to an agency for bags of soya beans for planting. In both cases the CBOs intend to share the crop among members and sell part to raise funds for their group. The team‘s limited evidence suggests that a study might be undertaken to identify gaps in the capacity of these groups to provide adequate information and counseling, absorb new members, and so forth. The objective would be to better develop an approach for addressing gaps and weaknesses among HIV-related CBOs and linkages within the system for use of HIV/AIDS funds coming into Malawi. Also, future FP and integrated community health programs should include local AIDS-related CBOs as a major target group, taking into account the achievements under CFPHS. CONCLUSIONS At this point, after reading the long litany of issues that affected the project, it is easy to forget the overall accomplishments. The CFPHS Project has been extremely successful: All Project Targets will be met or exceeded by the end of the Project Period, September 2011. CFPHS has been extremely successful in extending FP/HIV-AIDS Services to Rural Communities in eight Districts. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 39 Nonetheless, there is a great deal to do. In the team‘s mind, this project resembles a gleaming building that shows how it will enhance the skyline of a great city. However, this building is still supported by scaffolding, and would collapse of its own weight were that scaffolding taken away. The remaining months of this project, and any other follow-on project, need to focus on strengthening the structure so it can stand more on its own. These challenges must be met, particularly if there are plans to expand services in the near future. 40 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 41 APPENDIX 1: SCOPE OF WORK Global Health Technical Assistance Project, Task Order No. 01 GH Tech Contract No. GHS-I-00-05-00005-00 Statement of Work (Revised: 2-23-10) I. TITLE USAID/Malawi: Mid-Term Evaluation of Management Sciences for Health (MSH): Community￾Based Family Planning and HIV&AIDS (CFPHS) Project, Contract: GHS-I-00-07-00006-00 II. PERFORMANCE PERIOD The in-country evaluation shall be conducted during an approximately five-week period starting o/a April 8 to May 14, 2010. However, the actual dates of the project are subject to change based upon actual consultant availability. The entire duration of the project shall be February 2010–July 30, 2010. III. FUNDING SOURCE Mission. IV. PURPOSE This mid-term evaluation is a stocktaking exercise to assist USAID/Malawi and the Ministry of Health (MoH) in assessing the Management Sciences for Health (MSH) Project that is working on Community-Based Family Planning and HIV/AIDS Services (CFPHS). The evaluation will help to determine if the objectives agreed to are being achieved, progress made, and areas on which the Project and USAID Malawi need to invest more effort, so as to further the successful implementation of program. This evaluation is a required action under the Performance Based/Results Oriented agreement signed in October 2007 and will serve as the basis to inform the remaining two years of the program. This evaluation, planned for o/a April/May 2010, is intended to: The focus will be on determining the extent to which the objectives as defined in the contract and in relation to planned actions, and assess the likelihood of achieving them upon project completion taking into account the perspectives of the stakeholders and beneficiaries, and Determine the strengths and weaknesses of the existing program and approach. As a monitoring tool, this mid-term evaluation should provide information that can be useful in determining strengths and weaknesses of the program and its various components and help direct future program activities. In recognition of the fact that the project has appropriately emphasized improvement of systems and processes in the first years, the mid-term evaluation will focus on achievements and problems in these areas. The final evaluation will place greater emphasis on outcomes. V. BACKGROUND The purpose of the Community-Based Family Planning and HIV&AIDS Services task order is to support the GoM Ministry of Health (MoH) goal of promoting through informed choice, safer reproductive health (RH) practices by men, women, and young people, including increased use of 42 MALAWI CFPHS PROJECT: MID-TERM EVALUATION high-quality, accessible reproductive health services. The contract was awarded to Management Sciences for Health (MSH) in September 2007. The total estimated funding for the contract is $10,719,402. MSH is responsible for implementing integrated innovative approaches to community-based programs that promote access and utilization of family planning, reproductive health, and HIV/AIDS services in eight target districts of Balaka, Salima, Mangochi, Nkhotakota, Karonga, Phalombe, Chikwawa, and Kasungu. The districts mentioned are basically rural with pockets of semi-urban areas and the contractor is expected to address the needs of the district as a whole. The task order supports activities designed to reduce fertility and population growth, which is essential for attaining broad-based economic growth and lowering the risk of HIV/AIDS to mitigate the enormous impact on human resources and productivity. These inter-related elements contribute directly to the HPN program goal of Healthier Malawian Families and the overall Mission program goal of Poverty Reduction and Increased Food Security through Broad￾Based, Market-led Economic Growth. MSH Intervention Areas The program is an initiative under USAID/Malawi HPN Strategic Objective (SO) which is Increased Use of Improved Health Behaviors and Services. This SO contributes to the wellbeing of Malawians in three critical areas: (1) reducing fertility and population growth, which are essential for attaining broad-based economic growth, (2) lowering the risk of HIV/AIDS to mitigate the enormous impact on human resources and productivity, and 3) lowering infant and under-five mortality rates. Overall, the program goal is achieved through the following Intermediate Results (IRs): a. Increase access to community FP/RH and HIV/AIDS services: The Project has initiated community-based family planning services in eight districts and scaled up current operations by expanding coverage, access, and consistent use of FP/RH and HIV services. The focus for expansion is in rural and underserved areas and among high risk populations defined by high unmet demand for services or marginalized groups. The Project focus is on promoting consistent use of family planning services and looking for windows of opportunity to leverage increased access to HIV/AIDS services and utilization of testing and counseling, and positive living (i.e. dual protection). The Project also uses innovative approaches to expand contraceptive methods available to women through community-based distributors (CBDs), thereby increasing women‘s access to family planning (FP) in rural areas shall be strengthened. Access to long-term and permanent methods of contraception (LTPC) is problematic for most women and the contractor should address these issues. The contractor should improve access to long-term and permanent contraception to rural communities. Following approval of the community-based injectable by the Ministry of Health, the project is piloting provision of injectable contraceptives to poor, rural women at the community level through an alternative health delivery including use of a modified social franchising and public/private partnership approach as well as social marketing of oral contraceptives system (evaluation of the provision of injectables will be done separately). The contractor was also encouraged to come up with innovative approaches to meet the results outlined. CBDAs also include HIV/AIDS prevention messages, support testing and treatment-seeking and adherence behaviors. b. Strengthen contraceptive delivery systems, where voluntarism and informed choice are assured and quality is uncompromised. The Project focus is on achieving a balanced program that has community-based distribution (CBD) at the community level provided by frontline workers and clinical methods at all health MALAWI CFPHS PROJECT: MID-TERM EVALUATION 43 facilities within the targeted districts, offered by skilled providers. Clinical methods include sterilization, IUD, injectables, and Norplant where appropriate. c. Behavior change communication (BCC) While information is important, it is often not enough to change behaviors. BCC shall adequately portray family planning and HIV testing and treatment as mainstream health interventions, and influence the hearts and minds of individuals to make informed decisions about family planning and HIV/AIDS and promote them to their fellow community members. The contractor is working on promoting community involvement in generating the demand for services at the community level and adopting an engendered approach to stigma reduction. The Project is promoting the increase in demand for family planning and HIV/AIDS services, including BCCs, encompassing interpersonal communications, targeted mass media (posters, job aids, murals, etc.), and promotion of community involvement. The contractor is undertaking activities to revitalize family planning through expanding knowledge and awareness of family planning methods and integrating HIV/AIDS issues. The contractor is also expanding the communications portfolio to include generic BCC and service promotion. USAID/Malawi is already in full compliance with the Tiahrt provisions governing informed consent. Once availability and quality have been improved, the contractor should align clients‘ knowledge, attitudes, and practice more closely with the services that are offered. d. Quality of Services Improved The contractor works on enhancing the quality of family planning services, including training, interpersonal communications, quality assurance; incorporation of a gender approach into family planning; and HIV/AIDS service provision by training providers to identify signs of gender-based violence that should be addressed as part of family planning and HIV/AIDS counseling, record￾keeping, and M&E. Health worker performance improvement is an essential ingredient to enhancing the quality of FP/RH services, and should be a pillar of this program in order to provide back-up for referral by CBD agents. Within this context, the contractor shall invest in in￾service training for clinic-based staff where referrals are made in order to improve clinical and counseling skills and ultimately the quality of care provided, as well as linking to quality improvement activities for HIV support by USAID. e. Integrate family planning and HIV/AIDS and Sexually Transmitted Infections STI prevention The contractor achieves this IR through promotion of dual protection, encompassing condom promotion and other behavioral-change efforts to reduce pregnancy and STI/HIV risk and integration of family planning counseling and services (or referral for services) into HTC centers for women and men who wish to avoid future childbearing, including programs focused on mother to child transmission. The contractor should consider integration of services both ways HIV into FP as well as FP into HIV. f. Health sector capacity strengthened The contractor works on strengthening District and Community Provision and Management of FP/RH and HIV and AIDS Services by supporting the DHMT so that they provide their mandated supervisory and support functions to the health centers. By directing efforts towards this level, the program can create sustainable supervision and management capacity. The contractor is working in six of BASICS‘ districts. Both contractors are working with the same district health teams and, in addition, BASICS is working with the zonal offices which supervise the districts. It is therefore expected that there will be collaboration and coordination to avoid duplication and for leveraging and synergy. 44 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MSH Sub-Recipients MSH works in partnership with Population Services International (PSI) and Futures Group International (FGI). These two sub-recipients offer proven technical approaches and tools to work with the MoH to reposition FP and to improve access to HIV & AIDS services in rural communities of the eight target districts. Structure MSH/Malawi headquarters staff is based in Lilongwe with eight district health coordinators in each target district to facilitate activities. The DCs operate within the district health system and are located within the DHMT. In six districts where MSH overlaps with the BASICS project, the coordinator covers both projects. The coordinators work in collaboration with the district in overseeing community-based health activities in target communities. VI. SCOPE OF WORK The evaluation team is expected to carryout a mid-term evaluation of the Community-Based Family Planning and HIV/AIDS program in Malawi being implemented by Management Sciences for Health. The objectives of the Evaluation are to: Assess the extent to which program outputs and outcomes are being achieved; Assess and document the progress/extent to which the program outputs and outcomes are being achieved under the MSH program to support integrated community-based family planning and HIV/AIDS services; Assess and document the progress made through MSH and its sub-partners in improving access and quality of family planning and HIV services at community level; Assess progress made in health promotion and behavior change at the community level; Assess the effectiveness and efficiency of MSH‘s organizational structure in achieving program objectives; Determine progress towards achieving monitoring and evaluation (M&E) goals and targets (review M&E plans to guide program direction, monitor/evaluate project activities and contribute to USAID/Malawi/HPN results); and Make recommendations on strengthening successful efforts and adjustments/remedial actions that need to be considered to ensure sustainability of program outcomes, and institutionalization of policies and lessons learnt. VII. METHODOLOGY Evaluation Organization The evaluation team shall work under the supervision and guidance of the Contracting Officer‘s Technical Representative Technical Officer (COTR) for the MSH program. The COTR will organize all internal USAID meetings, including linking the team with other HPN team members overseeing family planning and HIV/AIDS programs for the health office. The methods to be used in completing this evaluation will include, but not be limited to: reviewing documentation, interviews, site visits, stakeholder meetings, etc. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 45 Document Review Prior to arriving in-country and conducting field work, the team will review various project documents and reports. The USAID/Malawi team will provide the relevant documents for review as soon as possible. Team Planning Meeting A two-day planning meeting will be held during the evaluation team‘s first two days in-country. This time will be used to clarify team roles and responsibilities, deliverables, development of tools and approach to the evaluation, and refinement of agenda. In the TPM the team will: Share background, experience, and expectations for the assignment; Formulate a common understanding of the assignment, clarifying team members‘ roles and responsibilities; Agree on the objectives and desired outcomes of the assignment; Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; and Develop data collection methods, instruments, tools and guidelines, and methodology, and develop an assessment timeline and strategy for achieving deliverables. Field Site Visits The MSH activities are focused in eight target districts; the evaluation team shall arrange to visit selected sites in each of the eight districts supported through MSH or a representative sampling covering the four zones (north, central, central east, south west, and south east). Decisions will be made in consultation with the COTR, Ministry of Health and the MSH Chief of Party (COP). Where possible, the evaluation team will be accompanied by a member of staff from USAID/Malawi and/or MSH. Field site visits should include visits to the district hospitals and CHAM facilities in the target districts if appropriate and surrounding HCs and communities. Meetings should be held with at least the following relevant officers at the Reproductive Health Unit; zonal health officers and DHMTs and MSH district staff; HC staff; HSAs; CBDAs, and community members. De-briefings Internal USAID/Malawi meetings will include, at a minimum: Initial organizational/introductory meeting at which the evaluation team will present an outline and explanation of the design of the evaluation; Mid-evaluation review with the COTR to outline progress and implementation problems; and Final evaluation debrief/summary of the data, draft recommendations, and report. An oral briefing meeting will be held with USAID/Malawi as either part of the team planning meeting or directly following it. The evaluation team shall propose and organize the evaluation process. The evaluation design and workplan shall be presented to the COTR and HPN team members for comments during the initial meeting with the Mission HPN team. A second meeting will be held with USAID/Malawi approximately half way through the in￾country work. This meeting will serve to outline progress and address any implementation problems. A third debriefing meeting will take place approximately four days before the evaluation team leaves Malawi. This meeting will be approved by USAID/Malawi and held with USAID/Malawi 46 MALAWI CFPHS PROJECT: MID-TERM EVALUATION and other key stakeholders after the site visit work is completed. The objective of the debrief will be to share the draft findings and recommendations, solicit comments and inputs, and clarify any remaining questions or issues before the team hands in the draft report and departs. Arrangement of Meetings The MSH COTR will arrange for an initial introductory meeting with appropriate MoH staff at the outset of the process. Where necessary, the COTR may participate in meetings with the MoH representatives and partners. A general list of relevant stakeholders and key partners will be provided to the evaluation team by the COTR at the time of arrival, but the evaluation team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so as to develop a comprehensive understanding of the program and services offered through the MSH agreement. The evaluation team will also need to review recent activities of the MoH and donors in order to understand the MSH program activities and develop an appropriate interview list. VIII. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORT The evaluation team will be composed of three technical experts in the following areas: Senior Evaluation Specialist/Team Leader (TL), Family Planning Specialist, and a Community Health/Behavior Change Specialist with a focus on HIV/AIDS. At various points, representatives from the Ministry of Health will participate in the review as observers. GH Tech will not be responsible for funding MoH participation. Roles of the Team Members: a. Senior Evaluation Specialist/Team Leader: The Team Leader (TL) will be responsible for overseeing the team and ultimately responsible for the submission of the final draft report to the Mission. S/he will provide team leadership, finalize the evaluation design, coordinate activities, evaluate the management strengthening component of MSH‘s support, plan and coordinate meetings and site visits, and be responsible for payments of local logistical needs and local staff working with the team. S/he will lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Malawi team and other major stakeholders and will consolidate reports from other evaluation team members and ensure that a draft report has been left with the Mission on departure. The Team Leader will take specific responsibility for assessing and analyzing the project‘s progress towards targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. The Team Leader should have experience in leading teams of international health experts and working with host country personnel. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required and experience in preparing high quality documents. The TL should have an advanced degree in public health or a related field with a minimum of five years of experience in management and evaluation of community-based and district level health programs, experience in leading teams of experts in health activities, experience in international health specifically dealing with capacity-building at district and community levels and should have worked extensively with USAID-supported FP/HIV programs. b. Family Planning/RH Specialist: One team member must be a Family Planning/RH Specialist. The FP/RH Specialist should have a post graduate degree in public health or related subject. S/he should have at least 10 years experience with reproductive health services design and implementation in developing countries and should have worked extensively with USAID-supported programs, preferably in the Southern Africa Region. S/he should be knowledgeable in community-based family planning program MALAWI CFPHS PROJECT: MID-TERM EVALUATION 47 assessment and evaluation methodologies, FP/HIV integration, and organizational and institutional capacity building. S/he should have extensive experience in conducting qualitative evaluations/assessments around FP/RH health service delivery. This person must identify programming gaps and recommend changes that would benefit the community-based integrated FP/HIV program of the MoH and recommend the current activities that justify continued support. S/he will also be responsible for the review of data-collection systems from the community and links with the district and HMISs. S/he should also examine MSH support for improving facility￾based FP services in order to expand the method mix and other internal systems within the health system that are essential to a well functioning FP program. As part of the management of a facility, the Specialist should review the progress towards implementing a quality assurance program, including infection-prevention efforts which will have been done in collaboration with the MoH roll-out plans. The Specialist should also examine the progress made towards repositioning family planning at the district level. This will include a review of the planning process at the district level, such as the development, use, and review of Detailed Implementation Plans developed by the DHMTs. c. Community Health/Behavior Change Specialist: One team member must be a community health and Behaviour Change Specialist. The Community Health/Behavior Change Specialist will be responsible for assessing the MSH work at the community level and its impact at the household level with particular attention on access and improved quality of family and HIV services and methods of community outreach. Responsibilities will include a review of progress made towards development, adaptation, and implementation of approaches to HIV/AIDS and family planning within the community to increase the knowledge of communities, and evaluating the progress made towards improving the integration of HIV into family planning programs at the community level. The Community Health Specialist will need to evaluate the appropriateness of messages and information developed to reach community members and assess implementation of community involvement in generating the demand for services at the community level and adopting an engendered approach to stigma reduction and the communications portfolio. This Specialist should have a master‘s degree in public health or related field with a minimum of five years experience designing and implementing large scale USAID-supported community based family planning and HIV programs in Africa, preferably in the Southern Africa region. Level of Effort (LOE) The contractor shall submit a proposed budget indicating salaries, international travel, in-country expenses, report printing and binding and miscellaneous direct costs. The budget shall be based on the following estimated level of effort (LOE): Task Team Leader Two other international consultants (days each) Local Consultant (Logistics Asst.) Background reading/preparation 5 days 3 days 4 days International travel- RT 4 days 4 days 0 days Team planning meeting and initial briefing with USAID/Malawi 3 days 3 days 3 days Field site visits/key informant interviews, meetings 18 days 18 days 14 days Mid-term and final debriefings 2 days 2 days 2 days 48 MALAWI CFPHS PROJECT: MID-TERM EVALUATION Task Team Leader Two other international consultants (days each) Local Consultant (Logistics Asst.) Report preparation, discussion and analysis in-country 5 days 5 days Draft Report revision & submission prior to team departure 4 days 4 days USAID/Malawi reviews draft report (10 working days) - Report revisions, based on Mission comments (out-of-country) 5 days 2 days GH Tech edits and formats final report (3–4 weeks) - Total est. LOE 46 days 41 days each 23 days IX. LOGISTICS A six-day workweek is authorized when the team is working in country. The team will work local holidays. The evaluation team will be responsible for all off-shore and in-country logistical support. This includes arranging and scheduling meetings, international and in-country travel (including vehicle rentals), hotel bookings, working/office space, computers, printing, photocopying. A local administrative assistant/secretary may be hired to arrange field visits, local travel, hotel, and appointments with stakeholders. GH Tech will be responsible for the following: Arranging travel in the U.S. and from the U.S. to overseas assignment location (country clearance, visa, plane tickets, hotel reservations, processing travel advance and expenses). Consultants are responsible for arranging in-country travel while overseas and ground transportation in the U.S.; Hiring a local logistics assistant to schedule travel, arrange meetings, and provide other support functions; Facilitating contact with USAID staff; Instruction and/or assistance with formatting charts, graphs, tables, and PowerPoint slides; and Arranging for editing/layout of final report. X. DELIVERABLES AND PRODUCTS The contractor deliverables shall include: 1. A written methodology plan (Evaluation design/operational workplan) during the pre￾evaluation meeting (due no later than fourth day in-country). MALAWI CFPHS PROJECT: MID-TERM EVALUATION 49 2. A draft report outline with possible issues for discussion during the mid-evaluation meeting (within two weeks of the start of the evaluation). 3. A Mission and partner debrief meeting that will be held around four days before the team‘s departure and prior to the submission of the draft report. 4. A draft report, due prior to the team leader‘s departure, will incorporate comments and suggestions from the debriefings. The Mission will have 10 days following the submission of the draft report to respond and provide written comments and feedback. 5. A final report (five hard copies and a CD-ROM) will be submitted as follows: The revised final unedited report will be provided to the Mission five days after the comments are received. Once the Mission signs off on the final unedited report, GH Tech will have the documents edited and formatted and will provide the final report to USAID/Malawi for distribution (five hard copies and CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. This will be a public document. The final report format shall include: – Executive summary, concisely summarizing critical elements of the main report, – Table of contents, – Introduction describing the purpose and objectives of the evaluation, – Background of the project, – Findings, – Conclusions and lessons learned, – Recommendations for improving the MSH program in the future, and – Other information relevant to the evaluation but not necessarily central to it may be included in an appendix. The report shall not exceed 30 pages, excluding the appendix. XI. MISSION AND/OR WASHINGTON CONTACT PEOPLE/PERSON Lilly Banda-Maliro, USAID/Malawi XII. COST ESTIMATE: TBD XIII. REFERENCES (PROJECT DOCUMENTS) Selected list of documents for review 1. MSH contract and modifications, 2. MSH quarterly reports, 3. MSH Baseline Assessment in Target Districts, 4. DMPA Evaluation Report, 5. MSH Performance Monitoring Plan (PMP), 50 MALAWI CFPHS PROJECT: MID-TERM EVALUATION 6. MoH Reproductive Health Strategy, 7. HIV/AIDS Strategic Framework, 8. 2004 MDHS, and 9. MICS 2006 MALAWI CFPHS PROJECT: MID-TERM EVALUATION 51 APPENDIX 2: PERSONS CONTACTED MALAWI U.S. Embassy/Malawi Ambassador Peter W. Bodde: U.S. Ambassador to Malawi U.S. Agency for International Development/Malawi Curt Reintsma: Mission Director Lilly Banda-Maliro: Deputy Team Leader, Office of Health, Population and Nutrition (HPN) Patrick J. Wesner: Program Officer Alisa Cameron: Team Leader, Office of HPN Martin D. Mtika: HIV Prevention Specialist, Office of HPN Beth Deutsch: Senior HIV Prevention Advisor Anteneh Worku: Senior Strategic Information Advisor Patricia Ziwa: Financial Program Management Specialist Martha Nanthoka: Program Management Specialist Ministry of Health Mr. Chris Kang‘ombe: Principal Secretary Dr. Chisale Mhango: Director, Reproductive Health Unit Sheila N. Bandazi: Director of Nursing Services Mtemwa Kalua Nyangulu: Program Officer, HIV Testing and Counseling (HTC) Austin Mnthambala: Deputy Director, HIV Unit Rudia Lungu: Public Health Laboratory Officer, HIV Unit Edwin M.F. Nkhono: Chief, Primary Health Care Coordinator Raphel Piringu: District Health Officer (Phalombe) Henry Chimbali: HIV Prevention and BCC, Health Education Unit Catherine Chiphazi: Child Health Specialist Sam Chirwa Senior Logistics Officer (HTSS) Management Sciences for Health/Malawi Rudi Thetard: Country Representative, MSH Mexon Nyirongo: Chief of Party, CFPHS Project Njuru Nganga: Deputy COP (MSH) Joyce Wachepa: FP Advisor (MSH) Flora Khomani: HIV/AIDS Advisor (MSH) Chimwemwe Msukwa: M&E Advisor (MSH) 52 MALAWI CFPHS PROJECT: MID-TERM EVALUATION Population Service International (PSI/Malawi) Ricki Orford: Resident Director Caroline Bakasa: Reproductive Health Program Manager Kongwani Kandeya: Family Planning and Communications Officer, Listeners Club, (Karonga) Isaac Salimu, FP Listening Club Officer, PSI (Salima and Nkhotakota) Patrick Magalasi FP Listening Club Officer, PSI (Mangochi and Balaka) Judith Njikho FP Listening Club Officer, PSI (Phalombe and Chikhwawa) Ricky Nyaleye Family Planning Communications and Listening Club Coordinator Futures Group Olive Mtema: Policy Specialist DELIVER, John Snow, Incorporated (JSI) Jayne Waweru: Country Director Elias Mwalabu: Assistant LMIS Associate Jhpiego Tambudzai Rashidi: Chief of Party, MCHIP Project & ACCESS/Malawi United Nations Population Fund Juliana M. Lunguzi: Program Officer, Reproductive Health National AIDS Commission Rev. Bernard Amos Malango: Chairman Family Health International Nick Ford: Country Director Christian Health Association of Malawi Desiree Mhango: Director of Health Programs Malawi Counseling and Resource Mobilization (MACRO) Martha Muyaso: Training Coordinator Intra Health Deliwe Malema: Country Director Qadria Muslim Association of Malawi Alison Liwanda: General Secretary Muslim Association of Malawi Saiti Buirhan Jambo: Executive Director Sufyan Ahmed Rashid: Director of Programs MALAWI CFPHS PROJECT: MID-TERM EVALUATION 53 Adventist Health Services Florence A. Chipungu: Director Joseph C. Mwandira: Program Manager Malawi College of Distant Education Jennefer Kennedy: Tikwere Radio Program Patricia Luhana: IRI Advisor Julie Kachasu: IRI Advisor Chifundo Clinic, Lilongwe N. Muriya: In-Charge Chikwawa District Sylvia Chinkwende: Hospital Matron Virginia Faeti: Family Planning Coordinator Rose Chonde: HTC Supervisor Kanyerere Gondwe: Pharmacy Technician Merenia Washa: Primary CBDA Supervisor Medson Boti: LTPM Provider Rex Bwanausi: Medical Assistant Judith Manyeje: LTPM Provider Haxston Kotakota: Clinical Officer Stanely Mphombo: LTPM Provider Esau Mtengowaminga: LTPM Provider 11 men: FGD Listerning Club Officer, PSI Headman and community members: Katunga Group Village Headman Mtombosola: Senior Local Leader 9 men, Jakison village: TA Katunga 6 women, Jakison village: TA Katunga Village Headman: Jakison Marko Evason: Local Leader, CBDA/HTC Joe Dunga: CBDA 10 women, 5 men: Madalitso FP Listeners Club CBO Group Villlage headman Fombe: Local Leader Power Eagles Drama Group: Drama Troupe Peter Musikuwa: Bishop of Chikwawa Catholic Diocese Mr. & Mrs. Chimoto: SDM providers Felix Makande: SCM trainer, Catechist, St. Mathews Parish Alexander Chipungu: Clinical Officer, BLM Christine Chimbuto: Nurse Midwife Technician 54 MALAWI CFPHS PROJECT: MID-TERM EVALUATION Karonga District Dr. Michael Kayange: District Health Officer Emily Phiri: District Nursing Officer Elias Phiri: Deputy District Environmental Health Officer Loyce Towera Mukhondiya: Family Planning Coordinator Christopher Singini: IEC Officer Kondwani Kandeya: FP Listeners Club Officer PSI Albert Mwanyongo: DMPA Provider / CBDA Primary Supervisor Zinaumaleka Nkhono: MSH District Coordinator Albert Mwanyongo: HSA & CBDA Supervisor, Lupembe HC Annex Njora: CBDA, Lupembe HC Mrs. Loyce Munkhondya Community health Nurse & CBDA Trainer Mwayi Mwahimba District Supervisor, HTC Joyce P. Wachepa: FP Advisor, MSH Mary Zulu: Nurse/MW Technician, Nyungwe HC Kasungu District Patrick Kachingwe: CBDA Rose Chinkhombeni: Radio Listeners club Philimon M. Phiri: RL Club Blessings Lungu: Listening Club Coordinator 5 women FGD: Community members 7 women waiting for their BTL Community members 9 women, 2 men: Radio Listening Club, Galika Richard Mwale: HSA and CBDA Primary Supervisor Mrs. Modesta Zgambo: Nurse/MW, LTPM provider, Kaluluma Dist. Hosp. Mrs. Victoria Zungu: Nurse/MW, LTPM provider, Kaluluma Dist. Hosp Happy Mbewe: HSA, DMPA provider, Chanakala HC Eusebia Phiri: Community Health Nurse, District Hospital Dr. Nkhambule: District Health Officer Margaret Chipeta: District Nursing Officer Ketwin Kondowe: District Environmental Health Officer Mr. Kudya: HTC Supervisor Mrs. Phiri: Family Planning Coordinator Victoria Mzuzngu: DMPA Supervisor Senior Chief Kaluluma: Traditional Leader Mangochi District Jaden Bendabenda: District Medical Officer JSE Chausa: District Environmental Health Officer Lonnie Mkwepere: District Nursing Officer MALAWI CFPHS PROJECT: MID-TERM EVALUATION 55 Mrs. A. Chuma: Accountant Mbiriyawanda Stone: HMIS E Mwagomba: Deputy Family Planning Coordinator Hackson Banda: Health Surveillance Assistant Chifundo Mambulu: Nurse, FP Coordinator Mrs. J Chausa: Nurse, HTC Coordinator Mrs. Nyalugwe: Nurse, LTPM Provider - BLM Chikosa Ngwira MSH Mangochi Vandross Chowe: MSH, Mangochi Martha Ilande: Nurse, LTPM Provider Rhoda Kandu: Senior, HSA Supervisor and DMPA Provider Arnold Mndalira: District Health Promotion/IEC Officer Patrick Magalasii: PSI FP Listening Club Officer 11 women: Community Members, FP Users 12 women, 7 men: Kwitunji Radio Listeners Club 8 women, 4 men: Mpita Drama Group Village Headman Kwitunji: Local Leader 5 women, 3 men: Tutendeuli Radio Listening Club CBO (Support group) Esmme Mussa: CBDA Hawa Singiramu: CBDA Mr. M Chabwera: HSA and CBDA Supervisor Anania Matenje: Medical Assistant Hope Kumwenda: Clinical Officer Fanuel Nganga: Clinical Officer and PAC trainer/provider Calista Mbayeni: SDM Trainer Saulos Chimbalange Salvaziyo: Deacon, Koche Parish Rozariyo Nkhoma: Catechist, Koche Parish Nkhotakota District Enock U. Ludzu: District Health Officer Piyo Gregory Dimba: Matron, St. Anne‘s Hospital Dr. Ludzu: District Health Officer Dorika Msiska: Family Planning Coordinator Mr. Banda: Medical Superintendent St. Anne‘s Hospital Peter Nyasulu: MSH District Coordinator Veronica Chisemphere: Hospital Matron Issac Salimu: PSI 5 women, 1 man, 3 young men: Community members in Area of a Recent Open Day Phalombe District Doreen Machinjili: MSH District Coordinator Dr. Piringu: District Health Officer 56 MALAWI CFPHS PROJECT: MID-TERM EVALUATION Andrew Saukira: HMIS Officer Frank Kapitapa: Pharmacy Technician Kotana Mtembo: Family Planning coordinator Mr Matumba: HTC Supervisor Ms Komwa: LTPM Provider/Pharmacy Supervisor Juliet Basikolo: Secondary CBDA Supervisor Chisomo Magola: Primary CBDA Supervisor Mr Mkandawire: CBDA/HTC Counselor Jane Likombola: Primary CBDA Supervisor 6 women: FGD Community members Group Village Headman, Namasolo: Local Leader Gladys Haleka: CBDA Dickens Kampira: HSA Judie Njikho: Village Headman Nagome: Listerning Club Officer, PSI 11 women, 3 men: Madalitso CBO FP Listeners Club 8 men: FGD 8 women, 5 men: FGD Community leaders and members Hope for Life: Drama Group Maulidi Likhanye: CBDA/HTC, SDM Provider Patrick Kaligomba: SDM Provider Catherine Kaligomba: SDM Provider Paul Franciso Makina: Catechist and SDM Provider Olive Mulaviwa: Catechist and SDM Provider Michael Mkhomeni: Director, Catechist Training Institute Charles Kaponya: Assistant Director, Catechist Training Institute Wisdom Makoka: HSA Phakuyeyi Zakeyu: Community Health Nurse, PAC provider Awema Zemani: HMIS Officer Salima District Elizabeth Chaloera: Nurse, FP Coordinator for District Amon Chagunda: CBDA Florence Bwanali: District Health Officer Dorothy Kabambe: District Nursing Officer Paul Chunga: District Environmental Health Officer Atwel Ndakala: HTC Supervisor Elizabeth Chalera: Family Planning Coordinator Mr. Chimalizeni: HSA DMPA Provider Makwinya Drama group Drama Group Amon Chimphepho: CBDA Kennedy Moyo: Primary Supervisor MALAWI CFPHS PROJECT: MID-TERM EVALUATION 57 Isaac Salimu: Listening Club Officer, PSI Ricky Nyalei: Communications and Listening Club Coordinator, PSI 8 members: Drama Troupe 58 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 59 APPENDIX 3: REFERENCES AND SOURCES CONSULTED Angotti, Nicole, Agatha Bula, Lauren Gaydosh, Eitan Zeev Kimchi, Rebecca L. Thornton, and Sara E. Yeatman. ―Increasing the acceptability of HIV counseling and testing with three C's: Convenience, confidentiality and credibility.‖ Social Science & Medicine, 68(12): 2263–2270, 2009. McHenry, Brigit. Operations and Policy Issues Affecting Community-based Distribution of Injectable Contraceptives. Futures Group, June 2008. Clark, Shelley, Michelle Poulin, and Hans-Peter Kohler. ―Marriage aspirations and HIV/AIDS in rural Malawi.‖ Journal of Marriage and the Family, 71: 396–416, 2009. Engender Health, Malawi Ministry of Population and Health, Christian Health Association of Malawi, Malawi Nurses and Midwives Council, Malawi Medical Council. Final Assessment, Long Term and Permanent Contraception (LTPC) and Post-Abortion Care (PAC) Programs 1999-2003, Malawi.September, 2003. Huber Douglas, Nika Saeedi, and Abdul Khalil Samadi. Achieving Success with Family Planning in Rural Afghanistan. Bulletin of the World Health Organization, 88:227–231, 2010. Futures Group International. Policy Landscape Analysis of Family Planning and HIV/AIDS Services. 2009. Health Policy Initiative and DELIVER Project. Analysis of the Operational Policy Barriers to Financing and Procuring Contraceptives in Malawi. Health Policy Initiative, Constella Futures and DELIVER Project, John Snow, Inc., Washington, D.C. and Arlington Va. 2008. International Planned Parenthood Federation, UNFPA, and the Coalition on Women and AIDS. 2006. Institute for Reproductive Health, Georgetown University, FAM Project. Website for Standard Days Method Resources. http://www.irh.org; http://www.irh.org/?q=resources; http://www.irh.org/sites/default/files/All%20about%20CycleBeads_ENG.pdf. Jansen, William H., Carmen Coles, and Ishrat Husain. An Assessment of Family Planning in Malawi. GH Tech. Washington, D.C., 2009. JHPIEGO Corporation. Reproductive Health/Family planning learning package for Malawi Reserve Education. November 2002. Kadzandira, John M. and Chilowa R. Wycliffe. The Role of Health Surveillance Assistants (HSAs) in the Delivery of Health Services and Immunization in Malawi, Centre for Social Concern. December 2001. Kasungu District Hospital Drug Formulary. October 2008. Letter from Office of The Secretary for Health, Provision of HIV Testing and Counseling Services by Community Based Distributing Agents. Dated 19 March 2009. Malawi Community-Based Family Planning and HIV/AIDS Project. Community-Based Provision of DMPA, Lessons learnt from Developing Countries with focus on Africa. January 2009. Malawi Office of the President and Cabinet Department of Economic Planning and Development. National Population Policy. 60 MALAWI CFPHS PROJECT: MID-TERM EVALUATION McSmith, D., K.S. Drapcho, G. Pirio, and S. Wilcox. Basics Project Mid-Term Evaluation, Task Order NO. GHA-1-06-04-00002-l00, for USAID/Malawi. March, 2010 (Draft). Malawi Ministry of Health. Senior Managements’s Decision at the Malawi Ministry of Health to Allow HSAs to Provide Injectable Contraceptives: Post-Morterm Investigation. July 2008. Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Monthly Report for Karonga. February 2010. Management Sciences for Health. Community-Based Family Planning and HIV/AIDS Services Project: Baseline Survey. January 2009. Management Science for Health-Malawi in collaboration with Ministry of Health District Health Office. Phalombe District Community Based Family Planning and HIV and AIDS Services profile (Undated). Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Trip reports, June 2008–September 2009. Management Science for Health. Chikhwawa District profile, Community Health Family Planning and HIV and AIDS Services Project (Undated). Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Quarterly Reports No.1–9, Annual Report October 2007–September 2008. Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Monthly Report, December 2009. Ministry of Health. Community Based Provision of Injectable Contraceptives by Health Surveillance Assistants: Participant’s Training Manual. Lilongwe, August 2008 (Draft). Ministry of Health. Community Based Injectable Contraceptive Services Guidelines. First Edition. Lilongwe, August 2008. Ministry of Health. Community Based Provision of Injectable Contraceptives by Health Surveillance Assistants: Participant’s Training Manual. Lilongwe, August 2008 (Draft). Ministry of Health. Guidelines for Community Initiatives for Reproductive Health. May 2007. Ministry of Health. Guidelines for Community Based Distribution of Contraceptives in Malawi. Lilongwe, April 2004. Ministry of Health. Health Surveillance Assistant Course Curriculum. Lilongwe, May 2009. Ministry of Health. Multiple Indicator Cluster Survey 2006: Malawi. National Statistical Office, United Nations Children‘s Fund. June 2008. Ministry of Health. Malawi National Reproductive Health Service Delivery Guidelines. Lilongwe, September 2007. Ministry of Health. National Health Policy. Draft 05. Ministry of Health. National Reproductive Health Strategy 2006-2010. June 2006. Ministry of Health, Reproductive Health Unit. Sexual and Reproductive Health and Rights. Annual Report, December 2007. Ministry of Health. National Quality Assurance Policy. May 2005. Ministry of Health. Reproductive Health Policy. February 2010. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 61 Ministry of Health, Reproductive Health Unit. Strategic Assessment on Issues Related to Unsafe Abortion in Malawi: Draft Report. Lilongwe, October 2009. Ministry of Health. Road Map for Accelerating the reduction of Maternal and Neonatal Mortality and Morbidity in Malawi. Second Version, July 2006. Ministry of Health-Reproductive Health Unit. Strategic Assessment on Issues Related to Unsafe Abortion in Malawi. Draft report, October 2009. Ministry of Health. National Sexual and Reproductive Health and Rights (SRHR) Policy. August 2009. Mkantawire, Peter. ―Involving Men and Women in Family Planning.‖ PSI Powerpoint presentation August 6, 2008, based on paper by R.T. Pierson, P. M. Ntala, of the Center for Social Research, University of Malawi. Mwandira Joseph. ―Adventist Health Services Family Planning & RH Project.‖ PowerPoint presentation, Blantyre, April 2010. National AIDS Commission. Proposed Door to Door HIV Testing and Counseling Model for scale-up in Malawi to achieve Universal Access. May 2009. National Statistical Office and ORC Macro. Malawi Demographic and Health Survey 2004. Zomba, Malawi, and Calverton, Maryland, December 2005. National Statistics Office. Summary Findings from the Malawi Demographic and Health Survey 2000 Kasungu District. Population Services International/Malawi. Community Health Family Planning and HIV and AIDS Services in Malawi Newsletter, 1st Issue. January to March 2010. Population Services International/Malawi. Community Health Family Planning and HIV and AIDS Services in Malawi. Quarterly Progress Report, January to March 2008, October to December 2008, April to June 2009. ―Report Card: HIV Prevention for Girls and Young Women, Malawi.‖ Retrieved March 26, 2010, from www.ippf.org/NR/rdonlyres/88147ACB-E804-4EAA-80BA-F98565A7466A/0/Malawi.pdf Reproductive Health Unit of the Ministry of Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Quarterly Progress Report February to June 2008, July to September 2008. Richardson F.M., M. Fahnestock, M. Bishop, P. Emmart, and B. McHenry. Community-based Distribution of Injectable Contraceptives in Malawi. Futures Group. Washington, D.C., 2009. Robertson, Joan, Margot Fahnestock, Alex Mkandawire, Jayne Waweru, Lelsie Patykewich, and Todd Dickenson. Analysis of the Operational Policy Barriers to Financing and Procuring Contraceptives in Malawi. Washington, D.C.: Health Policy Initiatives/Constella-Futures Group and Arlington, Va.; DELIVER Project/John Snow, 2008. The Status of Reproductive Health within the Sector Wide Approach Context Malawi: Towards Universal Access to Reproductive Health. March 2008 (no author, no source). USAID Africa Bureau, Office of Sustainable Development. Family Planning Program Review in Selected Countries in Sub-Saharan Africa. November 2009. USAID/Malawi. An Assessment of Family Planning in Malawi. William H. Hansen, Carmen Coles, and Ishrat Husain through the Global Health Technical Assistance Project, February 2009. 62 MALAWI CFPHS PROJECT: MID-TERM EVALUATION USAID/Malawi. Operations and Policy Issues affecting Community-Based Distribution of Injectable Contraceptives. 2008. USAID/Malawi. Draft Report, A Study of Integration of Family Planning and HIV and AIDS Services in Malawi. March 2010. USAID, Africa Bureau-Office of Sustainable Development. Family Planning Program Review in Selected Countries in Sub-Saharan Africa, Report Summary. November 2009. USAID/Malawi. Policy Landscape Analysis of Family Planning and HIV/AIDS Services. June 2009. USAID/Washington, D.C. Family Planning: A Key Component of Postabortion Care. Consensus Statement: International Federation of Gynecology and Obstetrics, International Confederation of Midwives, International Council of Nurses, and the United States Agency for International Development. Washington, D.C, 25 September 2009. http://www.usaid.gov/our_work/global_health/pop/techareas/pac/fp_component.pdf. World Health Organization, Department of Reproductive Health and Research (WHO/RHR), and Johns Hopkins Bloomberg School of Public Health/Center for Communication Programs (CCP), INFO Project. Family Planning: A Global Handbook for Providers (2008 update). Baltimore and Geneva: CCP and WHO, 2008. www.fphandbook.org Yeatman, Sara and Jenny Trinitapoli. Beyond Denomination: New Tests of Whether and How Religion Matters in Family Planning. No source, June 2007. PROJECT DOCUMENTS Akhter, Halida. ―Debriefing Note (draft) for COP, Community Health Family Planning and HIV and AIDS Services in Malawi. MSH Trip Report. 25 November 2009. Management Sciences for Health. Community Health Family Planning and HIV and AIDS Services. Implementation Plan. (Undated). Management Sciences for Health. Community Health Family Planning and HIV and AIDS Services Project Training Records. (Undated). Management Science for Health. ―Community Health Family Planning and HIV and AIDS Services.‖ In Malawi Monthly Reports to USAID, 2007–2009. Management Sciences for Health. Community-based Family Planning and HIV/AIDS Services in Malaw. Combined Quarterly/Annual Reports 1–9, December.2007 through December.2009. Management Science for Health. Salima Monthly Report on Community Family Planning and HIV and AIDS Service Project. December 2009. Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Annual Workplan, 1 October 2009–30 September 2010. Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi. Monthly Planning Report for Phalombe, October 2009. Management Sciences for Health. Community-based Family Planning and HIV/AIDS Services in Malawi: Performance Management and Evaluation Plan. Lilongwe, 2007. Management Sciences for Health, Malawi Project Team. Community Based Family Planning and HIV/AIDS Services Project. PowerPoint, MSH, Lilongwe, April 2010. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 63 Management Science for Health. Community Health Family Planning and HIV and AIDS Services in Malawi, Performance Management and Evaluation Plan, Period covered: October 2007 to September 2010. Management Sciences for Health. Detailed Implementation Plan: Community-Based Family Planning and HIV/AIDS Services (CFPHS) Programme. Period Covered: October 2007 to September 2010. Lilongwe, 2007. Mexon, Nyirongo and Rolex Tolani presented to Lilly Banda-Maliro. Detailed Implementation Plan Narrative. Community-based Family Planning and HIV/AIDS Services (SFPHS) Programme, October 2007 to September 2010. Two volumes: Narrative and Annexes. Lilongwe, Malawi, 2007. Minutes of COTR Meeting held at USAID Offices, 5 February, 2010. Population Services International. Community Health Family Planning and HIV and AIDS Services in Malawi Produced Behavior Change Communication, brochures and insert, with brochures accompanied by English translations (Undated). Population Services International. Zina Umanena radio drama episodes 1–26 in English (Undated). Population Services International. Community Health Family Planning and HIV and AIDS Services in Malawi. Monthly Reports to Management Science for Health, February 2008 through February 2010. USAID/Malawi. Community-Based Distribution of DMPA, Report on Stakeholders meeting held at Crossroads Hotel, Lilongwe, Malawi 15 July, 2008. USAID/Malawi. Community Health Family Planning and HIV and AIDS Services in Malawi Trip reports February 2009 to March 2010. USAID/Malawi. Amendment of Solicitation/Modification of Contract: Community Based FP and HIV/AIDS Services in Malawi: Task Order #GHS-I-03-00006-00. February 24, 2010. USAID/Malawi. Contract Award: Community Based FP and HIV/AIDS Services in Malawi: Task Order #GHS-I-03-00006-00. October 1, 2007. 64 MALAWI CFPHS PROJECT: MID-TERM EVALUATION MALAWI CFPHS PROJECT: MID-TERM EVALUATION 65 APPENDIX 4: TABLES TABLE 1. TRAINING IN FAMILY PLANNING AND HIV TESTING AND COUNSELING (HTC) BY CATEGORY OF HEALTH WORKER, GENDER AND METHOD IN THE 8 CFPHS PROJECT DISTRICTS DURING LIFE OF PROJECT 2007 TO THE MID-TERM EVALUATION, MAY, 2010. Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Male Female Total Total 1 Karonga 37 64 101 30 10 40 29 11 40 5 6 11 2 0 2 2 8 10 1 1 2 2 0 2 208 2 Kasungu 89 29 118 11 4 15 31 11 42 9 3 12 1 1 2 5 5 10 2 0 2 0 2 2 203 3 Nkhotakota 107 47 154 21 15 36 35 5 40 7 2 9 2 0 2 2 8 10 2 0 2 0 2 2 255 4 Salima 67 53 120 32 8 40 35 5 40 6 2 8 2 0 2 4 6 10 1 1 2 1 1 2 224 5 Balaka 73 47 120 19 16 35 20 20 40 8 3 11 0 0 0 1 10 11 1 1 2 2 0 2 221 6 Mangochi 59 49 108 13 7 20 58 21 79 5 6 11 2 0 2 2 6 8 1 1 2 0 2 2 232 7 Chikwawa 65 71 136 28 11 39 37 3 40 9 1 10 2 0 2 11 10 21 1 1 2 1 0 1 251 8 Phalombe 62 84 146 32 8 40 32 8 40 9 2 11 2 0 2 2 9 11 1 1 2 1 1 2 254 Total 559 444 1003 186 79 265 277 84 361 58 25 83 13 1 14 29 62 91 10 6 16 7 8 15 1848 District DMPA (HSAs) HTC Training (CBDAs) HTC Training (HSAs) TOT Natural Methods CBDA Supervisors LTPM (Jadelle) LTPM (Tubaligation and Vasectomy) CBDAs 66 MALAWI CFPHS PROJECT: MID-TERM EVALUATION TABLE 2. CFPHS INDICATORS: TARGETS AND ACTUALS TO MAY 2010 # Indicator Description Disagre￾gated by Yr1 Targets Yr1 Actuals Yr2 Targets Yr2 Actual Q9 & Q10 Actuals EoP Targets Total Actuals (To date) Family Planning & Reproductive Health Indicators 1 Number of new approaches successfully introduced through USG supported programmes NA 2 1 2 4 - 3 5 2 Couple years of protection (CYP) in USG supported programmes NA 73,316 54,386 80,000 118,738 73,689 225,148 246,813 Total 450 537 500 3,658 286 1,250 4,481 Male 225 310 200 962 138 625 1,410 Female 225 227 300 2,696 148 625 3,071 Total 25,000 9,444 30,000 490,645 229,518 1,500,000 729,607 Male 12,500 2,696 15,000 103,781 61,775 750,000 168,252 Female 12,500 6,748 15,000 386,864 167,743 750,000 561,355 Total 300,000 47,943 400,000 689,647 378,719 1,600,000 1,116,309 Male 150,000 15,115 200,000 169,549 109,641 800,000 294,305 Female 150,000 32,828 200,000 520,098 269,078 800,000 822,004 6 Number of policies or guidelines developed or changed to improve access to and use of FP and RH services NA 1 2 1 2 - 3 4 7 Number of USG assisted SDPs providing FP counselling or services NA 45 323 50 533 658 120 1,514 8 Number of USG assisted SDPs that experienced stocks outs of specific tracer medicines NA 5 5 - 52 75 5 132 HIV & AIDS Indicators - Total 55,000 24,237 750,000 430,467 381,925 750,000 836,629 Male 27,500 7,999 375,000 113,966 78,036 375,000 200,001 Female 27,500 16,238 375,000 316,501 303,889 375,000 636,628 Total 16 537 1,250 1,481 - 1,250 2,018 Male 8 310 625 871 - 625 1,181 Female 8 227 625 610 - 625 837 Number of people trained in FP and RH (with USG funds) both men and women 3 Number of counselling visits for FP and RH as a result of USG assistance 4 Number of people that have seen or heard a specific FP and RH message 5 CFPHS Project Indicators, Yearly Targets and Actuals (Yr 1, 2 and Q9 & 10) Number of people reached through community outreach that promotes HIV/AIDS prevention thorugh abstinence, being faithful or both 9 Number of People trained to promote HIV / AIDS prevention through other behavior change beyond abstinence, being faithful or both 10 MALAWI CFPHS PROJECT: MID-TERM EVALUATION 67 Total 55,000 5,813 65,000 419,222 383,571 70,000 808,606 Male 27,500 1,913 32,500 111,196 77,078 35,000 190,187 Female 27,500 3,900 32,500 308,026 306,493 35,000 618,419 Total 8 537 1,250 1,606 461 1,250 2,604 Male 4 310 625 955 95 625 1,360 Female 4 227 625 651 366 625 1,244 13 Number of Targetted condom service outlets NA 8 246 80 807 519 600 1,572 Total 250 537 1,250 1,509 1,480 1,250 3,526 Male 125 310 625 908 438 625 1,656 Female 125 227 625 601 1,042 625 1,870 15 Number of service outlets providing counseling and testing according to national and international standards NA 8 109 80 522 399 1,030 90 Total 16 21 25 79 - 25 100 Male 8 15 12 64 - 12 79 Female 8 6 13 15 - 13 21 Total 136,000 7,108 100,000 67,437 18,988 100,000 93,533 Male 68,000 1,927 50,000 26,433 7,221 50,000 35,581 Female 68,000 5,181 50,000 41,004 11,767 50,000 57,952 18 Number of local organizations provided with technical assistance for HIV institutional capacity building NA 35 - 30 71 9 60 80 Total 250 - 80 242 - 250 242 Male 125 - 40 127 - 125 127 Female 125 - 40 115 - 125 115 Notes: Under Indicator 1, the outputs are as follows: Under Indicator number 6, the outputs are as follows: 1 DMPA guidelines; 1 Community Based DMPA Policy 2 DMPA training materials (participant and training manuals); DMPA Job aids 2 Community Based DMPA Guidelines 3 Community based DMPA Logistics system 3 Policy on Social Marketing 4 Integrated tally sheet for FP and HTC 4 HIV and AIDS Integration (Work in Progress) Number of Individuals counseled and tested for HIV and received results (excluding TB) 17 Number of Individuals trained in HIV-related institutional capacity building 19 Number of People trained to promote HIV / AIDS prevention through abstence, being faithful, or both 12 Number of Individuals trained in HIV related stigma and descrimination reduction 14 Number of individuals trained in Counseling and Testing through FP Project 16 Number of individuals reached through community outreach that promotes HIV / AIDS prevention through other behavior change beyond abstinence, being faithful, or both 11 68 MALAWI CFPHS PROJECT: MID-TERM EVALUATION TABLE 3: EXAMPLES OF WOMEN'S AGE, NUMBER OF CHILDREN, AND CURRENT FP METHOD Woman's Age Number Children Method Used Prior Methods Used and/or Comments 17 1 none Child is less than 6 months old 23 3 Pill Had to drop out of secondary school when first became pregnant 24 1 Depo 25 3 Depo 27 4 Jadelle 29 4 Pill 30 4 Depo When Depo not available, uses female condom 30 4 Norplant Previously used Depo to space her children 31 4 TL Oral contraceptives for 12 years 31 6 TL When she discussed the method with her husband, he readily agreed 32 6 IUCD 32 3 Female condom Heavy bleeding from Depo so changed method 33 4 TL Depo led to heavy bleeding so switched method 33 4 Male condom Depo made her feel ill so switched 33 4 Depo 35 2 TL Couple wanted no more children. Has been in pain since the procedure was done some months ago. 40 6 TL Natural FP method 40 5 Depo Oral contractive MALAWI CFPHS PROJECT: MID-TERM EVALUATION 69 APPENDIX 5: EXPANDED RECOMMENDATIONS AND COMMENTS FOR STRENGTHENING SERVICE DELIVERY IMPROVING HSA AND CBDA PERFORMANCE AND QUALITY OF CARE The recommendation to develop job aids for HSAs, including a simple one-page protocol for managing side effects, should be a part of refresher training. Some HSAs are already providing pills to women who live closer to them than to the CBDA, referring others to CBDAs when they can better provide better access and resupply. This is very appropriate. CBDAs should also manage DMPA side effects with the same guidance and management for bleeding problems, particularly since they have oral contraceptives for managing prolonged or heavy bleeding, when appropriate. The same safeguards should be put in their protocol/job aid to ensure that the exceptional client who continues to have bleeding is referred to a higher level. Some CBDAs already know that oral contraceptives can be used in this way. The recommendation for HSAs to be trained in providing pills will also give them an appropriate option if there is a stock-out of DMPA to ensure that the women are adequately protected until DMPA stocks become available. HSA refresher training should include guidance as primary supervisors to strengthen these quality issues for CBDAs at monthly meetings. HSAs requested training in both oral contraceptives and HTC. Work load was not a complaint by HSAs and did not appear to be a constraint for those the team met. ASSESSING PERFORMANCE FOR DMPA SERVICES Given the strong preference for DMPA, HSA performance will likely be the main contribution of the CFPHS project for increasing contraceptive prevalence in Malawi. Sound documentation of performance and the elements of success could guide the national program for scaling up DMPA services for the 85% of the population in rural areas. Currently only a small fraction of the HSAs in each of the eight districts are trained to give DMPA. To assess their performance in relation to the total WCBA in the district would be a severe underestimate of their future potential for increasing CPR. Therefore, it will be important to measure HSA DMPA injections only for the WCBA in their catchment areas. The DMPA CYPs per 100 WCBA (in the HSA catchment area) can be a reasonable estimate of DMPA method-specific CPR. This could be aggregated for the district using total HSA CYPs for DMPA divided by the total WCBA in their combined areas. The epidemiologic principle is to sharpen the numerator and denominator of the measurement so that all in the denominator (WCBA) are eligible to be in the numerator (DMPA current users). Some of the family planning coordinators (e.g., Chikwawa District) have disaggregated the community-based services from HF services for pills, condoms, DMPA by month and by initial, repeat and restart clients. This data is another source for districts and CFPHS to assess more accurately the added contributions of the project over time in relation to services at the HFs. COMMUNITY-BASED DISTRIBUTION AGENTS: PROVIDING FAMILY PLANNING AND HIV TESTING AND COUNSELING The provision of home-based FP in combination with HTC is acceptable and welcomed by community leaders and couples in the few areas the team directly observed. Having a CBDA who is well-known in the community offering both makes it easier for women to be tested, since the 70 MALAWI CFPHS PROJECT: MID-TERM EVALUATION CBDA also provides FP (thereby reducing the risk of stigma, since she could be receiving either service). Pregnant women also prefer to be tested in their home rather than at the HC. There is anecdotal information that couple testing is increasing, with one verbal report that 50% of couples have been tested in some CBDA catchment areas. However, stock-outs of HIV test kits are severely hampering performance and are leading to deterioration of skills. HTC training for CBDAs should be conducted only when a good supply of test kits is available for them to immediately use their new skills. After the first cycle of pills, the woman is advised to visit the HC to have a medical assessment. Many women decide not to make the HC visit and CBDAs do not withhold pills (two cycles for repeat visits). This is appropriate. After using the checklist, there should be no need to refer, unless there was a contraindication or some special problem. Senior nurse trainers in the field also agreed that it was unnecessary and undesirable to have all women visit the HC after starting pills. The team recommends continuing to provide ongoing cycles of pills for women who decline to visit the HC after receiving the first cycle. This visit to the HC is not necessary (after completing the checklist), sends a negative message to the community, and is a waste of health resources. The guidance to visit the HC after one cycle should be discontinued. The simple written ―insert‖ on pills should include information on how to use, common non￾harmful side effects, and correction of common misconceptions—which CBDAs should provide to all pill-users for users, their husbands, and community members. Management of the most common side effects is often weak or lacking. CBDAs agree that bleeding irregularities are the most common complaints. Referral to the HC is frequent, though unnecessary and inadvisable for common non-harmful side effects—for which reassurance is the first line of management. A one-page job aid for the CBDAs on pills should also be developed to complement the written information to clients. The job aid will elaborate on management of common side effects and when to appropriately refer the exceptional client. This should be incorporated into refresher training and CBDA updates. It should also be used in regular supportive supervision by HSAs for helping CBDAs improve quality of care. CBDA RETENTION Sustainability and retention of CBDAs as volunteers is a challenge that has been noted in several previous programs (B. McHenry, 2008). CBDAs for other dedicated services in Malawi have been recruited and trained, only to have them drop out or discontinue working after the project ends. Sustainability of the CBDA program remains a big question for providers in the field and for MSH. There is no clear plan for retaining CBDAs newly recruited for FP and HTC. Some drop out to take other work. A few have been found non-productive due to poor initial selection and have been replaced. However, in general, the community selection process under the leadership of the village headman appears to be sound. The Brigit McHenry document, Operations and Policy Issues affecting Community-based Distribution of Injectable Contraceptives, includes the following expanded list of issues to consider for retaining CBDAs and for improving performance that is shown below. MALAWI CFPHS PROJECT: MID-TERM EVALUATION 71 Further exploration with the MoH is needed for how CBDAs can be retained. In the recently￾approved CFPHS project work plan, refresher training for CBDAs is planned, as well as added support for CBDAs. The specifics of support for CBDAs are currently being developed by the project. DATA COLLECTION SYSTEMS AND LINKS WITH THE HC AND DISTRICT HMIS HSAs and nurses both give DMPA at a remote health center when the nurse comes for a special day to see antenatal women, give immunizations, etc. One HSA complained that the DMPA he gives on this day is counted as performance for the nurse. One register for both was suggested as a way to avoid this conflict. SUPPORT FOR FACILITY-BASED FP SERVICES With regard to the problems of sterilizing trocars and packs for implants, CFPHS and the MoH should explore obtaining disposable trocars for Jadelle (and for future procurements of Sino￾Plant [Zairn]). The team understands UNFPA may be able to obtain these items. Alternatively, low-cost non-electric pressure cooker style sterilizers could be provided at the HC level where Jadelle or IUCDs can be provided. Basic training for these sterilizers at the HC will also be needed for the nurse. HEALTH SECTOR STRENGTHENING The District Implementation Plans (DIPs) include a large number of action items. The MSH DIPs are produced in concert with the DHO to ensure compatibility. However, some DHOs expressed the need for better coordination and more frequent meetings between the CFPHS district coordinator and the DHMT in relation to the two plans. This should result in better planning and problem-solving for ensuring sufficient contraceptive supplies and improved performance of the district and the CFPHS project. It would be advisable to hold more regular “Non-negotiable Requirements for Making Community-based Programs and Workers Better” (B. McHenry, June 2008) Clear tasks and functions of CHWs Appropriate recruitment and selection Standardized curriculum for initial training and continuing education Accountability and record-keeping Improvement of supportive supervision and quality of care Career prospects, remuneration, incentives Substantial and consistent provision of resources Involvement and reorientation of health service staff Leadership and advocacy (political, community, champions, etc.) Policy, guidelines, standards, regulation Community participation and management Evidence-based action Intersectoral collaboration 72 MALAWI CFPHS PROJECT: MID-TERM EVALUATION meetings (quarterly at least) with the DHO to ensure that the CFPHS implementation activities are in harmony with the DIP. INVOLVING THE FAITH-BASED COMMUNITY In support of introducing and strengthening the Standard Days Method and the use of CycleBeads, there will be a need for alternatives to the importation of CycleBeads, given the large interest. Local production of CycleBeads has not been successful in other countries, and the team is not suggesting this course for Malawi. The paper illustration of CycleBeads was well understood as a teaching aid (for the woman, her husband and the community) as well as a temporary way of practicing the method when CycleBeads are not available. The day of the week can be marked successively, rather than advancing a ring, starting with the first day of menstruation and following the standard instructions for CycleBeads. Muslims and Protestant leaders also consider this fertility awareness￾based method to be important as both a primary or back-up method. CFPHS should produce low￾cost paper illustrations of CycleBeads (which include the SDM instructions) for teaching and as a reminder for the couple. This has been found to be helpful for SDM introductions by the Institute for Reproductive Health (IRH) in other countries. With trained catechists, consider introducing an illustrated paper version for using SDM when CycleBeads are not available. Good illustrations of CycleBeads with instructions can be found in IRH training materials and slide presentations through the IRH web site: http://www.irh.org; http://www.irh.org/?q=resources; http://www.irh.org/sites/default/files/All%20about%20CycleBeads_ENG.pdf; For more information, please visit http://www.ghtechproject.com/resources Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com