THE ENHANCED HIV/AIDS PREVENTION AND IMPROVED FAMILY HEALTH PROGRAM (EHAP-IFH), MALAWI December 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by Gisele Maynard-Tucker, Janet Gruber, and Richard Pollard through the Global Health Technical Assistance Project. THE ENHANCED HIV/AIDS PREVENTION AND IMPROVED FAMILY HEALTH PROGRAM (EHAP-IFH), MALAWI DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 08-001-122) is available in printed or online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources.aspx. Documents are also made available through the Development Experience Clearinghouse (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. Malawi EHAP-IFH Program i ACRONYMS ABC Abstinence, Being Faithful, Condom use ACT Artemisinin-based combination therapy AIDS Acquired immune deficiency syndrome ART Antiretroviral treatment BC Behavior change BCC Behavior change communication BM Behavior maintenance CDC Centers for Disease Control and Prevention (U.S.) DAC District AIDS Committee (Malawi) DFID Department for International Development (U.K.) DHS Demographic and Health Survey DOD Department of Defense (U.S.) DSD Delayed sexual debut EHAP-IFH Enhanced HIV/AIDS Prevention and Improved Family Health Program FBO Faith-based organization FCP Faith Communities Program GOM Government of Malawi GTZ German Technical Cooperation HCT HIV counseling and testing HDI Human Development Index HIV Human immunodeficiency virus IEC Information, education, and communication IPC Interpersonal communication ITN Insecticide-treated mosquito net JHU John Hopkins University JICA Japan International Cooperation Agency KfW Kreditanstalt für Wiederaufbau (Germany) LLIN Long-lasting insecticide-treated mosquito net MDF Malawi Defense Force MDG Millennium Development Goal M&E Monitoring and evaluation MICS Multiple Indicator Cluster Survey MOH Ministry of Health MSI Marie Stopes International MVU Mobile Video Unit NAC National AIDS Commission (Malawi) NAPHAM National Association of People Living With HIV/AIDS in Malawi NGO Nongovernmental organization NMCP National Malaria Control Program (Malawi) ORS Oral rehydration salts OVC Orphans and vulnerable children PACT Program for Advancement of Commercial Technology PEPFAR President‟s Emergency Plan for AIDS Relief (U.S.) PLWHA People living with HIV/AIDS ii Malawi EHAP-IFH Program PMI President‟s Malaria Initiative (U.S.) PMTCT Prevention of mother-to-child transmission PSI Population Services International SOPIR Strategic Objective Performance Implementation Review (USAID) SOW Scope of Work SWAp Sectorwide Approach TOC Targeted Outreach Communications TRaC Track Results Continuously (PSI) UNDP United Nations Development Program UNFPA United Nations Population Fund UNICEF United Nations Children‟s Fund USAID United States Agency for International Development USG United States Government VAC Village AIDS Committee (Malawi) VPU Video Production Unit YA! Youth Alert! Malawi EHAP-IFH Program iii CONTENTS ACRONYMS..................................................................................................................... I EXECUTIVE SUMMARY................................................................................................. 1 I. INTRODUCTION ................................................................................................... 7 PURPOSE OF THE EVALUATION ................................................................................................................ 7 BACKGROUND ....................................................................................................................................................... 8 METHODOLOGY.................................................................................................................................................10 II. FINDINGS BY INTERVENTION AREA ............................................................... 11 BEHAVIOR CHANGE COMMUNICATION INTERVENTIONS ..................................................11 SOCIAL MARKETING INTERVENTIONS ..............................................................................................21 III. PSI MANAGEMENT AND SUSTAINABILITY STRATEGY ..................................27 MANAGEMENT ....................................................................................................................................................27 SUSTAINABILITY STRATEGY ......................................................................................................................27 IV. CONCLUSIONS AND RECOMMENDATIONS ................................................. 31 APPENDIX A. SCOPE OF WORK .....................................................................................................................35 APPENDIX B. PERSONS CONTACTED.......................................................................................................43 APPENDIX C. REFERENCES ..............................................................................................................................47 APPENDIX D. CROSSCUTTING ISSUES ......................................................................................................49 APPENDIX E. CALCULATING THE IMPACT OF PSI DISTRIBUTION OF ITNS/LLINS ..53 iv Malawi EHAP-IFH Program Malawi EHAP-IFH Program 1 EXECUTIVE SUMMARY INTRODUCTION This evaluation aims to determine the extent to which the Enhanced HIV/AIDS Prevention and Improved Family Health Program (EHAP-IFH) has achieved its objectives specific to HIV/AIDS, malaria, and diarrheal diseases and contributed to the overall United States Agency for International Development (USAID) Malawi Strategic Objective (SO) 8: Improved Health Status of Malawians. The Scope of Work (SOW) for the evaluation set out the following directives: 1. Assess the progress made in implementing the benchmarked activities 2. Assess the data gathering methods and validity of indicators 3. Identify the impact of EHAP-IFH related to behavior 4. Assess the level of population coverage in target areas 5. Evaluate the project‟s organizational system 6. Assess PSI/Malawi‟s technical leadership 7. Review the cost-effectiveness of EHAP-IFH interventions 8. Assess the various approaches used to promote sustainability 9. Assess gender dynamics that influence HIV/malaria/ORS-related behaviors This first evaluation of EHAP-IFH addresses the above questions, contextualizes findings, and provides conclusions specific to each component and the program as a whole. It also addresses EHAP-IFH lessons learned and makes recommendations for future interventions. The evaluation team consisted of three consultants from the Global Health Technical Assistance Project (GH Tech) with expertise in HIV, malaria, child health, and social marketing. The team visited Population Services International (PSI) offices and activities in Blantyre Urban, Blantyre Rural, and Lilongwe, Mulanje, Machinga, and Salima districts. The team met with representatives of USAID, the Government of Malawi (GOM), donor partners, and implementing organizations. PSI/MALAWI ACHIEVEMENTS In the case of several EHAP-IFH components, PSI/Malawi has significantly exceeded the targets set by USAID and contributed to better health outcomes in Malawi through its behavior change (BC) and social marketing activities. Many GOM, donor partner, and civil society stakeholders noted this contribution and recognize PSI/Malawi as a notable partner in national HIV/AIDS, malaria, and diarrheal disease prevention. Progress made in implementing the benchmarked activities: PSI/Malawi has broadly achieved EHAP-IFH objectives and met (and almost always exceeded) the annual 2007 USAID Strategic Objectives Performance Implementation Review (SOPIR) tracking indicators specific to program activities. Charts 1 and 2 and the body of the report provide details of targets reached. Data gathering methods and validity of indicators: PSI/Malawi has undertaken a considerable number of research studies linked to EHAP-IFH and other interventions. It implements a structured research process, including surveys, preparation of creative briefs, and pretesting of BC materials. However, at the outset of EHAP-IFH in 2005, PSI/Malawi did not undertake a pre-“track results continuously” (TRaC) qualitative study of behavioral determinants, which would have 2 Malawi EHAP-IFH Program allowed close tracking within subsequent TRaC and other studies. Therefore, PSI/Malawi has not developed BC process indicators specific to the program that would allow tracking of impacts specific to EHAP-IFH. Furthermore, issues arise regarding the validity of the indicators specific to sufficient disaggregation and gender analysis. (See appendix D.) Behavioral impact: EHAP-IFH has indisputably facilitated knowledge creation regarding HIV, malaria, and diarrheal disease prevention, and has played a major part in supporting BC and behavior maintenance (BM) through its media and interpersonal communication activities. Contribution to positive health outcomes can be measured through its percentage shares in its commodities markets. However, USAID has not required PSI/Malawi to track BC process indicators. Therefore, these impacts are difficult to disaggregate and attribute solely to EHAP-IFH. Population coverage in target areas:1 The weekly Youth Alert! Mix radio program has a national reach of potentially 3 million listeners in the 10- to 24-year age range. Data indicate there are 7,286,000 Malawians under age 18 (UNICEF 2006), so 3 million youth aged 10 to 24 represent a sizeable proportion of all young people. Assessment of the coverage of other EHAP-IFH BC/BM interventions awaits full national data. PSI/Malawi has distributed long-lasting insecticide-treated mosquito nets (LLINs) to every public sector health facility; this provides a strong proxy indicator of total malaria net access for pregnant women and under-5 children. Other PSI/Malawi social marketing commodities achieve broadly national coverage. Project organization: The evaluation team found that overall the organizational structures and systems supporting EHAP-IFH implementation reflect PSI/Malawi‟s strengths in managing reliable commodity distribution, providing technical assistance, and developing and rolling out BC interventions. The implementation of EHAP-IFH has also significantly enhanced PSI/Malawi‟s partnership with collaborating partners. PSI/Malawi has also leveraged USAID core support (e.g., helping to set up the Pakachere Institute, whose competencies include gender). To a lesser extent, PSI/Malawi has engaged with members of district assemblies; district, community, and village AIDS committees; civil society organizations such as the National Association of People Living with HIV/AIDS in Malawi (NAPHAM); and community-based organizations assisting orphans and vulnerable children. EHAP-IFH has not had a dedicated budget line to allow for extended focus. Technical leadership: PSI/Malawi has contributed to technical leadership to BC/BM materials and communications (e.g., through membership in technical working groups), and its inputs are widely appreciated. It has an effective partnership with the Johns Hopkins University BRIDGE project. There are effective synergies, coordination, and information sharing systems in place among PSI/Malawi and other partners in SO 8, such as Management Sciences for Health and the Program for Advancement of Commercial Technology (PACT). Close links and working partnerships are maintained with the National AIDS Commission (NAC), the Ministry of Health, the United Nations Population Fund (UNFPA), and the United Nations Children Fund (UNICEF). In addition, PSI has provided support to capacity development and training of EHAP-IFH volunteers and facilitators. Cost-effectiveness of interventions: PSI/Malawi does not track cost-effectiveness. The evaluation team has been able to make the following interim estimates in terms of beneficiary numbers reached (but not for quality of interventions, sustainability, etc.). Estimates indicate that the cost of BC 1 Malawi has just undertaken its first national census since 1998; when those data are available, PSI/Malawi should be able to provide figures on EHAP-IFH interventions regarding population coverage and disaggregated target groups. No such disaggregated data were available to the evaluation team, and while the 2008 Multiple Indicator Cluster Survey disaggregates to subdistrict level, its sample criteria do not allow for direct extrapolation relevant to EHAP-IFH target beneficiaries. Malawi EHAP-IFH Program 3 interventions per person in 2007 was $6 for Youth Alert!; $0.35 cents for Targeted Outreach Communications (TOC); and $12.75 for the pilot Faith Communities Program (FCP). Condoms cost about $24 per 150, and oral rehydration salts (ORS) $0.29 per sachet. PSI makes a small profit on insecticide-treated mosquito nets (ITNs), and retreatment kits sold for $4.29. It costs $0.75 to distribute each public sector LLIN. Sustainability approaches: PSI has implemented operational plans to reduce costs and streamline its distribution system by reducing its clients by some 40% by using core distributors. PSI has been successful in achieving full commodity and packaging cost recovery of ITNs and ORS but achieves only 38% cost recovery on condoms. It is planning to up-scale its condom product portfolio to improve cost recovery due to the substantial increase in public sector condom distribution. Gender dynamics: While EHAP-IFH includes implicit and ad hoc attention to gender issues, the program has not mainstreamed and systematized attention to the gender dynamics that underpin health and sexual attitudes and behaviors. The 2005 TRaC survey, the primary source of data to underpin EHAP-IFH BC/BM activities, is not sufficiently gender-aware and disaggregated in its analysis. The same is true of subsequent studies, both quantitative and qualitative. Gender analytic processes are not currently incorporated into data collection and analysis. (See appendix D.) KEY RECOMMENDATIONS Behavior Change Communications (BCC) All future PSI/Malawi interventions should segment activities and BC/BM communications into 10- to 14-, 15- to 19-, and 20- to 24-year age groups. The three Youth Alert! interventions should be considered for scale-up and continued funding. Continued support should be given to TOC, with its focus on high-risk groups and drivers of the epidemic. Consideration should be given to scaling up the FCP model, where BC/BM communication works through traditional and religious structures while challenging harmful societal and behavioral norms. PSI/Malawi should support creation and regular updating of a live referral register for HIV and other sexual and reproductive health needs. A register would support individuals‟ desired behavior change (e.g., the wish to have HIV counseling and testing). Condoms In view of the increased public sector free supply of condoms, PSI/Malawi should reposition its existing brand to cover its condom and packaging costs over time; it should continue its present plans to launch brand extensions at full commodity cost recovery. PSI/Malawi should work with the public sector to develop joint promotional activities and to ensure that free condoms distributed into the market are targeted better to those most in need under a total market approach. Malaria ITNs/LLINs PSI/Malawi and donors must prioritize resolution of PSI access to costs for public sector net distribution. 4 Malawi EHAP-IFH Program In view of the prospects for free public sector distribution of LLINs to all households, PSI needs to carefully assess its strategy to launch a commercial LLIN. This will help reduce leakage into the commercial sector but will add management and resource constraints if sales are to recover all costs. PSI should be employed to support malaria-component information, education, and communication activities to ensure correct use of nets and should consider accessing NAC funding in support of promoting the importance of LLIN use by people living with HIV/AIDS (PLWHA). PSI needs to work closely with UNICEF and USAID to find creative solutions to funding public sector net distribution costs. ORS and Other Commodities Thanzi sales growth requires major increases in promotional and BCC spending, with attention to preventive as well as curative behaviors. PSI/Malawi needs to increase rural commodity coverage through higher spending on BCC and promotional costs, which will strengthen rural demand generation balanced by responsive distribution systems. Gender Any future PSI/Malawi BC/BM interventions should further strengthen girls‟ capacity to communicate and negotiate. Gender should be more systematically mainstreamed throughout PSI/Malawi programs and activities. Gender analysis should be incorporated into all qualitative and quantitative research activities and inform all BC/BM message development. Research/Monitoring and Evaluation (M&E) Donors in the Malawian health sector should ensure all future BC/BM interventions have an adequate Q2 research/M&E budget line, with provision for disaggregated and gender analytic behavioral surveys and potential for participatory M&E. Technical assistance should be provided to PSI/Malawi for the further development of a qualitative approach to targeted BCC. Future BC programs should address the potential for including qualitative process indicators in addition to quantitative outcome indicators, which will track with more precision the actual impacts of specific interventions on health (see also annex 1). Sustainability Continue to achieve maximum cost recovery of commodities. Sustainability is dependent on assured national BCC funding and programming. PSI/Malawi should further strengthen its partnerships with national and international partners working on BCC in Malawi. PSI/Malawi should further develop its links with SWAp partners on BCC. Malawi EHAP-IFH Program 5 CHART 1: EHAP-IFH: Achievement of USAID SOPIR Objectives EHAP-IFH intervention 2007 # reached/served (% target achieved) (total year) 2008 # reached/served (% target achieved) (part year) 1. BCC interventions (in support of SOPIR indicators 2.1, 2.2, 2.3, 5.2, 8.1) 1A: Youth Alert! (YA!) YA! Mix radio program (2.1, 2.2, 5.2) No specified target; potentially 3 million listeners 10-24 No specified target; potentially 3 million listeners YA! Mix shows (2.1, 2.2, 5.2) 15,000 (750%) 6,000 (300%) YA! Listener Club (2.1, 2.2, 2.3, 5.2) 17,754 (296%) 16,305 (113%) YA! Listener Club Open Days (2.1, 2.2, 5.2) 21,000 (140%) 27,036 (96%) YA! Schools Program (2.1, 2.2, 2.3, 5.2) 103,421 (118%) 111,834 (120%) YA! Teacher/patron training (2.3) 2,954 (no specified target) 2,357 (92%) 1B: Targeted Outreach Communications (TOC) TOC: High-risk sites (5.2, 2.1, 2.2) 4,408 (109%) 5,951 (110%) TOC: High-risk group interventions (5.2, 2.1, 2.2) (between 100 and 114%) (between 112 and 153%) TOC: Drama shows (2.1, 2.2, 5.2, and also ITN and diarrhea) 201,600 (no % specified) 289,159 (115%) TOC: MVU shows 5.2, 2.1, 2.2, and also ITN and diarrhea) 123,000 (103%) 135,525 (103%) 1C: Pilot Faith Communities Program (FCP) FCP: Youth (2.1, 2.2, 5.2) 1,917 (123%) 1,800 (107%) FCP: Adults (2.1, 5.2, 2.2) 2,150 (180%) 2,400 (111%) FCP: OVC (8.1) 212 (141%) 150 (146%) FCP: Faith and committee leaders (5.2, 2.1 - 2.3) Faith leaders: 166 (332%) Committee leaders: 31 (155%) Faith leaders: 319 (160%) Committee leaders: 90 (141%) FCP: TOC 1,362 (170%) 1,600 (91%) 2. Targeted condom service outlets (5.1 and proxy for condom social marketing) SOPIR objective 1,167 (175.5 %) 338 (76.8 %) 3. Number of child diarrhea cases treated in USAID-assisted programs (EHAP-IFH and the Child Survival Program, both ORS and WaterGuard) SOPIR objective 598,620 (124%) 544,776 (72%) 4. Number of ITNs distributed purchased/subsidized with U.S. Government support SOPIR objective 185,375 (no target) 293,568 (34%) 5. Numbers trained in strategic information (M&E, surveillance, health management information systems) SOPIR objective 13.2 20 (100 %) 15 (100 %) 6. Numbers trained to promote HIV/AIDS prevention through abstinence and/or abstinence/be faithful messages (additional to those trained under YA! and the pilot FCP) SOPIR objective 2.3 197 (281%) 203 (76.9%) 6 Malawi EHAP-IFH Program CHART 2: EHAP-IFH Commodity Sales Performance 1. Condom sales and market share 2002 2003 2004 2005 2006 2007 2008 2009 Target 6,300,000 7,700,000 9,000,000 9,000,000 9,700,000 10,000,000 7,450,000 9,250,000 Achieved 7,255,804 8,414,316 8,563,494 8,852,949 9,201,726 9,057,033 Mkt. Share 34% 50% 29% 33% 28% 22% 2. Commercial ITN and retreatment kit sales ITNs Target 90,000 100,000 121,870 124,000 175,000 195,000 Achieved 79,134 87,467 116,880 109,955 103,165 157,482 Retreatment Target 60,000 69,000 76,500 100,000 110,000 194,000 220,000 250,000 Achieved 81,580 101,624 84,176 104,581 161,283 211,820 3. Public sector distribution ITNs Target 250,000 590,000 1,290,000 1,550,000 1,621,870 Achieved 372,911 1,069,845 1,364,843 1,472,720 1,874,089 842,251 2,500,000 (est.) 4. ORS sales Target 550,000 735,000 914,500 1,320,000 1,100,000 1,200,000 1,690,000 Achieved 829,560 1,582,560 1,197,592 1,024,547 982,548 1,246,584 Malawi EHAP-IFH Program 7 I. INTRODUCTION In July 2005, Population Services International (PSI)/Malawi received funding from the United States Agency for International Development (USAID) to implement the Enhanced HIV/AIDS Prevention and Improved Family Health Program (EHAP-IFH) in Malawi. The program focuses on continuing activities of the AIDSMark Project, with expanded attention to behavior change communications (BCC) and condom promotion targeting high-risk groups. EHAP-IFH was originally planned to run for 21 months to March 2007. Funding extensions have ensured EHAP￾IFH‟s continuation to the end of September 2008; a final six-month funding extension to March 2009, to close out EHAP-IFH field operations and prepare final reports, is to be decided imminently. Since July 2005, implementation of EHAP-IFH has been informed by external factors. These include the naming of Malawi as a nonfocus country of the President‟s Emergency Plan for AIDS Relief (PEPFAR) in late 2005, which resulted in PSI/Malawi reporting on interventions via Country Operating Plans under PEPFAR and tied to U.S. Government (USG) fiscal years. Another significant development was that in early 2007 Malawi was selected as a new target country of the President‟s Malaria Initiative (PMI); as a result, PSI/Malawi was requested to distribute 185,400 long-lasting insecticide-treated mosquito nets (LLINs) funded by a six-month extension to EHAP￾IFH. In mid-2007, PSI/Malawi was asked to prepare a 12-month cost extension to cover fiscal year (FY) 2008; this extension includes new funding for malaria prevention and treatment through the distribution of PMI-funded LLINs and joint development with the National Malaria Control Program (NMCP) of information, education, and communications (IEC) materials on artemisinin￾based combination therapy (ACT). The 2008 cost extension also covered oral rehydration salts (ORS) and includes funds for HIV/AIDS activities. In mid-2008, PSI/Malawi, USAID, and public sector partners agreed that activities linked to malaria prevention and treatment would continue to the end of FY 2009; funding has also been made available to allow final procurement of an additional 1 million ORS sachets. This FY 2009 cost extension assumes that PSI/Malawi can continue to cover a large proportion of its core costs, currently covered by EHAP-IFH through the HIV prevention component, by obtaining new USAID and National AIDS Commission (NAC) funding for HIV prevention. PURPOSE OF THE EVALUATION The purpose of this evaluation was to determine the extent to which EHAP-IFH has achieved its objectives specific to HIV in mitigating the impact of HIV/AIDS and to malaria and diarrheal diseases, thereby contributing to the overall USAID/Malawi Strategic Objective 8: Improved Health Status of Malawians. In the case of several EHAP-IFH components, PSI/Malawi has significantly exceeded the targets set by USAID. The Scope of Work (SOW) for the evaluation set out the following directives: 1. Assess the progress made in implementing the benchmarked activities 2. Assess the data gathering methods and validity of indicators 3. Identify the impact of EHAP-IFH related to behavior 4. Assess the level of population coverage in target areas 5. Evaluate the project‟s organizational system 8 Malawi EHAP-IFH Program 6. Assess PSI/Malawi‟s technical leadership 7. Review the cost-effectiveness of EHAP-IFH interventions 8. Assess the various approaches used to promote sustainability 9. Assess gender dynamics that influence HIV/malaria/ORS-related behaviors This first evaluation of the EHAP-IFH program addresses the above questions, contextualizes findings, and provides conclusions specific to each component and to the program as a whole. It also addresses EHAP-IFH lessons learned and makes recommendations for future interventions. BACKGROUND2 Malawi health and development situation Malawi is ranked 166 out of 177 countries on the human development index (UNDP HDI 2004); it is one of the 10 poorest countries in the world, and more than half its population lives in poverty, a situation which is only exacerbated by the impacts of HIV/AIDS. Malawians face many barriers to access to health services: 46% of the population lives farther than 5 km from a health facility of any description, and 35% of all health care is provided by faith-based organizations (FBOs) that charge fees. Life expectancy declined from 48 to 41 years between 1990 and 2000. The leading cause of disease burden is AIDS, followed by respiratory disease, malaria, diarrheal diseases, and perinatal conditions. Maternal mortality has increased from 620 to 984 deaths per 100,000 live births since 1992. The Government of Malawi (GOM) has instituted a number of strategies intended to reduce poverty and to improve health outcomes in line with the Paris Declaration and achievement of Millennium Development Goals (MDGs). These include the Poverty Reduction Strategy of the Malawi Growth and Development Strategy; within the health sector, the Sectorwide Approach (SWAp) seeks to implement the Essential Health Package (2004/5–2010/11). The NAC also operates a SWAp-type mechanism. The USG does not participate in SWAp mechanisms; it provides discrete partner funding, while engaging in joint planning with the GOM and other donor stakeholders. The HIV/AIDS situation in Malawi At least 14% of the total population of 12 million to 13 million Malawians (between 1.68 million and 1.82 million people) is estimated to be HIV positive. Ninety thousand Malawians become infected every year, with profound impacts on overall quality of life; individual, family, and community opportunities; HIV mitigation efforts; and national progress. Malawi has more than 1 million orphans (single and double), at least half of whom are orphans due to AIDS. As is the case in many other sub-Saharan African countries, Malawian women (and especially girls) are more vulnerable to HIV infection due to a combination of physiological and societal factors and deep-rooted gender inequalities. Young women aged 15 to 24 are nearly four times more likely to be 2 Data in this section are derived from a number of sources, including the NAC 2005 National HIV/AIDS Action Framework; the Malawi Demographic and Health Survey 2004; the UNICEF Malawi Multiple Indicator Cluster Survey 2006: Final Report; the DFID Country Assistance Plan 2007–2011 (draft); and the USAID and UNICEF Malawi websites. Information was also provided to the evaluation team during meetings with EHAP-IFH public sector, donor, and civil society partners. Malawi EHAP-IFH Program 9 infected with HIV than young men: HIV prevalence in 2005 among young men aged 15 to 24 was 2.1%, while among young women it was 9.1% (UNICEF data). Various ethnic groups continue to adhere to a number of customary practices that increase vulnerability to HIV infection, especially among women. These include widow and widower inheritance, death cleansing, enforced sexual initiation for girls (often by much older men), and wife and husband exchange. Women and girls bear the brunt of caring for HIV-positive family members, and their opportunity costs specific to gaining access to correct information and to health services, including HIV counseling and testing (HCT) and prevention of mother-to-child transmission (PMTCT) services, are invariably high. The child health situation in Malawi Although child mortality in Malawi remains high by global standards, the country has achieved significant progress in reducing child deaths. From 1990 to 2005, the under-5 mortality rate fell from 221 deaths per 1,000 live births to 125. The 2006 under-5 mortality rate of 120 deaths per 1,000 live births is the same as or lower than that of 30 other countries in sub-Saharan Africa. Malawi is on track to meet MDG 4 (reduce child mortality) and Target 5 (between 1990 and 2015, reduce the under-5 mortality rate by two-thirds). Success has been achieved by eliminating neonatal tetanus and polio, by achieving high measles vaccination coverage, and by increasing malaria prevention. Despite such successes, one in every eight children still dies before reaching his or her fifth birthday; malnutrition is widespread (nearly half of all children under 5 are stunted, and 22% are severely stunted); and PMTCT services are currently available to very few. Women give birth to an average of six children, one of the highest fertility rates in the region. Child mortality, fertility, and malnutrition rates are strongly associated with mothers‟ level of education and wealth. Dehydration as a result of severe diarrhea is a leading cause of illness and mortality among under-5 children in Malawi. It is in the above context that support to ORS procurement and promotion represents an integral component of EHAP-IFH. The malaria situation in Malawi Malaria is endemic in all parts of the country, with seasonal peaks between December and June. The majority of cases are caused by P. falciparum, but most are not confirmed. There is no evidence of a decline in malaria cases and deaths over the 2001–2006 period despite improvements in coverage with insecticide-treated mosquito nets (ITNs). The NMCP distributed more than 4.6 million ITNs between 2003 and 2006, and a 2006 survey found that 36% of households had at least one ITN. In 2006, 24% of febrile children under age 5 were treated with an antimalarial; ACT was adopted as the recommended method of treatment only in 2007. In addition, $23 million was spent on malaria control in 2006, provided by government, the Global Fund to Fight AIDS, Tuberculosis and Malaria, the World Bank, and United Nations agencies. USAID/Malawi USAID implements a significant program in Malawi, providing about $35 million annually in development assistance. One priority area of assistance is to improve care for HIV/AIDS and for health overall. USAID support to HIV and health is implemented through a range of partners, most notably the GOM, as well as through other USG agencies such as the Department of Defense (DOD), civil society organizations, and contractors. HIV prevention activities focus on the ABC approach (Abstinence, Being faithful, and correct and consistent use of Condoms, where appropriate). Condom sales to high-risk groups show steady rates 10 Malawi EHAP-IFH Program of increase. USAID/Malawi‟s focus on maternal and child health addresses malaria and diarrheal diseases. PSI/Malawi is a partner in all areas of USAID/Malawi‟s HIV and health interventions. METHODOLOGY The evaluation team consisted of three consultants with expertise in HIV, malaria, and child health contracted by GH Tech. After an initial briefing from USAID/Malawi chief technical officers and officers responsible for EHAP-IFH, the team left for Blantyre. Discussions were held with PSI/Malawi staff members; EHAP-IFH field workers, warehouse distributors, and wholesalers were visited; and marketing/cost analyses and surveys were conducted. Field visits were made to Mulanje District to observe the Youth Alert! Secondary Schools Program and to attend a session of the Youth Alert! Listener Club. The team visited Mpemba and Lirangwe in Blantyre Rural; observed Targeted Outreach Communications (TOC) and Faith Communities Program (FCP) activities; and interviewed program beneficiaries, health workers, and program volunteers. The team also attended a TOC night show in the Blantyre Kamba „hot zone‟ and visited PSI/Malawi „hot spots‟. Team members were able to participate in a Youth Alert! mega-show in Machinga and to attend a Listener Club Open Day in Siya-Siya community, Salima District. The team held meetings with public sector and donor partners in Lilongwe and also attended further TOC activities. The team held focus group discussions and key informant interviews and also made use of participant observation. PSI/Malawi staff members and program volunteers provided translation. Once data were collected, the evaluation team‟s findings were analyzed with regard to the nine objectives of the SOW and the targets attained concerning the USAID Strategic Objectives Performance Implementation Review (SOPIR) indicators. Following the analysis, the team considered the implications for future recommendations to be undertaken for EHAP-IFH components and PSI‟s operational sustainability. The team discussed work plans and outputs needed. The team met every other day to discuss the progress of their work and project issues. They exchanged findings and discussed methods and work plans. Members of the team met with the USAID Director and the Deputy Chief of the Mission to get feedback on their provisional findings, and debriefed with USAID EHAP-IFH officers in charge of the project and PSI staff to discuss findings, targeted indicators, and recommendations. The evaluation team reviewed a large number of program and PSI/Malawi documents, including the 2005 Track Results Continuously (TRaC) quantitative survey, qualitative studies, PSI creative briefs, and mass and mixed media materials created for EHAP-IFH. Quarterly reports to USAID and other USAID documents were also examined. The Malawi Demographic and Health Survey (DHS) 2004 was reviewed, as was the 2006 Multiple Indicator Cluster Survey (MICS). Other documents considered include the National HIV/AIDS Action Framework (2006), the Malawi National Condom Strategy (2005), and reports from other organizations engaged in HIV and other health preventive interventions, such as the Johns Hopkins University (JHU)-BRIDGE project (see appendix C). Malawi EHAP-IFH Program 11 II. FINDINGS BY INTERVENTION AREA BEHAVIOR CHANGE COMMUNICATION INTERVENTIONS Youth Alert! Findings Youth Alert! is a branded multiplatform behavior change communications (BCC) strategy developed to appeal specifically to young people. It started in 2001, targeting the 15- to 24-year age group. EHAP-IFH, of which Youth Alert! forms a core HIV prevention component, began in July 2005 as a follow-on from the AIDSMark project. Youth Alert! seeks to deepen correct knowledge and to promote behavior change (BC) and behavior maintenance (BM) through the effective use of mass and mixed media and interpersonal communication (IPC).3 It uses a primarily abstinence and be faithful approach, applying condom messages where necessary, to promote and reinforce correct knowledge of HIV prevention and mitigation, abstinence and secondary abstinence, delayed sexual debut (DSD), fidelity, reduction of partners, and safe sex. The 2005 PSI TRaC quantitative report and the 2007 Social Norms qualitative report have informed the development of Youth Alert! BC messages, as have PSI Dashboards, creative briefs, and pretesting of materials. Table 1: Youth Alert! Targets and Achievements Youth Alert! 2007 USAID Target Achieved % Target Achieved 2008 USAID Target Achieved (to 9/08) % Target Achieved Schools Program Students 87,200 103,421 118 93,500 111,834 120% Schools Program Teachers -------- 2,954 No defined target 2,550 2,357 92% Youth Alert! Mix (radio program) 3,000,000 No defined target: 3 million potential listeners in 10 to 24 age range 3,000,000 No defined target: 3 million potential listeners in 10 to 24 age range Listener Club 6,000 17,754 296 14,400 16,305 113% Listener Club Open Days 21,000 29,458 140 28,080 27,036 96% Live Shows 2,000 15,000 750 2,000 6,000 300% The 2008 TRaC study (being analyzed as of September 2008) will provide quantitative data on BC/BM specific to Youth Alert! and also to the TOC and FCP components of EHAP-IFH (see appendix D for PSI research issues). Due to its timing, its findings were not available to the evaluation team; only when the findings are disseminated will full appraisal of the degree to which EHAP-IFH has met the BC-specific USAID SOPIR indicators (2.1, 2.2, 2.3, and 5.2) be possible. 3 The MICS (2008) indicates the need for continued focus on knowledge creation as well as BC/BM: between 2000 and 2006, no more than 36% of young men and 41% of young women had comprehensive knowledge of HIV prevention. In addition, new cohorts constantly enter the target age range for EHAP-IFH HIV components. 12 Malawi EHAP-IFH Program Youth Alert! has been funded by USAID (under EHAP-IFH), the German government Kreditanstalt für Wiederaufbau (KfW), JHU, and PSI discretionary funding. JHU and PSI/Washington have been cofunding Youth Alert! activities since, respectively, March 2004 and July 2007. In 2007, JHU contributed $69,600 toward Youth Alert! activities, while PSI/Washington provided $55,100. A total of $971,325 was spent on Youth Alert! in 2007, of which $681,400 was financed by USAID under EHAP-IFH. PSI/Washington has pledged to provide $385,200 in support of Youth Alert! activities from July 2008 to June 2010. USAID currently funds the following Youth Alert! BCC activities: The Youth Alert! Mix weekly 30-minute radio program and its second broadcast The entirety of the Youth Alert! Schools Program The entirety of the Youth Alert! DSD campaign USAID currently cofunds the following Youth Alert! BCC activities under a PSI subagreement with the JHU-BRIDGE project: The Youth Alert! Listener Clubs in the eight JHU-BRIDGE districts4 The third weekly radio broadcast of Youth Alert! Mix each Thursday afternoon (when Listener Clubs meet across Malawi) Listener Club Open Days, where community members receive HIV IEC and BCC In addition, the development of the Youth Alert! Schools Program Teachers‟ Guide was facilitated by JHU-BRIDGE, as was teacher training. Youth Alert! interventions The first Youth Alert! intervention was the nationwide Secondary Schools Program, initiated in 2001 with the goal of imparting life skills and BC messages consistent with the Youth Alert! objectives. PSI states that by „using interactive methods such as presentations, role-plays, participative drama, debate and discussions, the Schools Program educates, guides, and motivates young people to adopt key risk avoidance and risk reduction behaviors‟. Despite its name, the Schools Program addresses out-of-school youth as well; in research, secondary school students evinced a desire for out-of￾school youth to be allowed to participate. The next step was the creation of Youth Alert! Magazine (more accurately, a one-off life skills manual); to date, 240,000 have been distributed to support and expand BC through IPC. The intention was to reinforce messages between Youth Alert! team visits. Due to budget constraints, Youth Alert! Magazine is only available in English, which PSI acknowledges limits its efficacy. There are currently five Youth Alert! Schools Program teams (two in each of the southern and central regions and one in the northern region); on average, each of the nearly 1,000 state and private secondary schools (which fewer than 15% of all Malawians of secondary-school age attend5 ) is visited once a year for Youth Alert! outreach and IPC activities. More than 360,000 students have 4 Balaka, Chikwawa, Kasungu, Mangochi, Mulanje, Mzimba, Ntcheu, and Salima districts. All are in the southern region and were chosen by USAID. 5 MICS 2008: Secondary school male net attendance ratio (2000–2006): 27% Secondary school female net attendance ratio (2000–2006): 26% [data from 2006]. Malawi EHAP-IFH Program 13 been reached during the entire period, and 2,954 teachers have been trained using the dedicated teachers guide. In 2003, the decision was taken to introduce a mass media component for Youth Alert! The weekly radio program Youth Alert! Mix began broadcasting in February of that year. As of mid-September 2008, 292 radio programs have been broadcast (each is repeated twice within a week, a total so far of 576 airings); 15 more programs are scheduled for 2008. The development of the Youth Alert! Mix radio program represented an expansion of the target beneficiaries to the 10- to 14-year age group. The Youth Alert! Listener Clubs were set up in 2004, then (as now) supported by JHU-BRIDGE. There are currently 200 Listener Clubs, with membership ranging from 10 to 24 years in age and including young people both in and out of school. The Listener Clubs are intended to reinforce the Youth Alert! Mix BC/BM messages through listening to the rebroadcast of Youth Alert! Mix, followed by interactive IPC. Each Listener Club session includes a „doable action‟, where members are encouraged to undertake an activity that reinforces the Youth Alert! Mix radio program messages. Discussions are led in each location by a male and a female Listener Club leader. All Listener Clubs have a Patron or Matron, a respected member of the local community for whom Youth Alert! has provided sensitization and training. PSI/Malawi Listener Club officers seek to visit each Listener Club every month to supervise and ensure the quality of discussions; this is often not possible, however, due to staffing and/or transport constraints. The current situation where Youth Alert! Mix and the Listener Clubs target the entire 10- to 24-year age range is recognized by PSI/Malawi as extremely challenging and less than ideal for optimally targeted BC/BM messages. Listener Club Open Days are occasionally held to sensitize community members, reinforce messages, and also to address problems such as poorly functioning Listener Clubs and adverse societal pressures (e.g., continued physical violence within the household; intergenerational sex). Listener Club Open Days are often attended by members of district AIDS committees (DACs), e.g., the district HIV/AIDS coordinator, and sometimes also by DAC community and village counterparts (members of community and village AIDS committees). JHU-BRIDGE district coordinators frequently participate. The evaluation team had the opportunity to visit an Open Day in Salima District (Siya-Siya community). This event demonstrated the degree to which PSI/Malawi seeks to embed its Youth Alert! approach and activities within traditional structures, to engage gatekeepers without whose permission such activities could not occur, and to move forward on BC issues. The event equally indicated the challenges faced by PSI/Malawi and other organizations in building and sustaining such links and the need for multisectoral partnerships. The „Real Man/Real Woman‟ DSD video and poster campaign began in 2005, focusing on abstinence/be faithful messages to lower-income groups and those describing themselves as religious. Its mandate has been to counter peer and intergenerational pressure that „real‟ men and women start being sexually active at a young age. The DSD campaign was linked into the Youth Alert! music competition, whose winners were featured in a one-hour TV program entitled „My Choice‟. To date, the NAC has funded and disseminated 500 copies of the DSD video to secondary schools; it has further funded 200 signed copies of „My Choice‟ for the deaf. All such materials were created in house at PSI/Malawi by the Video Production Unit (VPU), which works on behalf of all PSI/Malawi programs and activities. 14 Malawi EHAP-IFH Program Brand awareness activities (e.g., Youth Alert! Mix mega-shows that can draw crowds upwards of several thousand) take place regularly. Non-USAID- funded Youth Alert! activities KfW funded a pilot standalone Youth Alert! Peer Education (YAPE!) intervention in two districts, Mwanza and Balaka. This ended in 2007. A key lesson learned was that YAPE! was an expensive means of IPC, due to the constant need for peer educator training and refresher training; linked to this is the issue of peer educators and peers moving on or disengaging, with new age cohorts continuously requiring education. Youth Alert! Conclusions Youth Alert! interventions have reached significant numbers of young people and have supported and sustained knowledge creation and BC/BM related to HIV prevention and mitigation. Youth Alert! branding has achieved high recognition value and trust for its activities among young Malawians. This represents a significant achievement. In addition, the $6 cost per person reached in 2007 (for the entirety of Youth Alert! interventions) represents a cost-effective balance of resources and reach. It is also close to the international average for abstinence-focused interventions. Youth Alert! applies an integrated approach: Youth Alert! Mix, the Listener Clubs, and, as far as possible, the Schools Program address the same BC/BM issue at the same time. Thus mass media and IPC activities in early September 2008 focused on young people‟s ability to discuss sexual issues with parents. Such complementarity reinforces messaging and supports communication. Engagement with volunteer gatekeepers and beneficiaries represents another positive and participatory Youth Alert! approach, one that will serve to promote its BCC among target groups. There are issues of optimal reach and focus connected to the Schools Program component; these are addressed below in the recommendations. Youth Alert! Recommendations These are based on lessons learned by PSI/Malawi and observations made during the EHAP-IFH evaluation. It is strongly recommended that elements of Youth Alert! continue to be supported by the GOM and/or donor partners; its activities have undoubtedly contributed to both expanded HIV prevention knowledge creation and to BC/BM among young Malawians. Youth Alert! is currently not sustainable by PSI/Malawi without such external support. PSI/Malawi and existing/potential funding partners should re-evaluate indicators and include quality criteria and process indicators to track impact, as well as existing outcome indicators (target reach numbers). This is a common recommendation for any future work on current EHAP-IFH HIV components. It is essential that all future Youth Alert! interventions segment activities and BC/BM communications into 10- to 14-, 15- to 19-, and 20- to 24-year age groups. The following issues need to be speedily addressed and resolved: – All future Youth Alert! interventions should be more effectively informed by research that is closely disaggregated and analyzed and also supported by gender Malawi EHAP-IFH Program 15 analysis (this is a recommendation common to all EHAP-IFH BC/BM interventions). – Consideration should be given to PSI/Malawi support for the creation of a live referral register, where all relevant organizations commit to regular updating and information sharing. – PSI/Malawi should strategically plan for closer and more sustainable engagement with decentralized structures (district assemblies and lower) for Youth Alert! and all other BC programs. The best use of its comparative advantage would be to deepen its partnership with a group such as PACT, which has strengths in organizational capacity building and development. PSI/Malawi and PACT are currently in the process of submitting a proposal to the Global Fund‟s Round 7; this would in part address such issues. Targeted Outreach Communications (TOC) Findings The work of TOC spans all EHAP-IFH program areas (condom social marketing, HIV prevention, malaria, and diarrheal diseases). TOC engages in both BC/BM activities and provision of product information. All BC/BM interventions are “based on key determinants of behavior identified by research” (see appendix D for further discussion of issues linked to PSI/Malawi research). TOC employs four two-person teams (two in the southern region, one each in the central and northern regions), which cover even the most rural areas of Malawi. It also works with seven drama troupes, whose shows address all EHAP-IFH activities and target people aged 15 and older; community dramas disseminate HIV/AIDS IEC and BC messages as well as inform people about PSI/Malawi products. TOC has subcontracted its product promotion shows to two local commercial companies; these shows cover all PSI/Malawi products. Presentations by TOC‟s Mobile Video Unit (MVU) inform the general public about PSI/Malawi products. They target various groups, such as caregivers of children under age 5, when a Thanzi ORS video is being shown. MVU shows additionally address HIV prevention through knowledge creation and BC messages centered on safe sex and the use of the Chishango condom. Such shows also target high-risk groups, such as fishing communities and truck drivers, using interactive audiovisual shows with question-and-answer sessions. USAID funding to TOC under EHAP-IFH was $88,376 in 2007, representing 47% of the total program budget; the Japan International Cooperation Agency (JICA) provided $62,691 (33%), while the U.S. DOD provided $37,860 (20%). USAID funding for TOC under EHAP-IFH breaks down as follows: The funding of local salaries for three of the four TOC teams The funding of the two VPU technical specialists‟ salaries and other film production costs The funding of all drama troupe activities 16 Malawi EHAP-IFH Program Table 2: TOC Targets and Achievements Targeted Outreach Communications 2007 USAID Target (people) Achieved % Target Achieved 2008 USAID Target (people) Achieved (to 09/08) % Target Achieved Fishing Communities 7,200 7,350 102 7,800 11,924 153 Police HQ 1,400 1,800 128 3,000 4,721 157 Truck Drivers 400 400 100 400 449 112 High-Risk Sites 4,050 4,408 109 5,400 5,951 110 Drama Shows 201,600 - - 252,000 289,159 115 Product Promotion Shows 90,160 90,160 100 90,160 98,240 109 Mobile Video Unit Shows 120,000 123,000 103 132,000 135,525 103 World AIDS Day 10,000 23,116 231 10,000 23,116 231 Also: MDF Not funded by USAID 4,200 4,810 114 6,640 9,554 144 Since 2006, cofinancing for TOC has consisted of the following: JICA funding has been used to equip the VPU with cameras, computers, editing software, etc. JICA funding has been used to support some VPU film production costs. JICA funding has been used to cover vehicle and per diem costs for three of the four TOC teams (this funding ends in September 2008). Since mid-2007 one of the TOC teams has been funded with DOD support and has been working on Malawi Defense Force (MDF)-targeted activities. DOD funding for TOC has supported other MDF-specific activities and research. TOC interventions TOC HIV prevention work focuses on the high-risk groups listed in table 2. It also operates in high￾risk sites, identified by PSI through its definition of 18 „hot zones‟ across Malawi, within each of which are numerous „hot spots‟ and high-risk sites, e.g., bars, nightclubs, guesthouses, and depots. TOC does not focus on young people, as this target group is addressed through Youth Alert! and the FCP; however, its outreach among both the high-risk groups and in the high-risk sites does include youth. In addition, there has been some TOC contact with young sex workers. Malawi EHAP-IFH Program 17 The TOC EHAP-IFH malaria and child survival (ORS) activities are currently predominantly seasonal (October-November) and focus on malaria and child survival outreach. Outreach is concentrated in locations where caregivers and potential adult beneficiaries congregate, e.g., health facilities. TOC also supports FCP and Youth Alert! FCP support encompasses TOC officers showing VPU￾created abstinence-themed videos during training sessions. TOC assists Youth Alert! with community mobilization and awareness raising, e.g., for Youth Alert! live shows, such as the one held in mid-September 2008 in Machinga. TOC uses BC/BM materials that have all been developed using findings from the 2005 TRaC study; the 2008 TRaC study, currently being analyzed, will provide quantitative data on behavior change specific to TOC. Non-USAID funded TOC activities The U.S. DOD funds all TOC activities with the MDF; as mentioned above, in USAID‟s FY 2007 this amounted to $37,860, representing 20% of the total TOC budget for that year. Since February 2007, the film „A Soldier‟s Life,‟ created by PSI/Malawi‟s VPU for use by the MDF, has been disseminated widely via the TOC teams. Its creation was funded by DOD. More than 5,000 MDF personnel have seen the film since it was released; it focuses on the importance of knowing one‟s HIV status, mitigating stigma and discrimination, and living positively. TOC Conclusions TOC HIV activities target high-risk groups among whose members are to be found some of the epidemiological drivers of Malawi‟s HIV/AIDS epidemic. HIV prevention is acknowledged globally as requiring a balanced approach that encompasses both general populations and the activities of high-risk groups. Therefore, TOC represents an essential element of HIV prevention, working with groups of people who may well be difficult to access and who are often high risk in both attitudes and behaviors and resistant to prevention messages. TOC applies an imaginative, interactive approach to BC/BM communication. This is true especially of its HIV prevention activities in „hot zones‟ and among general populations. Its work focusing on malaria and diarrheal disease prevention applies a more didactic IEC approach by virtue of its target groups and the settings in which BCC activities are presented (e.g., health facilities attended by pregnant women). TOC Recommendations These are based on lessons learned by PSI/Malawi and observations made during the EHAP-IFH evaluation. TOC is not currently sustainable by PSI/Malawi without external funding. At the moment, malaria and ORS BCC activities are predominant during the rainy season (October-November); it is recommended that TOC outreach programs perform these activities all year long in order to reach the maximum number of caregivers and adult beneficiaries. TOC is somewhat ad hoc in its approach (covering both product placement and BC/BM); PSI/Malawi needs to consider priority activities among current interventions and plan strategically for a more targeted program: – Strategic planning should focus on the development of an integrated approach to 18 Malawi EHAP-IFH Program maximize PSI/Malawi‟s comparative advantage and existing (and commendable) TOC human resources; funding proposals should be developed based on such planning. – Decisions on a potential scaling-up of TOC should await strategic planning outcomes. There is a need for more systematic and disaggregated approaches to PSI/Malawi‟s HIV prevention work with high-risk groups. A manual should be created with separate modules for each group (PSI globally has experience in such activities, e.g., in Nigeria). As noted above under Youth Alert! recommendations, PSI/Malawi and existing/potential funding partners should re-evaluate indicators and include quality criteria and process indicators as well as target reach numbers (this is a common recommendation for all future BC work implemented by PSI/Malawi). PSI/Malawi might consider exploring opportunities for „social franchising‟ of TOC interventions, where PSI/Malawi moves more into a supervisory/quality assurance role. The Pilot Faith Communities Program (FCP) Findings The pilot FCP began its work in 2004 in two traditional authorities in Blantyre Rural, Mpemba, and Lirangwe. Its entry point to young people has been through traditional and religious gatekeepers, without whose ongoing engagement and endorsement PSI believes the program would not have survived. Adults are also provided with training, another entry point by which communal endorsement of discussion with young people is obtained. FCP faith committees comprising leaders from Christian denominations and the Islamic community support the program; leadership training has been provided to 197 members using a dedicated manual. The committees include PLWHA; the FCP and the committees consider their inclusion to be one of several actions that helped reduce stigma and discrimination in the pilot communities. USAID funding has ensured the implementation of the FCP; KfW cofunding previously supported transport costs. In 2007, FCP was wholly funded by USAID under EHAP-IFH. Total expenditure in that year was $72,800. FCP interventions The core FCP BC message is the promotion of abstinence for young people and abstinence and be faithful messages for adults. Condom use is discussed where essential. While the FCP disaggregates young people into two groups (ages 10 to 17 and 18 to 24), the program budget has not allowed for the development of separate materials for the two groups; this is acknowledged as a limitation. Malawi EHAP-IFH Program 19 Table 3: FCP Targets and Achievements Pilot Faith Communities Program 2007 USAID Target (people) Achieved % Target Achieved 2008 USAID Target (people) Achieved (to 09/08) % Target Achieved Youth 1,200 1,917 123 1,800 Total: 1,808 M: 909 F: 899 107 Adults 1,200 2,155 180 2,400 Total: 2,672 M: 880 F: 1,792 111 TOC 800 1,362 170 1,600 Total: 1,452 M: 644 F: 808 91 Faith Leaders 50 166 332 200 Total: 319 M: 148 F: 171 160 Committee Leaders 20 31 155 60 Total: 90 M: 61 F: 29 150 Orphans and Vulnerable Children 150 212 141 150 Total: 219 M: 121 F: 98 146 FCP BC/BM materials include three manuals (youth, adults, and faith leaders), posters, the DSD video, and other abstinence and abstinence/be faithful materials developed by the PSI VPU. One noteworthy feature of the FCP manuals is the relatively detailed and culturally aware focus on gender issues such as incest and gender-based and sexual violence, their links to both HIV transmission and human rights, and the need for the community to address such matters equitably. These materials have all been developed using findings from the 2005 TRaC study; the 2008 TRaC study, currently being analyzed, will provide quantitative data on behavior change relevant to the FCP. The FCP incorporates an element of qualitative monitoring and evaluation through question￾and-answer forms disaggregated by sex, age, and location; findings feed into subsequent training sessions. There is no provision in the FCP budget for end-line evaluation. The Lirangwe FCP faith committee provided anecdotal evidence of positive BC to the EHAP-IFH evaluation team. More people have been willing to have HCT, more people are now obtaining antiretroviral treatment (ART), and stigma and discrimination (including self-stigma) against people known or suspected to be HIV positive have declined. There have been fewer funerals and fewer children becoming orphaned or vulnerable; reasons given include better management of HIV, greater fidelity on the part of husbands, and perhaps also DSD among some young people. Currently, the FCP operates four vocational skills training centers for orphans and vulnerable children (OVC) close to Blantyre. As of September 2007, 306 young people had been trained, and 15 reported having gained employment after their graduation. FCP OVC activities in Lirangwe include the Nazareth Nursery School, staffed by volunteers, and carpentry and tailoring vocational training for older children. There is no public sector provision for OVC available in Lirangwe other 20 Malawi EHAP-IFH Program than adult ART administered in the home by the guardian system. No nutritional support can be provided to OVC by the FCP; some are known to be HIV-positive. The FCP would ideally like to expand its vocational work to include „starter packs‟ for trained OVC. The evaluation team was told that there is minimal stigma and discrimination of OVC in Lirangwe – the FCP work focuses not only on single/double orphans but also on other vulnerable children. In addition, extended families continue to care for OVC. PSI and NAC are currently discussing scaling up the FCP to Mangochi and Salima districts. The FCP is a member of the Malawi Interfaith AIDS Association, a network closely supported by NAC. Issues inhibiting further scale-up include the proliferation of FBOs that are unregulated (with concomitant quality concerns), absorptive capacity, and lack of funding. FCP Conclusions The FCP provides a model of gaining access to young people in socially conservative communities through traditional and religious gatekeepers. Its activities have revealed the potential for balancing respect for societal norms with challenges to harmful practices and unequal attitudes and behaviors. The FCP has thus engaged with gatekeepers while addressing issues such as stigma, care of OVC, and unequal gender practices. FCP committees testify to BC occurring in their community as a result of FCP interventions. The FCP is an exemplar of how to build gender awareness and promote gender equality. Its manuals provide detailed discussion of such matters and promote detailed attention among program beneficiaries. FCP Recommendations As with Youth Alert! and TOC, these are based on lessons learned by PSI/Malawi and observations made during the EHAP-IFH evaluation. The FCP may be sustainable through FBOs in terms of continued implementation of activities; however, any such continuation should be supported by PSI/Malawi through supervision, quality assurance, and inputs to data collection. The FCP approach to gaining entry to young people through gatekeeper sensitization, training, and support represents a model for scaling up in the Malawian context of conservative social mores and deep-rooted religious observance. The gender approach in FCP materials represents a starting point for further PSI/Malawi (and wider) attention to such issues. Any future FCP interventions should include funding for separating the youth manual and training sessions into two components that address the 10- to 14- and 15- to 24-year age groups. Malawi EHAP-IFH Program 21 SOCIAL MARKETING INTERVENTIONS Condom Social Marketing Table 4: Condom Sales and Market Share 2002 2003 2004 2005 2006 2007 2008 2009 Target 6,300,000 7,700,000 9,000,000 9,000,000 9,700,000 10,000,000 7,450,000 9,250,000 Achieved 7,255,804 8,414,316 8,563,494 8,852,949 9,201,726 9,057,033 Market Share 34% 50% 29% 33% 28% 22% Condom sales and market share were impacted by the introduction in 2004 of the Marie Stopes International (MSI) Manyuchi commercial condom, which achieved sales of just over 2 million by 2007; by increased distribution of MSI‟s free condom (2.5 million in 2007); and by the significant increase in free public sector condoms, which climbed from 14 million in 2005 to 35 million in 2007 and included free distribution through nongovernmental organizations (NGOs) in bars and other „hot‟ sites. This caused a distribution bottleneck in the PSI distribution system as uptake from retailers slowed and is reflected in below-target sales for 2007 and lower sales forecasts for 2008 onwards. All the same, PSI has weathered these competitive issues quite well, even though promotional spending has been reasonably modest ($139,000 in 2007) and, with the end of KfW funding, significantly reduced in 2008. Table 5: Key Results Indicators EHAP-IFH: Increase in correct and consistent condom use among 15- to 24-year-olds DHS 2000 DHS 2004 MICS 2006 Male 34% 47% 58% Female 32% 35% 40% PSI health impact data for 2007 sales HIV Cases Averted : 2,785 The key indicator of EHAP-IFH target beneficiaries indicates some significant success, to which the program has contributed, in increasing correct condom use among 15- to 24-year-olds. PSI/Washington also reports approximate health impacts from PSI‟s condom sales as noted for reference. Table 6: 2007 Expenditure in $ and Funding Sources $ Percentage Revenue Use USAID EHAP 889, 889 62% KfW 382,466 27% Revenues From Sales 154,393 11% Packaging and staff development TOTAL 1,426,749 22 Malawi EHAP-IFH Program PSI was successful in achieving cofunding for the Chishango condom at 27% of total expenditures in 2007 from KfW and at 11% of total spending in cost recovery from sales. Revenue from sales did not meet the raw cost of condoms supplied by USAID, which (assuming a cost of $0.03 per condom) would have been $276,051. However a sales increase in 2008 now notionally covers condom costs, although income continues to be used to cover packaging and other costs. Distribution and markups PSI achieved 82% urban and 54% rural coverage for the Chishango condom (2007). The primary issue in rural coverage is the disparity among districts with PSI zones 1, 2, and 4 requiring particular attention. PSI sells a pack of three condoms for about $0.06, and with a 16% distributor margin, 39% wholesaler margin, and 48% retail margin, the consumer pays about $0.14. These appear to be reasonable margins considering the very low cost of the product. Cost-effectiveness and cost recovery In 2007, total costs, including condom costs, per 150 condoms equaled just under $24.00. This is relatively high and includes about $4 per 150 for promotional and IEC costs. Commodity and packaging costs total $7.80/150 (including carton, pillow pack, dispenser, and leaflet). Revenue from sales in 2007 was $3/150 or 38% of condom and packaging costs. To fully recover condom and packaging costs, wholesale prices would need to rise from $0.06 for three to $0.16 and retail prices from $0.14 to about $0.38. PSI’s response to market conditions and sustainability – Condom marketing The shift to significant free condoms in the Malawi environment requires PSI to reposition itself from a supplier of very low-priced condoms to a more mid-market pricing structure to meet the needs of those who can afford, and prefer, to pay. PSI is planning to accomplish this through introducing brand extensions for Chishango, building on the brand‟s considerable awareness, or through introducing new brands. Research on these brands is underway and scheduled for completion in October. It is assumed that new or extension brands will be launched at full commodity and packaging cost recovery at least. It is noted that the introduction of higher-priced brands will also support some lowering of markups through the distribution system. PSI is seeking to gain better coverage in rural areas notably in zones 1, 2, and 4. It is noted that the cash-and-carry system between wholesalers and retailers makes the creation of demand at the retail shop or stall the key issue in expanding rural sales. The presence of free condoms in the marketplace is of concern. Resolutions to better segment markets between free condoms and social marketing condoms will lead to a more efficient use of public sector (and NGO) resources if those who can afford to pay do not have easy access to the public sector supply. Condom Social Marketing Recommendations PSI needs to improve coverage in low-coverage districts, notably in zones 1, 2, and 4. PSI would need additional promotional and advertising budgets to accomplish this. PSI needs to continue to plan annual incremental increases to its Chishango brand to more fully recover costs and to position its new brands at a somewhat higher price. Malawi EHAP-IFH Program 23 It is recommended that PSI work with the public sector to develop close working relations with them in respect to market segmentation strategies under a total market approach. Malaria Social Marketing Table 7: Commercial ITN and Retreatment Kit Sales 2002 2003 2004 2005 2006 2007 2008 2009 ITNs Target 90,000 100,000 121,870 124,000 175,000 195,000 Achieved 79,134 87,467 116,880 109,955 103,165 157,482 Retreatment Target 60,000 69,000 76,500 100,000 110,000 194,000 220,000 250,000 Achieved 81,580 101,624 84,176 104,581 161,283 211,820 PSI has achieved significant success with its untreated ITNs, achieving steady growth to meet sales targets that were revised upwards in 2004. Similar success has been achieved with sales of retreatment kits. Originally started by UNICEF and PSI in 2002 with funding (about $29 million) from the U.K. Department for International Development (DFID), the objective of the program was to distribute ITNs to all health centers in Malawi through district health management teams, with PSI receiving a small sales income to cover distribution costs. UNICEF regards this project as a very significant success. PSI accomplished national coverage by 2003 and, most importantly, succeeded in training health center staff to advise on correct usage. PSI engaged in significant advocacy and community mobilization activities as well as in mobilization and support of retreatment campaigns. UNICEF points out that this dual role is PSI‟s greatest strength. Subsequently, funding for nets has been supported through JICA (2005); German Technical Cooperation (GTZ) (2006 and 2007); PMI funding through EHAP-IFH (since 2007); and UNICEF through accessing Global Fund/SWAp funding (2008). In November 2006, the GOM shifted to LLINs provided free to all pregnant women and mothers of children under age 5. This meant that PSI could no longer cover part of its distribution costs from sales to health centers and has had to rely on support from GTZ and PMI and on funds from its income from commercial ITNs to support these costs. Table 8: Public Sector ITN and LLIN Distribution 2002 2003 2004 2005 2006 2007 2008 (est.) 2009 Target 250,000 590,000 1,290,000 1,550,000 1,621,8700 Achieved 372,911 1,069,845 1,364,843 1,472,720 1,874,089 842,251 2,500,000 24 Malawi EHAP-IFH Program Table 9: Select Indicators CDC in six sentinel sites Households with at least one ITN Children < 5 slept under ITN previous night 2005 36.6% 41.0% 2007 41.3% 51.3% PSI health impact date for 2007 nets distributed by PSI (see annex 4 for methodology) Malaria Cases Averted: 1,442,752 Malaria Deaths Averted: 10,535 Table 10: 2007 Expenditure in $ and Funding Sources – Commercial ITNs $ Percentage Revenue use USAID EHAP 665,506 30% UNICEF 8,738 - GTZ 910,435 42% Procurement of $749,950 Revenues from sales 604,299 28% Primarily procurement ex PSI Enterprise Fund TOTAL 2,188,978 In the evaluation year of 2007, delays in procurement of GOM ITNs led to a reduction in these nets delivered; the year is therefore not typical. In that year, PSI sustained commercial sales through revenues and required no additional donor inputs. All the above inputs, aside from sales revenue, were employed to support commodities and costs of public sector nets. PSI has developed significant additional funding in support of the GOM ITN program and, with only marginal promotional spending, was able to sustain commercial sector sales in our evaluation year of 2007 from $544,448 in sales revenue accrued under PSI‟s Enterprise Fund. It appears that demand is being sustained from spin-off of the general publicity relating to ITN use generated through the public sector, in which PSI plays a significant role. Distribution and markups Coverage is relatively low for commercial nets – 26% urban and 13% rural (2007). This is partly an aspect of PSI‟s minimal spending on the product at this time. PSI sells its retreatable net plus a retreatment kit for about $4.29. The nets sell for about $6.07 retail. Cost-effectiveness and cost recovery Commercial Nets: PSI procures nets at $3.40 and adds $0.66 for a price sticker, plastic bailer, and retreatment kit, for a total of $4.06. It sells nets at $4.29. PSI does not allocate a cost for distribution (the cost of distribution compared to its other product volumes is minimal) and spends no direct funds for promotional costs at this time. On this basis, the cost-effectiveness of the commercial net program must be regarded as optimal. Public Sector Nets: PSI estimates its costs for delivery of public sector nets to health centers at $0.75 per net including core costs. It is worth noting that the commercial sector markup for net distribution between the distributor and the end-purchaser is $1.78. This demonstrates the effectiveness of PSI‟s public sector net distribution. Malawi EHAP-IFH Program 25 PSI’s response to market conditions Commercial Nets: The present strategy for commercial nets has essentially been to fill gaps in public sector net distribution as these are targeted only to pregnant women and mothers of children under age 5 and are now free. Two shifts have occurred in public policy. ITNs have been replaced by LLINs, and the NMCP has decided to shift to national coverage of two nets per household, with a target of achieving this by 2010. This will require an initial 7 million nets for about 3.5 million households. PSI has responded to the government shift to LLINs by planning to launch its own LLIN net. This will require a higher level of pricing to achieve cost recovery (an additional $1.30) for these more expensive nets. The retail cost may be estimated at about $7.50 as against the ITN retail price of $6.07. This increased price plus the future plan to provide free nets to the whole population will impact PSI‟s capacity to gain volume sales and, therefore, sales efficiency. However, at the same time, even with national free coverage, some benefit can be gained through providing nets through the commercial sector (to replace damaged nets, reduce government costs to those who prefer to pay, etc.). Public Sector Nets: Government net procurement has shifted from direct donor support to SWAp mechanisms. This shift has not been smooth, however, resulting in delays in procurement that have been offset by UNICEF, PMI, GTZ, and others to continue to fund nets directly or negotiate with the SWAp mechanism to allow them to procure from these funds. The impact on PSI is that although they have applied for distribution cost funds to be received from the SWAp mechanism, difficulties exist in the tendering processes and application of contracts to non-Malawian organizations in support of core costs. PSI is continuing to explore its options in consultation with USAID and UNICEF. Malaria Social Marketing Recommendations PSI/Malawi and donors must prioritize the resolution of PSI access to costs for public sector net distribution. In view of the prospects for free public sector distribution of LLINs to all households, PSI needs to carefully assess its strategy to launch a commercial LLIN. This will help to reduce leakage into the commercial sector but will add management and resource constraints if sales are to recover all costs. PSI should be employed to support a malaria component of IEC activities to ensure correct ITN/LLIN use and should consider accessing NAC funding in support of promoting the importance of LLIN use by those with HIV/AIDS. PSI needs to work closely with UNICEF and USAID to find creative solutions to funding public sector net distribution costs. Oral Rehydration Salts Social Marketing PSI has achieved significant success with its ORS Thanzi brand and has consistently increased its targets for achievement since 2003. Sales were somewhat below these more ambitious targets in 2005 and 2006 but were well met in 2007 after the launch and promotion of the new low-osmolarity formula and pack and the increased promotional spending associated with the launch. 26 Malawi EHAP-IFH Program Table 11: ORS Sales 2002 2003 2004 2005 2006 2007 2008 Target 550,000 735,000 914,500 1,320,000 1,100,000 1,200,000 1,690,000 Achieved 829,560 1,582,560 1,197,592 1,024,547 982,548 1,246,584 Table 12: Key Results Indicators DHS surveys DHS 2000 DHS 2004 MICS 2006 Children <5 given ORS for diarrhea 47.9% 61.1% 55% PSI health impact data for 2007 sales (includes WaterGuard products) Diarrhea Cases Averted: 231,389 Deaths Averted: 296 The program was entirely funded by the EHAP-IFH project. In 2007, the project earned from revenue 21% of total costs, with spending for promotional and IEC of just over $70,000. Cost-effectiveness and cost recovery PSI recovers 94% of the cost of procurement and packaging ($0.0858 per sachet plus $0.001 for package carton or $0.087). It sells to retail at $0.082. Distributor margin is 15%, wholesale 36%, and retail 39%, giving a consumer price of $0.18. Distribution margins seem reasonable considering the relatively low price per sachet. Total costs less revenues equal $0.29 per sachet. Table 13: 2007 Expenditure in $ and Funding Sources $ Percentage Revenue use USAID EHAP 365,209 79% Revenues from sales 99,534 21% Procurement TOTAL 464,743 PSI’s response to market conditions There are no significant shifts in market conditions. The Ministry of Health (MOH) provides free ORS. However stock-outs do occur and, it appears that not all mothers are able to accumulate a pantry stock of ORS for future use. The availability of ORS in shops appears to be strongly recommended, not least for emergency procurement. PSI appears to have developed reasonable coverage for Thanzi at 83% urban and 63% rural (2007). Coverage in rural areas differs notably from district to district, and PSI is well aware of the need to concentrate on increased rural coverage in the lower-coverage districts. Recommendations PSI needs to increase rural coverage through higher spending on BCC and promotion to increase rural demand on the distribution system. Thanzi requires focus on rural demand generation balanced by responsive distribution systems. Thanzi sales growth requires major increases in promotional and BCC spending, with attention to preventive as well as curative behaviors. Malawi EHAP-IFH Program 27 III. PSI MANAGEMENT AND SUSTAINABILITY STRATEGY MANAGEMENT A review of PSI/Malawi‟s management structure and systems demonstrates good adherence to PSI‟s known effectiveness in its financial, administration, human resources, finance, and general management systems. PSI has 102 staff in its Blantyre head office; 40 (mainly packagers) in its central warehouse; and 38 in its three branches/depots in Lilongwe, Mzuzu, and Mangochi. In the last full evaluation year (2007), senior management was overseeing a total of 19 contracts with 13 donors, including USAID. Of these, only six were contracts worth over $1 million, and only nine were scheduled to continue beyond 2008. This situation, coupled with the shift of many donors to the SWAp mechanism, has put significant stress on management and has led to too many relatively small contracts that impede PSI from managing to its strength in large-scale programming. This environment also leads to difficulties in adequately recovering core costs from small contracts; potential SWAp funding has led, for example, to significant costs to keep older vehicles operative. PSI has a need to replace two pickup trucks for administration, one Land Cruiser for sales, two Land Cruisers for the MVU, and two Land Cruisers for its Youth Alert! teams. PSI/Malawi management needs to avoid small contracts that do not meet its core imperatives and to seek larger contracts through partnerships with other implementing agencies across the public sector and those agencies with access to non-NAC SWAp mechanisms in the MOH and international agencies, while continuing to access the NAC SWAp funding. SUSTAINABILITY STRATEGY Sustaining PSI‟s operations in Malawi needs to build on its strengths and capacities to manage to scale across commodities distribution and across the management of targeted BCC and community￾based initiatives coupled with mass media support. Cost recovery Cost recovery of basic commodities and packaging has been successfully achieved with ITNs and is close to being achieved for ORS. Condom cost recovery has not been a major strategic thrust in the past owing to imperatives to ensure very low-income consumer access. With relatively new inputs from a significantly expanded contribution of free condoms from the public sector, PSI can move to higher prices to fill an important gap between free and full-priced commercial brands. Higher pricing, however, should not be accomplished through significant and immediate price increases but through incremental increases of PSI‟s present brand and the introduction of more mid-priced brands or brand extensions. PSI is proceeding with the development of these brands and will require funding to launch them within the context of leading toward commodity cost sustainability for them in the medium term. 28 Malawi EHAP-IFH Program Condoms for HIV/AIDS prevention are to be financed through the recently bid USAID HIV/AIDS requests for applications. Additional funding, for youth activities particularly, is assumed to be available through the NAC SWAp mechanism and the Global Fund, but there is no realistic assessment of the time it will take for awards to be issued. PSI may require interim support to continue its youth activities from other donors. PSI should also seek to work with the public sector to come up with a total market approach for condoms; to better segment the distribution of free condoms and to more fully participate in the promotion of all condoms, whether free or not. Some technical assistance support to PSI or another contractor needs to be made available for this strategy to be accomplished. LLINs PSI should consider its options in respect to continuing with its ITN commercial brand and the launch of its LLIN commercial brand only after it has explored the epidemiologic need for these nets in view of the public sector‟s policy to move to total household coverage of two nets in the future. Cost recovery of commodities is, however, only a component part of the primary agenda to increase demand and coverage of BCC and social change activities. PSI can only be sustained as an organization through continued funding of expanded activities in these areas. PSI needs to continue to work with UNICEF to resolve the issue of its contract to distribute public sector LLINs, notably in view of the issues that (1) SWAp mechanisms outside of the NAC SWAp do not have a component that accepts grant applications, and (2) they allocate funds to partners, and PSI is not an allocated partner. However, NAC funds could be accessed to stress the importance of correct LLIN use by those living with HIV/AIDS. ORS PSI should seek to expand its promotional and BCC activities for ORS in order to create more demand, notably rural demand, in order to increase coverage. Present promotional and BCC spending is relatively modest for these longer-term goals to be achieved. Other products Expansions into other appropriate products (whether in support of public sector distribution, as with LLINs) such as ACT or ART drugs may be considered (as could the introduction of social marketing products such as birthing kits, zinc to the ORS program, or further family planning initiatives), once the present launch of existing products has been achieved. In all cases, however, the mandate to distribute commodities needs to be linked to clear strategies over the short and medium terms to aim for volume, rural distribution, and commodity cost recovery, with adequate promotional and BCC spending, to accomplish these aims. Operational cost-effectiveness Sustainability of PSI‟s activities is also a factor of operational cost efficiency. PSI does not appear to routinely track or report the cost-effectiveness of its component activities, and in this evaluation only approximate measures could be estimated. More emphasis is needed on analysis of cost￾effectiveness in order to better explore cost saving measures. Some analysis has been made to increase the cost-effectiveness of PSI‟s distribution system. PSI is reducing the number of its “client” wholesalers through a greater use of distributors. PSI reported that they expect to reduce their distribution costs, estimated at $500,000, by some 34% through this initiative. It must be noted, Malawi EHAP-IFH Program 29 however, that some increase in distribution markups, which may be passed on to consumers, may occur as a result of this strategy. Management and Sustainability Recommendations PSI/Malawi should continue to seek improvements in its operational efficiency through tracking component costs. PSI should continue to achieve maximum cost recovery of commodities within market realities, balanced against the availability of free public sector supplies, and work to collaborate with those supplies through total market approaches. PSI should expand its product and BCC portfolio but should build on its strengths in managing to scale and avoid pilot initiatives that are unlikely to be scaled up. PSI/Malawi should further strengthen its partnerships with national and international partners working on BCC in Malawi, including with those partners with access to MOH SWAp funding. 30 Malawi EHAP-IFH Program Malawi EHAP-IFH Program 31 IV. CONCLUSIONS AND RECOMMENDATIONS CONCLUSIONS To conclude, the team feels that PSI has contributed to the achievement of the objectives required by USAID SOPIR indicators and in doing so has contributed to the development and betterment of Malawians‟ family health. In addition, PSI has enhanced and strengthened EHAP-IFH with its ingenious ways of transmitting BCC messages about HIV prevention to youth and targeted rural and urban populations through mega-shows and other interventions. PSI‟s media work is largely appreciated by its partners and stakeholders, and PSI‟s messages about HIV prevention are undoubtedly saving many lives. Along with increased awareness among high-risk groups and young people of HIV prevention and PSI‟s capacity to affect and maintain behavior change, the nationwide distribution of commodities and training of health workers are also associated with improved family health – many more households use ITNs/LLINs, and more women are using ORS for their sick children. PSI‟s capacity to manage and scale up community-level distribution of commodities nationwide through the commercial sector to every health center in the country is widely acknowledged as having made significant contributions to HIV/AIDS, malaria, and diarrhea prevention. Overall, the team believes that the outcomes of the EHAP-IFH evaluation are very positive. The challenges discussed and recommendations suggested in this report are proposed as goals to help PSI build upon its achievements to date in order to obtain more sustainability for the future of its operations. RECOMMENDATIONS Youth Alert! It is strongly recommended that elements of Youth Alert! continue to be supported by the GOM and/or donor partners; its activities have undoubtedly contributed to both expanded HIV prevention knowledge creation and to BC/BM among young Malawians. Youth Alert! is currently not sustainable by PSI/Malawi without such external support. PSI/Malawi and existing or potential funding partners should re-evaluate indicators and include quality criteria and process indicators to track impact, as well as existing outcome indicators (target reach numbers). This is a common recommendation for any future work on current EHAP-IFH HIV components. It is essential that all future Youth Alert! interventions segment activities and BC/BM communications into 10- to 14-, 15- to 19-, and 20- to 24-year age groups. The following issues need to be speedily addressed and resolved: - All future Youth Alert! interventions should be more effectively informed by research that is closely disaggregated and analyzed and also supported by gender analysis (this is a recommendation common to all EHAP-IFH BC/BM interventions). - Consideration should be given to PSI/Malawi support for the creation of a live referral register, where all relevant organizations commit to regular updating and information sharing. 32 Malawi EHAP-IFH Program - PSI/Malawi should strategically plan for closer and more sustainable engagement with decentralized structures (district assemblies and lower) for Youth Alert! and all other BC programs. The best use of its comparative advantage would be to deepen its partnership with a group such as the Program for Advancement of Commercial Technology (PACT), which has strengths in organizational capacity building and development. PSI/Malawi and PACT are currently in the process of submitting a proposal to the Global Fund‟s Round 7; this would in part address such issues. Targeted Outreach Communications TOC is not currently sustainable by PSI/Malawi without external funding. It is recommended that TOC outreach programs perform malaria and ORS BCC activities all year long in order to reach the maximum number of caregivers and adult beneficiaries. TOC is somewhat ad hoc in its approach (covering both product placement and BC/BM); PSI/Malawi needs to consider priority activities among current interventions and plan strategically for a more targeted program: - Strategic planning should focus on the development of an integrated approach to maximize PSI/Malawi‟s comparative advantage and existing (and commendable) TOC human resources; funding proposals should be developed based on such planning. - Decisions on a potential scaling-up of TOC should await strategic planning outcomes. There is a need for more systematic and disaggregated approaches to PSI/Malawi‟s HIV prevention work with high-risk groups. A manual should be created with separate modules for each group (PSI globally has experience in such activities, e.g., in Nigeria). As noted above under Youth Alert! recommendations, PSI/Malawi and existing/potential funding partners should re-evaluate indicators and include quality criteria and process indicators as well as target reach numbers (this is a common recommendation for all future BC work implemented by PSI/Malawi). PSI/Malawi might consider exploring opportunities for „social franchising‟ of TOC interventions, where PSI/Malawi moves more into a supervisory/quality assurance role. The Pilot Faith Communities Program The FCP may be sustainable through FBOs in terms of continued implementation of activities; however, any such continuation should be supported by PSI/Malawi through supervision, quality assurance, and inputs to data collection. The FCP approach to gaining entry to young people through gatekeeper sensitization, training, and support represents a model for scaling up in the Malawian context of conservative social mores and deep-rooted religious observance. The gender approach in FCP materials represents a starting point for further PSI/Malawi (and wider) attention to such issues. Any future FCP interventions should include funding for separating the youth manual and training sessions into two components that address the 10- to 14- and 15- to 24-year age groups. Malawi EHAP-IFH Program 33 Condom Social Marketing PSI needs to improve coverage in low-coverage districts, notably in zones 1, 2, and 4. PSI would need additional promotional and advertising budgets to accomplish this. PSI needs to continue to plan annual incremental increases to its Chishango brand to more fully recover costs and to position its new brands at a somewhat higher price. It is recommended that PSI works with the public sector to develop close working relations with them in respect to market segmentation strategies under a total market approach. Malaria Social Marketing PSI/Malawi and donors must prioritize the resolution of PSI access to costs for public sector net distribution. In view of the prospects for free public sector distribution of LLINs to all households, PSI needs to carefully assess its strategy to launch a commercial LLIN. This will help to reduce leakage into the commercial sector but will add management and resource constraints if sales are to recover all costs. PSI should be employed to support a malaria component of IEC activities to ensure correct ITN/LLIN use and should consider accessing NAC funding in support of promoting the importance of LLIN use by those with HIV/AIDS. PSI needs to work closely with UNICEF and USAID to find creative solutions to funding public sector net distribution costs. PSI/Malawi and donors must prioritize the resolution of PSI access to costs for public sector net distribution. In view of the prospects for free net provision to all households, PSI should work with the Centers for Disease Control and Prevention to articulate the need for commercial LLINs and, if products are required, seek access to donated LLINs or other support to ensure affordability. In view of the prospects for free public sector distribution of LLINs to all households, PSI needs to carefully assess its strategy to launch a commercial LLIN. This will help to reduce leakage into the commercial sector but will add management and resource constraints if sales are to recover all costs. PSI should be employed to support a malaria component of IEC activities to ensure correct ITN/LLIN use and should consider accessing NAC funding in support of promoting the importance of LLIN use by those with HIV/AIDS. PSI needs to work closely with UNICEF and USAID to find creative solutions to funding public sector net distribution costs. Oral Rehydration Salts Social Marketing PSI needs to increase rural coverage through higher spending on BCC and promotion to increase rural demand on the distribution system. Thanzi requires focus on rural demand generation balanced by responsive distribution systems. Thanzi sales growth requires major increases in promotional and BCC spending, with attention to preventive as well as curative behaviors. 34 Malawi EHAP-IFH Program Management and Sustainability PSI/Malawi should continue to seek improvements in its operational efficiency through tracking component costs. PSI should continue to achieve maximum cost recovery of commodities within market realities, balanced against the availability of free public sector supplies, and work to collaborate with those supplies through total market approaches. PSI should expand its product and BCC portfolio but should build on its strengths in managing to scale and avoid pilot initiatives that are unlikely to be scaled up. PSI/Malawi should further strengthen its partnerships with national and international partners working on BCC in Malawi, including with those partners with access to MOH SWAp funding. Malawi EHAP-IFH Program 35 APPENDIX A. SCOPE OF WORK Program Evaluation of PSI Enhanced HIV/AIDS Prevention and Improved Family Health in Malawi Project (Revised/Final: 07-18-08) I. PURPOSE The purpose of this evaluation, planned for September 2008 is twofold: (1) Review, analyze, and evaluate the extent to which the PSI Enhanced HIV/AIDS Prevention and Improved Family Health in Malawi Program (EHAP-IFH) has achieved its stated objectives related to mitigating the impact of HIV/AIDS, malaria, and diarrheal diseases, thereby contributing to the overall USAID/Malawi Strategic Objective 8: Improved Health Status of Malawians. (2) Provide specific recommendations and lessons learned for future activities and directions that the Mission may wish to explore in designing future programs. II. BACKGROUND The EHAP-IFH cooperative agreement between Population Services International (PSI) and the U.S. Agency for International Development (USAID) began in July 2005 with the stated goal of improving the overall health status of low income and vulnerable Malawians through implementation of social marketing and behavior change communications (BCC) strategies. Interventions have been implemented through this agreement in an attempt to achieve the following outcomes: reduced HIV incidence, reduced malaria morbidity mortality among children under 5 years and pregnant women, and reduced morbidity and mortality caused by dehydration in children under 5. While the HIV, malaria, and child health components of the project have all focused on utilizing social marketing and BCC strategies to achieve their desired ends, the specific interventions and target populations for each component have been different. The HIV/AIDS prevention interventions have been primarily centered around two areas: 1) national social marketing to increase the correct and consistent condom use among sexually active young men aged 14 to 25 years, men aged 26-49 years who engage in casual sexual relationships (such as long-distance drivers), and female sex workers; and 2) branded behavior change messages and activities involving a combination of mass media and interpersonal communications with a focus on enabling young people to not only know have increased knowledge about safer sexual behaviors, but also to be able to enact safer sexual behaviors related to abstinence, delayed sexual debut, and correct condom use. The primary purpose of malaria prevention interventions funded under the agreement has been to increase the adoption and appropriate use of quality insecticide treated nets, with a focus on those most vulnerable to the effects of malaria: pregnant women and children under five years of age. At the same time, PSI/Malawi has also continued efforts to increase ITN use amongst the general population via collaborations with the MOH to facilitate the distribution of nets to district health offices and onto community-based ITN committees, as well as via the continued marketing and distribution of the PSI Chitetezo brand ITNs through commercial outlets nationwide. 36 Malawi EHAP-IFH Program The purpose of the social marketing of oral rehydration salts funded under the interventions has been to increase the public‟s adoption and appropriate use of oral rehydration salts (ORS) and oral rehydration therapy (ORT). Activities have focused on ensuring sustained high access to ORS through commercial sector outlets and increased informed demand for both PSI‟s Thanzi branded ORS and generic ORS through communications campaigns. An important secondary goal of the agreement also was to promote sustainability by developing the capacity of local staff, improving product cost recovery, improving cost efficiency, expanding the role of the private sector in achieving health objectives, and continuing to diversify the program‟s funding and range of interventions. The agreement was initially scheduled to end in September 2007, but has been extended until September 2008. A second and final extension is planned to continue the project through March 2009. III. EVALUATION OBJECTIVES AND ILLUSTRATIVE EVALUATION QUESTIONS The evaluation shall assess and analyze the following: 1. What progress has been made in implementing the benchmarked activities negotiated in the agreement and annual workplans and in achieving yearly targets. To what degree has the project achieved its objectives related to HIV, malaria, and child health? 2. Have the data gathering methods used by the project been reasonable for monitoring progress and indicators? Are indicators appropriate and/or valid? 3. What has been the likely effectiveness in facilitating behavior change with regards to each of the priority behaviors targeted for HIV, malaria, and diarrheal disease? To what degree can adoption of behaviors of interest such as use of ORS, bednets, or condoms be attributable to the work of the EHAP-IFH project versus other factors that may have influence such behaviors? Is there strong evidence that increased sales of products, both generic and branded, such as bednets or condoms, correlates strongly with increased and appropriate usage of tehse products by the taregt populations? 4. What level of population-based coverage has the project achieved with its interventions in targets areas, looking specifically at coverage by interpersonal communications campaigns, mass media, and marketing of the products of interest? Did the program reach a high proportion of the populations in Malawi and also achieve high coverage of various subgroups of interest such children under 5, pregnant women, youth, and high-risk groups? 5. What has been the effectiveness and efficiency of the project‟s organizational system (i.e. administering grants, providing technical assistance, building capacity, liaising with NAC, MOH, etc)? Review staff composition and capacity, project systems and procedures? 6. Has project has provided technical leadership within Malawi and facilitated synergy, coordination, and information sharing among: USAID/Malawi team, NGOs, sub-partners, other SO8 partners, other donors and the Government of Malawi? Is it linking activities between these groups and are there opportunities to provide feedback on activities? Has technical assistance provided by the project staff 7. What has been the estimated cost-effectiveness of the various interventions implemented by the EHAP-IFH project relative to the number of beneficiaries reached and estimated impact Malawi EHAP-IFH Program 37 on behavior change? Based upon the cost, does it seem feasible to attempt nationwide scale￾up of particular interventions that were successful? 8. Has the project achieved its goal of promoting sustainability through various approaches including developing the capacity of local staff, improving product cost recovery, improving cost efficiency, expanding the role of the private sector in achieving health objectives, and continuing to diversify the program‟s funding and range of interventions. 9. Assess how well the project activities are addressing the underlying gender dynamics that influence behaviors related to HIV/AIDS, malaria and diarrheal disease. The following specific areas shall be considered by the evaluation team: What has been the overall impact of the EHAP-IFH project? How effective was the approach for communication and behavior change behaviors related to abstinence, partner reduction, condom use, bednet use, bednet re-treatment, and use of ORS? Were there differences in effectiveness in reaching various populations (i.e. youth, adult men, women, high-risk groups)? How effective was the project‟s system to measure progress towards program objectives? Was the quality of data collected and the reporting system up to standard? Was decision-making evidence-based? What is the potential for scale-up or expanding the impact of interventions of the EHAP-IFH? To what extent can EHAP-IFH project achievements be sustained without USAID/Malawi assistance? Is there demand in the communities for program activities to continue? IV. METHODOLOGY The evaluation must take into consideration the current political/social environment of Malawi, Malawi‟s Sector Wide Approach to Programming in the health sector (SWAp), the HIV/AIDS epidemic, the National Action Framework (NAF) for HIV/AIDS, the decentralization process, program coverage, strategic partnerships, community participation, youth involvement and participation of people living with HIV/AIDS in the programs. The Evaluation team shall use facilitative methods and activities that will enhance collaboration and dialogue among counterparts particularly the MoH, NAC and implementing partners involved in the project. The Evaluation team shall work in collaboration with the CTO for the EHAP-IHF Project. The CTO will organize all internal USAID meetings including linking the team with HPN team leader. The Evaluation team shall propose and organize the evaluation process in collaboration with the CTO. The evaluation design and work plan shall be presented to the HPN team members for comments during the initial meeting with the Mission HPN team. The CTO in collaboration with the PSI Chief of Party will arrange for an initial introductory meeting with appropriate stakeholders at the outset of the process. When appropriate and at the discretion of the Evaluation Team, the CTO may participate in meetings with relevant stakeholders and partners in order to preserve the independence of the evaluation. A general list of relevant stakeholders and key partners will be provided to the Evaluation team by the CTO at the time of arrival but the Evaluation Team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so as to develop a comprehensive understanding of the program and services offered through EHAP-IFH agreement. 38 Malawi EHAP-IFH Program Internal USAID/Malawi meetings will include, at a minimum: Initial organizational/introductory meeting at which the Evaluation Team will present an outline and explanation of the design of the Evaluation; Mid-evaluation review with the HPN Team leader and CTO to outline progress and implementation problems; and Final Evaluation debrief/summary of the data, draft recommendations and report. Prior to conducting field work, the evaluation team will review existing literature and data, including program strategies, quarterly reports, cooperative agreements and modifications and other reports and documents reflecting the EHAP-IHF projects‟ work in Malawi. Document Review USAID/Malawi will provide the evaluation team with the key documents prior to the start of the in country work. All team members will review these documents in preparation for the initial team planning meeting. Prior to conducting field work, the evaluation team will review existing literature and data, including program strategies, quarterly reports, cooperative agreements and modifications and other reports and documents reflecting JHU/CCP Bridge work in Malawi. Team Planning Meeting A two-day team planning meeting will be held in Malawi before the evaluation begins. This meeting will allow USAID to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will: - clarify team members‟ roles and responsibilities, - establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion, - review and develop final evaluation questions - review and finalize the assignment timeline and share with USAID, - develop data collection methods, instruments, tools and guidelines, - review and clarify any logistical and administrative procedures for the assignment, - develop a preliminary draft outline of the team‟s report, and - assign drafting responsibilities for the final report. Internal USAID/Malawi meetings will include, at a minimum: Initial organizational/introductory meeting at which the Evaluation Team will present an outline and explanation of the design of the Evaluation (refer to the TPM noted above); Mid-evaluation review with the HPN Team leader and CTO to outline progress and implementation problems; and Final Evaluation debrief/summary of the data, draft recommendations and report. Field visits/Interviews: The Evaluation Team shall arrange to visit selected sites in consultation with the CTO and the PSI Chief of Party. The Evaluation Team may be accompanied by a member of staff from USAID/Malawi. The site visits will involve interviews with Malawian government Malawi EHAP-IFH Program 39 staff, health workers, NAC, NMCP, implementing partners, PLWHA, and both individuals and communities who were targeted to benefit from the project. Key informant interviews will be conducted in each region. The Evaluation Team will conduct interviews with donor organizations, selected NGOs, and other key respondents identified during the planning meeting List of Documents: The team will gather information and resources/tools that have been developed by the partners and provide the mission with a small clearinghouse of resources related to HIV/AIDS, malaria and diarrheal disease prevention in Malawi. V. TEAM COMPOSITION Team Leader: Should have an MPH or related post graduate degree in health or any applicable social sciences field. S/he should have at least 5 years senior level experience working in health systems programs in a developing country, and extensive expertise in BCC and social marketing, as well broad knowledge of HIV, malaria, and child health program. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required. The Team Leader should also have experience in leading evaluation teams and preparing high quality evaluation reports. The Team Leader will take specific responsibility for assessing and analyzing the organization‟s progress towards quantitative targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Malawi team and other major partners and will be responsible for finalizing the evaluation report with the assistance of other team members. HIV/AIDS Specialist: This specialist should have wide experience in HIV/AIDS at various levels. S/he will need to analyze the program within the context of the NAF and the national BCC strategy their implementation. The expert should have a post graduate degree in public health or related field with a minimum of five years experience working with USAID-supported programs, preferably in Southern Africa Region. Malaria and Child Health Specialist: This specialist should have substantial experience in malaria and child health programs and BCC. S/he will analyze the malaria and child health interventions PSI has implemented in Malawi. The team member should have a post graduate degree in a health￾related field or related field with a minimum of five years experience working with USAID￾supported programs in developing countries. VI. TIMELINE and LOE USAID/Malawi anticipates that the period of performance of this assessment will be approximately 37 days. This would include preparation days, in-country work in Lilongwe and the regions, and 40 Malawi EHAP-IFH Program report writing and finalization. The evaluation will begin in September 2008 and be completed within one month. The following is a sample timeline. Task/Deliverable LOE Period of Performance Review background documents & offshore preparation work. 3 days September 2 - 4 Travel to Malawi 2-3 days Sept 5-7 Team Planning Meeting 2 days Sept 8 - 9 Meet with USAID/Malawi SO 8 team 1 day Sept 10 Information and data collection 15 days Sept 11 - 27 Discussion, analysis and draft assessment report in￾country 5 days Sept 29 – Oct 3 Debrief meetings with SO 8 team and key stakeholders 1 day Oct 4 Depart Malawi 2 days Oct 5 - 6 USAID & Partners provide comments on draft Oct 6 - 10 Team reviews comments & revises report (TL – 5 days; members – 3 days) 5 days/3 days Oct 13 - 17 USAID completes final review Oct. 20 - 24 GH Tech edits/formats report Oct 27 – Nov 25 TOTAL # days 37 days/34 days o/a Sept 2 – o/a Nov 25 VII. LOGISTICAL SUPPORT A six day work week is authorized when working in country. The evaluation team will be responsible for all off-shore and in-country logistical support. This includes arranging and scheduling meetings (with exception to previously mentioned meetings and initial introductory meetings and field trips), international and in-country travel, hotel bookings, working/office space, computers, printing and photocopying. A local administrative assistant/secretary may be hired to arrange field visits, local travel, hotel and appointments with stakeholders. In addition, the Evaluation Team Leader is responsible for draft and final report development, as well as other eligible expenses associated with the completion of the assignment. VIII. DELIVERABLES Work Plan: The contractor will submit a detailed written work plan before end of week one of work. Methodology Plan: A written methodology plan (evaluation design/operational work plan) will be prepared during the Team Planning Meeting and discussed with USAID prior to implementation. Debriefing with USAID and partners: The team will debrief with USAID and partners prior to submission of the draft report and the team‟s departure from country. The team will consider USAID and stakeholder comments and revise the draft report accordingly, if necessary. After the debrief meeting, the evaluation team shall incorporate oral comments received from USAID and stakeholders Draft evaluation report: should be completed prior to the Team Leader‟s departure from Malawi. The written report should clearly describe findings, conclusions and recommendations (using the Malawi EHAP-IFH Program 41 report format provided in “IX. Reporting Requirements” below). USAID will provide comments to GH Tech on the draft report within 5 working days of submission. Final report: will incorporate Mission and partner comments and suggestions. The draft final report should be completed within 5 days after USAID provides its feedback on the draft report incorporating the comments received from the review of the draft and sent to the Mission. The final report (excluding executive summary and annexes) should be no more than 30 pages. After the final but unedited draft report has been reviewed by USAID, GH Tech will have the documents edited and formatted, and will provide the final report to USAID/Malawi for distribution (2 hard copies and a CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. IX. REPORTING REQUIREMENTS The findings from the evaluation will be presented in a draft report at a full briefing with USAID/Malawi and possibly at a follow-up meeting with key stakeholders. The format for the evaluation report is as follows: Executive Summary –concisely state the most salient findings and recommendations (2 pp); Table of Contents (1 pp) Introduction – purpose, audience, and synopsis of task (1 pp); Background – brief overview of MSH project in Malawi, USAID program strategy and activities implemented in response to the problem, brief description of MSH, purpose of the evaluation (2-3 pp); Methodology – describe evaluation methods, including constraints and gaps (1 pp); Findings/Conclusions/Recommendations – for each objective area; and also include data quality and reporting system that should present verification of spot checks, issues and outcome (17- 20 pp); Issues – provide a list of key technical and/or administrative, if any (1-2 pp); Future Directions (2-3 pp); References (including bibliographical documentation, meetings, interviews and focus group discussions); Annexes – useful for covering evaluation methods, schedules, interview lists and tables – should be succinct, pertinent and readable. The final version of the evaluation report will be submitted to USAID/Malawi in hard copy as well as electronically. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins 1” top/bottom and left/right. The report shall not exceed 30 pages, excluding references and annexes. After the debrief meeting, the evaluation team shall incorporate oral comments received from USAID and stakeholders. Within 2 weeks of receiving oral comments, the evaluation team leader will send 2nd draft to the Mission for final, written, comments. Within 2 weeks of receiving the final comments from the Mission, the Evaluation Team, through the Evaluation Team Leader, shall send the final report: 2 hard copies and a CD ROM. 42 Malawi EHAP-IFH Program X. OVERSIGHT AND MANAGEMENT The GH Tech team will work under the direction of USAID/Malawi. Responsibilities: USAID/Malawi and/or JHU/CCP will be responsible for the following: Obtain country clearances for travel. Coordinate and facilitate initial assessment-related field trips, interviews, and meetings. GH Tech will be responsible for the following technical and logistical support: Identify and recruit team members – international and local consultants. Provide funds to the team for all in-country logistics. Provide administrative and management support to the team while on assignment Provide support and editing services for the preparation of the final versions of the deliverables Documents for Review: EHAP-IHF Cooperative Agreement USAID/Malawi Operational Plan for 2007 USG COP USAID/Malawi Performance Monitoring Plan Work Process Analysis Report Technical Assistance Reports Variety of published project materials Quarterly Reports Project Work plans National Action Framework National BCC strategy Quality Assurance Policy Malawi EHAP-IFH Program 43 APPENDIX B. PERSONS CONTACTED USAID, PSI, GOM and Donor Partners Organization Name Position US Embassy Kevin Sullivan Deputy Chief of Mission USAID Curt Reintsma USAID Director USAID Richard Kimball General Development Officer USAID Patrick Wesner Program Development Officer USAID Matthew Barnhart HIV/AIDS Advisor USAID Catherine Brokenshire-Scott Strategic Information Liaison Advisor USAID Catherine Chiphazi HPN Team Child Health Specialist USAID Alisa Cameron Health Team Leader USAID Catherine Wolf PMI Advisor USAID Lilly Banda-Maliru HPN Team Deputy Team Leader PSI John Justino Resident Director PSI Sarah Makunganya Gibson Director of Communications PSI Collins Mussa Director of Sales & Marketing PSI Alfred Zulu Director of Administration & HR PSI Jephta Mtema Director of Finance PSI Pastor David Mussa FCP Program Manager PSI Maria Mahowe FCP Communications Officer PSI Yvonne Kabwila TOC Program Manager PSI Marvin Mbwana Executive Producer YA! Mix PSI Martin Kalima YA! Mix Co-Producer/Presenter PSI Carolyne Maziya YA! Mix Co-Producer/Presenter PSI Victor Gama Youth Alert! Program Manager PSI Ricky Nyaleye Listener Club Coordinator PSI Judith Puikho Listener Club Officer PSI Emmanuel Frank Kadango Youth Alert! Officer PSI Eliana Gutierrez-Amo Program Manager PSI Richwell Tambuli YA! Communications Officer PSI Hadworic Keith Pundani Coordinator, VPU PSI Patrick Njawala Video Development Officer, VPU PSI Velia Manyonga Research Unit Coordinator PSI Philip Mkandawire Research Officer PSI Memory Mwasi Executive Assistant PSI Charles Yuma Senior Product & Marketing Services Manager PSI Robert Mahala Health Facilities Program Assistant PSI Ellias Matemba Warehouse Supervisor PSI Mary Baloyi Product Manager, Child Survival PSI Francis G Khonyongwa Child Survival Program Assistant PSI Franklin Allan John Senior Branch Operations Manager (Lilongwe) PSI Patrick Magalasi Listener Club Coordinator, Salima District PSI Lauteni Chatsika Youth Alert! Focal Person, Siya-Siya community, Salima District Pakachere Institute of Health & Development Simon Sikwese Executive Director 44 Malawi EHAP-IFH Program Organization Name Position Communications CDC Carl Campbell Director, CDC Malawi Malaria Program Malaria Alert Centre Chimwemwe Silwimba Quality Control Officer National Malaria Control Programme John A Chiphwanya Acting Programme Manager JHU-BRIDGE Ms. Glory Mkandawire Chief of Party JHU-BRIDGE Lisa Basalla BRIDGE Research Fellow JHU-BRIDGE Lawrence Chulu District Coordinator, Salima National AIDS Commission Robert M Chizimba Head of BCI Ministry of Health, SWAp Secretariat Dr Anne Phoya Director, SWAp Secretariat Ministry of Health, Reproductive Health Unit Hans Katengeza RH Officer Ministry of Health, HIV Unit Mr. Eliya PMTCT Coordinator Ministry of Health, Preventive Health Unit Member of staff UNICEF Ketema Bizuneh Health Manager UNICEF Karen Doll Manda Head, OVC and Child Protection Management Sciences for Health Mexon Nyirongo Chief of Party PACT Malawi Matthew Tiedemann Chief of Party, Community REACH Program PACT Malawi Kate Musimwa Senior Program Manager, Community REACH Program PACT Malawi Patrick Phoso Program Officer Malawi Police Force (HQ) Edward Chatsalira (Senior Asst Commissioner) HIV Program Coordinator Malawi Police Force (HQ) Alfred Mbewe (Sub-Inspector) HIV Program Officer PSI partners 4 commercial wholesalers and 8 commercial retailers Malawi EHAP-IFH Program 45 EHAP-IFH Civil Society Partners Location Contact person/s Position and connection to EHAP￾IFH Milonga Secondary School, Mulanje District YA! Schools Program meeting participants Nkanda School, Mulanje District George Temani YA! Listener Club Patron and teacher Nkanda School, Mulanje District Club leaders and members of the YA! Listener Club Mpemba Health Centre, Blantyre District ANC & MCH Clinic clients Mpemba Health Centre, Blantyre District Mrs. Wemah Mbalame Senior Community Health Nurse and Nurse-in-Charge Mpemba Health Centre, Blantyre District Flora Ngalauka Health Surveillance Assistant Mpemba Health Centre, Blantyre District Alice Bonya Health Surveillance Assistant Mpemba Health Centre, Blantyre District Victoria Chunba Health Surveillance Assistant Mpemba Health Centre, Blantyre District Lucia Gluedeza Health Surveillance Assistant District Medical Office, Blantyre Nyokase Kaimila District IMCI Coordinator District Medical Office, Blantyre S. Sambo District ITN Coordinator Queen Elizabeth Hospital, Blantyre Christina Mbiza Nursing Officer, Pediatric Dept Queen Elizabeth Hospital, Blantyre Catherine Chande Sister-in-Charge, Pediatric Dept Lirangwe, Blantyre Rural Adamu S Chiroko Member of FCP Faith Committee (Church of Christ BT) Lirangwe, Blantyre Rural Alex Bengo Member of FCP Faith Committee (Word of Life Church) Lirangwe, Blantyre Rural Yesaya Denja Member of FCP Faith Committee (UPC) Lirangwe, Blantyre Rural Mouren Edson Member of FCP Faith Committee (Church of Christ) Lirangwe, Blantyre Rural Ethel Mwalwanda Member of FCP Faith Committee (CCAP) Lirangwe, Blantyre Rural David Maluwa Member of FCP Faith Committee (Mpingo wa Khristu) Lirangwe, Blantyre Rural Anne Bazale Member of FCP Faith Committee (Abraham Church) Lirangwe, Blantyre Rural Volunteer teachers and Orphans and Vulnerable Children, Nazareth Nursery School, FCP NAPHAM, Blantyre Davis Halare District Coordinator NAPHAM, Blantyre James Misi Senior Counselor Siya-Siya Community, Salima District Members of the Village Development Committee and the VAC 46 Malawi EHAP-IFH Program Malawi EHAP-IFH Program 47 APPENDIX C. REFERENCES GOM, donor partner and civil society documents Ministry of Health (2005): Malawi: National Condom Strategy. National AIDS Commission & National Statistical Office (Malawi) (2005): National HIV/AIDS Action Framework. NAC. National Statistical Office (Malawi) & ORC Macro (2005) Malawi Demographic and Health Survey 2004. National Statistical Office (Malawi) & UNICEF (2008) Malawi Multiple Indicator Cluster Survey 2006. Final Report. DFID (2007): Country Assistance Plan 2007-2011 (draft). Data from USAID and UNICEF Malawi websites. REACH Trust Malawi documents. Other CSO documents. PSI/Malawi reports and other documents PSI/Malawi (2005a): Quarterly Progress Report. PSI/Malawi (2005b): Quarterly Progress Report. PSI/Malawi (2005c): Quarterly Progress Report. PSI/Malawi (2005d): Quarterly Progress Report. PSI/Malawi (2006a): Quarterly Progress Report. PSI/Malawi (2006b): Quarterly Progress Report. PSI/Malawi (2006c): Quarterly Progress Report. PSI/Malawi (2006d): Quarterly Progress Report. PSI/Malawi (2006e): Project TRaC: HIV/AIDS. The PSI Dashboard. PSI/Malawi (2006f): Project TRaC: malaria and diarrheal diseases. The PSI Dashboard. PSI/Malawi (2006g): Project MAP: coverage, quality of coverage, and national penetration. The PSI Dashboard. PSI/Malawi (2007a): Quarterly Progress Report. PSI/Malawi (2007b): Quarterly Progress Report. PSI/Malawi (2007c): Quarterly Progress Report. PSI/Malawi (2007d): Quarterly Progress Report. PSI/Malawi (2007e): Beliefs and social norms associated with HIV/AIDS – report. A qualitative study conducted in sampled districts. PSI/Malawi (2007f): Project MAP: Round 2 study. PSI/Malawi (2007g): Malawi Defence Force: firing messages against HIV/AIDS, stigma and discrimination. PSI/Malawi (2007h): Increasing health impact through targeting of condom sales in Malawi. PSI/Malawi (2008a): Quarterly Progress Report. PSI/Malawi (2008b): Quarterly Progress Report. In addition: PSI/Malawi Extension documents. A number of PSI/Malawi Creative Briefs. PSI/Malawi EHAP-IFH materials EHAP-IFH Behavior Change components Youth Alert! Program briefs – programs # 261 – 302 2008 Calendar 2008 Khakani Boo! (Stay Cool) pocket calendar 48 Malawi EHAP-IFH Program Maonekedwe apusitsa (can‟t tell by looking) (Girl/Boy) Posters Khalani Boo! Posters Live Show Promotional Posters Real Man/Real Woman posters Youth Alert! Real Man/Real Woman Music Video TV Magazine Show Youth Alert! Real Man/Real Woman Music Videos Youth Alert! My Choice Faith Communities Program (FCP) Promoting HIV/AIDS prevention among Malawi‟s Faith Communities since 2004 FCP Leadership Dynamics Training Manual FCP Adults‟ Training Manual FCP Youth Training Manual FCO Khalani ozindikira (Be wise) posters Commodities Chitetezo Net LLIN packaging with IEC messaging Consistent net use flyer – „Usiku uliwonse chaka chilichonse‟ Consistent net use posters – „Usiku uliwonse chaka chilichonse‟ Also: National Malaria Control Programme and PSI/Malawi in partnership with US President‟s Malaria Initiative „ Mwana akadwala malungo…‟ Thanzi ORS posters PSI Malawi Thanzi DVD „Mwana akatsegula m‟mimba…‟ Miscellaneous Copy of PSI radio adverts for Youth Alert!, Thanzi, M‟mbwerera Copy of Delayed Sexual Debut Real Man/Real Woman Music Competition Winning Entries PSI/Malawi materials produced with leveraged funding PSI/Malawi – Malawi Defence Force „A Soldier‟s Life‟ (US Dept. of Defense) Various Pakachere materials, e.g. its (not dated) Living positively with HIV/AIDS manual and its documents on gender-based and sexual violence. UNFPA female condom materials IDP Project materials and partner training manuals PSI Discretionary Funding materials for Family Planning products P&G POU water treatment materials KfW Chishango print and radio materials Malawi EHAP-IFH Program 49 Appendix D. Crosscutting issues 1. Gender While the EHAP-IFH evaluation team found clear evidence of commitment on the part of PSI/Malawi to gender-equal approaches and to the creation of gender-sensitive BC/BM messages and materials for use in mass and mid media and IPC, there is scope for more systematic mainstreaming and focus. PSI/Malawi acknowledges that while EHAP-IFH has sought to address gender issues in all program components, more gender-specific attention is required for both BCC development and implementation and in research and M&E. Gender analysis of PSI/Malawi quantitative and qualitative research is broadly lacking: gender analysis should not be confused with in-depth disaggregation during data analysis (although that too is essential). The term refers to matters such as a detailed examination of issues of agency (or its absence), (self) empowerment, the potentially different perceptions and applications of people‟s social capital, and an understanding also that „gender‟ can encompass many and varied attitudes and behaviors, e.g. that on occasion certain men may be more gender sensitive and equitable than many women. Because the 2005 PSI/Malawi TRaC survey (in particular) and other studies such as the 2007 qualitative social norms report have been the foundation upon which all EHAP-IFH messages have been built, the initial absence of detailed analysis based on gender may have considerable ramifications. HIV/AIDS prevention is widely acknowledged as a challenge where understanding of, and attention to, gender norms, attitudes and behaviors and agency are essential. Similarly, opportunity costs and societal barriers in terms of access to services and commodities (e.g. nets) may well differ considerably for men and women. Therefore, when the 2007 social norms study reports in aggregate („participants‟), e.g. on beliefs regarding HIV transmission, this represents a failure adequately to address potential gender differences that may have significant bearing on targeted and effective BC/BM messaging. The 2006 PSI/Malawi net coverage survey also reports largely in aggregate, and does not consider in detail issues of gender-specific perceptions, access, knowledge, etc. The evaluation team is aware of the capacity constraints in Malawi in terms of gender expertise. Nonetheless, PSI/Malawi should seek to strengthen and mainstream gender focus in all its activities, and gender analysis should be incorporated into all research and M&E activities. One existing PSI/Malawi and EHAP-IFH example of relative gender focus is the Faith Communities Program, whose manuals incorporate a degree of detailed and gender-aware approaches. Possible external resources include the Pakachere Institute and Management Sciences for Health, both with expertise in gender issues of relevance to Malawi. 2. Research PSI/Malawi did not undertake a pre-TRaC qualitative study of behavioral determinants. While PSI has tracked USAID outcome indicators, it has not been required by USAID to track process indicators regarding behavior change and maintenance. Therefore, while PSI has fulfilled USAID indicators, no indicators unique to EHAP-IFH were developed at the outset of the program. PSI/Malawi is of the view that is it is very difficult to disentangle with any certainty the specific impacts of its interventions on behavior change and maintenance among the target beneficiaries. 50 Malawi EHAP-IFH Program Furthermore, there are issues regarding the validity of the indicators specific to sufficient disaggregation and gender analysis. As discussed in the section on gender issues, it is essential that PSI/Malawi incorporate more structured disaggregation and closer analysis of both quantitative and qualitative data. PSI/Malawi is soon to introduce its new FOCUS research methodology, which targets an individual and seeks to define behavioral archetypes within the context of their environment(s). FOCUS will be used to develop statements encapsulating social norms; the intention is that these will be used to create behavior change and maintenance messages. If FOCUS is to be effective, current levels of analysis will need to be deepened and strengthened. The evaluation team recommends that PSI/Malawi consider the development and use of both process and outcome indicators for future measurement of its program impacts. Process and Outcome Indicators Indicators are used to track progress and change achieved by a project or program. In any project, indicators depend on that project‟s objectives. While objectives tell use what the project plans to achieve, indicators tell use how to measure if those objectives are achieved. In most projects there are two types of indicators. Process indicators describe the project‟s stage of implementation and progress in completing planned activities, e.g., support to behavior change. Outcome indicators describe the level of impact or change achieved through the activities. They are referred to as outcome indicators because they indicate the (usually quantitative) outcomes that are achieved through the project‟s intervention. The acronym QQT is often used as shorthand to define why both process and outcome indicators are necessary to enable more complex and complete measurement of any one intervention‟s inputs. QQT stands for quantity, quality, and time: the inputs, quality of interventions, time taken to implement the interventions, and the length of time the impacts of those interventions have been or may be sustained. Process Indicators Process indicators are usually identified through baseline qualitative research and are measured through qualitative research and participatory M&E. Their relative achievement in terms of impacts that can be directly or partially attributable to any one activity is measured through ongoing M&E and mid-term and end-line studies. Outcome Indicators Outcome indicators show the results of project activities; directly linked to objectives and expected impacts or results. Counting numbers reached and/or trained by project activities, the number of condoms distributed, etc., does not allow for complete assessment of the quality of inputs and the mid-term or long-term success or failure of a specific intervention and sustained support to behavior change and maintenance. PSI Nigeria (the Society for Family Health) has developed process indicators for use in its 7-year (2001-2008), DFID-funded Promoting Sexual and Reproductive Health for HIV&AIDS Prevention program. Lessons learned from that program may be useful to PSI. Malawi EHAP-IFH Program 51 3. GIPA and MIPA Greater, more meaningful involvement of PLWHA has increasingly come to the fore since the first UNGASS on HIV and AIDS in 2001. While PSI/Malawi undoubtedly applies an equitable approach in its work and organizational ethos, and is determined to ensure the complete absence of stigma and discrimination in the workplace, there is nonetheless scope for more integrated and coherent mainstreaming of GIPA/MIPA. For example, the evaluation team found no evidence of explicit PSI/Malawi company policy specifying appropriate involvement of PLWHA (whether disclosed status staff members or from the target beneficiary categories) in research or BC/BM materials￾development activities. There appears to be a similar absence of explicit, consistent PLWHA involvement in outreach activities under any of the three EHAP-IFH HIV prevention components. 4. Referral links PSI/Malawi should consider its future role in supporting the strengthening of referral mechanisms specific to addressing the needs of people wishing to be tested for HIV and having access to other services. None of the EHAP-IFH components incorporates a systematic element of referral information for beneficiaries: such referral might be to the nearest center for HIV Counseling and Testing, local PLWHA Support Groups, youth-friendly health facilities, reproductive health and family planning services, etc. While members of the various EHAP-IFH teams (TOC, Youth Alert! Schools Program, the FCP, etc.) will often provide such information based on personal knowledge, this is ad hoc and site- and situation-specific. Yet the EHAP-IFH evaluation team found evidence of a clear need for referral information among program target beneficiaries, and a realization among PSI/Malawi staff members that a live referral register would be a valued asset. Such views were echoed during a meeting with the Blantyre branch of NAPHAM (the National Association of PLWHA in Malawi). Attention to such referral links is stated by NAC to be in line with the imminent review of the National HIV/AIDS Action Framework 52 Malawi EHAP-IFH Program Malawi EHAP-IFH Program 53 Appendix E. Calculating the impact of PSI distribution of ITNs/LLINs PSI is developing a series of tools to evaluate the health impact of its interventions and services worldwide. The first stage in this process is the development of a working and evolving mathematical model to estimate the burden of disease and relative impact of PSIs work in each country. For malaria, the model uses a deterministic epidemiological framework to estimate the disease burden in a country and the impact of introducing PSI products. These products are both preventative products: ITN, LLIN, and insecticide treatments for nets; and therapeutic products: PPT (pre-packaged therapy) for malaria (both ACT and non-ACT) in infants, children, and adults. Estimating country-specific disease burden The first step of the model is to allocate each country into one of four groups based on relative transmission intensity of the disease. The four subgroups for transmission intensity come from the recent series of papers on estimating the burden of malaria globally by Hay et al (Nature, 2004): Hypo-, Meso-, Hyper- and Holo-endemic. The incidence and mortality rates from these four subgroups are then translated into four geographical malaria-endemic regions: sub-Saharan Africa (central), sub-Saharan Africa (southern), South Asia, East Asia, and Latin & Central America. For each region data is taken from the Snow et al working paper for DCPP on disease burden in the following parameters: population at risk, incidence of malaria (clinical episodes), relative risk of severe disease, and mortality rate from malaria (deaths). Each population is broken down into demographic subgroups: children 0-4 years, children 5-14 years, pregnant women, and all other adults. Finally, due to overwhelming evidence that the burden of malaria sits predominantly in rural areas, we took a meta-analysis of a series of studies across different regions that estimated the relative burden between rural and urban areas and weighted it across the rural/urban divide. The model then combines the demographic subgroups in absolute numbers with the rates from the regional datasets to estimate the absolute burden in terms of mortality and morbidity by demographic sub-group for each country. The final stage of the modeling process is the translation 54 Malawi EHAP-IFH Program of the diverse measures of health impact into a total figure for burden measured in DALYs. For this process the methodology is followed from the original DCPP report published in 2000, incorporating any updates from the more recent DCPP version 2 published in 2006. Estimating the impact of interventions The second component of the model estimates the impact of each intervention on alleviating the burden from the disease in a specific population. This involves estimating the likely impact from preventative interventions (i.e. nets and treatment of nets) on the likelihood of disease outcomes: episodes, severe episodes, and deaths. The first step is to translate the consumption of a product by a household into its use within that household. Data taken from a selection of published and PSI surveys on net use enabled us to develop a proxy table of effective net use in an average household. The next stage is to incorporate empirical evidence of the protective efficacy associated with use of a net, which in this case been drawn solely from the Cochrane review referenced in the DCPP final document (Lengeler, 2004). This quotes estimates of protective efficacy for treated and untreated nets and for insecticide treatment of nets across different regions. This data is then incorporated into the model alongside the previously estimated burden of malaria in terms of episodes of disease, episodes of severe disease and deaths from malaria. Finally we have to incorporate the „effective lifespan‟ of each product. For example, LLIN have a lifespan of between three and five years, whereas some net treatments only have an effective lifespan of six months. Testing of the model and results The final exercise in the development of the deterministic version was the comparison of model predictions against published and unpublished results. The totals for DALY burden from sub￾Saharan Africa match closely to the recent DCPP study and for Asia we have matched more closely to the estimates from the series of papers by Hay & Snow in Nature, and the Journal of Infectious Disease (Snow et al, 2004; Hay et al, 2005), which showed that the burden in Asia was underestimated by DCPP due to the use of passive surveillance especially in East Asia. For more information, please visit http://www.ghtechproject.com/resources.aspx Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com