BRIDGE PROJECT FINAL EVALUATION SEPTEMBER 2008 This publication was produced for review by the United States Agency for International Development/Malawi. It was prepared by Jean Capps and Sandra Wilcox through the Global Health Technical Assistance Project. BRIDGE PROJECT FINAL EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 08-001-120) is available in printed or online versions. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources/. Documents are also made available through the Development Experience Clearinghouse (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-0. BRIDGE PROJECT FINAL EVALUATION i ACKNOWLEDGEMENTS The final evaluation team would like to acknowledge the generous support it received in planning and organizing this evaluation from the Johns Hopkins University and Save the Children staff of the BRIDGE Project. Glory Mkandawire, chief of party, Pius Nakoma, community mobilization and capacity-building coordinator, and the project‘s seven district coordinators gave generously of their time and support in accompanying the team on field visits, answering questions, and providing insight into the BRIDGE Project‘s experience. The team would also like to thank Triza Kakhobwe, behavior change intervention coordinator, Tellina Matabwa, administrative manager, and Mutheliso Luhana, finance manager, for their support and willingness to answer numerous questions about the project. Thanks are due to the members of the National AIDS Commission and the Malawi Ministry of Health‘s Health Education Unit, who gave freely of their time in answering numerous questions and providing feedback. Special thanks are also due to Alisa Cameron, USAID/Malawi Health Team leader, and Matthew Barnhart, USAID/Malawi HIV/AIDS lead, not only for arranging for the BRIDGE Project Final Evaluation but also for providing insight and support to the evaluation from the donor‘s perspective. Thanks also go to the other USAID Health Team members who assisted the evaluation team: Catherine Brokenshire-Scott, strategic information liaison advisor, and Lilly Banda-Maliro, reproductive health specialist. The team would also like to thank the numerous people (listed in Annex 4) from communities, radio stations, media agencies, nongovernmental organizations, faith-based and community￾based organizations, District AIDS Coordinating Committees, Community and Village AIDS Committees, cooperating agencies, schools, youth clubs, listening clubs, and support groups who were interviewed for this report and took a serious interest in sharing their BRIDGE experience with the team. Finally, the team thanks Charles LaDuca and the other members of the GH Tech team for all their assistance with organization, assignment preparation, logistics, and final organization of the report. Despite the best efforts of the BRIDGE Project and USAID, factual errors may appear in the report. They must be considered the responsibility of the authors, who have tried to grasp a complex environment in a very short time. ii BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION iii ACRONYMS ARV Antiretroviral drugs BCC Behavior change communication BCI Behavior change intervention CA Cooperating agency CAC Community AIDS Committee CBO Community-based organization CCP Center for Communication Programs COP Chief of party CSW Commercial sex worker DACC District AIDS Coordinating Committee DHMT District Health Management Team DHS Demographic and Health Survey DIP District Implementation Plans FBO Faith-based organization GH Tech Global Health Technical Assistance Project GOM Government of Malawi HCT HIV counseling and testing HEU Health Education Unit HPN Health, Population, and Nutrition IEC Information, education, and communication JHU Johns Hopkins University MANASO Malawi Network of AIDS Service Organizations MBC Malawi Broadcasting Corporation M&E Monitoring and evaluation MDHS Malawi Health and Demographic Survey MMC Media Materials Clearinghouse MOHP Ministry of Health and Population NAC National AIDS Commission NBCI National Behavior Change Intervention Strategy NGO Nongovernmental organization NYCOM National Youth Council of Malawi PAC Public Affairs Commission PEPFAR President‘s Emergency Plan for AIDS Relief PLWHA Person living with HIV/AIDS PMTCT Prevention of mother-to-child transmission (HIV) PMP Performance monitoring plan iv BRIDGE PROJECT FINAL EVALUATION PSI Population Services International PVO Private voluntary organization RPA Risk Perception Attitude Framework SC/US Save the Children, U.S. SE Structural-environmental model SO Strategic objective SOW Scope of work SRH Sexual and reproductive health STI Sexually transmitted infections TWG Technical working group USG U.S. government VAC Village AIDS Committee VCT Voluntary HIV counseling and testing YAM Youth Alert Mix BRIDGE PROJECT FINAL EVALUATION v CONTENTS ACKNOWLEDGEMENTS ................................................................................................ i ACRONYMS ...................................................................................................................iii EXECUTIVE SUMMARY ................................................................................................ vii INTRODUCTION ............................................................................................................. 1 FINDINGS AND RECOMMENDATIONS ........................................................................ 5 CONCLUSIONS AND FUTURE DIRECTIONS ............................................................. 27 ANNEX 1: FIELD SITE VISIT SCHEDULE ................................................................... 29 ANNEX 2. MALAWI BRIDGE EVALUATION QUESTIONNAIRE.................................. 31 ANNEX 3. CULTURAL PRACTICES ADDRESSED IN BRIDGE COMMUNITY-BASED ACTIVITIES ............................................................. 35 ANNEX 4. PERSONS CONTACTED AND INTERVIEWED .......................................... 37 ANNEX 5. REFERENCES ............................................................................................ 43 ANNEX 6. SCOPE OF WORK ...................................................................................... 45 vi BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION vii EXECUTIVE SUMMARY A final evaluation of the BRIDGE Behavior Change Initiative HIV/AIDS Project led by the Johns Hopkins University Center for Communication Programs (JHU/CCP) and Save the Children, U.S. (SC/US) was conducted in Malawi August 4–30, 2008. The objectives were to determine progress in achieving the results outlined in its work plans and then to formulate recommendations and identify lessons learned for use in future activities USAID/Malawi may wish to explore. The evaluation team consisted of two consultants from the GH Tech Project. After an initial briefing from the USAID officer in charge and BRIDGE project staff, the team conducted field visits to Mangochi, Salima, Mzimba, Kasungu, Ntcheu, Mulanje, Chikwawa, and Balaka districts and Blantyre city August 8–August 20. During the visits the team interviewed youth groups, cultural committees, parent groups, AIDS support groups, District AIDS Coordinating Committees (DACCs), Community AIDS Committees (CACs), Village AIDS Committees (VACs), community-based organizations (CBOs), radio listening groups, collaborating faith￾based (FBO) and nongovernmental organizations (NGO), school staff, and others. In addition, numerous interviews were conducted in Blantyre and Lilongwe with representatives from radio stations, media firms, NGOs, cooperating agencies, and government ministries. Follow-up in￾depth interviews were also conducted with BRIDGE and USAID staff. In general the team found that BRIDGE is achieving most of its stated objectives. Finding after completing its baseline survey that the initial intention of targeting high-risk groups was not appropriate, the project changed its strategy to target the general population, with an emphasis on both youth and adults. The four pillars of the BRIDGE strategic framework (based on the structural-environmental (S-E) model for behavior change) are nation, community, capacity building, and coordination. BRIDGE‘s multilevel approach is appropriate; it facilitates creation of norms and interpersonal support for adhering to protective behaviors. The evaluators found evidence that dissemination of consistent messages to different audiences is creating synergy that is reinforcing intent to adopt behaviors protecting against HIV. This is particularly evident in the radio listening groups, radio support groups, youth clubs, interfaith groups, and district and community committees. There is reliable evidence that the various groups working on HIV/AIDS prevention and control have harmonized their messages and are building capacity for efficient production of communication materials and dissemination of messages. Among the most successful campaigns are the Radio Diary program (people living with HIV and AIDS [PLWHAs]), Youth Alert, the Tisankhenji Girls radio program, and the three phases of the Nditha campaign. The unifying themes of openness, hope, and self-efficacy (Nditha), male responsibility (Bambo Wachitsanzo), and raising risk awareness are effectively reaching their target audiences. Initial emphasis was given to youth, and in the later campaigns added weight has been given to adults. The quality of the radio broadcasts is high. Radio Diaries and HIV/AIDS content in the Youth Alert and Tisankhenji programs are reaching audience segments that are a critical link to the community, offering messages of hope and empowerment. These are cited by community members and radio station staff as essential in changing behaviors. The facilitator guides for Youth Alert and Tisankhenji clubs are well written. For continued behavior change, technical capacity building will continue to be needed at the radio stations, in leader/teacher training, and in managerial and programmatic areas at the district levels. viii BRIDGE PROJECT FINAL EVALUATION According to its midterm surveys, the project seems to be on track to meet the indicators cited in the agreement and measured at baseline related to improvements in knowledge (more than 60%); risk perception (more than 100%); and self-efficacy (20–40%) among youth and adults. In addition, 17 percent of youth and 43 percent of adults feel more confident in talking with partners about condoms. Other indicators show improvements of 10–20 percent in HIV testing in three out of four districts. In addition, a 2006 study by Population Services International (PSI) shows that 58 percent of youth who have high exposure to Youth Alert programs and clubs use condoms. Although initially there were some data quality problems with measurement of the indicators for the President‘s Emergency Plan for AIDS Relief (PEPFAR), these were corrected. The project uses PEPFAR data to monitor progress and make management decisions about program direction. It also used the mid-term evaluation survey results to confirm that the audience was ready for the Risk Awareness Campaign phase of the project to be introduced. However, the actual impact of the project on behavior is not being measured. Consistent with its theory-based approach, the project is measuring knowledge, intention, and perceptions, but not whether behaviors have actually changed. Also, because there is no district-wide coverage data, it is difficult to measure the broad public health impact of activities focused on four to seven traditional authorities in each district. MANAGEMENT The BRIDGE project is a driving force for behavior change intervention (BCI) in Malawi at all levels and is pushing the BCI agenda in close cooperation with the National AIDS Commission (NAC) and the Health Education Unit (HEU) of the Ministry of Health and Population (MOHP). BRIDGE deserves credit for building capacity in Malawi. All project staff are Malawian. The project has provided a career track for a Malawian professional as chief of party (COP). District officers are competent, knowledgeable, and a resource that is valued by their counterparts. BRIDGE has brought important technical assistance and capacity to partners from both project and headquarters staff. There is a good complementary relationship between the JHU/CCP and SC/US; the organizations clearly function as a team in addressing both national and community components. Systems and procedures are in place for personnel, subgrants, subawards, and technical activities. However, the fact that many administrative and financial transactions need approval or are managed directly from Baltimore and the requirement that USAID approve many activities have caused delays and programming inflexibility. BRIDGE effectively operates at the national level through working groups, task forces, collaborating partners, and other networking activities to create a coordinated and harmonized BCI approach. The project assures that messages are consistent and that all agencies are promoting the same HIV prevention concepts—the core BCI concepts (hope, openness, Nditha) have been very successful. GENERAL RECOMMENDATIONS BRIDGE is the lead NGO working on BCI in Malawi. After it ends, the team recommends that the NAC and the governments of Malawi and the United States support a similar project BRIDGE PROJECT FINAL EVALUATION ix to extend HIV BCI and continue promoting efficacy and risk interventions. State-of-the-art, evidence-based best practices in HIV prevention are not yet demonstrated. This presents an opportunity for BRIDGE to demonstrate its successes. In order to scale up, there is a need to increase funding and staff and leverage resources for non–HIV/AIDS-related programming, such as education, reproductive health, and Food for Peace. BCI media activities directed to both youth and adults should continue. There is a need to continuously reinforce messages and to scale up implementation throughout districts. Building capacity in radio stations, media partners, community-based theater, and advertising agencies also should continue. It is recommended that Tisankhenji school activity be expanded by linking it to girls‘ education, radio learning programs, and gender activities. This program is having a noticeable impact on young girls. The Radio Diaries program should be continued; it is reaching a wide audience, reducing stigma and discrimination, and encouraging openness. However, the themes could be expanded to cover pregnancy, prevention of mother-to-child transmission (PMTCT), multiple partners, and mothers who talk about HIV. Youth Alert clubs, which have been in place for four years, are having an impact. They should continue but be segmented by age. They could also be linked to education programs and adolescent reproductive health, with BRIDGE providing technical assistance. A system is needed to monitor the quality of BCI training and tools once these are turned over to collaborating and implementing partners. As time goes on, BRIDGE will need to prioritize because it must respond to more and more requests for technical assistance as other demands and workload increase. Depending on the focus of future projects, it may be necessary to clarify the priority of technical assistance compared to other activities. BRIDGE staff members are interested in forming a local BCI organization similar to JHU/CCP that will be eligible for NAC funding. Technical assistance from JHU or another agency would help them formulate a vision for such an organization and understand the organizational and administrative requirements. Monitoring and evaluation (M&E) capacity needs to be increased, and NAC, the Government of Malawi (GOM), the U.S. government (USG), BRIDGE, and other development partners need a shared vision for M&E that goes beyond collecting and reporting quality data. It is necessary to better define the relation between self-efficacy, risk assessment, intention to change, and whether change occurs (behavioral impact). x BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION 1 INTRODUCTION The main purpose of this final evaluation is to provide a basis for USAID/Malawi‘s Strategic Objective 8 (SO 8) team to assess progress made by the Johns Hopkins University Center for Communication Programs (JHU/CCP) and Save the Children‘s (SC/US) BRIDGE Behavior Change Initiative Project in mitigating the impact of HIV/AIDS in Malawi. A second purpose is to identify lessons learned and make recommendations for USAID/Malawi to explore in designing future programs. This report addresses a series of questions about the success of the BRIDGE Project in building institutional capacity in target districts and among communication agencies. It also reviews the potential for scale up of project activities and the sustainability of BRIDGE achievements. This external evaluation offers an opportunity to highlight project strengths, discuss weaknesses, and explore options for future programs. It also offers an opportunity to make recommendations that will enhance results before the project ends. BACKGROUND HIV/AIDS in Malawi When it began in 2003, the BRIDGE project was designed to address two conflicting realities: More than 93 percent of Malawians had the knowledge necessary to prevent HIV infection. Nevertheless, most Malawians did not engage in preventive behaviors. These conflicting realities are commonly referred to as the KAP gap: the gap between knowledge, attitudes, and practices. According to the 2000 Demographic and Health Survey (DHS), in the previous year 18 percent of married men had engaged in extramarital sex and 25 percent of unmarried men had multiple partners. Only 15 percent of men and 5 percent of women reported using a condom in their last sexual encounter. Social stigmas, gender inequalities, poverty, and little access to basic services are a few of the obstacles that prevent Malawians from adopting behaviors that will lower their risk of being infected with HIV. Malawi is one of the 10 countries worldwide that are most affected by HIV/AIDS. The HIV prevalence rate is estimated to be 12 percent among those who are 15 to 49,1 and the gap between male and female prevalence is smaller than in other countries in sub-Saharan Africa. AIDS is now the leading cause of death in adults. The National AIDS Commission (NAC) estimates that there are 70,000 new cases of AIDS each year in Malawi. While youth continue to be at high risk, the age-specific HIV rates point to a need for prevention efforts to also target adult populations. HIV prevalence in urban areas is significantly higher than in rural areas, and rates are highest across the south, where approximately half the population resides. Low rates of male circumcision, the cultural practices of certain ethnic groups, and the low status of women are believed to contribute to HIV transmission. Early marriage is another HIV risk factor for women. 1 Malawi Demographic and Health Survey, 2004; HIV Sero-survey in Antenatal Clinics, 2007. 2 BRIDGE PROJECT FINAL EVALUATION The Government of Malawi (GOM) has responded to the KAP gap by developing a national behavior change intervention (NBCI) strategy that not only reinforces the knowledge of citizens about how to change their practices but also discusses how they can act on this knowledge. Through its SO 8, USAID/Malawi supports the NBCI. By addressing the need to change the behavior of youth especially and the population generally, USAID Malawi hopes to delay sexual debut, increase condom use, and decrease the number of sexual partners in order to reduce new HIV infections. The BRIDGE Project Response In 2003 funding was awarded to JHU/CCP and SC/US for the BRIDGE Behavior Change Initiative (BCI) Project. The purpose of the project is to aid and coordinate stakeholders as they implement the NBCI strategy. JHU and SC/US identified the need for Malawi to operationalize the NBCI strategy to create a bridge between strategy and action. The project builds the capacity of behavior change agents while coordinating and mobilizing community and national stakeholders as they implement the NBCI. The expectation was that as these stakeholders implement evidence-based BCI, Malawians would begin to act on their HIV prevention knowledge. Outlets like the media, community leaders, and national celebrities are being used to model and reinforce changed behavior and support NBCI movements. It was anticipated that the BRIDGE project would reduce high-risk behavior and increase the use of HIV prevention practices (SO indicators), particularly among youth and the general population, and thus ultimately reduce the number of new HIV infections. When the project was originally conceived the objective was to focus on ―bridge‖ populations and ―targets of opportunity.‖ ―Bridge‖ populations were understood to be those individuals who engage in risky behavior, become infected, and transmit HIV to lower-risk partners. ―Targets of opportunity‖ are those who are already infected or at high risk of infection due to proximity to bridge populations (youth, lower-risk partners, etc.). Once the project began in Malawi, JHU realized through its own data analysis and experience in other countries that Malawi was undergoing a generalized HIV/AIDS epidemic that affected the whole population. At that point the project reoriented its approach and determined that the BCI strategy should be directed to the general population but with particular attention to youth because of their vulnerability. Many of the messages do draw attention to ―bridge‖ and ―target of opportunity‖ groups, but these are not the focus of activity. The BRIDGE project uses four strategies to achieve its objectives: Support an enabling environment at the national level by harmonized action plans, using unifying themes such as ―openness and hope‖ and core tools to support on-the-ground mobilization. Mobilize the social system at the district and community level through a grants program, implementation of transformative BCIs and of a community action cycle, and workshops that enable change agents to play a powerful persuasive role. Reach individual targets of opportunity, such as bridge groups, youth, men, and clients through a range of innovative approaches, such as sports intervention and youth events. Increase the capacity of government, NGOs, CBOs, FBOs, and community-based influentials to plan, coordinate, and implement effective BCI. BRIDGE PROJECT FINAL EVALUATION 3 Underlying these strategies are the principles and values BRIDGE uses as a theme for project interventions: Belief in a better future (hope) Risk is shared by everyone (personalized risk) I can STOP AIDS (personal responsibility, action, self-efficacy) Discussion about HIV/AIDS (openness, destigmatization) Gender equity (girls‘ empowerment and changed men‘s behavior) Emphasizing positives (action orientation, community assets, positive role modeling). The BRIDGE project, based in Lilongwe, is led by the JHU/CCP chief of party (COP), Glory Mkandawire. JHU/CCP is leading the project and provides all behavior change technical assistance; the SC/US role has been to provide technical assistance in grants management and community mobilization. The project‘s technical staff consists of the COP, a BCI coordinator, a youth coordinator, an M&E officer, an HIV/AIDS specialist, a community mobilization and capacity building coordinator, two program officers, and eight district coordinators. The project is being implemented in eight districts: Mzimba, Kasungu, Ntcheu, Salima, Mangochi, Balaka, Mulanje, and Chikwawa. METHODOLOGY At the request of USAID/Malawi, the Global Health Technical Assistance Project (GH Tech) recruited a two-person team to conduct a final evaluation of the BRIDGE project. The team brought expertise in HIV/AIDS, behavior change communication (BCC), population-based programming, community mobilization, capacity building, management, and evaluation. Before going to Malawi, the team conducted telephone discussions and reviewed project and other materials provided to them by GH Tech and USAID/Malawi. Upon arrival in Malawi the team was briefed by the USAID HIV/AIDS lead officer on the issues to be explored and by JHU/CCP and SC/US staff on BRIDGE project activities. Staff also helped the team to compile a list of stakeholders, plan field visits, and arrange meetings with local groups. The team drafted questionnaires for use in for interviews and focus group discussions with stakeholders (Annex 2). Between August 8th and 21st, the team visited all eight project districts to interview youth groups, cultural committees, parents groups, AIDS support groups, District AIDS Coordinating Committees (DACCs), Community AIDS Committees (CACs) and Village AIDS Committees (VACs), community-based organizations (CBOs), radio listening groups, FBO and NGO collaborating partners, school staffs, and others. One team member spent two and a half days in Blantyre city interviewing NGO/CBO agencies, cooperating agencies (CAs), radio stations, advertising agencies, media firms, and community theater training groups (see Annex 4). Upon returning to Lilongwe, the team met with the HEU and HIV/AIDS units at the MOHP. Finally, USAID Health, Population, and Nutrition (HPN) staff were interviewed to gain their perspective on the history and performance of the project. (See Annex 1 for the team‘s schedule.) 4 BRIDGE PROJECT FINAL EVALUATION Constraints Because the end-of-project survey has not yet been conducted, the evaluators were unable to analyze their qualitative project findings in terms of quantitative data. They were thus unable to document that implementation based on theory has actually changed behaviors or to determine the extent of possible changed behaviors beyond the immediate project areas. The evaluation was limited to the midterm survey results, self-reported behaviors, observation, and interviews. Turnover of both project and collaborating partner staff meant that many of those interviewed did not have enough experience with the project to answer evaluation questions about the project‘s history in a region or reasons for changes in project direction. The only technical staff who had been with the project from the beginning were the COP and two district coordinators. However, the COP accompanied the team to the southern districts and also made herself available throughout the evaluation period to clarify issues. Although key individuals had been with the project only briefly, the evaluation was comprehensive. However, this constraint may have resulted in some misunderstandings of the project, such as project accomplishments or roles for key organizations, that the evaluators were unable to discover or verify. However, the team organized interviews and focus groups to include as much participation as possible subject to the availability of project partners and stakeholders. BRIDGE PROJECT FINAL EVALUATION 5 FINDINGS AND RECOMMENDATIONS This report responds primarily to questions posed in the final evaluation scope of work (SOW). The SOW questions and the evaluators‘ responses are presented in separate sections. The team presented additional detailed findings in its oral debriefings with the USAID Mission, the project, and its partners. 1. Assess progress made in implementing the project and achieving yearly targets and if the project achieved its objectives. Review the suitability of the design and effectiveness of BRIDGE components in helping community and national stakeholders implement the NBCI strategy. TABLE 1. PROPOSED AND ACTUAL CHANGES TO INTERMEDIATE TARGET INDICATORS (Baseline to First and Second Midterm) Indicator Baseline Value Target (%↑ from baseline) Midterm 1 (% ↑ from baseline) Midterm 2 (% ↑ from baseline) Knowledge % youth who score above 75% on a knowledge test about HIV prevention and transmission % adults who score above 75% on a knowledge test about HIV prevention and transmission 44 30 55 (25%) 39 (30%) 72.3 (64.3%) 69.6 (132%) 71.6 (62.7%) 60.6 (102%) Risk perceptions % youth who believe they could become HIV positive in the next year % adults who believe they could become HIV positive in the next year 17 21 20.4 (20%) 25.2 (20%) 39.7a (133%) 44.3 a (111%) 41.4a (144%) 49.9a (138%) Self-efficacy % youth who “definitely can” remain abstinent until they are married % youth who “definitely can” have sex with only their partner while in a relationship % youth who are “generally” or “strongly” confident that they can use a condom every time they have sex % married adults who “definitely can” have sex only with their spouse 75 78 66 70 82.5 (10%) 85.8 (10%) 79.2 (20%) 77 (10%) 96.1b (28%) 87.9b (12.6%) 80.0 (21.2%) 97.6b (39.4%) 58.0b (-22.7%) 91.5b (17.3%) 83.7b (26.8%) 94.1b (34.4%) Interpersonal communication % youth who are confident they can talk about the use of condoms with their sexual partner % adults who are confident they can talk about the use of condoms with their sexual partner 76 55 83.6 (10%) 66 (20%) 86.6 (13.9%) 79.0 (43.6%) 89.4 (17.6%) 79.1 (43.8%) a The wording of the answer choices was somewhat different at baseline and at midterm. Risk perception was asked as “could happen” or “could not happen” at baseline, and at midterm as “not at all likely,” “somewhat likely,” and “very likely.” In the table, “somewhat likely” and “very likely” are combined. b Figures for midterm represent responses that “mostly agreed” or “strongly agreed” to the given statement. At baseline, response scales were “generally agree” and “agree very strongly.” 6 BRIDGE PROJECT FINAL EVALUATION As can be seen from Table 1, the project had either met or surpassed its key targets by the time of the two midterm surveys (MTS1–2006, MTS2–2007). The indicators measure knowledge, risk perceptions, and perceptions of self-efficacy in accord with the structural-environmental (S-E) theoretical model and the project approach. Of particular note is that among both youth and adults the perception of the risk of contracting HIV/AIDS increased more than 100 percent. Moreover, the self-efficacy findings showed significant increases in the percentage of youth who believe that they can abstain from sex or use condoms and of adults who believe they can be faithful to their spouses. These findings of increased risk awareness and greater self-efficacy prompted the project to move forward with its last campaign using themes to reduce the risk of contracting HIV/AIDS, such as lowering alcohol use and reducing mother-to-child HIV transmission and intergenerational sex. The midterm surveys also found notable improvements in the HIV testing rates in three of the four districts surveyed. Although the midterm survey data on condom use was not available at the time of the evaluation, a PSI survey in 20062 found that 58 percent of youth with high exposure to Youth Alert programs and 51 percent of youth with medium exposure reported regularly using condoms, compared to only 40 percent of those who had little or no exposure to the program. The BRIDGE project is an HIV/AIDS prevention program designed to change the way Malawians think and act in relation to HIV. The baseline survey and other background research pointed out the need for a behavioral approach that could address the pervasive sense of fatalism and low perception of HIV risk present in Malawi when the project began. To address this, the project chose as its theoretical basis the S-E model, which maintains that behavior change requires interventions at national, community, and individual levels. For individuals to change, they need knowledge, skills, hope, and self-efficacy; they also need reinforcement and behavior modeling from their social system—peers, family, and authority figures—in a supportive and responsive environment dedicated to behavior change (see Figure 1). 2 PSI Malawi Project TRaC – HIV/AIDS. The PSI Dashboard. March 2006. Research Division, Population Services International. Malawi. BRIDGE PROJECT FINAL EVALUATION 7 Figure 1. Structural-Environmental Model for Behavior Change Interventions in Malawi Structural-Environmental Model for Behavior Change Interventions in Malawi National Enabling Environment • Increased capacity • Resource allocation • Political will • Policy changes Community Structures • Leadership • Social norms • Collective efficacy • Participation equity • Information equity • Priorities consensus • Ownership • Social capital • Responsive services Individual • Knowledge • Attitudes • Perceived risk • Perceived self-efficacy • Perceived social support • Perceived stigma • Personal advocacy • Life skills • Demand for services Intermediate Result 8.2: Reduced new HIV infections Advocacy Community Mobilization Capacity Building Interpersonal Communication Mass Media Strategic Behavior Change Intervention Initial Outcomes Behavioral Outcomes Supportive Environment • Multisectoral partnership • Coordination • Institutional performance • Resource acquisition • Media support Community Structures • Collective action • Reduced unhealthy traditional practices • Improved gender relationships • Quality/friendly services • Increased client volume Individual Behaviors • Sexual abstinence, delay, or partner reduction • Condom use • Appropriate service use • Accepting behaviors Sustainable Outcome The baseline and midterm surveys assessed knowledge of HIV/AIDS, stigma, communication, sociocultural factors related to current health and sex practices, risk perceptions, and control/efficacy. According to behavior change theory and the S-E model, risk perceptions and control/efficacy components are essential to behavior change strategies. Analysis of these variables helped to identify ―bridge‖ populations and strategic communications objectives for accomplishing the desired behavior change. Keeping this in mind, the project successfully formulated strategies to counter both the sense of fatalism present in Malawi and the perception of most Malawians that they were at low risk of contracting HIV. As a first step the project produced the Nditha campaign focusing on the unifying themes of hope and self-efficacy. Evidence from the midterm surveys and this evaluation indicates that these messages are reaching targeted audiences—75 percent of individuals surveyed knew the Nditha slogans. There is similar evidence that messages from the subsequent two campaigns, on male involvement (Bambo Wachizanzo) and risk awareness (alcohol, MTCT, and intergenerational sex), are also reaching target audiences. These campaigns are consistent with and helped activate the NBCI. They encourage Malawians to take small, doable actions to prevent HIV. The NBCI strategy was also addressed through many of BRIDGE‘s community mobilization activities (see below, questions 5 and 6) and the initiatives BRIDGE supported in youth congresses through the National Youth Council of Malawi (NYCOM), the work the Public Affairs Commission (PAC) did with adults, and the training of PLWHA, parents groups, and the media. 8 BRIDGE PROJECT FINAL EVALUATION In general, the reviewers believe, the project is designed correctly based on research and theory that backs up its behavior change strategies. From the surveys and this qualitative review, the strategies seem to be succeeding. People are more knowledgeable about HIV prevention, a greater number perceive themselves to be at risk of the disease, and more of them feel confident in making decisions about abstinence, being faithful to one partner, and condom use. Also, as demonstrated by the Nditha campaigns, more individuals and communities feel empowered to take steps to prevent HIV. There are not many effective BCI programs demonstrating success in HIV/AIDS prevention; the state-of-the-art evidence base is weak in this area. Because the BRIDGE project has delved into and tried to change the root causes of behaviors that promote HIV infection, it offers promising alternative strategies for combating spread of the disease in Africa. Feedback from community members across districts, age groups, and both genders revealed perceptions that teenage pregnancies had decreased, health facility circumcisions have increased (in areas where circumcision is practiced), there was ―less moving around‖ by youth at night, and school drop￾outs and several ―harmful cultural practices‖ have decreased. Because BRIDGE‘s M&E system is not designed to capture these effects, verifying the extent of these changes will require additional investigation. Promising self-reports of decreased alcohol consumption and fidelity by men seeking to become Bambo Wachitsanzo also need further verification. Informants were unequivocal in endorsing the ―positive‖ behavioral approach that encourages communities and individuals to ―move towards a better life‖ and away from ―danger and death.‖ The approach emphasizes that individual and group empowerment, goal orientation, and the individual‘s ability to succeed even after failure to become a community role model are more rewarding than the short-term pleasures of risky encounters. This approach differs significantly from the more cerebral, guilt-producing approaches of many other prevention programs. It was supported by campaigns and community activities to create an enabling environment that encourages and supports individual behavior change. BRIDGE‘s research discovered, and the evaluation field work supported the discovery, that HIV/AIDS prevention activities in Malawi need to be demedicalized and integrated more completely into Malawian daily life. Recommendation BRIDGE is the lead organization working on BCI for HIV prevention in Malawi. After it ends, a similar project should extend HIV BCI and continue building on efficacy and risk interventions, with more thorough measurement of actual behavior change. State-of-the-art, evidence-based best practices in HIV prevention have not yet been demonstrated. This presents an opportunity for BRIDGE to build on its successes. 2. Assess whether the BRIDGE Project is meeting the benchmarked activities negotiated in the agreement: Are data-gathering methods reasonable for monitoring progress and indicators? If not, why not? Are indicators appropriate and valid? If not, why not? Quantitative measurement of changes in the requisite behavior (self-efficacy, risk perception, and intention, specifically condom use) and the consistency of responses from informants in all eight districts indicate that behavior is actually changing. Project indicators, both the initial ones and those that were added when PEPFAR was introduced to Malawi, only partially capture the impact of the project‘s approach. Regrettably, methods of proving the effectiveness of specific prevention approaches are still not well-defined. Uptake of HIV counseling and testing (HCT), BRIDGE PROJECT FINAL EVALUATION 9 the only behavioral indicator measured during the program, has increased from the baseline. Later age of sexual debut, sex with regular partners, and condom use will be measured in the end-of-program quantitative survey scheduled to begin in October 2008 (see Table 1 above). Analysis for this evaluation could have benefited if population-based data had been obtained from collection methods (such as lot quality assurance samples) that were recommended in the BRIDGE external midterm evaluation. The original project M&E plan did not provide for collecting this type of data.3 BRIDGE‘s data collection methods and indicators had to change several times during the course of the project. In 2003 HIV/AIDS indicators were significantly different from those introduced when the Mission was drawing up its performance monitoring plan (PMP) and when the COP was introduced after 2005. No M&E position was specified in the project organogram or budget. Once need for this expertise was identified, the job was exceptionally difficult to fill. One promising hire did not meet expectations and was terminated. Discussions with the research director at JHU/CCP in Baltimore confirmed that project data collection and reporting was of necessity focused more on monitoring and reporting than on formal evaluation (though there will be a number of reports analyzing specific components of the program at the end, including the Hope Kit Assessment (completed) and the Tisenkenji Listening Clubs). BRIDGE consistently adapted to changing requirements (including changes in indicators and in the reporting period, from July to October) and responded to data quality requirements when a data audit identified a significant error. The Mission strategic information officer stated that BRIDGE was responsive to feedback and made the necessary corrections. Whether indicators and data collection are appropriate is a matter of perspective. If the question is, ―Did BRIDGE collect data, report on targets and analyze reasons for meeting, or not meeting targets identified in the larger USAID HIV/AIDS program?‖ the answer is yes. On the other hand, both PEPFAR and the Mission‘s PMP have used process indicators since the project‘s inception. Process indicators do not answer the questions generated in a Results Framework: ―Did the project activities lead to attainment of the Intermediate Results that would ultimately lead to meeting the goals and objectives in the framework?‖ The objective of the BRIDGE research design was to demonstrate that the prerequisites of behavior change had been attained, not that the behavior itself had already changed. The evidence strongly suggests that the desired changes are taking place, but the extent and type of these changes is not yet known. Geographic representation: Work in the North, Central, and South regions, rather than total population coverage, were the original objective of both the Mission (SO 8) and the BRIDGE strategy. The external midterm evaluation highlighted the need to scale up and develop an exit strategy before the project ends. The uncertainty of funding (only two more years was the option in 2005) and severe budget restrictions especially at the district and community level made extensive coverage of the entire population virtually impossible. BRIDGE was able to scale up to only two more traditional authorities in each district, leaving total coverage of community activities at less than 50 percent of the population except for the seven out of nine traditional authorities in Mangochi District, where Save the Children built upon the previous STEPS project. Nevertheless, the media campaigns and wide dissemination of some BRIDGE activities, notably the Hope Kit, have had impact beyond the communities targeted. 3 Capps, Haider, and Wilcox, BRIDGE Project Midterm Evaluation, July 2005. 10 BRIDGE PROJECT FINAL EVALUATION BRIDGE invested significant effort in learning about underlying societal values related to what a ―good man‖ does; it found that the stereotype of the Malawian man as one who values sex over relationships and leaves his wife to care for household and children did not hold. During evaluation field discussions, men confirmed that they valued the opportunity to find love and good communication at home above the short-term thrill of a drunken encounter with a sex worker. This information corresponds with findings of organizations that have implemented the popular Stepping Stones programs in other countries. Understanding the real motivations that can move men and women toward healthier and happier behaviors and away from what seem ultimately to be unfulfilling coping mechanisms shows great promise as a basis for designing effective prevention strategies. Additional research is needed to pinpoint the essential elements of these approaches as a basis for identifying indicators that better capture and measure effective prevention strategies. 3. Assess and analyze the likely effectiveness of BRIDGE Project ability to facilitate behavior change via their four pillar strategies: Are individual pillars appropriate and effective? Are BCIs sustainable? Are unifying themes effective? Is project technical assistance state￾of-the-art? BRIDGE found after completing its baseline survey and reviewing other documentation that its initial plan to target high-risk groups was not appropriate and changed its strategy to target the general population. The four pillars of the BRIDGE strategy implement the S-E model that outlines intervention levels: national, community, coordination, and capacity building. This multilevel approach is reasonable and appropriate. It facilitates creation of social norms and interpersonal support for adhering to HIV protective behaviors. The pillars are essential elements of an effective health BCI approach. The evaluators found evidence that dissemination of consistent messages to different groups is creating synergy that is reinforcing intent to adopt HIV protective behaviors. This was particularly evident in the radio listening groups (PLWHA support groups, Tisankhenji girls 10–14, and the PSI Youth Alert clubs) but was also evident in interfaith groups and the district, community, and village committees. There was additional evidence that messages were reinforced through the Radio Dairy programs, the Hope Kit, and the three Nditha campaigns. These activities reach national and community as well as individual levels and involve building capacity for use of community tools such as the Hope Kit and the capacity of radio producers, diarists, and advertising agency personnel. There is evidence that messages have been harmonized among various groups working on HIV/AIDS prevention and control. BRIDGE has provided technical assistance and materials to multiple national and community partners interviewed by the evaluators, among them the PAC (Public Affairs Committee); MANASO (Malawi Network of AIDS Service Organizations); MIAA (Malawi Interfaith AIDS Association); Napham; Manet Plus; NASFAM; and NYCOM (National Youth Council of Malawi). The unifying themes of hope and self-efficacy (Nditha) are effectively reaching target audiences. The quality of the radio broadcasts is high. The HIV/AIDS content of the three Nditha campaigns, Radio Diaries, and the Youth Alert and Tisankhenji programs are reaching audience segments that are critical links to the community with messages of hope and empowerment. Community members cite these programs as important in changing attitudes and behaviors at the community and household levels. Technical capacity building will continue to be needed at radio BRIDGE PROJECT FINAL EVALUATION 11 stations and in the schools and youth groups forming listening clubs. The Radio Diaries programs that feature PLWHA life stories were cited as particularly effective in reducing stigma and discrimination. They have also been effective in encouraging people to seek voluntary counseling and testing (VCT) services. With regard to sustaining BCIs, the midterm evaluation team suggested that BRIDGE highlight a critical pathway that showed the sequence of events necessary to make an impact using individual pillars and unifying themes. The team stated that without a critical path to demonstrate how the four pillars play the reinforcing role envisioned in the conceptual framework, attribution of the strategies would be difficult, as would determining the resources needed to replicate activities. For example, if BRIDGE could quantify and cost out each critical step in developing such programs as the media campaigns or Hope Kit training and dissemination, donors could be approached for funding to replicate them. This critical pathway has not been mapped. It would be useful if the project could do so for key project components before it ends. Although sustainability was not directly addressed in project plans and documents, discussions with staff and partners showed that BRIDGE is working with Malawian implementing and collaborating partners (MANASO, PAC, NYCOM, NASFAM, African Media Portal, Top AD, Galaxy Media, Business Eye, EXP Momentum, etc.) and providing technical assistance to enable them to carry out BCI activities after BRIDGE closes. An example is the dissemination of the Hope Kits and the training being done through MANASO and the Peace Corps. However, partners made it clear that even if many of them can continue the current activities once they have the resources, they would still need technical updates, assistance, and expertise from an organization like BRIDGE. They pointed out that BRIDGE has provided leadership in BCI for prevention that is not available from any other agency. Because it takes time to change behaviors, constant reinforcement of the message is required. The midterm evaluation report in 2005 also pointed out the need for an exit strategy for turning over district and community activities. The final evaluation team was surprised to find that there appeared to be no awareness at all that the BRIDGE project would be ending. Sustainability depends on a successful strategy for turning major program elements over to Malawian partners, continuing to integrate them into NAC and HEU activities, and ensuring continuing technical oversight and quality control of BCIs. Recommendations It would be useful for BRIDGE to document the critical pathway of successful strategies employed in national and community media activities and in other activities. It should specify the resources required for local partners to replicate the activities. A plan is needed for transitioning BCI program elements to local or international partners before the project ends. Some activities can be designed by groups currently part of the BRIDGE program (DACCs, CACs, VACs, and CBOs) and inserted into annual District Implementation Plans (DIPs) to access NAC funding. Only activities included in the DIP can be funded with NAC funds. In principle, a certain percentage of NAC funding is now supposed to be devoted to prevention. BRIDGE should advocate with DACCs to scale up prevention activities started in the project as part of the DIPs. Inclusion in the DIP of programs and the parties responsible would be one indicator of sustainability. 12 BRIDGE PROJECT FINAL EVALUATION Local radio stations need to be reinforced to do their own HIV/AIDS programming. 4. Assess what level of population-based coverage the BRIDGE project achieved with its interventions in target districts. Did the program reach a high proportion of the populations in the areas where it worked and achieve high coverage of various subgroups, such as men, women, youth, and high-risk groups? As discussed under Question 2, the uncertainty of funding and severely restricted district and community budgets made extensive coverage of the population virtually impossible. Nevertheless, the media campaigns and wide dissemination of some BRIDGE project activities, notably the Hope Kit, have had impact beyond specific communities and targeted districts. Though high-risk groups per se were not the focus of the project, in the last years of the program more specific attention has been given to prisoners and pregnant women. However, widespread coverage of both groups is not anticipated before the current funding ends. Demand for many project activities, e.g., Tisenkhenji Listening Clubs, Girls Congresses, Community Dramas, Nditha sports, etc., far exceed the capacity of the project to ever meet the demand. Many community-based structures supported by BRIDGE, e.g., CACs, VACs, CBOs, and youth clubs, have already envisioned ways they can take the message to other communities and to traditional authorities beyond the current limited number in each district. Several CACs expressed great appreciation for the opportunity to participate in exchange visits with a CAC in Mangochi District where they were able to see a model CAC in action and learn how to write proposals to get NAC funding. Recommendation BRIDGE, USAID, NAC, and the GOM should decide which community-based BCI activities should be scaled up and help secure the financial and logistic support needed to do so. Great benefits could be obtained from very modest investments. CBOs have opportunities to apply for funds from NAC through the DACCs, but that is not necessarily true case for youth clubs, groups of traditional leaders, or groups of Agogos (grandmothers). 5. Assess and analyze the effectiveness of the BRIDGE Project to facilitate implementation of the NBCI: BRIDGE’s effectiveness in coordinating BCIs; effectiveness in implementation of community mobilization; effectiveness of capacity-building efforts; and NAC and HEU interest in BRIDGE scale-up despite resource constraints. BRIDGE is widely acknowledged to be the driving force in prevention through BCI in Malawi. The BCI office of NAC, as with most national coordination offices, sets NBCI policy guidelines, and it views BRIDGE as the partner that is often called upon to ―make it happen.‖ The HEU in the past has implemented the health-related components of the NBCI but has recently had to limit its role to promoting HCT and PMTCT. It is widely recognized that the ―C‖ in HCT is very weak, representing significant missed prevention opportunities. Prevention activities implemented by other programs (e.g., Umoyo Networks Post-Test Clubs and PMTCT support groups) have not been scaled up through the HEU. Male circumcision, female condoms, and post-exposure prophylaxis have not yet been given significant attention. The HEU acknowledges the deficits in its program capacity and has not been able to compensate for the loss of its technical assistance consultant in 2006. It is severely understaffed for meeting the BRIDGE PROJECT FINAL EVALUATION 13 significant competing needs from such programs as immunization campaigns and the recently launched Roll Back Malaria campaign. It is generally recognized that without additional help, the HEU will not be able to ramp up additional prevention activities that require MOHP participation. A large amount of capacity assistance is needed to address this deficit. BRIDGE project activities specifically address sexual debut delay, partner reduction, and condom use as well as access to HIV/AIDS and sexual reproductive health services as specified in the NCBI. BRIDGE does this by working at the national level through the BCI Technical Working Group and by providing technical assistance to numerous national, district, and community partners. BRIDGE was heavily involved in developing the NBCI. The BRIDGE project is designed to address barriers raised by the community, support systems, government policies, cultural practices, gender relations, and spirituality/religion. Low self-risk perception, limited condom use, and having multiple sexual partners are barriers identified in the NBCI that are specifically addressed by BRIDGE activities. BRIDGE uses Save the Children‘s Community Action Cycle approach (Figure 2) for community mobilization, which is based on decades of experience in Malawi and globally. Figure 2. The Community Action Cycle Traditional authorities (chiefs) and groups of headmen are the entry points into communities. DACCs, CACs, and VACs were already in place throughout the country, but most of them were not active until BRIDGE started. Each CAC and VAC has BCI and youth subcommittees that correlate with the same committees in the DACC. Other donor programs are working with these same structures, especially in care and support and programs for orphans and vulnerable children, but only BRIDGE focuses on prevention through behavior change. Skills that have 14 BRIDGE PROJECT FINAL EVALUATION been gained in its mobilization to prevent HIV/AIDS are often generic and can be applied to wider social and economic needs. While each activity in itself may not be very expensive (Hope Kits cost less than $50 each and can be made with local materials), community activities tend to be labor-intensive. Although the budget was not sufficient to scale up these activities district-wide, the community vibrancy that BRIDGE has observed and will try to document before the program ends has inspired many groups supported by the project to develop their own scale-up plans. Chiefs have been approached by other chiefs to bring their community mobilization and cultural practice BCI skills to neighboring traditional authority areas. However, they and others currently lack support (primarily for bicycles for transportation) to reach neighboring communities. Opportunities for scaling up to districts that are not currently BRIDGE districts depend on whether mechanisms for collaborating and disseminating the lessons learned in BRIDGE community mobilization can be formalized, funded, and staffed. Malawi has an abundance of US-based private voluntary organizations (PVOs) working in a variety of maternal and child health and reproductive health programs throughout Malawi. They are familiar with the sharing and collaboration necessary to scale up models that have been proven to work. Most of these organizations are skilled in conducting population-based monitoring and impact surveys that could add to the data that is so valued by the Mission and global PEPFAR programs. Recommendations Because understaffing at the MOHP/HEU is a major barrier to moving forward in health￾related prevention activities, USAID should consider providing technical assistance directly to the HEU similar to that currently provided to the ministry HCT and PMTCT programs. Over time USAID should advocate with the GOM to increase HEU staffing and training for HIV/AIDS prevention. Without significant improvements in program capacity and integration with other MOH functions, PMTCT and HCT will languish, and more controversial approaches, such as those related to male circumcision, will have greater difficulty in gaining acceptance. Starting with the DIP development process that begins in October 2008, BRIDGE should make a determined effort to ensure that scaled-up prevention efforts in communities are part of the prevention plans and budget for the coming year. BRIDGE should also provide some sort of written guidance (standard operating procedures, manual, guide, etc.) to leave with the DACCs to help orient new members to BCI and community mobilization as their membership changes. DACCs should be advised to consult the manuals when the DIPs are developed each year. Otherwise, DACC institutional memory will be lost. Including plans to use HIV/AIDS vehicles that have been provided to each district in the DIP may encourage the vehicles to be available, since the plan will be developed well in advance of the activities. Leveraging costs across activities and development sectors will help decrease financial and logistic costs. Many structures that have had their capacity built through umbrellas, BRIDGE, and other activities are already able to plan and implement community mobilization activities. Support for these efforts will encourage other grassroots development initiatives, because the skills gained are transferable. On the other hand, if all funding must come through the cumbersome and time-consuming CBO grant-making process, some of these groups may have their BRIDGE PROJECT FINAL EVALUATION 15 enthusiasm dashed by the unfamiliar bureaucratic processes. It is in the interest of USAID to offer support if transparency and accountability for the dispensing and use of NAC funds should become an issue. Capacity-building support need not necessarily come from organizations with HIV/AIDS technical expertise but might be accessed through either civil society or democracy and governance funding support mechanisms. USAID/Malawi should carefully consider a funding mechanism so that other US-based PVOs can scale up BRIDGE-tested community-based prevention BCI activities in areas of Malawi where SC/US is not currently working. BRIDGE, or a BRIDGE-like project, could provide the technical assistance and leadership to train these organizations in approaches that have worked for them. This is likely to work especially well where prevention could be added when the PVO is already working with DACCs and local NGOs in community OVC or home-based care programs. 6. Assess and analyze the effectiveness and efficiency of the BRIDGE Project’s organizational system (administering grants, providing technical assistance, building capacity, and liaising with NAC). Review staff composition and capacity, project systems and procedures, relationship between JHU/CCP and SC/US. The staff of the BRIDGE project consists of a chief of party, a BCI and a youth coordinator, an M&E officer, a program officer for Girls Congresses and listening clubs, a program officer for BCI tools, a finance manager and assistant, and an administrative manager (along with receptionist and drivers), all of whom are supported by JHU/CCP. SC/US supports a part-time HIV/AIDS specialist, a community mobilization and capacity building coordinator, and seven district coordinators. The project also receives significant guidance from JHU/CCP headquarters in finance, research, and technical assistance. Many of the staff have advanced degrees and have received additional training from the project to meet job requirements. The project holds weekly staff meetings, monthly meetings with district staff, and regular meetings or conference calls with implementing partners to review and plan activities. As a testament to BRIDGE‘s commitment to capacity building, the present COP was promoted from her position as the BCI coordinator. It is noteworthy that the entire project team is Malawian. The organizational structure and staff skills are appropriate for most project activities described in the work plan, but one concern raised during the evaluation is the lack of enough staff at the district level to manage all work plan activities, which include BCI activities like the Hope Kit, follow-up campaign activities, work with radio diarists, mobilization of community and village committees, and oversight of listening groups and school programs. To complete all these tasks, there are only seven district coordinators for eight districts. Moreover, the Youth Alert program, run in collaboration with PSI, only has one coordinator for two districts (50 clubs). The original plan was to implement the project in the districts where the SC/US STEPS program was already at work, and to use the STEPS staff to complement BRIDGE activities. Later the STEPS districts would be paired with neighboring districts to expand the model. However, by the time the project was funded, USAID/Malawi had decided to focus on eight districts, of which only two overlapped with STEPS; thus BRIDGE was not able to build on local experience, and synergies between the projects were lost. At the time of the midterm evaluation, because BRIDGE was only active in four districts, the activities were manageable with eight district staff. Now there are eight districts and only seven staff. The midterm evaluators had recommended 16 BRIDGE PROJECT FINAL EVALUATION that the project either not expand to the full eight districts or acquire resources for additional staff. Apparently, because of the way the project was extended, it was not possible to consider this recommendation. JHU/CCP and SC/US have a good working relationship and function well as a team, with each managing tasks suited to their areas of expertise: SC/US manages district and community mobilization and JHU/CCP oversees BCC technical inputs, national communications activities, and research. JHU/CCP appreciates SC/US‘s longstanding experience mobilizing Malawian communities, and SC/US appreciates the technical BCC expertise and capacity strengthening JHU provides. BRIDGE relies primarily on JHU/CCP policy and procedure documents to guide its activities. A procedures manual outlines administrative and financial protocols. Administrative procedures are consistent with USAID regulations on kinds of vendors, numbers of estimates required, and what can be funded in-country or through headquarters in Baltimore. Other procedures cover approval processes and authority, procurement and purchasing, computer and information systems, and subcontracting. A personnel manual governs hiring and personnel management, policies, benefits such as insurance, workers compensation, and the pension scheme, and employment regulations. The systems and procedures are appropriate for implementing project activities and consistent with USAID requirements. There may be concerns about the efficiency of procedures that often require approval or direct funding from Baltimore, which takes time. The project has to acquire approvals from three structures: USAID, JHU, and the JHU/CCP. The university is known to demand that procedures be strictly followed for funding of activities, which has led to delays in implementation. This was particularly evident in fall 2007 when JHU implemented a new financial accounting system with more stringent procedures, which the project had to adopt in a short period of time. In the second year of the project JHU/CCP administered small grants to CBOs and NGOs to facilitate community mobilization. This process proved to be more cumbersome than anticipated. Using the Umoyo Network model, JHU initiated a process of orienting organizations about proposal writing to familiarize CBOs, FBOs, and NGOs to the JHU/BRIDGE format and requirements. Six seed grants were awarded to youth clubs, CACs, and orphan support organizations in three districts. With the advent of the NAC grants program in year three and the presence of NAC umbrella organizations to manage the grants programs, JHU/CCP phased out the small grants process. The BRIDGE project has awarded subgrants to implementing partners in order to expand their reach in getting BCC messages to target audiences. For example, BRIDGE worked with PSI to add 200 listening groups to the Youth Alert Mix (YAM) program. Other grantees include the PAC, which has received consistent funding for BCI activities conducted through its member organizations. In recent years BRIDGE has provided subgrants to MANASO so that it can provide Hope Kit training to CBOs and NGOs in project districts. Partners continue to have difficulties in producing successful proposals, which delays start-up of activities. Subgrants have also been awarded to agencies supporting media activities, such as Galaxy Media, Top Ad, African Media Portal (AMP), and Business Eye. Some of the grantees have had particular difficulties in meeting JHU‘s strict financial reporting requirements with regard to third-party expenditures. This has delayed budget allocations, as did BRIDGE PROJECT FINAL EVALUATION 17 the new financial system in 2007. Fortunately, the COP was able to obtain permission to advance payments to partners from local funds. While it is understandable that new agencies would have difficulty adjusting to the reporting requirements, it is not entirely clear why some of the agencies that have been working with BRIDGE for a while are still having difficulties. It also seems that the number of small subcontracts to media agencies has been difficult to track. When asked about this the COP indicated that if BRIDGE was to be awarded a new BCI project, it would look for a partner in Blantyre, where most of the media agencies are, that could manage all the media tasks now being done by different agencies. A major focus of BRIDGE‘s activity has been to build local capacity to carry out behavior change initiatives outlined in the project and the national strategy. BRIDGE‘s role has been to connect community and national activities through capacity building. A major focus of technical assistance has been on organizations receiving subawards, which are of strategic importance because they have a large number of members and networks throughout the country. Many of their central staff have received BCI training and fully comprehend the concepts and the national strategy. BRIDGE also provides technical assistance to other Malawian organizations, such as NAC and the MOHP HEU and HIV/AIDS Unit. BRIDGE reports participating in and providing technical assistance at numerous workshops, congresses, working groups, task forces, and other activities. Since the midterm BRIDGE has added a program officer to oversee development of tools and materials, but the process is taking longer than anticipated and it is difficult to efficiently meet all the demands. It appears to be a challenge, given the size of the staff, to balance the technical assistance demands with other project responsibilities. Recommendations Future BCI projects should consider having at least double the staffing at district levels to oversee activities. The project should also consider working in districts that are less dispersed where it can take advantage of synergies between districts. Local agencies applying for subawards seem to need substantial assistance in proposal development and financial reporting. If future programs are to have sizeable subgrant activities, they need to arrange for more technical assistance in these areas. To date BRIDGE has been the main technical expert working in HIV/AIDS BCIs. One of the challenges is balancing demands for technical assistance to other organizations with regular project responsibilities. If the Mission undertakes a similar project in the future, it should make it clear how much priority should be given to technical assistance to nonproject agencies, and staff the project accordingly. 7. Assess whether the BRIDGE Project has facilitated synergy, coordination, and information￾sharing among USAID/Malawi, NGOs, subpartners, other SO 8 partners, other donors; and the Government of Malawi. Is it linking BCIs between these groups and are there opportunities to provide feedback on activities? If so, how has it done this? The BRIDGE project has successfully brought BCI expertise and findings to the attention of other agencies and donors working on HIV/AIDS in Malawi. At the start of the project, it held several national and district meetings, often in coordination with NAC, describing baseline findings on HIV/AIDS behavioral practices. The information was well received and helped to 18 BRIDGE PROJECT FINAL EVALUATION bring the HIV/AIDS community on board with BCI objectives. The midterm survey results were shared at the national level on March 2008 in order to discuss BCI progress and next steps. As part of their efforts to operationalize the BCI strategy and create an enabling environment, project staff have cultivated collegial relationships with other agencies, particularly the BCI Unit at the NAC and the HEU and HIV/AIDS Unit at the MOHP. Although the MOHP/HEU worked closely with BRIDGE in preparing BCIs during the early years of the project, its involvement has lessened. Because the HEU is being asked to respond to a variety of other programs, including malaria and tuberculosis, and has also lost its external technical advisor, it has less time to devote to HIV. Early in the project BRIDGE led efforts to establish coherent and integrated BCIs among organizations by conducting a Message Harmonization Workshop with NGOs, GOM agencies, and district officers. Workshop participants formulated a series of messages for different audience that partners could use in their media and community outreach programs. BRIDGE has continued to offer message development workshops through different collaborating organizations, and it also led efforts to create coordinated core BCI materials and tools for use by partners that draw on baseline and midterm survey results and build on existing best practice materials—it has hired a program officer specifically responsible for this activity. The three versions of the Hope Kit that correspond to the three campaigns are being used successfully by several partners as well as the Peace Corps. Additional materials are posters, leaflets, good guy certificates, scratch cards, and T-shirts. Other coordination functions include BRIDGE‘s efforts to harmonize work plans and collaborate and provide input through technical working groups with NAC, the MOHP, and other CAs. BRIDGE disseminates its experiences through technical working groups and the annual NAC Best Practices Conferences. This year, three of its abstracts were accepted. BRIDGE also participates in PEPFAR‘s annual HIV implementation meetings, making presentations and submitting posters about its BCI activities. BRIDGE has worked with SO 8 partners in a variety of capacities, such as the subaward with PSI to organize and manage 200 listening groups that follow up on PSIs Youth Alert radio program, where HIV/AIDS is discussed together with other prevention activities, such as Nditha sports. BRIDGE and PSI have also collaborated on training for Peace Corps volunteers: BRIDGE gives training on use of Hope Kits and PSI on Youth Alert and Water Guard, all of which are tools volunteers use in the work they do with communities and schools. BRIDGE has also worked with FHI in an NAC-supported effort to create a tool kit that deals with youth behaviors. Together BRIDGE and FHI assessed youth information needs, highlighted messages and tools needed, and did Nditha sports cards with HIV prevention messages. FHI is going to produce the message cards, which are still at NAC, as part of the tool kit. However, to speed use of the cards, BRIDGE has printed them and given them to the youth groups. They have also printed 10,000 copies of the sports newsletter, but are anxious to have the whole tool kit to work with. When the Umoyo Network was operating, BRIDGE worked with it in BCI training and provided message development workshops and Hope Kit training for the partners. The Peace Corps and the I-Life Food Security programs are using BCI tools, especially the Hope Kit. Relationships with reproductive health and safe motherhood are less clear because several other donors are supporting MOHP activities in the BRIDGE districts. Clearly there are linkages between the BCI BRIDGE PROJECT FINAL EVALUATION 19 HIV/AIDS prevention activities in the program that could be continued and strengthened for synergies with other USAID/Malawi and donor programs in a variety of sectors; CAC/VAC/CBO structures can provide foundations for other community health and development programs. Another way that BRIDGE influences BCI policy and links with partners has been through its input to Global Fund applications (Rounds 6 and 7). It advocated for specific activities, such as expansion of the Tisankhenji and Youth Alert radio listening clubs into every traditional authority (5 clubs in each of 300). BRIDGE also pushed BCC education through interactive mass media, such as the EXP roadshows and Nanzikambe training of community theater activists in all 300 traditional authorities. There is an opportunity to make these activities sustainable through Global Fund resources, but so far the activities have not started. BRIDGE also worked with the U.K. Department for International Development in development of a youth campaign. They created radio spots and posters and DFID brought over the Manchester United football team to highlight the events. BRIDGE also played a key role on the publicity committee promoting the MOHP‘s HIV/AIDS National Testing Week, working with UNFPA, UNDP, and NGOs. BRIDGE also provides leadership to a wide range of national and international partners on such BCI-related functions as World AIDS Day and the Candlelight Memorial held this year in Malawi. BRIDGE has done a good job of working at the national level through working groups, task forces, and other networking activities to ensure a coordinated and harmonized BCI approach with consistent messages and with different agencies all promoting the same HIV prevention concepts. BRIDGE has done well in making tools like its Hope Kits available for dissemination through numerous partners. The only problem is that once BRIDGE has given partners the training and materials to replicate, there is no follow-up or quality control because BRIDGE is not overseeing dissemination. BRIDGE has been active in bringing its research and survey results to the attention of other agencies working in HIV/AIDS prevention in Malawi and drawing attention to norms and values affecting behavior. Also, because BRIDGE is addressing behaviors stemming from this system of values, it is showing how they can be changed. Recommendations There is an opportunity to have the Global Fund move forward with some of the BRIDGE listening group and community mass media outreach activities. It is recommended that BRIDGE help the NAC to get this done quickly, and also help organize the programs to make sure that implementation structures are in place before the project closes in 2009. It is recommended that BRIDGE work with partners to set up a system for regular training and distribution of Hope Kits and to monitor and evaluate its effectiveness. USAID Malawi should encourage BRIDGE to present some of the methodologies to other programs supported by the Mission, because many BCI principles are generic enough that they can be adapted to other programs. 20 BRIDGE PROJECT FINAL EVALUATION 8. Assess the costs of the various interventions implemented by the BRIDGE project relative to the number of beneficiaries reached and the estimated impact on behavior change. Based upon the cost in a few districts, does it seem feasible to attempt nationwide scale-up of particular interventions that were successful? Is there enough cost data from the few districts covered to extrapolate to assess the feasibility of national scale-up? If yes, then recommend steps to develop guidance for scale-up. Unfortunately, neither team member is an economist or a specialist in cost-benefit analysis. Although the team did try to acquire data on beneficiaries served in the districts where BRIDGE is currently working in order to calculate the cost per beneficiary, this information was not available during the evaluation period. The project did offer to see what information it could get from district and municipal data systems if it was needed by USAID after the evaluation. However, this would entail more work for the already overburdened district coordinators and would take weeks. It would then require a skilled individual (probably from Baltimore) to calculate and report the cost-benefit ratios. Based on the project budgets and costs submitted from Baltimore and some calculations of activities by district submitted by SC/US, the evaluators were able to ascertain from a preliminary review that project costs are quite reasonable. About half the annual budgets were awarded to subpartners to conduct district and community activities and create mass media programs. The rest of the budget covers labor, consultants, travel, supplies, and equipment and the cost of numerous training workshops. Based on the budget documents supplied, if the project were to scale up activities from the four traditional authorities in each district where they are currently operating, to cover the whole district would at least double the current cost. SC estimates that over the next five years, scale-up would cost $900,000 to $1,100,000 per district (about $200,000 a year). Thus the cost for 28 districts would be roughly $28 million over five years. The project estimates that the cost of production, training of trainers, and follow-up of 50 trainings for the three Hope Kit activities averages $116,000 to $195,957 for the eight districts. For scale-up these figures would need to be multiplied by the total number of districts in the country, and then the costs for Hope Kits ($20– $50 each) would need to be added. Currently other partner organizations are underwriting many of the training and production costs and expanding use of the Hope Kit through their own networks. It may be possible to continue such arrangements. The lack of clear evidence about which behavior change approaches are most effective in preventing HIV/AIDS (see response to Question 5) makes it very difficult to decide which activities should be scaled up. BRIDGE gave the evaluation team estimated costs for some activities. Based on current epidemiologic estimates, those activities directed toward reducing multiple concurrent partners that link newly infected people with multiple sexual networks should be given priority. Male awareness, risk awareness (including alcohol and drugs), and harmful cultural practices are also important to address. A thoughtful district and community scale-up strategy with cost estimates could best be done by BRIDGE in consultation with the DACCs and their field staff. Recommendation BRIDGE should conduct scale-up planning and cost-estimate workshops in its districts and provide the estimates with the end-of-project reports. BRIDGE PROJECT FINAL EVALUATION 21 9. Assess how well project activities are addressing the underlying gender dynamics that influence behaviors in HIV/AIDS. The BRIDGE program gender equity in HIV/AIDS prevention activities could also prove valuable in other gender-oriented health and development programs, especially in reproductive health, safe motherhood, education, food security, civil society, and gender-based violence. Some activities, such as youth clubs, open days, Hope Kit, and Nditha campaigns, are implemented with both genders. These are sometimes further broken down by age and developmental stage, such as young adolescent girls (Girls Congresses, Tisenkenji) or adult men and women (Garage Parties). Some are appropriately directed more toward women, e.g., Have a Healthy Baby PMTCT activities. Some activities directly address gender roles, including the gender packet in the Hope Kit and African Transformations; others are interwoven in the overall activity (Nditha sports, listening clubs). BRIDGE uses multiple communication techniques and follows up with face-to-face discussions. It gives examples of positive role models to raise awareness that, for instance, both men and women can do a wide variety of activities and jobs or engage in domestic activities normally associated with the other sex. The Bambo Wachitsanzo—positive male role models of the Ideal Guy or Great Dad— being presented in national mass media campaigns and reinforced by community activities, such as the Hope Kits, seem to be very effective in shifting male social norms and identifying individual risk, especially when combined with the personal empowerment promoted by Nditha. The Bambo Wachitsanzo has a supportive, mutually satisfying relationship with his wife and children. His characteristics directly address reasons men say contribute to them having multiple partners or being violent with their partners. These factors, which are also addressed in the new risk awareness campaign, include not consuming alcohol to excess, talking and communicating with wives and children, and assisting with household work and child care. Men say that the campaign has given them greater appreciation for the hard work and stress their female partners undergo in the home. Some communities reported that polygamy is being reduced as a result of the campaign. Couples counseling through Garage Parties and group discussions has shown both men and women how to have closer relationships that include fulfilling sexual relationships resulting from better communication and understanding of normal sexual behavior. Garage Parties take their names from the auto repair places in many towns where men often congregate. The name implies that the men are bringing their spouses to the ―men‘s club,‖ including them in discussions that have traditionally excluded women. Church guilds in some districts have been very enthusiastic about conducting these programs. BRIDGE activities like Nditha sports, YAM radio listening clubs, and community dramas present coercive and intergenerational sex in an unfavorable light. The Bambo Wachitsanzo campaign and engaging male influentials, especially traditional and religious leaders, in all aspects of the program have been bold steps in working through community structures to address male norms and behaviors. The evaluation team was struck by the touching testimonials from chiefs, religious leaders, and other influential men about how they had ―learned how hard their partners worked,‖ ―learned they had a right to take care of their children,‖ and learned how they could help to end unacceptable male behavior, such as intergenerational sex with adolescent girls, sex between male teachers and their students, parents who force adolescent daughters into marriage, or domestic abuse. Although conducted in only 22 BRIDGE PROJECT FINAL EVALUATION limited numbers in each district, Garage Parties give married couples guidance on how they can have more supportive and rewarding relationships with each other. Improving communication is the primary emphasis, but frank and open discussion of sexual needs and behaviors was cited by men as explicitly dealing with their needs for ―variety‖ and ―conversation‖ that they cite as reasons they look for sex in the bars. Factual, nonjudgmental adult reproductive health education for couples has motivated men to allow pregnant partners to rest during pregnancy. Reproductive health counseling has also recommended reducing the traditional prolonged postpartum abstinence period (6 months to 2 years) that was another major reason why men would go outside of the household for sex. Participants said that once they knew the waiting period was only six weeks, they could wait that long, thus avoiding a period when very high-risk behavior often took place. Men also cited as extremely helpful the direct talk about the relationships between excess alcohol and marijuana consumption and risky behavior and providing acceptable techniques on how to leave a bar before they reach the point where their judgment is too impaired. One man in Kasungu said, ―I can now say ‗well, I‘ve had enough, now it is time for me to go home to my wife.‘‖ CBO and VAC members say men now carry their wives on the back of their bicycles— something that was not common before the BRIDGE project. Women in every district echoed the sentiments expressed in this case study from the BRIDGE Annual Report: Perhaps the most glowing testimony to the difference the training made comes from Mrs. Phiri herself (Fostina Kanyasko): ―My husband has really changed since he came back from Mangochi District where he attended a training on Bambo Wachitsanzo. Today we walk together to attend church and we also came together to attend this meeting today [a community BW meeting in Mzimba]. After 2-1/2 years, last week we went to the market together to buy household items. How I wish more men were trained to become Bambo Wachitsanzo! Chiefs are the most influential male members of the community. They introduce community￾based activities, and CACs and VACs are organized around the catchment areas of traditional authorities and groups of headmen. The project has found chiefs to be extremely supportive in organizing communities and gaining participation from all sectors. The chiefs expressed sincere gratitude for the help they have received in addressing HIV/AIDS. At the beginning of the program, they were feeling overwhelmed and helpless as large numbers of their community members died, leaving widows and orphans. They are extremely pleased that they can use their positions to take action in preventing HIV/AIDS and helping PLWHAs. They state they use peer pressure when one of their fellow chiefs fails to discourage cultural practices (e.g., hyena, wife inheritance, and sexual cleansing, explained in Annex 3) that have been identified as feeding the epidemic. They feel strongly that the training they received from BRIDGE was invaluable in helping them deal with the consequences of the HIV/AIDS epidemic in their communities. Circumcision is not universal in Malawi, especially among Christians (70% of the population), though it is practiced in the majority of Muslim communities. Male initiators cite training from BRIDGE as the reason they no longer use one razor blade to circumcise several boys and do not encourage boys to ―try out‖ sex after initiation. They have also decreased the age of the circumcision camps to around 6 or 7, when follow-up sexual activity would be less appropriate. Initiation ceremonies are risky for the child who is initiated, even if they are no longer BRIDGE PROJECT FINAL EVALUATION 23 encouraged to engage in sexual behavior. In some areas, initiation celebrations involve significant drinking and dancing and often result in high-risk sexual activity that would be prohibited at other times. Chiefs trained by BRIDGE in Mangochi district took it upon themselves to shift the traditional dances to daytime specifically to discourage high-risk behavior by removing the anonymity provided by darkness. Male circumcision as an HIV/AIDS prevention measure has not yet been adopted into the national BCI strategy. PAC has conducted several Bambo Wachitsanzo trainings for FBO partners. They have also supported open days, community discussions, and Caravans of Faith. The PAC Bambo Wachitsanzo initiative aims to stimulate communities to create a supportive environment for young men to adopt more gender-equitable attitudes and behaviors; strengthen their capacity to plan BCIs in FBOs; critically assess traditional gender norms, roles, and ways in which they treat their sexual partners; acquire knowledge and skills to help men to take an active role in HIV prevention; explore issues of masculinity and how men can use masculinity to positively influence HIV prevention within FBOs; and acquire leadership and facilitation skills so that how they can lead discussions in their religious groups. Responding to the external midterm evaluation recommendation, BRIDGE created activities specifically to teach assertiveness to young adolescent girls (age 10–14) so they can avoid coercive behavior by focusing on the barriers that pregnancy, marriage, and HIV/AIDS could pose to reaching their goals, especially by causing them to drop out of school. Girls Congresses and the Tisenkenji Radio Listening clubs focus more on positive role models and having goals for the future than on sex education. Interviews with young girls reveal that both activities are highly valued, and peers are clamoring to join. They have also generated demand for similar activities for young boys. Girls Congresses in some districts have been conducted either in partnership with or facilitated by the Malawian Girl Guides Association. Tisankhenji‘s ―plucky‖ heroine, Almanafi, and her adventures appear to change the young adolescent girl‘s image from passive to being able to take action to achieve her dreams. Listening Club members who were interviewed said that fewer girls were dropping out of school due to pregnancy, and some girls can resist parental pressure to get married too young. ―When those boys [bicycle taxi drivers] ask us for sex in exchange for taking us home from school we tell them, ‗I have a future and I‘m NOT FOR SALE!‘‖ – Salima Listening Club member. The Girls Congresses invite female role models (teachers, nurses, technical workers) that show the girls alternatives to early marriage and a life over which they have little control. JHU/CCP will build upon the BRIDGE experiences in the activities of the new ―Go Girls‖ initiative that specifically targets girls‘ vulnerability. Grandmother (Agogo) programs, which were also created in response to a midterm evaluation recommendation, have proven very popular where they have been piloted. Elder women are the source of information on sexual development for both adolescents and married couples, even in areas where initiation ceremonies are not very common. They counsel married couples about sex during and after pregnancy, including the consequences of having partners outside the relationship. Although they lack formal education, they are highly respected and influential. Agogos who were interviewed were very enthusiastic about being able to teach young people about ways to avoid HIV/AIDS. 24 BRIDGE PROJECT FINAL EVALUATION Community discussions and dramas also address women‘s legal protection by using a rights￾based perspective and raising awareness of the laws against domestic violence, encouraging reporting to both official and traditional authorities, and confirming that police will take action against offenders if violence is reported. Some communities reported that youth involved in BRIDGE activities have sometimes intervened when there is violence in the family. Female poverty is the major remaining challenge driving behaviors that encourage HIV/AIDS transmission in communities. The need for money was cited as the major barrier to addressing transactional sex and early marriage in prevention activities. In Salima, widows have been linked with income-generating projects, but project partners expressed distress about a household of adolescent sex workers living near the bars in the Boma. Youth in Chikwawa District said that adolescent girls were often the ones who approach older men to enter into sexual relationships in exchange for money and clothes, not the other way around. They said that the girls are sometimes pushed into this by their parents and thought they would stop if they had other ways of getting money. BRIDGE trained a FINCA-affiliated farming organization in use of the Hope Kit. Most of the members are men who belong to agricultural clubs throughout the country. So far Nasfam has distributed kits to and trained over 1,000 clubs. Nasfam also bought them solar-powered radios to listen to BRIDGE-sponsored radio programs. The members have been particularly interested in the Tisankhenji broadcasts that target young girls, stating, ―We need to have our wives and daughters listen to this!‖ VACs and CBOs are engaged in income-generating activities that can teach valuable skills, such as gardening, animal husbandry, and crafts, but currently these activities are not specifically directed toward increasing women‘s incomes. Moreover, discouraging wife inheritance actually reduces a widow‘s resources, especially when she is ―allowed‖ to return to her (usually very poor) family home. When she does so, the husband‘s family retains all the household property, except in Mangochi, where husbands typically move into homes owned by wives. Follow-up investigation of the economic status of widows who are not ―inherited‖ as the social norm shifts away from that practice is needed, and programs directed to sustaining or improving the economic status of widows will probably also be needed. Recommendations Many of the gender-oriented BRIDGE activities showed great promise for translation into other development activities where USAID is mainstreaming a gender perspective, such as reproductive health, gender-based violence education, food security, civil society/democracy, etc. Other donors, such as UNICEF and UNFPA, may be able to support components of some of the successful programs that would be too expensive for HIV/AIDS funding alone. It will require vigilance to keep the focus of certain programs, especially those targeted to a specific demographic sector, e.g., young adolescent girls. There is high demand for Boys Congresses like those for girls. There was a history of Youth Congresses, but the participatory and interactive nature of Girls Congresses is different from the way those activities were organized. BRIDGE‘s approaches to reinforcing adult relationships based on important underlying values seems to have struck a resonant chord in communities in the project areas. The BRIDGE PROJECT FINAL EVALUATION 25 approaches share characteristics with another promising ―B‖ program, Stepping Stones, but are also reinforced by changing community social norms and mass media campaigns. Consistent reports of real behavior change throughout the program districts, along with the unequivocal acceptance of the messages and the changes that go along with them, underscore the need for professional social science and public health investigation to document that the changes have gone beyond changing intentions to actually changing behavior. Because the coverage area and sample sizes are limited, the 2009 DHS may not be sensitive enough to detect and document significant differences that have in fact occurred. Given the dearth of high-quality evidence-based prevention program strategies, USAID PEPFAR programs could benefit from thorough investigation of the impact of the BRIDGE BCI approaches. It has become a truism that poverty is driving much of the global HIV/AIDS epidemic. Many changes promoted for sound epidemiologic reasons, such as elimination of wife inheritance, might be having unintended negative consequences and driving women and children deeper into poverty, as was reported to the evaluation team. Follow-up on possible side effects of desired prevention behavior changes is needed and the results shared in forums of those working on HIV/AIDS prevention in Southern Africa. 26 BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION 27 CONCLUSIONS AND FUTURE DIRECTIONS BRIDGE has successfully implemented its S-E theoretical model in a real-world situation. Project activities have helped both youth and adults to move along the continuum from fatalism to self-efficacy to risk recognition to intention to engage in certain behaviors. Quantitative assessments to measure actual behaviors have been few, but self-reports from communities in the program state that higher percentages of youth are abstaining from sex or are using condoms, teenage pregnancy has decreased, men are spending more time with their families, and the number of their partners has been reduced. Communities also report less domestic violence. However, it was not possible for this evaluation to measure the extent to which these changes have actually occurred. HIV/AIDS prevention is clearly not solely the responsibility of the health system. Much more effort is needed to mainstream it into other sectors, especially education, community development, and social welfare. RECOMMENDATIONS FOR FUTURE DIRECTIONS Many BCI approaches introduced through the BRIDGE project may apply to other USAID programs, especially education, reproductive health, maternal and child health, gender-based violence reduction, and food security. BRIDGE has already guided inclusion of radio listening groups and mass media interactive outreach shows into the Global Fund proposal that was recently funded. The objective is to expand these activities nationwide. Concerns about the transparency of and accountability for NAC funds distributed through District Assemblies might be alleviated with capacity-building directly for DACCs in governance and financial management; this would not necessarily require use of HIV/AIDS program funds because the skills are cross-cutting. Because of the dearth of evidence-based approaches in HIV/AIDS programs, especially in the areas of partner reduction and fidelity (the ―B‖), operations research to document which BCI approaches are most effective in preventing HIV/AIDS transmission is desperately needed. USAID/Malawi should seek funds to verify whether the effects observers believe the BRIDGE BCIs are having have in fact taken place. It appears from current plans that USAID/Malawi intends to focus future prevention activities on adults, as is consistent with HIV prevalence patterns. However, youth still need attention. BRIDGE has made great progress in reaching them and has demonstrated a continuing need for youth programs. It is recommended that before the project ends BRIDGE assures that its youth BCI programming be transferred to an organization that can continue building these activities. 28 BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION 29 ANNEX 1: FIELD SITE VISIT SCHEDULE WORK ITINERARY FOR DISTRICT VISITS—EOPE DATE DISTRICT ACTIVITY WHO? TIME PLACE 8/8/2008 Salima Traveling to Salima Sandra & Jean 0730 Hrs In transit 8/8/2008 Salima Meeting DACC Sandra & Jean 0900-1100 Hrs District Assembly 8/8/2008 Salima Tisankenji Listening Club activities & meeting parents Sandra & Jean 1400-1600 Hrs Boma 9/8/2008 Salima Meeting Khombedza CAC Jean 1000-1200 Hrs Khombedza 9/8/2008 Salima PLWHAs support group activities Sandra 1000-1200 Hrs Khombedza 9/8/2008 Traveling to Lilongwe Sandra & Jean 1400-1600 Hrs Intransit 10/8/2008 Mzimba Traveling to Mzimba Sandra & Jean 1300-1600 Hrs Intransit 11/8/2008 Mzimba Meeting DACC Jean 0900-1100 Hrs Touvirane 11/8/2008 Mzimba PAC men’s group Bambo wachitsanzo activities Sandra 0900-1100 Hrs Kazombe 11/8/2008 Mzimba Meeting district partners Sandra 1400-1600 Hrs Touvirane 11/8/2008 Mzimba Grandmothers meeting Jean 1400-1600 Hrs Manyamula 12/8/2008 Mzimba Meeting Luwerezi CAC (cultural issues discussion) Sandra & Jean 1000 -1200 Hrs Luwerezi 12/8/2008 Traveling to Lilongwe Sandra & Jean 1400-1700 Hrs In transit 13/08/2008 Ntcheu Traveling to Ntcheu Sandra & Jean 0700-0900 Hrs In transit 13/08/2008 Ntcheu Meeting Njolomole CAC Sandra 0900-1100 Hrs Njolomole 13/08/2008 Ntcheu Meeting DACC Jean 0930-1100 Hrs Boma 13/08/2008 Ntcheu Njolomole Youth activities (Interactive drama) Jean 1300-1500 Hrs Njolomole 13/08/2008 Ntcheu Mpepozinai EXP Road Show Sandra 1400-1600 Mphepozinai 13/08/2008 Traveling to Zomba to spend the night Sandra 13/08/2008 Traveling to Mangochi Jean 1600-1800 Hrs Mangochi 14/08/2008 Meeting with initiators Jean 0930-1500hrs Mangochi 14/08/2008 Traveling from Mangochi to Blantyre Jean 1530Hrs Mangochi 30 BRIDGE PROJECT FINAL EVALUATION 14/08/2008 Traveling from Zombs to Blantyre Sandra 0800-1200 Hrs In transit 14/08/2008 Blantyre Meeting with Galaxy & radio producers Sandra 1300-1700 Hrs Galaxy office 15/08/2008 Blantyre Traveling to Mulanje; meeting with DACC Jean 0800-1200Hrs Mulanje 15/08/2008 Traveling to Blantyre; Meeting with MANASO Jean 1230-1700Hrs MANASO office 15/08/2008 Blantyre Meeting with PSI Sandra 0830-1000 Hrs PSI office 15/08/2008 Blantyre Meeting with Nanzikambe Sandra 1030-1200Hrs Nanzikambe office 15/08/2008 Blantyre Meeting with Business Eye Sandra 1330-1500Hrs Business Eye office 15/08/2008 Blantyre Meeting with Top Ad Sandra 1600-1700Hrs Top Ad office 16/08/2008 Blantyre Meeting with AMP Sandra 0900-1000 Hrs AMP office 16/08/2008 Meeting with EXP Sandra 1030-1200 Hrs EXP office 16/08/2008 Traveling back to Lilongwe Sandra 1230Hrs 16/08/2008 Traveling to Mulanje Jean 0700-0900 Hrs In transit 16/08/2008 Mulanje PSI listening club Jean 1400-1600 Hrs Mulanje 18/08/2008 Traveling to Chikwawa Jean 0700-0830 Hrs In transit 18/08/2008 Chikwawa Hope kit activities for prisoners Jean 0900-1100 Hrs Boma 18/08/2008 Chikwawa Meeting with CAC (harmful cultural practices) Jean 1400-1600 Hrs Kakoma 18/08/2008 Traveling back to Blantyre Jean 1700-1800 Hrs In transit 19/08/2008 Traveling to Balaka Jean 0730-0930 Hrs In transit 19/08/2008 Balaka Nditha sports activities Jean 1000-1300 Hrs Balaka 19/08/2008 Traveling to Lilongwe Jean 1400-18000 Hrs In transit 20/08/2008 Traveling to Kasungu Jean 0730-0930 Hrs In transit 20/08/2008 Kasungu Chipili CAC activities Jean 1100-1400 Hrs Chipili 20/08/2008 Traveling to Lilongwe Jean 1400-1700 Hrs In transit BRIDGE PROJECT FINAL EVALUATION 31 ANNEX 2. MALAWI BRIDGE EVALUATION QUESTIONNAIRE COMMUNITY 1. Have you heard any information about how to prevent HIV/AIDS? If yes, how did you hear about it? Can you recall the message? Can you name anything specifically that the messages said that you could do to protect yourself from HIV/AIDS? What about other people? 2. What are you doing differently now as a result of hearing these messages? What are other people doing as a result of these messages? 2 a. Are there things that are keeping you (barriers) from taking those actions? 3. Do you think you can do anything to keep from getting HIV/ AIDS? What about other people? Are there people who would support you (them) in these actions? People who would not approve of your (their) actions? 4. After listening to the messages or other information about HIV/AIDS, did you talk to anyone or did anyone talk to you about the messages? 5. Have you heard about ―listening groups‖? If yes: What (problems?) did you discuss in the listening groups? How can the listening group be more helpful to you? 6. Do you think these prevention messages about HIV/AIDS should continue? If yes: Are there prevention messages you think should be heard that are not currently promoted? 7. Have you heard of the ―BRIDGE‖ Project? If yes, what is the purpose of the project? YOUTH 1. Do you currently believe that you are personally at risk for HIV/AIDS? What actions do you currently take to prevent HIV/AIDS? How did you learn about these actions? 32 BRIDGE PROJECT FINAL EVALUATION 2. Are you currently involved in any group or organization that is undertaking HIV/AIDS prevention activities? (listening groups, Nditha sports) 3. Have you seen any T shirts with messages about ―Big Brother‖ and HIV/AIDS? What were the messages? 4. Do you read the Youth Link Newsletter? If yes: Have you seen the Help Line page and the page where young people living with HIV/AIDS share their experiences? 5. Are there behaviors that you are encouraged to do that you have difficulty implementing? What could help you to do these behaviors? ADULTS 1. Can you name the behaviors to prevent HIV/AIDS that are promoted by the BRIDGE project? 2. Are you doing anything differently since you heard about those behaviors? 3. Do you have suggestions for how to communicate messages about these behaviors better? 4. Have you noticed any changes in your communities since the BRIDGE project came here? 5. Are there barriers to implementing some of the things the project is encouraging you to do? Is there anything that can be done to overcome these barriers? DACCS, CACS, VACS 1. How do you interact with the BRIDGE project? What specific behaviors does the project promote? How are you promoting them within your organization as a result of the BRIDGE project? 2. What is your role in the project (training, recruiting volunteers, promoting condoms, providing grants)? 3. What do you think are the strengths and weaknesses of the project? What is different now from before the BRIDGE project? BRIDGE PROJECT FINAL EVALUATION 33 4. People know about prevention of HIV/AIDS but don‘t change their behavior. Why? How do you think the BRIDGE project is addressing these reasons? 5. How has BRIDGE addressed harmful beliefs or traditions related to HIV/AIDS? How effective has that been? 6. Have you heard any messages about how to prevent HIV/AIDS? What did you think when you heard the messages? Did you take any actions after you heard the messages? Is there anything that the project could do that would help encourage more people to change their behaviors to prevent HIV/AIDS? NGOS, FBOS, MANASO, PAC 1. How does your partnership with the BRIDGE project work? Do you participate in the subgrants program? How is it working for you? How often do you meet? How much did you contribute in designing your activity with the Bridge project? The work plan? The budget? 2. What capacity building have you received through BRIDGE? (Save? Manaso? PAC?) 3. What inputs did you receive from BRIDGE in designing programs? developing work plans? identifying training needs? 4. What do you see as the strengths and weaknesses of the BRIDGE project? 5. Do you have any recommendations for the future? NATIONAL LEVEL STAKEHOLDERS 1. What assumptions do you have about the way to address HIV/AIDS prevention and reduce HIV/AIDS risk? 2. What are the major sources of funding for HIV/AIDS prevention programs in Malawi? 34 BRIDGE PROJECT FINAL EVALUATION 3. How well is the process working in NAC support to the DACCs (update on implementation plan with DACC)? How are resources mobilized from the central to the district level? 4. Is there multisectoral collaboration between donors, agencies, and programs? What is the BRIDGE project‘s role in this, if any? Was funding given to BRIDGE to provide technical assistance to NAC‘s BCI? 5. What is your overall impression of the BRIDGE project? Do you think it is effective? What are its strengths and weaknesses? 6. What is the government already doing in HIV/AIDS media programming? 7. What policy reforms are needed to make the media more effective? Are you doing other HA programming? Does MBC have the capacity to do its own programming? Develop new health programs? Is anyone else supporting radio health programs? BRIDGE PROJECT FINAL EVALUATION 35 ANNEX 3. CULTURAL PRACTICES ADDRESSED IN BRIDGE COMMUNITY-BASED ACTIVITIES (Changes reported relate only to communities where BRIDGE has been working.) Bonus wife: The younger sister of a man‘s wife who is ―given‖ to him as a reward because he is judged to be a good husband. Circumcision: practiced in most Muslim communities but to a much lesser extent in Christian communities. Self-reports of men being circumcised can be misleading because sometimes ―initiation‖ is used interchangeably with ―circumcision.‖ Moreover, circumcision does not necessarily mean complete removal of the foreskin of the penis. Dry sex: Women place plant materials in their vaginas to dry up secretions. Forced marriage of adolescent daughter: Girls can be provided as a wife by parents in exchange for money, property, or animals. The parents force girls as young as 14 or 15 to drop out of school. Tisenkhenji Listening Clubs and Girls Congresses train girls in assertiveness skills and setting goals for their future. There were several reports of girls who have negotiated out of being married off. A patron of the NDITHA sports youth club in Balaka intervened with a girl‘s parents to prevent a forced marriage. Hyena: A surrogate male hired by the husband or wife where it is assumed the husband is infertile. Either the husband (most common) or wife hires another man to have sex with and hopefully impregnate the woman. Communities report good success in significantly diminishing or eliminating the activity, sometimes aided by men who have acted as hyenas because many of them were dying from AIDS. Incest: Father/daughter or uncle/niece sexual relations were reported to have diminished in Chikwawa. BRIDGE CAC reported this practice has been significantly reduced. Initiation ceremonies: Coming of age activities that can include circumcision (boys) or female genital cutting (girls, but rare) along with instruction from elders in hygiene, sexual behavior, respect for elders, and general behavior. In the past, several boys were circumcised with the same instrument. The tradition in many areas was also for youth of both genders to seek an older member of the opposite sex to gain experience after the initiation. In addition, drinking and dancing by the community after initiations have been opportunities for wide-spread casual sex among the celebrants. Working with traditional and religious authorities, adjustments were made in the time and activities for initiation celebrations. Initiation camps take place at younger ages (around age 7 or 8), where follow-up sex is less appropriate. More boys are taken to health centers for circumcision, and traditional initiators are using one razor blade per boy. 36 BRIDGE PROJECT FINAL EVALUATION Post-partum abstinence period: Traditionally 6 months to 2 years, depending on the location. This is a high-risk period because men seek extramarital partners due to the long wait. Communities reported men were going for additional partners less often after they learned that they only needed to abstain for 6 weeks. Polygamy: One man taking more than one wife. Communities reported that relationship-building and improved role models in Bambo Wachitsanzo require a man to spend more time with his wife and children, and having more than one wife makes this difficult. Sexual cleansing: Sexual relations for multiple spiritual reasons, also related to periods of sexual abstinence for certain situations. In the past these relations were not necessarily with one‘s partner. Communities have adapted to keep the cleansing rituals within the family. Spouse exchange: It was reported in Chikwawa and Kasungu Districts that groups of friends or relatives would exchange partners. CACs said the practice was being discontinued due to the danger of transmitting HIV/AIDS. Sugar daddy: intergenerational sex, not common in every district In Salima, the PLWHA support group intervened with an older man and young girl when they saw this happening. They went to the girl‘s parents, who were very grateful. The man was embarrassed and the relationship ended. Transactional sex between traditional healers and female clients: Communities reported it was common practice for traditional healers to request sex in exchange for treatments. Traditional healers were included in many BCI trainings in communities. Traditional authorities and headmen were also trained to forbid these practices in their areas. Wife inheritance: The practice where a widow would be ―inherited‖ by a male relative of the dead husband. Because of the high death rate from HIV and the low percentage of women knowing their status, this causes a high risk for transmission. Refusal to be inherited risked the woman being chased from her home and losing all her possessions and her children. Many communities reported they had discontinued wife inheritance and now ―allow‖ the woman to return to her family‘s home. Most communities were content with the change, and traditional authorities were enforcing the woman‘s right not to be inherited. There were reports, however, that uninherited widows were now poorer than before their husbands died. BRIDGE PROJECT FINAL EVALUATION 37 ANNEX 4. PERSONS CONTACTED AND INTERVIEWED NATIONAL AIDS COMMISSION Bridget Chibwana, Director of Programs Chris Teleka, Communications Officer Maria Mukwala, Community Officer Eriam Kamanga, Information Officer BRIDGE STAFF, LILONGWE Tellina Matabwa, Administrative Manager Meruyn Mwale, Financial Assistant Triza Kakhobwa, BCI Coordinator Muthelisa Luhana, Finance Manager Assana Magombo Program Officer, Go Girls Initiative Josephine Mkandawire, Program Officer Pius Nakoma, Community Mobilization/Capacity Building Coordinator (Save the Children) Tinkhani Chisala, Receptionist Joel Suzi, Program Officer, Youth Lisa Butsama Glory Mkandawire, Chief of Party SAVE THE CHILDREN Jeanne Russel, Deputy Director, Programs Chris Mzambe, Program Manager, ECD/HIV BRIDGE DISTRICT COORDINATORS John Masi, Balaka/Ntcheu Districts Linley Sakhama, Kasungu District Memory Kaleso, Mangochi District Edda Mwamadi, Mulanje District Mary Kumwenda, Mzimba District Mumderanji Zipangani, Chikwawa District Lawrence Chulu, Salima District 38 BRIDGE PROJECT FINAL EVALUATION IMPLEMENTING PARTNERS African Media Portal Randy Martins, Blantyre Business Eye Fred Muphuwa, Blantyre EXP Momentum Allen Mukwenha, Blantyre Galaxy Media Consultants Benson Nkhoma-Somba, Blantyre MANASO Francina Nyirenda –Director; Akuzike Tasowana, BCI Officer Nanzikambe Francis Mmaka, Blantyre Mbhatso Kamanga Haman Kameza Population Services International Ricky Nyale – YAM Cooridnator Judy Ngikha – Mulanje/Chikwawa LC Officer, Blantyre Public Affairs Committee, Lilongwe Robert Phiri, Director Sophia Nthenda Luke Edward Top Advertising Mr. Kalonjeka, Blantyre Mr. Chipiriro Mzanda, Blantyre Malawi Broadcast Corp. Blantyre Vialema, Presidential Producer Radio Islam, Blantyre Rashid Mapila, Station Manager Mcpherson Maiwlana, Producer Transworld Radio, Lilongwe Joseph Kazembe, Producer Jacqueline Nhlema, Producer Bangwe Township - Blantyre Radio Diary Listening Support Group NTCHEU DISTRICT, DACC Pius Myenya, BLM Yonna Kalaundi, COOM Harold Chilikultali, CHC Picaid Chikafa, SOM Godknows Maseko, CHC Mepherson Kapalamula, Concern Universal Maureen Faith Mwakgyoka BRIDGE PROJECT FINAL EVALUATION 39 Racheal Banda, CHC Acksow Chandhca, Ntcheu District Hospital Mike Makalande, Social Welfare Kinglsley Chafaugh, World Vision International Godknows L.T. Maseko, Health Aubrey E. Mgoko, Water Department Marjorie Fundi, YANDO Hassan Jissah, GSCOM Marko Kantawa, CRWB Henry Mugangu, District Assembly M.P. Hchapa, DIAC Marued Kapuzang‘ona, District Assembly Mpepozinai Road Show, EXP NJOLOMOLE, CAC Njolomole Youth Club NDITHA Sports Community Drama SALIMA DISTRICT DACC Executive Committee Members Salima DACC Technical Subcommittees Members BCC Subcommittee Mr. Juhomulawa, Health/MOH Owen Chataifka, Health/MOH Piliran Chagaza, NICE Joseph Friday, Education George Kanyemea, SAMALA Edith Chilomba, Anglican Church Youth Subcommittee Thomas Muyande, Education Madalitso Mpurga, Social Welfare Charles Chitengu, Salima AIDS Support Org, (SASO) Christopher Dicksons Phiri, SASO 40 BRIDGE PROJECT FINAL EVALUATION Yamikani Doctor, Kaone Youth Organization Salima AIDS Support Organization (SASO) SAMALA George Kanyemea, Director Steve Mfune, Director TISENKHENJI RADIO LISTENER’S CLUB Tisenkhenji Radio Listeners Club members Listener‘s club matrons Representative of Salima District School System KHOMBEDZA COMMUNITY AIDS COORDINATING COMMITTEE (CAC) PLWHA SUPPORT GROUP MZIMBA DISTRICT DACC Rose Kayira, Community Mobilization District Coordinator Mishek Fombe, M&E District Coordinator Orpan Thera, Department of Housing MZIMBA DISTRICT PARTNERS Ishmael Ngurube, Mzimba Youth Association George Rungue, Tovirani Thomas, Every Child Andrew Chapusa, Action Aid Iino Tokutro, JICA Hitomi Nada, JICA Florence Chipeta, Chaputa Women‘s Guild Kazomba FBO (PAC Affiliate), Mzikubora TA - Bambo Wachitsanzo Activities Agogo Group Luwerezi CAC BCI, Youth, Resource Mobilization Committee Members NDITHA Sports BRIDGE PROJECT FINAL EVALUATION 41 MANGOCHI DISTRICT Female Initiator‘s Focus Group Discussion Male Initiator‘s Focus Group Discussion Chief‘s Focus Group Discussion Traditional Birth Attendant Focus Group Discussion MULANGE DISTRICT DACC Executive Committee Youth Alert Listing Club CHIKWAWA DISTRICT District Prison HIV/AIDS Peer Education Group Kakoma CAC Discussion on Cultural Practices BALAKA DISTRICT DACC Executive Committee NDITA Sports staff and patrons NDITA Sports Youth members KASUNGU DISTRICT Chipili CAC members PEACE CORPS Victoria McCarthy Edith Mkawa COLLABORATING PARTNERS Robert Ngaiyaye, MIAA Mr. Limbe, MACRO Winfred Maluwa, SWAM, Maya Miko, FPAM Victor Kamanga, Manet Plus Clement Vaumje, World Vision Andrew Naamakhoma, NASFAM 42 BRIDGE PROJECT FINAL EVALUATION BRIDGE PROJECT FINAL EVALUATION 43 ANNEX 5. REFERENCES 1. Government of Malawi, National HIV/AIDS Action Framework 2005–2009, May 2005. 2. __________, National AIDS Commission, National HIV Testing and Counseling Week, Technical Report, November 2007. 3. Jewkes, R. ―Impact of Stepping Stones on Incidence of HIV and HSV-2 and Sexual Behavior in Rural South Africa: Cluster Randomized Controlled Trial,‖ British Medical Journal (BMJ), 337: 2008:a506 4. Johns Hopkins CCP/ BRIDGE Project, Annual Report: October 2006 to September 2007. October 31, 2007. 5. __________, Annual Report: October 2005 to September 2006. October 31, 2007. 6. __________, Annual Report: October 2004 to September 2005. October 31, 2007. 7. __________, Annual Workplan: Year One, 2003. 8. __________, Annual Workplan: Year Three (Optional Year 1), July 1, 2005 – June 30 2006. 9. __________, ―Bambo Wachitsanzo, A Hope Kit Update: Malawi Users Guide,‖ 2007. 10. __________, ―Have a Healthy Baby, A Hope Kit Update: Malawi Users Guide: 2007. 11. __________, Program Description (Optional Year 3&4) 1 October 2007 – 30th June 2009, 2007. 12. __________, Revised Monitoring & Evaluation Plan, January 2005. 13. Malawi Human Rights Commission, Cultural Practices and their Impact on Enjoyment of Human Rights, Particularly Women and Children. 14. Malawi Network of AIDS Service Organizations (MANASO), Report on Hope Kit Assessment, June 2008. 15. National AIDS Council of Zimbabwe, National Behavior Change Strategy for Prevention of Sexual Transmission of HIV 2006–2010. 16. Opoku, J., and J. Williamson, ―Community Action and the Test of Time: A Summary of the Retrospective Study of Community Mobilization and Community Feedback, Africa 2010 Project,‖ Academy for Educational Development and USAID Displaced Children and Orphans Fund (DCOP), February 2008. 44 BRIDGE PROJECT FINAL EVALUATION 17. Poulsen, M., and L. De Aguero, ―Addressing Multiple Concurrent Partnerships and Cross￾generational/Informal Transactional Sex: Programmatic Models for HIV Prevention,‖ Office of the Global AIDS Coordinator, PEPFAR, 2007. 18. Republic of Malawi, ―National Behaviour Change Interventions Strategy (HIV/AIDS/SRH),‖ Draft for Final Review, 18 September 2002. 19. Republic of Malawi, ―Report of the Malawi Triangulation Project: Synthesis of Data on Trends in the National and Local HIV Epidemics and the Reach and Intensity of Prevention Efforts,‖ 2006. 20. Rimal, R., A. Greiner, et al., ―Interpersonal Discussion as an Amplifier of Campaign Effects: Experience from a Micro-credit Loan Program to Promote HIV Prevention and Financial Independence Among Women in Malawi,‖ paper presented to the International Communication Association, Montreal, Canada, May 2008. 21. Rimal, R., A. Creel, et al., ―Applying Social Marketing Principles to Understand the Effects of the Radio Diaries Program in Reducing HIV/AIDS Stigma in Malawi,‖ Health Marketing Quarterly. 22. Gibson, Sarah, ―The Role of Context and Rationalizations in Understanding Behavior in Relation to HIV Prevention in Rural Malawi: A Qualitative Analysis.‖ MPH Capstone Project. May 2006 23. PSI/ Malawi Project TRaC, ―HIV/AIDS. The PSI Dashboard,‖ Malawi March 2006. BRIDGE PROJECT FINAL EVALUATION 45 ANNEX 6. SCOPE OF WORK I. PURPOSE The purpose of this evaluation, planned for August 2008, is twofold: (1) Review, analyze and evaluate the impact of JHU/CCP Bridge Behavior Change intervention against project objectives and USAID/Malawi Strategic Objective 8, and overall contribution to mitigating the impact of HIV/AIDS in Malawi. (2) Provide specific recommendations and lessons learnt for future activities and directions that the Mission may wish to explore in designing future programs. II. BACKGROUND The BRIDGE Project is a partnership between Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (CCP) and Save the Children Federation (SC) designed to conduct high visibility and high impact program activities around HIV prevention. The JHU/CCP Bridge HIV/AIDS prevention program is a vision for the future that aims to change the way Malawians think and speak about HIV/AIDS and more importantly, how they act. Its goal is to bring about mini revolution that is not only possible but essential. JHU/CCP and its partner SC developed a Behavior Change Intervention program that has been engaging Malawians to move from knowledge to preventive actions; assist stakeholders to move from strategy to coordinated implementation; help communities to move from present to more hopeful future. The JHU/CCP Bridge project, funded by the U.S. Agency for International Development (USAID) began in July 2003 and was initially supposed to end in June 2005. JHU/CCP Bridge project enjoyed two extended option years from July 2005 to June 2006 and July 2006 to June 2007. The JHU/CCP award has just been granted an extension to June 2009. The project approach reflects state of the art behavior change models and lessons learned from the Africa region. It is based on the structural environmental theoretical framework that calls for multi-level /dimensional interventions to influence norms and change behavior. The John Hopkins and its partner Save the Children Federation, together bring behavior change leadership spanning years of field experience to achieve transformation. The project contributes to USAID‘s fundamental objective ―Health Behaviors and Services Improved‖ by influencing sexual practices such as increasing abstinence and the ‗ideational‘ factors and psychological variables that are theoretically and empirically documented to influence these behaviors. JHU/CCP focused its intervention on four areas: 1) creating an enabling environment at national level; 2) mobilizing social systems at the district/community levels; 3)reaching individual targets of opportunity including youth and bridge groups; and 4) increasing capacity of government, NGOs, CBOs and FBOs to plan, coordinate and implement effective BCIs. In five years of implementing the project, Bridge maintained its focus and broadened its activities by expanding and building on investments made with modest innovations continued within the parameters of the current project mandate. Based on formative research, and applying the Risk Perception Attitude (RPA) framework, BRIDGE identified as its primary strategy the need to establish both the individual and collective sense that HIV is preventable (self-efficacy). 46 BRIDGE PROJECT FINAL EVALUATION Since the program‘s inception in Malawi, JHU/CCP/Bridge made the following progress: Nditha! National mass media campaign - Over 759 airings of 9 Nditha! (―I Can Do It‖) and Bambo Wachitsanzo (―Great Guy!‖) radio spots (in Chichewa, Yao & Tumbuka); - Over 170,000 people reached through Bambo Wachitsanzo community festivals (with Exp.) in emphasis districts; - 2,000 Nditha Bandanas, 5,000 ballpens, 100 t-shirts and with HIV prevention messages and slogans produced and distributed at outreach activities; - 4, 000 Nditha! posters and 15,000 leaflets reprinted - 20,000 Bambo Wachitsanzo Certificates reprinted and distributed through community events. Involving Youth - Over 600 teachers and community youth leaders trained in managing listening clubs using Tisankhenji and Sara materials for girl‘s empowerment and HIV prevention in 8 emphasis districts; - 200 Youth Alert! listening clubs supported, reaching over 16,000 participants during activity year; - 25,000 youth participated in community-based festivals focusing on life skills, girl‘s empowerment and HIV prevention sponsored by Youth Alert!; - 14, 30-minute programs of new ―Tisankheni‖ radio program for very young adolescents broadcast and linked to communities through AIDS Toto clubs and community youth groups; - 12,000 copies of Nditha! Sparks scenarios printed and distributed to schools and youth groups nationwide; - Over 20,000 youth reached through ―Nditha! Sports‖ activities in 4 BRIDGE districts and open day activities. Community Mobilization and Capacity Building - Over 92,000 participants in 55 community initiated Open Days in eight emphasis districts. - 92 HIV prevention and community mobilization trainings conducted in BRIDGE emphasis districts, building grass-roots HIV prevention skills in over 2560 community influentials. Hope Kit - Finalized development and production of 1600 Hope Kits ―Bambo Wachitsanzo‖ updates; - Over 700 Bambo Wachitsanzo facilitators trained and update kits distributed in 4 emphasis districts, training will continue in additional districts during FY08. III. EVALUATION OBJECTIVES AND ILLUSTRATIVE EVALUATION QUESTIONS The evaluation shall: 1. Assess progress made in implementing the project and achieving yearly targets and estimate if the project achieved its objectives. Review the suitability of the project design and effectiveness of BRIDGE components in helping community and national stakeholders implement National Behaviour Change Interventions Strategy for HIV/AIDS and Sexual Reproductive Health (NBCI). BRIDGE PROJECT FINAL EVALUATION 47 2. Assess whether the BRIDGE Project is meeting its benchmarked activities negotiated in the agreement: Are data gathering methods reasonable for monitoring progress and indicators? If not, why not? Are indicators appropriate and/or valid? If not, why not. 3. Assess and analyze the likely effectiveness of BRIDGE Project ability to facilitate behavior change via their four pillar strategies: Are individual pillars appropriate and effective? Are BCIs sustainable? Are unifying themes effective? Is project technical assistance state-of-the￾art? 4. Assess what level of population-based coverage the BRIDGE project achieved with its interventions in targets districts. Did the program reach a high proportion of the populations in the areas where it worked and achieve high coverage of various subgroups such men, women, youth, and high-risk groups? 5. Assess and analyze the effectiveness of the BRIDGE Project to facilitate implementation of the NBCI: BRIDGE‘s effectiveness in coordinating BCIs; effectiveness in implementation of community mobilization; effectiveness of capacity-building efforts; and NAC and HEU‘s interest in BRIDGE scale-up despite resource constraints. 6. Assess and analyze the effectiveness and efficiency of the BRIDGE Project‘s organizational system (i.e. administering grants, providing technical assistance, building capacity, liaising with NAC, NBCI). Review staff composition and capacity, project systems and procedures, relationship between JHU/CCP and SC/US. 7. Assess whether the BRIDGE Project has facilitated synergy, coordination, and information sharing among: USAID/Malawi team, NGOs, sub-partners, other SO8 partners, other donors and the Government of Malawi. Is it linking BCIs between these groups and are there opportunities to provide feedback on activities? If so, how has it done this? 8. Assess the costs of the various interventions implemented by the BRIDGE project relative to the number of beneficiaries reached and estimated impact on behavior change. Based upon the cost in a few districts, does it seem feasible to attempt nationwide scale-up of particular interventions that were successful? Is there enough cost data from the few districts covered to extrapolate to assess the feasibility to implement national scale up? If yes, then recommend steps to develop guidance for scale-up. 9. Assess how well the project activities are addressing the underlying gender dynamics that influence behaviors related to HIV/AIDS. The following specific areas shall be considered by the evaluation team: What has been the overall impact of JHU/CCP Bridge project? To what extent has institutional capacity building in target districts been achieved? How much is attributable to JHU/CCP/Bridge? How effective was the system to measure progress towards program objectives? Was the quality of data collected and the reporting system up to standard? Was decision-making evidence-based? What is the potential for scale-up or expanding the impact of intervention areas? To what extent can JHU/CCP/Bridge achievements be sustained without USAID/Malawi assistance? How effective was the approach for community mobilization? Is there demand in the community for program activities to continue? How effective was the approach for communication and behavior change? Were there differences in effectiveness in reaching various populations (i.e. youth, adult men, women, high-risk groups)? 48 BRIDGE PROJECT FINAL EVALUATION IV. METHODOLOGY The evaluators should consider a range of possible methods and approaches for collecting and analyzing the information which is required to assess the evaluation objectives. Data collection methodologies will be discussed with, and approved by the USAID at the start of the evaluation. The evaluation must take into consideration the current political/social environment of Malawi, the HIV/AIDS epidemic, the National Action Framework (NAF), decentralization process, program coverage, strategic partnerships, community participation, youth involvement and PLHIV participation in programs. The Evaluation team shall use facilitative methods and activities that will enhance collaboration and dialogue among counterparts particularly NAC and JHU/CCP/Bridge partners. The Evaluation team shall work in collaboration with the CTO for the JHU/CCP Project. The CTO will organize all internal USAID meetings including linking the team with HPN team leader. The Evaluation team shall propose and organize the evaluation process in collaboration with the CTO. The evaluation design and work plan shall be presented to the HPN team members for comments after the Team Planning Meeting. The CTO in collaboration with the JHU/CCP/Bridge Chief of Party will arrange for an initial introductory meeting with appropriate stakeholders at the outset of the process. When appropriate the CTO may participate in meetings with relevant stakeholders and partners. A general list of relevant stakeholders and key partners will be provided to the Evaluation team by the CTO at the time of arrival but the Evaluation Team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so as to develop a comprehensive understanding of the program and services offered through JHU/CCP/Bridge agreement. The final methodology and work plan will be developed as a product of the Team Planning Meeting (TPM) and shared with the Mission prior to application. Document Review USAID/Malawi will provide the evaluation team with the key documents prior to the start of the in country work. All team members will review these documents in preparation for the initial team planning meeting. Prior to conducting field work, the evaluation team will review existing literature and data, including program strategies, quarterly reports, cooperative agreements and modifications and other reports and documents reflecting JHU/CCP Bridge work in Malawi. Team Planning Meeting A two-day team planning meeting will be held in Malawi before the evaluation begins. This meeting will allow USAID to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will: – clarify team members‘ roles and responsibilities, – establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion, – review and develop final evaluation questions – review and finalize the assignment timeline and share with USAID, BRIDGE PROJECT FINAL EVALUATION 49 – develop data collection methods, instruments, tools and guidelines, – review and clarify any logistical and administrative procedures for the assignment, – develop a preliminary draft outline of the team‘s report, and – assign drafting responsibilities for the final report. Internal USAID/Malawi meetings will include, at a minimum: Initial organizational/introductory meeting at which the Evaluation Team will present an outline and explanation of the design of the Evaluation (refer to the TPM noted above); Mid-evaluation review with the HPN Team leader and CTO to outline progress and implementation problems; and Final Evaluation debrief - summary of the data, draft recommendations and draft report. Field visits/Key Informant Interviews: JHU/CCP/Bridge activities are focused in eight target districts as well as at a national level. The Evaluation Team shall arrange to visit selected sites in consultation with the CTO and the JHU Chief of Party. The evaluation team shall arrange to visit selected sites in each of the eight districts (the exact sites to be selected during the Team Planning Meeting (TPM) depending on consultant time and resource availability) supported through JHU/CCP/Bridge to cover the three regions of the country (North, Central and South). The Evaluation Team may be accompanied by a member of staff from USAID/Malawi, as appropriate. The site visits will involve interviews with NAC, partners, PLHIV, Youth and Communities. Key informant interviews will be conducted in each region. The Evaluation Team will conduct interviews with donor organizations, selected NGOs, and other key respondents identified during the planning meeting. List of Documents: The team will gather information and resources/tools that have been developed by the partners and provide the mission with a small clearinghouse of resources related to HIV/AIDS prevention in Malawi. Wrap up and debriefing: At the conclusion of the field visits, there will be a debrief meeting at USAID/Malawi. The purpose of the meeting will be to share findings and get final inputs before preparing the report. V. TEAM COMPOSITION Team Leader/ HIV/AIDS Technical Advisor: Should have an MPH or related post graduate degree in health or any applicable social sciences field. S/he should have at least 5 years senior level experience working in health systems programs in a developing country. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required. The Team Leader should also have experience in leading evaluation teams and preparing high quality documents. This specialist should have wide experience in implementation of HIV prevention programs in Africa at various levels. 50 BRIDGE PROJECT FINAL EVALUATION The Team Leader will take specific responsibility for assessing and analyzing the organization‘s progress towards quantitative targets, factors for such performance, benefits/impact of the strategies, and compare with other possible options. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the USAID/Malawi team and other major partners. HIV/AIDS and Behavior Change Communication Specialist: This specialist should have wide experience in implementation of HIV prevention programs in Africa at various levels S/he will need to analyze the program within the context of the NAF and the national BCC strategy their implementation. The BCC Specialist should have substantial experience in Behavior Change Communication. S/he will analyze the behavior change interventions in Malawi in general and BCC intervention by Bridge. The team member should have a post graduate degree in Health Promotion Sciences or related field with a minimum of five years experience working with USAID-supported Behavior Change programs in developing countries. VI. TIMELINE AND LOE USAID/Malawi anticipates that the period of performance of this assessment will be 28 days. This would include preparation days, in-country work in Lilongwe and the regions, and report writing and finalization. The evaluation will begin in August 2008 and will complete its in￾country work within a month of arriving. The following is a sample timeline. Task/Deliverable Duration 1. Review background documents & offshore preparation work. 3 days 2. Travel to Malawi 2 days 3. Team Planning Meeting and meeting with USAID/Malawi SO 8 team 2 days 4. Information and data collection. Includes interviews with key informants (including partners and USAID staff) and site visits. 16 days 5. Discussion, analysis and draft evaluation report in country 4 days 6. Debrief meetings with SO 8 team and key stakeholders (preliminary draft report due to mission) 1 day 7. Depart Malawi 2 days 8. Continued writing of draft report (out of country) 10 days 9. USAID & partners provide comments on draft report (out of country) – 5 days 10. Team reviews and revises report – final 5 days BRIDGE PROJECT FINAL EVALUATION 51 Total Estimated LOE = 45 days (Team Leader; 35 days team member) VII. LOGISTICAL SUPPORT A six day work week is authorized when working in country. The evaluation team will be responsible for all off-shore and in-country logistical support. This includes arranging and scheduling meetings (with exception to previously mentioned meetings and initial introductory meetings and field trips), international and in-country travel, hotel bookings, working/office space, computers, printing and photocopying. A local administrative assistant/secretary may be hired to arrange field visits, local travel, hotel and appointments with stakeholders. In addition, the Evaluation Team Leader is responsible for draft and final report development. VIII. DELIVERABLES 1. Work Plan: The contractor will submit a detailed written work plan before end of week one of work. 2. Methodology Plan: A written methodology plan (evaluation design/operational work plan) will be prepared during the Team Planning Meeting and discussed with USAID prior to implementation. 3. Debriefing with USAID and partners: The team will debrief with USAID and partners prior to submission of the draft report and the team‘s departure from country. The team will consider USAID and stakeholder comments and revise the draft report accordingly, as appropriate. After the debrief meeting, the evaluation team shall incorporate oral comments received from USAID and stakeholders 4. Draft evaluation report should be completed prior to the Team Leader‘s departure from Malawi. The written report should clearly describe findings, conclusions and recommendations (using the report format provided in ―IX. Reporting Requirements‖ below). USAID will provide comment on the draft report within 5 working days of submission. 5. A final report that incorporates the team responses to Mission comments and suggestions. The draft final report should be completed within 5 days after USAID provides its feedback on the draft report incorporating the comments received from the review of the draft and sent to the Mission. The final report (excluding executive summary and annexes including List of citations: List of all reviewed/cited sources in final report) should be no more than 30 pages. After the final but unedited draft report has been reviewed by USAID, GH Tech will have the documents edited and formatted, and will provide the final report to USAID/Malawi for distribution (2 hard copies and a CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. IX. REPORTING REQUIREMENTS The findings from the evaluation will be presented in a draft report at a full briefing with USAID/Malawi and possibly at a follow-up meeting with key stakeholders. The format for the evaluation report is as follows: Executive Summary –state the most salient findings & recommendations (2 pp); Table of Contents (1 pp); Introduction – purpose, audience, and synopsis of task (1 pp); 52 BRIDGE PROJECT FINAL EVALUATION Background – brief overview of MSH project in Malawi, USAID program strategy and activities implemented in response to the problem, brief description of MSH, purpose of the evaluation (2-3 pp); Methodology – describe evaluation methods, including constraints and gaps (1 pp); Findings/Conclusions/Recommendations – for each objective area; and also include data quality and reporting system that should present verification of spot checks, issues and outcome (17-20 pp); Issues – provide a list of key technical and/or administrative, if any (1-2 pp); Future Directions (2-3 pp); References (including bibliographical documentation, meetings, interviews and focus group discussions); Annexes – useful for covering evaluation methods, schedules, interview lists and tables – should be succinct, pertinent and readable. The final version of the evaluation report will be submitted to USAID/Malawi in hard copy as well as electronically. The report format should be restricted to Microsoft products and 12-point type font should be used throughout the body of the report, with page margins 1‖ top/bottom and left/right. The report shall not exceed 30 pages, excluding references and annexes. X. OVERSIGHT AND MANAGEMENT The GH Tech team will work under the direction of USAID/Malawi HIV/AIDS Lead Matthew Barnhart. Responsibilities: USAID/Malawi and/or JHU/CCP will be responsible for the following: Obtain country clearances for travel. Coordinate and facilitate initial assessment-related field trips, interviews, and meetings. GH Tech will be responsible for the following technical and logistical support: Identify and recruit team members – international and local consultants. Provide funds to the team for all in-country logistics. Provide administrative, operations and logistical support to the team while on assignment Provide support and editing services for the preparation of the final versions of the deliverables For more information, please visit http://www.ghtechproject.com/resources/ Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com