USAID/NAMIBIA: NAWA LIFE TRUST COOPERATIVE AGREEMENT END OF PROJECT EVALUATION December 2010 This publication was produced for review by the United States Agency for International Development. It was prepared by Ruth Bessinger and Willis Odek through the Global Health Technical Assistance Project. USAID/NAMIBIA: NAWA LIFE TRUST COOPERATIVE AGREEMENT END OF PROJECT EVALUATION DISCLAIMER The views of the authors expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 10-01-425) is available in printed or online versions. Online documents can be located in the GH Tech web site library at http://resources.ghtechproject.net/. Documents are also made available through the Development Experience Clearing House (http://dec.usaid.gov/). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION i CONTENTS ACRONYMS..................................................................................................................... III EXECUTIVE SUMMARY..................................................................................................V Introduction....................................................................................................................................................................v Key Findings....................................................................................................................................................................v Recommendations.......................................................................................................................................................vi I. BACKGROUND AND INTRODUCTION .............................................................. 1 The HIV Epidemic in Namibia................................................................................................................................... 1 The NAWA Life Trust Project................................................................................................................................. 1 Overview of HIV Communication Interventions 2007-2010............................................................................ 2 Purpose and Objectives of the Evaluation ............................................................................................................. 4 II. CAPACITY BUILDING ............................................................................................. 7 National Level............................................................................................................................................................... 7 Community Level......................................................................................................................................................... 8 III. STANDARDIZATION AND SCALE-UP OF BCC MODELS ............................. 11 Developing Messages and Campaigns...................................................................................................................11 Populations Reached by BCC .................................................................................................................................13 IV. RESEARCH, MONITORING AND EVALUATION ............................................. 17 Applied Research .......................................................................................................................................................17 Monitoring and Evaluation .......................................................................................................................................17 V. LESSONS LEARNED AND RECOMMENDATIONS........................................... 21 Achievements, Challenges, and Lessons Learned...............................................................................................21 Recommendations for Moving Forward...............................................................................................................23 ANNEX A. SCOPE OF WORK............................................................................................ 27 ANNEX B. PERSONS CONTACTED................................................................................. 33 ANNEX C. LIST OF DOCUMENTS REVIEWED.............................................................. 37 ANNEX D. KEY INFORMANT INTERVIEW GUIDES...................................................... 39 ANNEX E. LIST OF REFERENCES.................................................................................... 45 FIGURES Figure 1. Pathways Conceptual Framework for the NawaLife Program............................................................18 ii NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION iii ACRONYMS ART Antiretroviral therapy BCC Behavior change communication CAA Catholic AIDS Action CACOC Constituency AIDS Coordinating Committee CAF Community Action Forums CBOS Central Bureau of Statistics CORD Coalition of Responsible Drinking CT Counseling and Testing for HIV DAC District AIDS Coordinator GFATM Global Fund for AIDS, Tuberculosis, and Malaria IEC Information, Education, and Communications IPC Interpersonal communication JHU Johns Hopkins University JHU-CCP Johns Hopkins Center for Communication Programs JHU-HCP Johns Hopkins Health Communications Partnerships M&E Monitoring and evaluation MCPs Multiple concurrent partnerships MICT Ministry of Information and Communication Technology MoHSS Ministry of Health and Social Services NGO Non-governmental organization NLT Nawa Life Trust NPC National Planning Commission NSF National Strategic Framework OVC Orphans and vulnerable children PEPFAR President’s Emergency Plan for AIDS Relief PLHIV People living with HIV and AIDS PMP Performance monitoring plan PMTCT Prevention of mother-to-child transmission RACOC Regional AIDS Coordinating Committee SFH Society for Family Health TB Tuberculosis UN United Nations USAID United States Agency for International Development iv NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION v EXECUTIVE SUMMARY INTRODUCTION In October, 2007, Nawa Life Trust (NLT) was awarded a three-year cooperative agreement from the United States Agency for International Development (USAID). This cooperative agreement called for NLT to conduct three activities: (1) design and implement mass media, community-based, and interpersonal behavior change communication (IPC BCC) interventions; (2) build the capacity of community-based and faith-based non-governmental organizations (NGOs) and Government of Namibia (the Government) Ministries in behavior change communication through the provision of technical support; and (3) conduct applied research to guide program planners, implementers and evaluators about health communications interventions. This evaluation sought to assist USAID to determine the performance and effectiveness of the Nawa Life Trust cooperative agreement, as well as to inform the implementation of USAID’s future competitive award for a follow-on project for national-level communication and community-level prevention activities. The objectives of the follow-on project are to (a) strengthen HIV communication in public and private institutions; (b) implement a package of IPC BCC interventions; and (c) strengthen regional capacity to coordinate and implement HIV-prevention interventions. Data collection for the evaluation was conducted from October 17 to November 3, 2010 through key informant interviews with community volunteers and staff at NGOs, local and national government offices, international agencies, and other organizations working in HIV prevention, treatment, care and support at national and local levels. KEY FINDINGS Capacity Building During this project phase, NLT successfully managed its transition from a field office of the Johns Hopkins Center for Communication Programs (JHU-CCP) to a full-fledged Namibian organization. The organization hired additional staff and conducted a variety of trainings to orient them to the new organization, developed management systems and, in 2009, conducted a strategic planning exercise to align with its new status. At the community level, NLT trained regional project coordinators to provide supportive supervision to community volunteers organized in groups called Community Action Forums (CAFs). NLT also directly trained the CAFs to strengthen their capacity. All together, these efforts helped position NLT to contribute to capacity building for BCC strategy development at the national level and to strengthen implementation at the community level. Both governmental and non￾governmental organizations involved in HIV response highly regard NLT’s expertise in developing cutting-edge and effective BCC campaigns. At the national level, NLT has made substantial contributions to the Take Control initiative through its work with the Ministry of Communication and Information Technology (MICT). NLT effectively served in the coordination role within the Take Control Committee for HIV communication efforts through various technical working groups, such as the prevention technical advisory committee and male circumcision task force, and is noted to have pushed the agenda for, and rallied stakeholders around, different HIV communication campaigns while working with some key Government ministries such as MICT and MoHSS to help prepare them and address the challenges these ministries faced. vi NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION The major challenges for NLT’s HIV communication capacity-building efforts were an excessive demand from the MICT for hands-on coordination and, at the community level, low numbers of dedicated volunteers needed for increased HIV IPC BCC interventions. Standardization and Scale-up of BCC Models Informants for this evaluation generally regarded NLT as the lead organization on mass media for BCC in Namibia. From 2007 to the present, NLT has been a key contributor in identifying appropriate messages and designing the national HIV communication campaigns, Be There to Care; Break the Chain; Be Strong, Get Tested; and Stand Up Against Alcohol Abuse. By encouraging the participation of different stakeholders, including grassroots communities, in HIV communication materials development, NLT has been able to achieve buy-in and national ownership of HIV communication campaigns and IPC BCC tools. Partners acknowledge that the consensus reached on recent campaigns such as ―Break the Chain‖ and ―Stand Up Against Alcohol Abuse‖ are largely due to an appropriate framework emphasizing participation by as many stakeholders as possible and to multi-layered and multi-channel messaging applied by NLT in developing HIV communication strategy and materials. The scale-up of IPC BCC interventions, however, still faces some challenges. While different partners are using a standard set of tools, it is likely that the quality of implementation may vary across partners because of different levels of training and experience. In addition, existing tools are not available in local languages, a problem particularly pertinent to those conducting outreach in more rural communities. The reach to rural communities also remains limited. Research, Monitoring and Evaluation NLT has made excellent use of formative research to design and fine-tune mass media campaigns and information, education and communication (IEC) messages. Their campaigns and materials are widely perceived as being professional, of the highest quality, and on target in terms of their messaging. This can partly be attributed to NLT’s methodical approach of using research to guide the development and roll￾out of HIV BCC. Research carried out by NLT has provided strategic information for the identification of key drivers of the HIV epidemic in Namibia. Many partners acknowledge the importance of this research, having used some of the research findings in designing their own activities. NLT lacks a results-based monitoring and evaluation (M&E) plan to enable tracking of progress towards meeting its objectives and to showcase its achievements beyond the limited set of the President’s Emergency Plan for AIDS Relief (PEPFAR) indicators. While there has been a huge investment in community-level BCC efforts, there are no data to assess either the coverage of these efforts in terms of reaching at-risk populations or their effectiveness at the community level. A performance monitoring plan that is results oriented; includes a logical framework, a minimum set of indicators, data collection and use plans; and can be implemented with the available resources and organizational capacity will be critical moving forward. RECOMMENDATIONS Strengthening HIV Communication Continue building capacity of MICT to coordinate national HIV communication interventions NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION vii This objective stands to gain from the hiring of an HIV/AIDS Resource person within the MICT, whose funding request the ministry is reported to have made to the GFATM. However, because technical positions within Government ministries conduct inherently governmental work, USAID’s position was for such positions to be funded by the Government of the Republic of Namibia rather than by donors. Clarification of roles between MICT and the follow-on project in as far as technical assistance is concerned is also necessary to reduce the MICT’s dependency on external hands-on support. Continue with technical assistance to the national BCC strategy development and implementation Work from the previous project has confirmed that by following an appropriate framework, it is possible to achieve consensus on and ownership of HIV IPC BCC messages and tools at both national and regional levels. The follow-on project should build on this momentum to continue providing leadership in developing and implementing national BCC strategies. Strengthen linkages with workplaces (private sector and government) in BCC strategy development and implementation There has been limited incorporation of HIV wellness programs within both the public and private sectors in terms of both utilizing data available and implementing or adapting BCC interventions within these settings in the previous project. Greater linkages with workplaces should, therefore, be explored and strengthened. Implementation of a Package of IPC BCC Interventions Support a training organization to standardize training on IPC BCC tools for organizations involved in various public health interventions to facilitate scale-up A number of organizations support community volunteer programs on a variety of public health issues. To reduce the challenging demand upon one organization to scale up IPC BCC in communities, training should be provided to a variety of community volunteers to transform them into resources for the communities they serve. Expand reach of IPC interventions to underserved areas The follow-on project should prioritize sites based upon evidence on the drivers of the epidemic and consider growth into peri-urban and rural communities that have not been reached. Strengthening Regional Capacity to Coordinate and Implement Build the capacity of HIV/AIDS focal persons in HIV communication The National Strategic Framework proposes creation of the position of HIV/AIDS focal point persons within line ministries and at regional levels. The follow-on project, once operational, should work within this structure to build the capacity of these focal persons on HIV communication, including existing and new mass media, IPC materials and tools. Strengthen capacity of regional AIDS coordination mechanisms for community-level BCC implementation The follow-on project could work to build the capacity of the HIV focal persons to develop operational plans for the implementation of BCC programs within the regions. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 1 I. BACKGROUND AND INTRODUCTION THE HIV EPIDEMIC IN NAMIBIA Namibia is one of the wealthiest countries in Africa, due largely to its mining industry (NPC 2006). While classified by the World Bank as an upper middle-income country, Namibia also has the most unequal distribution of resources in the world, with more than one in four households living in poverty (CBOS, 2008). Its population is young—with 40% under the age of 15—and diverse—with over 10 ethnic groups (NPC, 2006). Two-thirds of the population is rural, and the majority relies on subsistence agriculture or herding. The country also has a high unemployment rate, estimated at 51% in 2008 (CBOS, 2008). These are important underlying factors that may contribute to an HIV/AIDS epidemic and shape the national response to it. Namibia has a generalized and mature epidemic, with HIV primarily transmitted through heterosexual sex. It is estimated that the HIV prevalence in the general population aged 15 to 49 years was 13.3% in 2008-09 and that 174,000 adults and children were living with HIV in 2009. Available data from the National HIV Sentinel Surveys indicate that HIV prevalence among pregnant women attending ante-natal clinics rapidly increased from 4% in 1992 to 18% in 2008. HIV prevalence among pregnant women reached a peak of 22.3% in 2002 before it started showing some signs of stabilization and a statistically significant decline to the current percentage of 17.8% (NSF, 2010/2011). According to the recently developed 2010/11 National Strategic Framework for HIV/AIDS, Namibia has identified the following as being among the key drivers of the HIV epidemic:  Multiple and concurrent partnerships  Inter-generational sex  Early sexual debut  Alcohol abuse  Mobility and migration  Transactional sex  Low and inconsistent use of condoms  Low rate of male circumcision  Low levels of HIV counseling and testing  Gender and income inequality THE NAWA LIFE TRUST PROJECT In October, 2007, Nawa Life Trust (NLT) was awarded a three-year cooperative agreement from the United States Agency for International Development (USAID). This cooperative agreement called for NLT to conduct activities in three areas: 1. Design and implement mass media, community-based, and interpersonal behavior change communication interventions; 2. Build the capacity of community-based and faith-based non-governmental organizations (NGOs) and Government of Namibia ministries in behavior change communication through the provision of technical support; and 2 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 3. Conduct applied research to guide program planners, implementers and evaluators about health communications interventions. This cooperative agreement was a follow-on to a program implemented by the former Namibian Field Office of Johns Hopkins University Health Communication Partnership (JHU HCP) from April 2003 to 2007. In 2006, the Field Office registered itself as a Namibian NGO, with the vision of improving the quality of life for all people in Namibia through innovative behavioral change communication (BCC). Since then, NLT has made significant progress in the area of organizational development and has established itself as an independent and fully functioning Namibian organization. The objectives1 for the NLT cooperative agreement awarded in 2007 were as follows:  Strengthen the capacity of Namibian organizations to design and implement health communication programs.  Strengthen the capacity of the Ministry of Information and Communication Technology (MICT) and other institutions to develop strategic plans for innovative HIV/AIDS communications.  Standardize and scale up proven HIV/AIDS and tuberculosis (TB) communication models.  Use an effective and appropriate mix of electronic and non-electronic media and community-based and interpersonal communications techniques to reach communities and most-at-risk audiences.  Expand accurate messages and portrayals about HIV and AIDS to address stigma.  Conduct applied research and routine monitoring and evaluation (M&E) of communications activities to support program learning to guide future communications interventions. OVERVIEW OF HIV COMMUNICATION INTERVENTIONS 2007-2010 Over the project life, NLT undertook HIV communication interventions at both the national and community level. National-level Activities At the national level, NLT contributed to HIV communication efforts through the umbrella ―Take Control‖ HIV/AIDS media campaign. ―Take Control‖ is a Government-led initiative, comprising NGOs, faith-based organizations, United Nations (UN) agencies, line ministries, and development partners housed at MICT. Through this forum, NLT spearheaded mass media HIV communication campaigns under the following themes: Be There to Care (2007) ―Be There to Care‖ was a mass media campaign with a focus on family support and disclosure of HIV status. Break the Chain (2009) The ―Break the Chain‖ campaign, launched in the last quarter of 2009, focused on the concept of sexual networks and HIV risks through multiple concurrent partnerships (MCPs). Using multi-media (mass media, information, education and communication [IEC] materials and interpersonal communication) and multi-level (national and community) approaches, the campaign aimed to raise awareness of HIV risks 1 As paraphrased in the Scope of Work for this evaluation (see Annex A). NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 3 within sexual networks and to promote strategies to reduce those risks within such networks. Messages focused on HIV prevention self-efficacy by breaking away from networks through partner reduction and consistent condom use. Be Strong, Get Tested (2010) Under the stewardship of the Ministry of Health and Social Services (MoHSS), NLT provided technical support for the development and implementation of a multi-media ―Be Strong, Get Tested‖ generic campaign in the first quarter of 2010. The campaign encouraged knowledge of HIV status as a basis for behavior change. National testing days were used as an important avenue for rolling out this campaign. This campaign also reinforced NLT’s work carried over from the previous project period to create demand for HIV counseling and testing at New Start HIV counseling and testing centers. Stand Up Against Alcohol Abuse (2010) The Take Control Committee identified alcohol abuse as a driver of the Namibia HIV epidemic and, in 2006, agreed to develop the ―Alcohol AIDS/HIV Campaign,‖ which focused on awareness communication. The campaign was linked to the Coalition for Responsible Drinking (CORD) initiative led by the MoHSS. Together with CORD and other partners, NLT launched a follow-up mass media and interpersonal communication campaign in 2010, themed Stand Up Against Alcohol Abuse to reinforce norms of low-risk drinking and encourage greater corporate social responsibility among key stakeholders in the alcohol industry. Male Circumcision (2010) Through the Namibian Male Circumcision Task Force, NLT in 2010 assisted with the development of a communication strategy and an initial informational pack for use by clients, healthcare workers, gatekeepers at the community level, decision makers and media workers on male circumcision as an HIV prevention strategy. The MoHSS’s approval of these communication materials and their launch was pending at the time of this evaluation. Community-level Activities At the community level, NLT implements three outreach and behavior change communication (BCC) activities. NawaOutreach NawaOutreach activities were carried over from NLT’s previous project phase. As originally implemented, NawaOutreach involved the use of community volunteers organized in groups called Community Action Forums (CAFs) to create demand for HIV services such as counseling and testing and anti-retroviral therapy (ART). In early 2010, a program shift was made from community mobilization and IEC to community-oriented BCC through interpersonal communication approaches. Tools such as a Flannelogram, interactive videos, and picture codes are being used by NLT to support the BCC activities of CAF members. NawaCinema NawaCinema was designed to generate participatory discussions on issues relating to HIV/AIDS prevention, care and treatment in a socially comfortable and entertaining way. Meant for an older audience but open to young people as well, NawaCinema utilizes existing videos to address sensitive topics surrounding HIV and AIDS. Standardized BCC outreach programs have been developed to ensure the appropriate dosage of different messages for targeted groups. For example, BCC outreach using the 4 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Three and a Half Lives of Philip Wetu video requires four contacts with each targeted group within one month. NawaSport NawaSport is a behavior change intervention created to engage young men 15 to 35 years of age, a group noted to be underrepresented in HIV/AIDS prevention and care interventions in Namibia. The program taps into the excitement of soccer to capture the attention of young people in an environment where they feel comfortable exploring serious and sensitive issues. In this program, community members are identified and trained in the NawaSport Coaches Guide. The trained coaches pass on the messages to players and other young men from the communities. The NawaSport Coaches guide was updated in the project period under review to incorporate evidence on key drivers of the HIV epidemic in Namibia and now consists of 14 BCC program sessions. PURPOSE AND OBJECTIVES OF THE EVALUATION This evaluation sought to assist USAID to determine the performance and effectiveness of the NLT cooperative agreement, as well as to inform the implementation of USAID’s future competitive award for a follow-on project for national-level communication and community-level prevention activities. This follow-on project will address three objectives2: (1) strengthen public and private Namibian communication institutions for public health interventions; (2) implement a package of interpersonal and community-oriented BCC interventions for HIV prevention; and (3) support regional capacity to implement HIV prevention interventions. The specific objectives of the evaluation were to:  Assess why progress toward the agreement’s planned results has been positive or negative, and identify and analyze unintended consequences and effects of assistance activities.  Assess the extent to which NLT was able to build capacity for the Government of Namibia ministries and community organizations.  Assess the extent to which NLT monitored and evaluated the outcomes and impact of the range of communication activities supported under the project.  Assess to what extent NLT used a strategic and sustainable mix of communication channels to reach diverse populations of persons at risk.  Assess the implications of making a transition from JHU to NLT for project implementation, management and financial savings.  Identify lessons learned and make recommendations to guide the implementation of the follow-on project. 2 As indicated in the Scope of Work for this evaluation (see Annex A). NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 5 The evaluation was conducted from October 18 to November 4 by a two-person team of international consultants. The Team brought expertise in HIV BCC, applied research, and M&E. After briefings with the USAID HIV Prevention Team and senior management of NLT, the Team conducted a series of interviews with community volunteers, as well as with staff at NGOs, local and national government offices, international agencies, and others working in HIV prevention, treatment, care and support at national and local levels. The Team also interviewed NLT staff. In addition to conducting interviews with national-level informants in Windhoek, the Team visited Omaruru, Oniipa, Oshakati, Oshikuku, Otjiwarongo, Rehoboth, Swakopmund, and Walvis Bay. 6 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 7 II. CAPACITY BUILDING NLT’s capacity-building objectives in the period under review were targeted at both the national and community level. At the national level, NLT sought to strengthen the capacity of MICT for effective HIV communication through the national Take Control Committee. The key areas identified for capacity building included HIV communication strategy design, materials development and pre-testing, and campaign implementation and monitoring. Additionally, the project aimed to strengthen national capacity for planning, implementation and evaluation of HIV communication interventions. At the community level, NLT’s capacity-building objectives were to increase outreach for service utilization, strengthen community-level BCC interventions, and address gender and other social factors related to HIV prevention, treatment and care, such as stigma and alcohol abuse. The community component was to be strengthened through recruitment and further training and provision of supportive supervision to community volunteers. NLT was also to provide capacity building in effective BCC to other organizations working at the community level. NATIONAL LEVEL Discussions with various informants during this evaluation confirmed that NLT had made substantial contributions to the Take Control initiative through its work with MICT. NLT is perceived as the technical experts in HIV communication, with the capacity to design and implement high-quality campaigns, partly because of its understanding of both mass media and targeted communities. NLT has worked to build capacity within MICT through the development of HIV communication operational plans. It has also trained national- and regional-level MICT staff in the use of IPC materials. Through the Take Control Prevention Advisory Committee, NLT provided technical assistance to the MICT in the development of a warehouse to facilitate the collection, distribution, and use of the various HIV communication materials available in Namibia. At the time of this evaluation, however, the warehouse was not operating at its optimal capacity. Discussions with NLT, MICT and some regional AIDS coordination mechanisms indicated that for the warehouse to become operational, it needs additional efforts to stock it with relevant materials, create awareness about its existence at local levels, and ensure an efficient distribution system to the regions. NLT effectively served the coordination role within the Take Control Committee for HIV communication efforts through various technical working groups, such as the Prevention Technical Advisory Committee and Male Circumcision Task Force. NLT is noted to have pushed the agenda for, and rallied different stakeholders around, different HIV communication campaigns even at times when key Government ministries (such as MICT and MoHSS) appeared reluctant or indifferent. Many of the informants in this evaluation whose organizations were represented in the Take Control Committee noted that NLT often surpassed expectations with regard to the coordination of HIV communication efforts, sometimes to the extent of being perceived as an arm of the MICT rather than an independent civil society organization. Indeed, MICT informants for this evaluation described NLT as being in the ―driver’s seat for HIV communication‖ by identifying communication themes and bringing partners together to develop mass media campaigns. Furthermore, NLT was among several organizations involved in developing Namibia’s National Strategic Framework, which is to replace the Namibia Medium-Term Plan III on HIV. This process is soon to be finalized. 8 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Other highlights of NLT’s contribution to national HIV communication capacity building include the following:  Training broadcasters of radio programs to more effectively give HIV messages to the community.  Building the capacity of Regional AIDS Coordination Committees (RACOCs) to understand HIV messages and to contribute to HIV communication materials development.  Participating in RACOCs’ HIV communication operational planning and implementation.  Supporting regional IEC committees in implementing HIV communication through training, talk shows, and regional-level meetings. NLT has taken a hands-on national HIV communication coordinator role, which might not be a sustainable way to proceed. NLT expressed the opinion that MICT regarded it more as a collaborator with significant collective responsibilities for coordinating national HIV communication interventions, rather than as a technical assistance provider. During the evaluation, MICT explained that it had made a request in the most recent Global Fund for AIDS, Tuberculosis and Malaria (GFATM) proposal to fund the position of an HIV/AIDS Resource person within the ministry. USAID’s position, however, was for such technical staff within the ministries to be covered by the Government of Namibia rather than by donor funds for sustainability purposes. Nevertheless, the availability of a staff with technical understanding of HIV/AIDS responses within MICT would enable NLT to serve its role as a technical assistance provider to the ministry rather than as the national coordinator of HIV communication interventions. Even then, the roles of both NLT and MICT in the coordination of national HIV communication efforts need to be clarified. COMMUNITY LEVEL While the original plan envisaged that NLT would provide capacity building for other organizations for community-level BCC activities, USAID transferred much of this mandate to C-Change in 2009. NLT’s transition in 2007 to a local organization also required strengthening its internal capacity and organizational development before it could effectively build capacity among other organizations. In late 2008 and early 2009, NLT worked extensively with a consultant arranged through the C-Change project to integrate NLT’s mass media and community components and strengthen its ability to do effective IPC. This then led to hiring and training regional coordinators to provide supportive supervision for community volunteers at its project sites and to training its CAF members in IPC in 2009. At the community level, NLT and other stakeholders have provided a variety of training opportunities to community volunteers on subjects such as personal growth, HIV/AIDS, basic story writing, alcohol awareness, voluntary counseling and testing, treatment literacy and prevention of mother-to-child transmission (PMTCT) of HIV. National Community Action conferences were organized attended by community volunteers from all project sites to build their capacity for community mobilization, interpersonal communication and collection of M&E data. In the past year, NLT has built the capacity of CAF members to implement effective IPC activities in their communities through the NawaOutreach program. These community volunteers are dedicated to their tasks and are well respected in their communities for the work that they are doing. The CAF members have received extensive training and support from NLT headquarters and regional staff. More experienced members work with newly trained ones to help them become more confident and effective IPC experts. Until this past year, CAF members were conducting more traditional IEC activities, which were not very interactive and did not require multiple sessions with the same groups. During interviews with CAF NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 9 members, they indicated that the IPC sessions were much more effective in getting people engaged in HIV and gender issues. They felt that by participating in the discussions, the participants are better able to think through their own issues, perceptions, and actions. In addition to conducting group sessions, some volunteers do follow-up with community members, either in their homes or via telephone or short message services, to ensure that the needed and appropriate support or services are received. These community volunteers also network and coordinate with government, non-government, and private sector organizations in their communities. They are often seen as the IPC experts in the community and as the only people equipped to do effective IPC. They are also perceived as the source of the most up-to-date and accurate information about HIV and efforts to stem the epidemic. The IPC efforts of the CAF members were described by the organizations working in the same areas covered by NLT (e.g., Red Cross, Catholic AIDS Action, IntraHealth, and local faith-based organizations) as a way to boost their own efforts. The CAFs complemented the activities of these other organizations to address HIV-related issues by creating a knowledge base and by making it easier to talk about HIV (in church, for example) and to disclose HIV status, thereby facilitating access to treatment and care services. In principle, coordination of CAF members and regional NLT staff with other partners formally occurs through local governments, including the RACOC and Constituency AIDS Coordinating Committee (CACOC). At this time, however, many of these coordinating mechanisms are either not functional (e.g., CACOC) or meet too infrequently (e.g., RACOC). In most cases, RACOCs consist of only one person, a community liaison, who may not have sufficient time or capacity to coordinate BCC activities. Other organizations thus often look to NLT to take the lead in BCC at the regional level. The CAF volunteers attend meetings of the RACOCs and other local coordinating committees and use these forums to share information and coordinate activities. Most coordination and collaboration, however, occurs more informally through the personal relationships that CAF members have developed with partners. For example, CAF members have sought out staff at hospitals and counseling and testing sites, and from within local government, in order to promote collaboration on activities such as national HIV testing and World AIDS Day. Volunteers have also been pro-active in approaching other organizations—such as churches, people living with HIV and AIDS (PLHIV) support groups, clinics, police and prisons—to identify groups for IPC sessions and coordinate activities. In smaller towns, this more informal partnering appears to be an effective way to collaborate, particularly with non-governmental partners. In the absence of a functioning coordinating body, however, government officials in particular may be less informed of what is going on in their communities. The benefits of NLT’s capacity building for community-level BCC activities have not been limited to their own program sites and activities. Highlights of the community-level spillover effects of these interventions include the following:  NLT’s community volunteers share tools and IPC approaches with others and complement the work of other agencies through their innovative IPC approaches.  Some of NLT’s community volunteers also serve as volunteers in other programs, thus transferring their skills to those programs. 10 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION  In some instances, community volunteers employed in the government sector have brought their skills and approaches into the workplace through peer education. As a CAF member who also works as a schoolteacher and counselor noted: ―NLT enhances what I do. It has brought me knowledge that I would not have had otherwise, and allows me to bring my skills and knowledge to the students.‖ This same community volunteer has been asked to travel to Sweden to share the NawaSport approach and is returning to Sweden to share other IPC approaches and tools she has learned to use through NLT. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 11 III. STANDARDIZATION AND SCALE-UP OF BCC MODELS One of NLT’s objectives has been to standardize and scale up proven HIV/AIDS and TB communication models. As noted in the previous chapter, NLT has taken a lead role in developing BCC to support the national response to HIV/AIDS in Namibia through its work with MICT and other partners. Over the last three years, the organization has led and participated in technical working groups, coordinated with governmental and non-governmental partners, and assisted in the development and implementation of mass media campaigns and community-level BCC strategies to support scale-up of effective models. DEVELOPING MESSAGES AND CAMPAIGNS Targeting Messages Representatives of organizations interviewed for this evaluation described NLT as the lead organization on mass media for BCC in Namibia. From 2007 to the present, NLT has been a key contributor in identifying appropriate messages and in designing the national campaigns, Be There to Care; Break the Chain; Be Strong, Get Tested; and Stand Up Against Alcohol Abuse. Over time, the process for campaign development has evolved to include more buy-in of stakeholders and increasing levels of collaboration and coordination. NLT staff members have worked closely with partners to standardize mass media messages and develop broader support for the national BCC strategy. Consensus-building around identifying the drivers of the epidemic over the past year has resulted in NLT leading the development of the MCP campaign, the first campaign where broad consensus was reached. Work on the drivers of the epidemic has also encouraged a much more evidence-based approach to BCC campaign development. Within the prevention technical advisory committee, NLT presented its own ideas on how to address issues such MCP and alcohol use, shared findings from regions, and worked closely with members of the Prevention Technical Advisory Committee to incorporate feedback. It has been able to use formative research to shape and frame the messages. As the lead of the mass media effort, NLT has pushed for coordination among partners (on the Take Control campaign, for example). NLT’s successful project proposal to USAID included a strong conceptual framework (see Chapter 4 of the proposal) for HIV prevention programming, which included the role of direct factors (such as MCP and lack of consistent condom use) and indirect factors (such as gender issues and alcohol abuse). Despite the fact that there was no consensus around drivers of the epidemic in Namibia around which to build BCC programs when the project began this phase in 2007, campaigns that NLT helped to develop addressed some of the key issues presented in this framework and are supported by the newer campaigns. Experiences around message development, particularly with the more recent campaigns, indicate that NLT is an organization that understands BCC. For example, the re-launched Stand Up Against Alcohol Abuse campaign has been augmented by IPC materials such as picture codes and integrated with other campaigns such as Break the Chain to take the message beyond awareness creation to challenging people to change HIV risk behaviors. 12 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Campaign Development Recognizing the limitations of individual classic BCC theories—health belief model, reasoned action, diffusion of innovation, social marketing—arising from their focus on the individual, linear approach and rational actor assumptions, NLT uses a participatory, multi-layer and multi-channel communications development framework. NLT has been very methodical in designing and implementing the campaigns. It has conducted community assessments to identify whether the messages are pertinent and appropriate, have brought in partners to help guide design, and have conducted research to assess how well the campaign and its messages have been received. NLT observes that this approach has been helpful in identifying and addressing sensitive socio-cultural and structural drivers of the HIV epidemic in Namibia, such as multiple concurrent partnerships and alcohol abuse. All the stakeholders interviewed believed that NLT should continue to play a key role in HIV communication development in the future. As discussed in the previous chapter, MICT does not have the capacity to develop mass media campaigns and materials. The partners widely agree that NLT produces materials that meet the needs of partners working in BCC. Many of the organizations interviewed for this evaluation suggested that NLT should also play an active role in the adaptation of BCC materials at the regional level. Combining Mass Media and IPC In the previous project phase and in the early period of the project under review, NLT’s approach to HIV communication was more focused on social mobilization with IEC and mass media rather than IPC. In the second half of 2009, following USAID’s suggestion for more IPC-oriented activities, NLT invested resources in building the capacity of its staff in IPC approaches to BCC and trained CAF members in IPC. It also put into place a CAF development plan to transfer skills to CAF volunteers as a way to build volunteer capacity and commitment to the project and reduce turnover. NLT’s practical integration of IPC and mass media occurred with the launch of the Multiple Concurrent Partnerships Campaign towards the end of 2009. Informants from organizations in the community indicated that in the past, groups just came in to talk about HIV and give out pamphlets, which the informants did not think was effective. With the more recent IPC approach adopted by CAFs, they report that the CAF members do fun and interactive sessions with their groups and that these sessions really engage the participants, promote discussions, and change how the participants view the issues and their own behavior. Messages delivered in the community through events and interpersonal communication activities go hand in hand with mass media messages being delivered through the radio, television, posters billboards and other IEC materials. While NLT has invested significant efforts towards the development of the mass media campaigns, fewer resources, coupled with the challenges inherent in developing and implementing a new IPC program, have resulted in less intensity of BCC interventions at the community level. The extent of IPC through community volunteers remains limited due to a limited number of sites with CAFs present and the smaller than anticipated number of volunteers per CAF. There are several tools developed by various partners that are being used in Namibia to support the recent MCP campaign. These include the Flannelogram, the MCP Pictogram, and the Three and a Half Lives of Philip Wetu video. These tools appeared in the offices of, and were reported as being used by, the organizations involved in HIV outreach visited by the Team. The CAF members indicated that these tools worked and that they enjoyed engaging their audiences by using them. The leaders of organizations with whom CAFs do group sessions indicated that the sessions NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 13 are fun and help overcome ―HIV/AIDS fatigue.‖ Volunteers report that by using the tools, they can better help the community understand the issues and internalize them, and address their own beliefs and behaviors. The standardization of tools across partners doing community outreach and the integration of tools with national mass media campaigns are great achievements. Challenges do exist, however. While different partners are using a standard set of tools, it is likely that implementation quality may vary across partners due to different levels of training and experience. In addition, tools are not available in local languages, a problem that is particularly pertinent for those conducting outreach in more rural communities. In many cases, CAF members must translate the tools on the spot, risking miscommunication. POPULATIONS REACHED BY BCC Mass Media Campaigns Due to the absence of population-based data that captures campaign exposure, it is not possible to estimate the proportion of the population reached by the national mass media campaigns. The efforts that have gone into mass media over the past three years, however, have been extensive and have led to television and print advertisements, radio advertisements, billboards, and numerous IEC materials. Community members living in the towns visited by the Team report that the campaigns do reach the communities through radio and television. In particular, it was noted that the broadcast of radio programs in local languages seems to be an effective means of targeting different populations. Availability of other media and IEC materials in local languages, however, is limited. In addition to language barriers, cultural differences between ethnic groups also play a role in the acceptability of BCC materials. As noted by several informants, however, these mass media channels are much less effective in reaching populations in towns in rural areas that the Team visited. NLT has used mass media campaigns, particularly radio, to address special needs among PLHIV and to address the stigma associated with HIV and AIDS. In collaboration with Positive Vibes, NLT has built the capacity of networks of PLHIV through training and the provision of equipment to produce radio programs addressing positive health, dignity and prevention (also known as Prevention with Positives) messages through the Wings of Life radio program. The radio programs were started as part of NLT’s treatment literacy program in the previous project phase and were strengthened under the current project to combine topical discussions, live call-in, interactive sessions, and tips for positive living. The programs are relayed in local languages. Informants from networks of PLHIV commended these radio programs for their role in encouraging treatment adherence, involving people living with HIV in prevention activities, and tackling stigma. The Be There to Care mass media campaign launched in 2008 also generated discussion about the HIV stigma at the community level. Community-level IPC At the time of this report, CAFs were operating at 16 sites in 11 regions. When the CAFs were formed to perform community mobilization activities, it was anticipated that each CAF would have about 12 members, but, over time, there has been attrition of CAF members as some have found employment and others have just dropped out. With the switch to an IPC approach this past year and the need for community volunteers to have more intensive training and skills in BCC, NLT anticipated that the number of members per CAF would be smaller: the seven CAFs visited for this evaluation had an average of four to five members each. In addition to the small membership, turnover among the volunteers has been a challenge for NLT and has hampered efforts to scale up activities in communities. 14 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NLT has a reporting system that captures IPC and other community outreach activities. However, the Team was not able to look at trends in outreach over time due to a shift from IEC to more intensive IPC in the past year (meaning fewer people are reached with greater intensity). Nonetheless, it is clear that CAFs are reaching only a small number of people within their communities. Through the NawaOutreach Program, CAF volunteers have worked effectively with church groups (such as choirs within the Evangelical Lutheran Church in the Republic of Namibia), home-based care support groups affiliated with Catholic Action for AIDS and other organizations, PLHIV support groups, military, police, prisoners and others. The majority of IPC group activities that NawaOutreach volunteers conduct are with church groups, and most church groups reached are predominantly female. According to the CAF volunteers, these groups do represent a cross-section of the community in terms of risk factors, and discussions with group members indicate that practices such as MCP are not uncommon. Many of the groups are found in churches that serve the most vulnerable sections of the community (such as Block E in Rehoboth and Kuiseb in Walvis Bay). These are communities towards the edge of town or informal settlements where poverty and unemployment is high and reflect those areas where the most vulnerable in the community are found. Men are a more challenging population to reach than women, partly because they are less likely to seek health services and less likely to participate in group activities. NawaCinema and NawaSport, which also provide opportunities for community outreach and IPC, were reported by NLT partners as reaching the male population. NawaSport in particular has been very effective at reaching men, especially those in the younger age groups (15-35). In addition, IPC sessions through NawaOutreach have been conducted at some places of employment, including the military and fisheries, and have been more effective at reaching men. According the MoHSS, almost one-third (31%) of new infections are expected to occur among youths aged 15 to 24, with women accounting for over two-thirds of new infections among youth (MoHSS, 2009). While church groups are composed of women of all ages, there are no specific activities for young women. There is a demand within communities for outreach activities to have a greater focus on young girls, and there is even a demand for NawaSport for school girls. There seems to be inconsistencies across CAF volunteers as to what types of IPC activities can be done with young women. While NLT’s mandate is to target the general population, some CAF members are responding to requests from their communities to reach out to young people. For example, some CAF members have conducted IPC activities in schools (with a focus on healthy relationships). It is unclear whether or how the existing tools are adapted and whether new approaches are needed to effectively reach young girls in particular with the messages that are most appropriate. As previously noted, CAF members also use IPC materials to reach out to networks of people living with HIV. As some of the CAF members also live with HIV, they believe that involvement in IPC activities is helping reduce HIV-related stigma. The Be There to Care mass media campaign generated discussion around HIV stigma at the community level. However, some informants were of the view that the timing of the campaign and its limited integration with the drivers of the epidemic might have portrayed the messages too softly, thereby limiting its effectiveness in challenging stigma-inducing social norms. CAF members observed that even campaigns that do not address stigma directly—such as Break the Chain, NawaSports and NawaCinema—created opportunities for open and respectful discussion among community members on the continuum of HIV prevention, treatment, care and support, thus reducing stigma. As a result of these efforts, informants indicated that community members were more open to talking about HIV, there was less fear of being associated with a person living with HIV, and people were more willing to come forward for HIV counseling and testing on national testing days. MoHSS staff interviewed for this evaluation also observed that the level of uptake of antiretroviral therapy (ART) had improved and that they believed this was due, in part, to efforts to reduce stigma. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 15 Traditional leaders and healers were also more open to talking about HIV and asked for training on BCC. A church minister interviewed for this evaluation explained that CAF members, through their regular outreach and IPC activities with various groups in the church, had made it easier to talk about HIV in church, reduce stigma and encourage disclosure of HIV status. However, informants noted that men were less likely than women to disclose their HIV status. At the time of this evaluation, Positive Vibes and NLT were working on improving IPC materials for use with PLHIV. Prototype facilitator manuals and flipcharts were under review. 16 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 17 IV. RESEARCH, MONITORING AND EVALUATION Over the past three years, NLT has aimed to conduct applied research and routine M&E of communications activities to support program learning and guide future communications interventions. Applied research is an important and necessary aspect of the design and implementation of behavior change communication programs and activities. Research activities, in conjunction with routine reporting systems, also allow a program to monitor and evaluate whether or not the program is making progress in meeting its objectives. This chapter will assess NLT’s efforts to use applied research in the design and implementation of its mass media and IPC activities, as well as whether NLT was able to monitor and evaluate its progress. APPLIED RESEARCH NLT has undertaken several formative research activities, primarily focus group discussions, to inform campaign design and implementation, including the development of messages. Some examples are:  NLT commissioned a series of focus group discussions on social support for people living with HIV to inform the Take Control Care and Support phase.  A survey of male attitudes, motivators and fears related to HIV testing was used to inform the development of campaign materials to promote HIV counselling and testing among men.  Research to pre-test and refine materials was conducted for the ART Treatment Literacy and Male Circumcision campaigns. After the recent MCP campaign was launched, NLT assessed its early effects through focus group discussions. The research helped NLT assess the level of campaign exposure as well as attitudes, knowledge and practices and thus establish if the campaign is likely to achieve its strategic goals. NLT has also been very responsive to informal (not from research) feedback on its various mass media activities. For example, NLT explained that it changed an MCP radio spot to clarify risks in sexual networks and to address the potential misconception of the campaign as an attack on traditional polygamous marital unions among some audience members. Other efforts to evaluate activities include an evaluation of NawaSport that focused on the coaches’ capacity to serve as community resources for BCC. While NLT has made good use of research to develop and refine mass media campaigns and IEC materials, much of the research that has been done has been outsourced to a professional research firm. Due to the costs involved, this has limited the amount of research that could be conducted. NLT has been building the research capacity of its staff to improve technical expertise in designing and analyzing research findings. The consolidation of this capacity should allow NLT to reduce the amount and type of research activities that need to be outsourced and provide the organization with greater control over the conduct of research activities. This benefit, of course, needs to be weighted against the possible bias (perceived or real) that would be introduced when an organization conducts research to evaluate its own products and their effectiveness. MONITORING AND EVALUATION The NLT project proposal submitted to USAID in 2007 included a clear conceptual framework for the project (see Figure 1 below). This framework illustrates underlying factors for the HIV epidemic, the communication interventions that the project will undertake, and the expected outcomes or results that are to be achieved. The performance monitoring plan (PMP) developed by NLT, also in 2007, is not aligned to this framework and is not being used by NLT to track its progress towards achieving the initial and behavioral outcomes (using the terminology from this framework) that the project’s activities 18 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION hope to address. The PMP that NLT developed does not present a clear, logical framework or a list of key indicators that could provide a framework for its M&E activities. Figure 1. Pathways Conceptual Framework for the NawaLife Program (Source: NLT Project Proposal, 2007) NLT’s approach to monitoring work at the community level has evolved over time from simply a system to collect data on the numbers of people reached (and trained) to a system that now captures the quality and intensity of outreach activities, as well as a description of successes, challenges, and opportunities for improvement. It is only within the past year that NLT has had a dedicated M&E Officer, who has been able to implement such a system and then train staff and volunteers in how to use it. These new Outreach Monitoring Forms, completed by CAF members, allow staff at the regional and headquarters level to monitor activities more closely, and provides an opportunity for program learning. Completing these forms also provides CAF members an opportunity to review and discuss sessions and make improvements for subsequent ones. NLT has worked hard with CAF members to improve the quality of the descriptive information provided on these forms though the lack of English language abilities and the educational attainment among some volunteers still limit what they can do. The M&E staff has also put data quality checks into place to reduce errors in indicators of persons reached and to reduce double-counting, in order to improve the quality of the data provided to USAID for PEPFAR reporting. As noted, NLT does have a monthly reporting system that includes PEPFAR indicators, among them the number of persons trained and reached with outreach. Due to the shift in approach by NLT from one￾off community mobilization and IEC sessions to more intensive IPC with multiple sessions and because NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 19 of changes in the way that double-counting has evolved over time, a meaningful assessment in trends in these indicators is not appropriate and therefore is not a part of this report. When the project operated under JHU HCP, a series of household surveys were undertaken in the catchment areas around ART clinics. The CAFs were formed in the areas of the communities where the need was thought to be greatest (based on the community assessments conducted by NLT). These areas did not always overlap with the catchment areas of the ART clinics. Thus, these surveys were not an effective means of monitoring the communities in which NLT-supported IEC and IPC activities were being undertaken. NLT discontinued these surveys in 2007 but did not allocate resources towards a new set of surveys. As a consequence, NLT is not able to assess the effects (positive or negative) of its BCC efforts at the community level in a quantitative manner. This includes understanding the reach of its programs (i.e., coverage of IPC) as well as the effectiveness of IPC in the community. At this juncture, there is only anecdotal evidence to indicate the reach and effectiveness of IPC. The findings from the surveys conducted under the JHU phase of the project have been used by NLT and its partners to design and target interventions. In fact, these data were used in a report commissioned for the MoHSS on the drivers of the HIV epidemic in Namibia. Several informants commented that they wish that similar data had been collected under this most recent phase. In the absence of a robust project M&E system, assessing results at the community and individual level can only be done through qualitative means. For example, at the end of each IPC session, the groups create an action plan. The CAF volunteers can follow up with the group at the next session to see what progress has been made on the plan since the previous one. This information is often anecdotal and is not recorded or compiled in any systematic way. With increasing capacity in M&E within the project, NLT hopes to more systematically track and report on these more qualitative measures of success. It is also considering going back to groups that have been reached through IPC several months later at select sites to assess whether changes have occurred. Discussions with NLT staff indicate that it is moving towards a more results-based approach when thinking about M&E. For example, they have recently been working to triangulate monthly data regarding counseling and testing for HIV with information on the occurrence of mass media events and campaigns in order to assess whether mass media can be linked to increased demand for counseling and testing. Discussions around follow-up of groups (mentioned in the previous paragraph) also indicate this shift. 20 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 21 V. LESSONS LEARNED AND RECOMMENDATIONS Over the past three years, NLT has become a full-fledged and independent Namibian NGO that is highly regarded within the Namibian HIV response community. Since its inception, the organization has put tremendous effort into hiring staff, building staff capacity in BCC and the various aspects of organizational management, and developing organizational systems and structures. At the same time, the organization was programming activities at the national level and expanding their community-level efforts. NLT’s technical expertise in designing BCC materials is highly recognized among public and private organizations working in HIV. Several organizations expressed an interest in using NLT’s services, which would broaden NLT’s base and increase its sustainability beyond donor funding. As a Namibian organization, NLT is more accepted by local stakeholders, who expressed confidence in NLT’s capacity for developing cutting-edge HIV communication interventions. ACHIEVEMENTS, CHALLENGES, AND LESSONS LEARNED Capacity Building Through its partnership with MICT, NLT has taken on significant responsibilities for coordination of the national HIV communication interventions. The staff at MICT almost regard NLT as their technical arm for HIV communication. In working closely with MICT, NLT has laid the groundwork for MICT to build its own capacity in HIV communication. As a result of this partnership, MICT has a growing appreciation for HIV communication activities and a greater awareness that HIV is an important part of their work. Training MICT staff in national and regional offices has also better prepared the ministry to take on a greater role in coordinating and implementing HIV BCC activities. NLT recognizes the challenge of their current arrangement with MICT in terms of NLT taking more of a leadership role within the MICT in HIV, rather than just serving as a technical assistance provider. Consequently, NLT has urged MICT to hire an HIV/AIDS resource person, which the ministry explained it planned to cover with resources to be obtained from GFATM. A similar position was created within the Ministry of Health Department of Special Programs through the support of the U.S. Centers for Disease Prevention and Control, which greatly increased the ability of the Ministry of Health to coordinate activities and take on a leadership role in the national HIV response. USAID, however, expressed the opinion that, in the interest of sustainability, such positions should be covered by the Government of Namibia rather than by donors. The availability of an HIV/AIDS resource person within MICT should make it possible for NLT to serve as a technical assistance provider to the ministry, rather than as the national coordinator of HIV communication interventions. Finally, a clarification of the roles of both NLT and MICT in coordinating national HIV communication efforts is needed. NLT’s greatest success in terms of capacity building is in the formation of the CAFs. Made up of dedicated community volunteers, the CAFs are able to implement IPC with groups in their communities through NawaOutreach. They also reach the population through NawaCinema and NawaSport events. The CAF members work pro-actively with community partners—such as health facilities, local government and other NGOs—to coordinate activities, create synergies of efforts to reach populations with different services, and collaborate on larger events, such as national HIV testing days. Efforts to build the capacity of CAFs, particularly in IPC as this was a new approach introduced to the project last year, has come at the expense of scaling up community outreach. Effective IPC requires extensive training and supportive supervision to ensure that CAF members are able to deliver quality 22 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION work. Although CAFs are empowered to solve their own problems, support from regional coordinators, who are currently over stretched, is still needed. CAFs are not formally linked to health facilities, yet better linkages could help build the CAF’s capacity to better meet the needs of their communities. A major challenge with the NLT’s approach to community-level IPC is the use of volunteers. CAFs frequently lose members as they find employment or lose interest in continuing to work with the organization. As the resources put towards capacity building of CAF members are great, the turnover of members has negative consequences for NLT’s efficiency, as well as for the quality of IPC in the community. NLT has looked at strategies such as skill development and short-term employment of CAF members to do administrative tasks as a way to keep CAF members involved. CAF members also indicated that non-monetary incentives—such as certificates in areas in which they have been trained, umbrellas for sun protection, and high quality T-shirts—reinforce their status and role in the community and are highly valued. Those members who have stayed as part of the CAF are very dedicated and take great pride in their work. Standardization and Scale-Up of BCC Models NLT has spearheaded a consultative and collaborative process to develop national HIV communication campaigns. NLT’s strength is in HIV communication in terms of applying a participatory multi-layer and multi-channel HIV communication framework. The resulting campaigns receive broad buy-in and demonstrate technical expertise in campaign design and implementation. NLT has worked with partners to standardize IPC tools and ensure integration of messages between these tools and the national campaigns. The high level of political commitment for the recent MCP campaign speaks to NLT’s success. The HIV communication campaigns that NLT has supported are well appreciated among partners working at the national level and in the community. Even though there was limited integration across communication campaigns in the earlier phase of the project, with one campaign finishing to be followed by a whole new message, greater integration of campaigns has started with more recent campaigns such as Break the Chain and Stand Up Against Alcohol Abuse. NLT has not been able to meet their own targets pertaining to scale-up of outreach, and, more recently, IPC at the community level. This may not be surprising given the maturing of the organization and the many demands that were made upon it. In addition, NLT has been challenged by the high turnover of CAF members, field office infrastructure and providing the supportive supervision required for effective community-level IPC interventions. Scale-up has also been affected by resource limitations. Many CAFs would like to expand to communities where there is little or no outreach, but transport allowances and the logistical challenges of moving equipment for outreach limit the communities they can reach. Experiences from community-level health behavioral promotional interventions suggest that linkage between community volunteers and local health facilities is critical for their effectiveness. Such linkage can facilitate supportive supervision and M&E for quality assurance and continuous learning purposes. Existing linkages between the CAFs and health services in NLT’s areas of operation are largely based on personal relationships (or CAFs being recognized by some providers for their activities in the community) rather than on formal referral and networking arrangements. NLT may want to forge a formal working relationship between its community volunteers and local health facilities. While stakeholders in communities, including CAF members, would like to see NLT expand its current outreach, NLT’s current strength is not in implementing large-scale community IPC. NLT may be better placed at focusing its efforts on the development and standardization of BCC materials rather than on NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 23 large-scale implementation at the community level. However, NLT should continue to use CAF members as a resource in the development and adaptation of cutting-edge BCC materials. Research, Monitoring and Evaluation NLT has made excellent use of formative research to design and fine-tune mass media campaigns and IEC messages. Its campaigns and materials are widely perceived as being professional, of the highest quality, and targeted in terms of messaging. This can partly be attributed to NLT’s methodical approach to using research to guide their development and roll-out. The organization lacks a results-based M&E plan that allows for tracking of progress towards meeting objectives. Any follow-on projects undertaken should have a performance monitoring plan that is results-oriented; includes a logical framework, a minimum set of indicators, data collection and use plans; and can be implemented with the available resources and organizational capacity. NLT does collect and report a short set of PEPFAR indicators, but these do not reflect the project’s efforts, and there are few data that allow NLT to demonstrate its progress and achievements. A project of this magnitude, one which implements activities that contribute to changes in knowledge, attitude, and behaviors in the population, should include plans for data collection at the community level as part of its M&E plan (e.g., through small cross-sectional surveys or follow-up on groups). In addition, triangulation of data from different sources, as NLT is now considering, can be an effective but less resource-intensive way to link project activities with changes in service utilization and other outcomes of interest. RECOMMENDATIONS FOR MOVING FORWARD Our recommendations for the way forward for further support to HIV communication in Namibia are presented below under each of the three objectives identified by USAID for the follow-on project. Strengthening HIV Communication in Public and Private Institutions Recommendation 1: Continue building capacity of MICT to coordinate national HIV communication interventions MICT remains mandated to coordinate HIV communication interventions in Namibia in the National Strategic Framework, which is yet to become operational. While NLT has made some achievements in strengthening the capacity of MICT through the umbrella ―Take Control‖ initiative, on-going technical assistance will be required to enhance MICT’s coordination capacity. The effectiveness of future capacity building for MICT will be enhanced by the following factors:  The hiring of an HIV/AIDS resource person within the MICT to provide knowledge and expertise in HIV interventions and reduce MICT’s reliance on outside assistance for coordination.  Clarification of roles between MICT and the new HIV Communication Project. The role of the follow-on project should be recognized as that of providing technical assistance to the MICT to coordinate HIV communication activities at both national and regional levels. Actual coordination of HIV communication—including convening relevant meetings, managing the warehouse established for HIV communication materials, and advocating for existing and new HIV communication interventions—should be the mandate of MICT. 24 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Recommendation 2: Continue with technical assistance to the national BCC strategy development and implementation NLT’s strength in rallying different stakeholders around HIV communication strategy development is well recognized, as can be seen in the consensus achieved on messaging and IPC tools for recent HIV BCC campaigns such as Break the Chain and Stand Up Against Alcohol Abuse. Because of the diversity of stakeholders and capacities for HIV BCC development and implementation within Namibia, the new project should build on this earned reputation to continue providing leadership to the development and implementation of national BCC strategies. Recommendation 3: Strengthen linkages with workplaces (private sector and government) in BCC strategy development and implementation Through its community component, the HIV communication development process in Namibia has demonstrated expertise in involving grassroots organizations in strategy development. However, there has been limited incorporation of HIV wellness programs within both the public and private sectors in terms of both utilizing available data and implementing or adapting BCC interventions within these settings. Greater linkages with workplaces should, therefore, be explored and strengthened. Implementation of a Package of IPC BCC Interventions Recommendation 1: Work with a training organization to standardize training on IPC BCC tools for organizations involved in various public health interventions to facilitate scale-up A number of organizations support community volunteer programs on a variety of public health issues. To reduce the challenging demand upon one organization to scale up IPC BCC in communities, training should be provided to a variety of community volunteers to transform them into resources for the communities they serve. Many NGOs already collaborate on the ground and express an interest in extending IPC BCC training to their own volunteers. The follow-on project should, therefore, support an organization to standardize training on existing and new IPC tools to community volunteers across partners and regions. Recommendation 2: Expand reach of IPC interventions to underserved areas The reach of NLT’s IPC BCC interventions has been limited by the project design, which centered community activities in public sector HIV treatment sites. The follow-on project should prioritize sites based upon evidence on the drivers of the epidemic and consider growth into peri-urban and rural communities that have not been reached. The incorporation of these hitherto underrepresented communities has the potential to enrich the development and adaptation of IPC BCC materials. Strengthening Regional Capacity to Coordinate and Implement HIV Prevention Interventions Recommendation 1: Build the capacity of HIV/AIDS focal persons in HIV communication The National Strategic Framework proposes the creation of the position of HIV/AIDS focal point persons within line ministries and at regional levels. The follow-on project should work within this structure, once operational, to build the capacity of these focal persons on HIV communication, including existing and new mass media, IPC materials and tools. Training of HIV/AIDS focal persons on the existing tools will not only ensure standardization and quality but also increase capacity to implement effective IPC activities. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 25 Recommendation 2: Strengthen capacity of regional AIDS coordination mechanisms for community-level BCC implementation Effective community-level implementation of BCC and IPC interventions hinges on a strong coordination framework. Currently, the mandate of HIV/AIDS focal persons to coordinate HIV/AIDS activities within their local and regional governments is unclear. Capacity and support from management is also often lacking. The follow-on project could work to build the capacity of these focal persons to develop operational plans for the implementation of BCC programs within the regions. 26 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 27 ANNEX A. SCOPE OF WORK Global Health Technical Assistance Project (GH Tech) Contract No. GHS-I-00-05-00005-00 SCOPE OF WORK (Draft 0.3 23-Sept-2010) I. TITLE: USAID/Namibia End of Project Evaluation for Nawa Life Trust Cooperative Agreement. Contract: GH Tech GHS-I-00-05-00005-00. II. PERFORMANCE PERIOD: Not including time for preparation and completion of report, two to three weeks in-country, to begin October 25, 2010. III. FUNDING SOURCE: This assignment will be funded by USAID/Namibia. IV. PURPOSE AND OBJECTIVES: The US Agency for International development (USAID) awarded Nawa Life Trust a Local Cooperative Agreement number: 690-A-00-07-00-00103-00 in October 2007 with an end date of December 31, 2010 for a planned Life of Project of $7,826,522. The agreement focused on prevention in the technical areas of Prevention abstinence and being faithful, Prevention with condoms and other, Palliative Care, Strategic Information, Other/Policy Development and System Strengthening, and Counseling and Testing. Sub-partners included: Ibis, Johns Hopkins University, Matters and Means, and Research Facilitation Services. This cooperative agreement was a follow-on exception to competition requirements to a $6 million program implemented by the former Namibian field office of Johns Hopkins University Health Communication Partnership (JHU HCP) from April 2003- 2007. The Cooperative Agreement awarded to Nawa Life Trust called for three main activities: 1) Design and implement mass media, community-based, and interpersonal behavior change communications interventions; 2) Build the capacity of community-based and faith-based non-governmental organizations and Government of Namibia Ministries in behavior change communication through the provision of technical support; and 3) Conduct applied research to guide program planners, implementers and evaluators about health communications interventions. As paraphrased from the June 2007 program description for the cooperative agreement, the six objectives for Nawa Life Trust cooperative agreement were:  Strengthen the capacity of Namibian organizations to design and implement health communication programs;  Strengthen the capacity of the Ministry of Information and Communication Technology (MICT) and other institutions to develop strategic plans for innovative HIV/AIDS communications.  Standardize and scale up proven HIV/AIDS and TB communication models;  Use an effective and appropriate mix of electronic and non-electronic media and community based and interpersonal communications techniques to reach communities and most at risk audiences;  Expand accurate messages and portrayals about HIV and AIDS to address stigma;  Conduct applied research and routine M&E of communications activities to support program learning to guide future communications interventions; 28 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION To achieve these objectives, Nawa Life has conducted activities at the national level and in eleven regions (Erongo, Hardap, Karas, Khomas, Kunene, Kavango, Omaheke, Omusati, Oshana, Oshikoto, and Otjozondjupa). Nawa Life Trust’s activities include:  Media: As a member of Take Control, the Government of Namibia’s stakeholder group hosted by MICT, Nawa Life Trust designed and implemented major national media campaigns on: 1) Stigma/care, 2) Multiple Concurrent Partnerships, 3) Male Engagement in HCT and 4) Stand Up to Alcohol Abuse.  Collaboration with community service organizations: Assisted in the coordination of messages from media campaigns to be integrated into outreach activities with support of C-Change, IntraHealth and PACT and non-USA funded NGOs.  Community Activities: a) ―NawaOutreach‖ on gender equity, life skills and HIV/AIDS awareness; b) ―Community Action Forums‖ reaching individuals with HIV prevention activities; and c) ―Nawasport‖ activities across 8 regions (street squad & soccer tournaments etc.). Goal of the evaluation: This evaluation seeks to assist USAID to determine the performance and effectiveness of the Nawa Life Trust cooperative agreement, as well as inform the implementation of USAID’s future competitive award for a follow-on project for national level communication and community-level prevention activities. The follow-on project will address three objectives: strengthen public and private Namibian communication institutions for public health interventions; implement a package of interpersonal and community oriented BCC interventions for HIV prevention; and support regional capacity to implement HIV prevention interventions. Objectives of the evaluation include:  To assess why progress toward the agreement’s planned results has been positive or negative; identify and analyze unintended consequences and effects of assistance activities.  To assess the extent to which Nawa Life Trust was able to build capacity for Government of Namibia ministries and community organizations.  To assess the extent to which Nawa Life Trust monitored and evaluated the outcomes and impact of the range of communication activities supported under the project.  To assess to what extent Nawa Life Trust used a strategic and sustainable mix of communication channels to reach diverse populations of persons at risk.  To assess the implications of making a transition from JHU-CCP to Nawa Life Trust for project implementation, management and financial savings.  To identify lessons learned and make recommendations to guide the follow-on project implementation. Key implementation issues: The evaluation may require concurrence of Government of Namibia counterpart Ministries, such as the Ministry of Information and Communication (MICT), Ministry of Regional and Local Government (MRLG) and Ministry of Health and Social Services (MOHSS). USAID/Namibia will inform/provide this concurrence to GH Tech if team members need it for work in￾country. In addition, per a recent audit by the Office of Inspector General, (See ―Audit of USAID/Namibia’s efforts to address crucial shortages of trained HIV/AIDS Health Workers,‖ Audit Report No. 9-000-10-00X-P, July 1, 2010), this evaluation will be required to define the Nawa Life Trust project’s contribution to health systems strengthening through HRH investments, as well as identify Nawa Life Trust’s contribution to sub-partners or to the Government of Namibia in the area of organizational capacity. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 29 Period under review for the evaluation: From start of project (October 2007) to the Annual Progress Report for 2010 (September 30, 2010). V. SCOPE OF WORK Illustrative Key Questions to be addressed by the team: Guiding Evaluation Questions 1. What was the performance and effectiveness of Nawa Life Trust (NLT) as measured against the six objectives of the 2007 USAID cooperative agreement? Why has NLT progress toward planned results been positive or negative? 2. To what extent did NLT monitor and evaluate the outcomes and impact of communication activities? How has NLT used project data and nationally available data that were collected over the course of the project? 3. To what extent did NLT use a strategic and sustainable mix of communication channels to reach diverse populations of persons at risk? 4. How well has NLT served the needs of different ―customer‖ and or/demographic groups (For example, rural, urban, young men, young women, clients in health facilities) 5. Did NLT activities result in referral services impacting customer’s sexual behaviors on multiple concurrent partnerships or health seeking behavior to HIV counseling and testing or alcohol abuse interventions? 6. Did NLT use a theoretical framework or behavior change theory-based approach to guide the design, implementation and evaluation of HIV-prevention communication programs? If so, what was the theoretical basis for NLT programs and how was this theory used? 7. Did NLT use a clearly defined logic model (also referred to as a program impact pathway) to guide the design, implementation and evaluation of NLT communication programs? 8. What were the implications of making the transition from being a JHU-CCP affiliate to a locally independent NGO for project implementation, management and financial savings? 9. What are key lessons learned and key recommendations that can inform the implementation of USAID’s future competitive award for a follow-on project for national level communication and community-level prevention activities? The response to this question should address each of the three objectives of the 2010 competitive follow-on award: i) strengthen public and private Namibian communication institutions for public health interventions; ii) implement a package of interpersonal and community oriented BCC interventions for HIV prevention; and iii) support regional capacity to implement HIV prevention interventions. 10. What has been NLT’s contribution to health systems strengthening through investments in human resources for health? 11. What has been NLT’s contribution to sub-partners, community organizations and/or to the Government of Namibia in the area of organizational capacity? Performance information sources—Items 1-6 to be sent to GH Tech as soon as possible before in￾country work begins. 1. Baseline assessments for program implementation (NDHS 06/7; others) 2. All available examples of NLT applied research to guide communication interventions. 3. Country Operational Plan FY07, FY08 and FY09 narratives 4. Work plans and PMP 30 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 5. Quarterly, semi-annual and annual progress reports 6. Financial report and pipelines 7. Media coverage survey data for all types of media: radio, TV, print. 8. Ministry of Health and Social Services reports on NLT activities 9. Any signed agreement with local partners 10. Key informants interviews 11. Field visits and direct observations VI. METHODOLOGY The evaluation team will use a variety of methods for collecting and analyzing qualitative and quantitative information and data. The methods to be used in completing this evaluation will include, but not be limited to: reviewing documentation, interviews, site visits, stakeholder meetings, etc. Drawing on experiences in other PEPFAR countries, USG Namibia will seek the assistance of external consultants, headquarters, host country and local USG counterparts to conduct the assessment. The following essential elements should be included in the methodology as well as any additional methods proposed by the team: Document Review Prior to arriving in country and conducting field work, the team will review various project documents and reports. Lists of key documents are included in Sections V and XIII. The USAID/Namibia team will provide the relevant documents for review as soon as possible. Team Planning Meeting A two-day Team Planning Meeting (TPM) will be held during the evaluation team’s first two days in￾country with USAID staff. This time will be used to clarify team roles and responsibilities, deliverables, development of tools and approach to the evaluation, and refinement of agenda. In the TPM the team will:  share background, experience, and expectations for the assignment  formulate a common understanding of the assignment, clarifying team members’ roles and responsibilities  agree on the objectives and desired outcomes of the assignment  establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion  develop data collection methods, instruments, tools and guidelines, and methodology and develop an assessment timeline and strategy for achieving deliverables  develop a draft report outline for Mission review and approval In-depth Discussions with USAID/Namibia and project staff Key Informant Interviews The team will conduct structured interviews with the project staff, and key partners including the MOH and NGOs, other donors, implementing partners, and other stakeholders. To ensure that comparable information is collected during interviews, the team will develop standard guides reflecting the questions posed by the evaluation scope of work. NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 31 Field Site Visits: The team will coordinate with USAID/Namibia to prepare for and conduct site visits while in-country, and to interview key informants at these sites. Assuming a two person team, over a period of 9 days, one member of the team will conduct site-visits in two northern regions to be determined. During the 9 days when one team member is making site visits in the two Northern regions, the other team member will conduct site visits in Erongo Region and Hardap Region as well as site-visits in the Windhoek area, with two days travel. USAID/Namibia will arrange for all required in-country transport. USAID/Namibia will make arrangements for accommodations as needed. 32 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 33 ANNEX B. PERSONS CONTACTED NAMIBIA NawaLife Trust Staff Nahum Gorelick, Director Salen Engelbrecht, Deputy Director Bastian Schwarz, Media Manager Abigail Solomons, Monitoring and Evaluation (M&E) Officer Ester Andreas, Regional Coordinator, Oshana Region (Oshikuku) Joey Heita, Regional Coordinator, Hardap Region (Rehoboth) Erica Oxurus, Regional Coordinator, Otjozondjupa Region (Otjiwarongo) Harold Kandjii, Program Officer, Acting Regional Coordinator, Erongo Region NawaLife Trust Community Action Forums (CAF) Oshikuku CAF, Omusati Region Oniipa CAF, Oshikoto Region Oshakati CAF, Oshana Region Otjiwarongo CAF, Otjozondjupa Region Omaruru CAF, Erongo Region Walvis Bay CAF, Erongo Region Rehoboth CAF, Hardap Region DONORS United States Agency for International Development (USAID) Melissa Jones, Director, HIV/AIDS and Health Office Brad Corner, HIV/AIDS Prevention Adviser Nabil Alsoufi, DLI Health Officer Helen Cho Renee Liebenberg United Nations Children’s Fund (UNICEF) Rushnan Murtaza, Chief Education HIV Prevention & Mitigation Joint United Nations Program on HIV/AIDS (UNAIDS) Henk van Renterghem, UNAIDS Country Director US Centres for Disease Control and Prevention, Namibia Nick de Luca, Prevention Advisor John Pitman, Technical Advisor, Health Communications and Blood Safety GOVERNMENT MINISTRIES Ministry of Information and Communication Technology (MICT) Fednedy Kabunga Kabunga, Manager, Campaigns (Print Media Affairs) 34 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Ministry of Health and Social Services (MoHSS) Manni Khandji, Coordinator, MoHSS/CORD (Coalition on Responsible Drinking) Frieda Katuta, National Prevention Coordinator, Department of Social Programmes (MoHSS/DSP) Yvonne Januarie, Principal Social Worker, Rehoboth Renate Shoombe, Regional HIV/AIDS M&E Coordinator, Oshana Region Liina Emvula, Chief Social Worker, Oshakati Ministry of Regional, Local Government and Housing (MoRLGH) Connie Podewiltz, HIV/AIDS Focal person NON-GOVERNMENTAL ORGANIZATIONS Jane Shityuwete, Director, LifeLine/ChildLine Reggie Mouton, ChildLine Manager Libet Maloney, HIV Prevention & BCC Director, IntraHealth Elizabeth Burleigh, Country Director, C-Change Stephanie van der Walt, Technical Advisor, C-Change Selvester ―Lolo‖ Gorase, President, Namibia Football Players Union, NawaSport Ambassador Kirsten Moeller-Jensen, Director, Development Aid from People to People (DAPP) Protasus Aipanda, Division Commander, DAPP, Oshana Region Melody Chipadze, Division Commander, DAPP, Oshana Region Ian Maxwell, Deputy Chief of Party, Pact Namibia Nelson Prada, Program Manager, Pact Namibia Mike Haidula, Regional Manager, Society for Family Health Jeremiah Shetunyenga, Coordinator, TONATA network of people living with HIV Sakeus Angula, Regional Coordinator, Red Cross, Oshana Region Peter Njuki, Medical Doctor, Onadjokwe Hospital, Shanamutango, IntraHealth George Ekandjo, National Coordinator, Skillz Namibia/Grassroots Soccer David Nghikelwa, Community Mobilizer, Catholic AIDS Action Efraim Lipinge, Regional Manager, Catholic AIDS Action Casper W. Erichson, Director, Positive Vibes Pastor Mberira, New Start Manager, Otjiwarongo Damina Conde, Regional Coordinator, DAPP-TCE, Otjiwarongo Maria Napolo, BCC Officer, Society for Family Health, Walvis Bay Mr. Kaveta, Catholic AIDS Action, Walvis Bay David Novak, Medicos del Mundo, Rehoboth REGIONAL AIDS COORDINATION MECHANISMS AND LOCAL ORGANIZATIONS Councillor Edward Alfred Wambo, Rehoboth East Constituency AIDS Coordination Committee Paul Skrywer, HIV/AIDS Focal person, Rehoboth Town Council/ Rehoboth AIDS Association Julius Mbuende, Life-Change Centre, Oshana Region. Fillipine Chivoro, HIV Prevention Officer for Onandjokwe District Fillipus Angula, Finance and Administration Manager, Oshikuku Village Council Petros Endjambi, Councillor, Oshikuku Constituency AIDS Coordination Committee Julius Shuuya, Chief Clerk, Oshikuku Constituency AIDS Coordination Committee Marianne N. Tangeni, Coordinator, Ondangwa Constituency AIDS Coordination Committee Johannes Kuutondokwa, NawaSport Coach Edward Thaniseb, Associate Pastor, Evangelical Lutheran Church John Hinda, Multipurpose Center Supervisor, Otjiwaringo Hendrina Kashipolo, M&E Officer, Otjiwarongo Regional AIDS Coordination Committee Naomi Muinjo and Roswitha Kaura, Omaruru Municipality NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 35 Ms. Aune, HIV Point Person, Prison Services, Omaruru Sister Tijanda, Nurse, Omaruru Hospital HIV Counselling and Testing Clinic Ronnie Neib, District AIDS Coordinator, Walvis Bay 36 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 37 ANNEX C. LIST OF DOCUMENTS REVIEWED NLT WORK PLANS AND ANNUAL REPORTS (RFA) No. 390-07-008-HIV/AIDS Emergency Plan Activities in Namibia. Project Proposal. NawaLife Work Plan Narrative Report (FY 2007) Work Plan Narrative Report COP 2008. Compiled for submission to USAID V 2.0 16th Jan 09 NawaLife Trust Work Plan - Nov 15th, 2009 to Sept 30th, 2010 NawaLife Trust Work Plan - Nov 15th, 2009 to Dec 31st, 2010 JHU/HCP Namibia and NawaLife Trust, Annual Report 1 October 2006 – 30 September 2007 NLT. Annual Progress Report for FY09 Quarter 4. (Jul 2009 – Sep 2009) Progress Report for FY10 Quarter 3 (April 2010 – June 2010) Progress Report for FY10 Quarter 2 (Jan 2010 – Mar 2010) & Semi-Annual Program Results (SAPR) (Oct 2009 - Mar 2010) MONITORING AND REPORTING Performance Monitoring Plan. Cooperative Agreement. USAID Namibia, NawaLife Trust, 2007-2008 Monitoring Tool for MCP Campaign Activities 2009 – 2010 Nawa Sports Monthly Reporting Form Trip Report (Gobabis) Media Flowplan Summary Activity Report. NawaOutreach, June 2010. Activity Report. NawaSport, August, 2010. Activity Report. Break the Chain Long Distance Transport Activations DATE Event Report. Take Control: MCP Windhoek Show Stand, September– October 2009. Event Report. Ongwediva Annual Trade Fair. DATE Event Report. Take Control ―Media Against HIV‖ Partnership Launch June 12, 2008 Data Quality Assurance Assessment, June 2010 RESEARCH REPORTS NawaCinema Pilot Evaluation, Rehoboth, Oshakati and Kasote, 2007 Focus Group Report, Men and HIV Testing, March 2009 Take control care and support phase, Formative Investigation. A report on a series of Focus Group Discussions on Social Support for People living with HIV, 2007 Annual review of New Start Demand Creation Activities 2008 (Presentation) Focus Group Report. Alcohol AIDS HIV Campaign. 2009 Campaign Pre-Testing Focus Group Report. Multiple And Concurrent Partnerships, Campaign Pre￾Testing. 2009 Early campaign impacts. A qualitative evaluation of the immediate responses to Namibia’s campaign on 38 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Multiple and Concurrent Partnerships prepared by NawaLife Trust for the Take Control National HIV & AIDS Media Campaign May 2010 Media Planning Brief ―Take Control - Break the Chain (MCP)‖ NATIONAL DOCUMENTS Male Circumcision for HIV Prevention. Communication Strategy, Feb 2009 National Condom Strategy 2010 -2015 Reducing Multiple Concurrent Partnerships in Namibia. Social and Behaviour Change Communication Strategy for the National MCP Campaign. May 2009 HIV/AIDs in Namibia: Behavioral and Contextual Factors Driving the Epidemic National Strategic Framework for HIV and AIDS 2010/11-2015/16 United Nations General Assembly Special Session (UNGASS) Country Report Reporting Period 2008 –2009 OTHER USAID Namibia Sexual Prevention Portfolio Review, Draft 4/24/09 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 39 ANNEX D. KEY INFORMANT INTERVIEW GUIDES Key Informant Interview Guide NawaLife Trust Evaluation Draft Oct 19, 2010 National Level Partners Government (MICT/MOH) 1) How has the ministry collaborated with NLT on HIV communication activities since the project began in 2007? 2) How has the ministry benefitted from the collaboration with NLT in terms of designing and implementing HIV mass media campaigns? (probe: messages, strategy, materials, implementing, evaluating) 3) What was the process followed by NLT and other stakeholders for developing the different communication campaigns? (probe: target population, message, desired behavior change) 4) Has NLT’s HIV mass media communication strategy been effective in reaching populations at risk with appropriate HIV prevention messages? a. Have the messages been appropriately targeted to the drivers of the epidemic? b. Are there populations who have not been reached by mass media and that should be? 5) How have research findings generated by NLT been used by the ministry in planning and designing health communication campaigns and materials at both the national and community level? 6) What HIV communication strategies have been used at the community level to enhance the reach and effectiveness of the national program? 7) Have community outreach activities been an effective way to reach at risk populations with HIV prevention messages (probe: right population, right message, address expressed needs, right type/mix of activities)? 8) How has NLT contributed to communication interventions around reducing stigma at the national and community level? b) How successful have these campaigns been at addressing stigma at both the national and community levels? 9) What has worked at the national level in terms of mass media campaigns for HIV prevention? (right message? right format? delivery?) What has not worked? What can be improved? 10) What has worked at the community level in terms of behavior change interventions? (types of events, mode of delivery, role of CAFS, messages, etc) What has not worked? What can be improved? 11) What are the key lessons learned from NLT supported communication activities since 2007? 12) Has coordinating/collaborating with NLT strengthened the ministry’s capacity to implement effective HIV communication programs? In what way? (Probe: Design, Implementation, evaluation) 13) What do you see as the key areas of support for effective HIV communication interventions at the national and community level in the future? (probe: skills, coordination, best practices) 14) In 2007, NLT transitioned from being part of Johns Hopkins Communication Program to being a local organization. How did this transition affect NLTs contribution to HIV communication activities in the country? (probe: effectiveness, perception) NGO/INGO 1) How has your organization collaborated with NLT on HIV communication activities since the project began in 2007? 40 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 2) Were activities on which you collaborated adequately supported by NLT and other partners? (Probe: resources, technical assistance) 3) How has your organization benefitted from the collaboration with NLT? (Probe: planning, implementing, evaluating) 4) Think of the communities that you work with. Have NLT’s HIV communication activities been an effective way to disseminate HIV messages? (Probe: appropriateness of messages, mode of delivery, community level strategies) a. What has worked? b. What has not worked? c. What can be improved? 5) Regarding the communities that you work with, what is needed for effective HIV communication activities at the national and community level? 6) What do you see as the role of NLT in HIV communication in Namibia going forward? 7) In 2007, NLT transitioned from being part of Johns Hopkins Communication Program to being a local organization. How did this transition affect NLTs contribution to HIV communication activities in the country? (probe: effectiveness, perception) Donors/UN/CDC 1) How has your organization collaborated with NLT on HIV communication activities since the project began in 2007? 2) How does your organization prioritize the HIV responses to support? (Probe: based on what data or information) 3) How has your organization benefitted from the collaboration with NLT in terms of designing and implementing mass media campaigns? (probe: messages, strategy, materials, implementing, evaluating) 4) Has NLT’s HIV mass media communication strategy been effective in reaching populations at risk with appropriate HIV prevention messages? a. Have the messages been appropriately target to the drivers of the epidemic? b. Are there populations who have not been reached by mass media and that should be? 5) Have community outreach activities been an effective way to reach at risk populations with HIV prevention messages? 6) How has NLT contributed to communication interventions around reducing stigma at the national and community level? b) How successful have these campaigns been at addressing stigma at both the national and community levels? 7) In 2007, NLT transitioned from being part of Johns Hopkins Communication Program to being a local organization. Did this transition affect your organization’s ability to coordinate/collaborate with NLT in HIV communication activities in the country? (probe: effectiveness, perception) 8) What has worked at the national level in terms of mass media campaigns for HIV prevention? (probe: right message, right format, delivery) What has not worked? What can be improved? 9) What has worked at the community level in terms of behavior change interventions? (probe: types of events, mode of delivery, role of CAFS, messages) What has not worked? What can be improved? 10) What do you see as the key areas of support to the government for effective HIV communication interventions at the national and community level in the future? (probe: skills, coordination, best practices) NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 41 Regional/Administrative Partners (RAC, CACOC, MORLGH, Authorities) 1) How has the ministry/regional/local authorities collaborated with NLT on HIV communication activities since the project began in 2007? 2) Have you received materials from NLT campaigns? If so, how have they been used? 3) How has NLT work on communication and messaging supported coordination of community partners in the region? 4) How has the ministry/regional/local AIDS program benefitted from the collaboration with NLT in terms of and implementing HIV communication activities? a. Capacity to Design b. Capacity to implement c. Capacity to Monitor and evaluate 5) Has NLT’s HIV mass media communication strategy been effective in reaching populations at risk with appropriate HIV prevention messages? a. Have the messages been appropriately target to the drivers of the epidemic? b. Are there populations who have not been reached by mass media and that should be? 6) Have community outreach activities in the region been an effective way to reach at risk populations with HIV prevention messages (probe: right population, right message, address expressed needs, right type/mix of activities)? 7) Are you collecting information around health beliefs and attitudinal changes within communities? 8) What do you see as the key areas of support to the regional/local government for effective HIV communication interventions in communities? (probe: skills, coordination, best practices) Community Level Partners CAFs 1) When was the CAF formed and what is its membership? 2) What activities does the CAF do and how are they conducted? (probe: one on one, group, venue, frequency of contact, population targeted) 3) How well has the CAF met the needs of this community? Have needs expressed by the community been met in terms of understanding the availability of services such as CT, PMTCT, ART, how to reduce risk of HIV infection? 4) How well are you able to reach those most at risk? 5) What have CAFs done to reduce stigma in communities? Have PLHIV been involved in designing and implementing community activities? 6) Can you describe the linkage between the CAFs and the local health care facility? 7) How do the local health facilities support the activities of the CAF? 8) How do CAFs ensure that persons referred for health services such as (CT, ART, PMTCT) receive the services? 9) How does the CAF use its activity reports and feedback from the community to refine and plan for future activities? 10) What has worked at the community level in terms of behavior change interventions? (types of events?, mode of delivery, role of CAFS, messages, etc) What has not worked? What can be improved? 11) What support is needed by the CAFs to help the community reduce risky behavior and make better use of HIV services at health facilities? 42 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION Implementing Partners NGO, Government 1) What HIV-interventions does your organization do in the community? 2) How has your organization collaborated with NLT on HIV communication activities since the project began in 2007? 3) How has your organization benefitted from the collaboration with NLT? 4) How involved is your organization with the CAF? (probe: planning, implementing activities, information sharing) 5) Think of the communities that you work with. Has NLT’s HIV communication activities been an effective way to disseminate HIV messages? (probe: behavior change, health service utilization, stigma) : a. What has worked? b. What has not worked? c. What can be improved? 6) Regarding the communities that you work with, what is needed for effective HIV community level interventions? Health Facilities 1) Can you describe the linkage between the health care facility and the CAF? (probe: planning, implementing activities, information sharing) 2) How does the health facility support the activities of the CAF? 3) How has staff at this health facility benefitted from the NLT HIV communication program? (Probe: training of staff, IEC materials, involvement in outreach activities) 4) Is there a referral mechanism between the CAFs and the health facility? How does the health facility ensure that persons referred by CAF receive HIV services? (Probe: CT, PMTCT, ART) 5) Is there information sharing between the CAF and the health facility? Does the health facility use reports or feedback from the CAF to strengthen HIV services? 6) What has worked at the community level in terms of behavior change interventions? (types of events?, mode of delivery, role of CAFS, messages, etc) a. What has not worked? b. What can be improved? 7) What support is needed at the health facility level to help the community reduce risk behavior and make better use of HIV services? NawaLife Staff 1. How would you assess your performance in achieving each of the six objectives in the 2007 cooperative agreement? What were your biggest achievements, and what is the evidence for these results? (Focus on objectives related to capacity strengthening) 2. Can you describe the process you followed in designing the messages for the different communication campaigns? What specific factors guided your approach? 3. To what extent did Nawa Life Trust monitor and evaluated the outcomes and impact of its activities? 4. What is the way forward in terms of formative research for campaign development? 5. What training have CAFs and regional coordinators received? 6. What is the appropriate role of NLT is supporting MICT in HIV communication activities? 7. What has been the role of NLT and its partners in developing BCC materials and training of community volunteers? NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 43 8. What were the implications of making the transition from being a JHU-CCP affiliate to a locally independent NGO for project implementation, management and financial savings? 9. What are key lessons learned and key recommendations that you have to support effective national and community-level prevention activities? 44 NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION NAWA LIFE TRUST COOPERATIVE AGREEMENT: END OF PROJECT EVALUATION 45 ANNEX E. LIST OF REFERENCES National Planning Commission (NPC). 2006. Namibia Household Income and Expenditure Survey. Windhoek, Namibia: Central Bureau of Statistics, 2003-2004. Central Bureau of Statistics. Namibia Labor Survey Report. Windhoek: 2008. Ministry of Health and Social Services. 2008/09 Estimates and Projections of the Impact of HIV/AIDS in Namibia, Windhoek, 2009. Government of Namibia. National Strategic Framework for HIV and AIDS 2010/11-2015/16, 2010 For more information, please visit: http://resources.ghtechproject.net