FINAL EVALUATION OF THE HEALTH COMMUNICATIONS PARTNERSHIP (HCP II) PROJECT FINAL REPORT April 19, 2013 This document was produced for review by the United States Agency for International Development (USAID/Uganda). It was prepared by i-Train and Evaluate Center (i-TEC). Page i FINAL EVALUATION OF THE HEALTH COMMUNICATIONS PARTNERSHIP PROJECT (HCP II) Final Report Submitted by Contracted under Order Number: AID-617-O-12-00013 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. I- TRAIN AND EVALUATE CENTER (I-TEC) Plot 19, Ntinda View Crescent - Naguru P.O Box 918, Kampala – Uganda Phone: +256 – 772 – 193860/ +256-775-772771 Fax: +256 - 312 - 283585 E-mail: rrwampororo@evaltrain.com; Kasingye@aol.com Website: www.evaltrain.com Page ii TABLE OF CONTENTS THE I-TEC EVALUATION TEAM......................................................................................................................... V EXECUTIVE SUMMARY.......................................................................................................................................... IX 1.0 INTRODUCTION................................................................................................................................................1 a) Social and Behavioral Change Communication..................................................................... 1 b) HCP II Project Summary ....................................................................................................... 2 c) Evaluation Purpose ............................................................................................................... 2 2.0 EVALUATION DESIGN AND METHODOLOGY..............................................................................3 2.1 Approach and Methodology for Addressing the Key Evaluation Questions ................................. 3 2.1.1 The Selection of Districts and Target Population....................................................................................................................... 4 2.1.2 Data Analysis Used in Getting to the Answers........................................................................................................................... 4 2.1.3 Evaluation Data Limitations/Constraints....................................................................................................................................... 4 3.0 HCP II FINAL EVALUATION FINDINGS............................................................................................6 3.1 Extent of Achievement of HCP II Project Results .................................................................. 6 3.1.1 Mapping Achievements to HCP II Projects Results Framework........................................................................................... 6 3.2 Progress towards Intermediate Result 1 ............................................................................... 9 3.2.1 Design & Implementation of Effective Communication Strategies....................................................................................... 9 3.2.2 Increased Knowledge and Behavior Change............................................................................................................................ 10 3.3 Progress towards Supportive Social Environments (Intermediate Result 2) ........................17 3.3.1 Public Support for Services and Practices that Promote Health ........................................................................................ 17 3.3.2 Youth supportive structures ............................................................................................................................................................ 18 3.4 Effectiveness of Project Approaches ...................................................................................19 3.4.1 Effectiveness of the One Focal Communication Project Supporting Different Implementing Partners ................. 19 3.4.2 Effectiveness of Knowledge and Behavioral Change-Related Approaches................................................................... 20 3.4.3 Behavioral Change Communication Campaigns and Health Service Delivery ............................................................. 23 3.4.4 Generating Opportunities for Leadership and Professional Development (GOLD) ..................................................... 24 3.4.5 Radio Distance Learning (RDL) for Village Health Teams ................................................................................................... 25 3.5 Impact of HCP II on the capacity of local IPs and the MoH (Evaluation Question 3) ............26 3.5.1 Capacity of Government through MoH........................................................................................................................................ 26 3.5.2 Capacity of Local Partners .............................................................................................................................................................. 27 3.5.3 Capacity for Communication/Media Organizations................................................................................................................. 28 4.0 LESSONS LEARNED................................................................................................................................28 4.1 Implications for Future Designs, Capacity Enhancement, & Partnerships ...........................28 5.0 CONCLUSIONS............................................................................................................................................29 6.0 RECOMMENDATIONS .................................................................................................................................31 6.1 Project Design, Strategies, and Approaches .......................................................................31 6.2 Implementation & Achievement of Results ..........................................................................31 6.3 Strengthened Institutional Capacity .....................................................................................31 7.0 ANNEXES ............................................................................................................................................................32 Annex 7.1: Excerpt from the HCP II Final Evaluation Scope of Work ..............................................32 Annex 7.2: Detailed Methodology, List of Selected Districts, & Summary of FGDs .........................34 Annex 7.3: List of HCP II PMP Indicators & Trend Data from 2008 - 2012 ......................................40 Annex 7.4: Detailed Communication Strategies ..............................................................................44 Page iii Annex 7.5: HCP II Programs & their Communication Channels ......................................................46 Annex 7.6: Summary Trend Data FP, SMC, HIV, and Malaria (2006 – 2012) ................................. 48 Annex 7.7: HCP II Programs vis-à-vis Strategic Approaches ..........................................................50 Annex 7.8: List of USAID & USG-funded IPs on HCP II Project ......................................................52 Annex 7.9: Quotes from FGDs and KIIs ..........................................................................................54 Annex 7.10: Introduction to KII and FGDs Guides............................................................................................................. 57 Annex 7.10A: KII Guide 1 – USAID Managers/CORs ..................................................................58 Annex 7.10C: KII Guide 3 - HCP II Program Staff ..........................................................................60 Annex 7.10D: KII Guide 4 – Communication/Media Organizations ..................................................63 Annex 7.10E: KII Guide 5 - Government Partners ...........................................................................64 Annex 7.10F: Observation and Exit Interview Guide .....................................................................65 Annex 7.11: FOCUS GROUP DISCUSSIONS – GUIDES..................................................................................................... 66 Annex 7.11A: FGD Interview Guide 1 - Youths ............................................................................66 Annex 7.11B: FGD Interview Guide 2 – Adults ...............................................................................68 Annex 7.11C: FGD Interview Guide 3 - Village Health Team (VHT) ...............................................70 Annex 7.11D: FGD Guide 4 - People Living with HIV/AIDS (PLWHA) ............................................71 Annex 7.12: Post-Training/Internship Evaluation for GOLD Program: .............................................73 LIST OF TABLES Table 1: FY 2012 Performance (Target Versus Actual) ...................................................................................... 6 Table 2: Change in Comprehensive Knowledge on HIV by Gender and Residence............................................ 11 Table 3: Comprehensive Knowledge on HIV and HCP ll Campaign/Communication Medium ..................................... 12 Table 4: Number of sexual partners and comprehensive knowledge on HIV .................................................... 12 Table 5: SMC and Stand Proud, Get Circumcised Campaign ............................................................................ 14 Table 6: Association between use of Family Planning Methods and selected HCP .................................................. 15 Table 7: Data on Policy-Related SMC, Alcohol, and TB- 2008 – 2012............................................................... 18 LIST OF FIGURES Figure 1: Triangulation of Methods ................................................................................................................ 3 Figure 2: HCP II Result Framework-Performance Mapping Based on FY 2012 Targets ........................................ 7 Figure 3: Core Messages from Studies & the Campaigns they Informed ........................................................ 10 Figure 4: Trend in Comprehensive Knowledge on HIV/AIDS ......................................................................... 11 Figure 5: Trends in Knowledge on Benefits of SMC in reducing chances of acquiring HIV/AIDS ......................... 13 Figure 6: Male Circumcision by Region..................................................................................................... 14 Figure 7: Knowledge about the spread of TB.............................................................................................. 16 Page iv Figure 8: Knowledge on Symptoms of TB Mentioned by Respondents ........................................................... 16 Figure 9: How messages from Rock Point 256 influenced actions ....................................................................... 21 Figure 10: Actions taken from the Knowledge gained from Go Together, Know Together Campaign ............................ 22 Figure 11: Messages heard by Respondents about the Nurse Mildred Campaign.................................................... 22 Page v ACKNOWLEDGEMENTS i-TEC team wishes to express its deep appreciation for the guidance and accessibility provided by the USAID/Uganda Mission to various relevant stakeholders who were involved in the HCP II project implementation. These included but were not limited to the government and institutional partners, implementing partners, GOLD (Generating Opportunities for Leaders and Professional Development) program graduates, community members who also willingly participated in the focus group discussions, and HCP II senior project management and staff. Specifically, our special thanks go to USAID/Uganda Mission Director, Ms. Leslie Reed and the HCP II Final Evaluation Task Managers (Ms. May Mwaka, Ms. Rhobbinah Ssempebwa, and Ms. Charmaine Matovu) who provided us with technical feedback. The Mission team was involved as key informants during the design stage. They reviewed our evaluation instruments and responded to some of our evaluation questions. They also provided comments and observations on the draft report. Special thanks also go to HCP II staff both at Headquarters and in the field, especially the Chief of Party, Ms. Cheryl Lettenmaier, without whose unwavering support it would not have been possible to conduct and complete this evaluation exercise successfully. The HCP II team were also instrumental in providing the program overview, reviewing and harmonizing the program’s secondary data, and following up with locating the respondents included in this evaluation field exercise. Our appreciation goes to Dr. Esther Kaggwa and her team for providing us with the data generated from the survey conducted by John Hopkins University (JHU) on the “Evaluation of Behavior Change Interventions in Uganda” without which our findings would not have been as rigorous. The extra quantitative data helped us to triangulate findings from different sources and validate many of our conclusions. Our sincere thanks also go to all the stakeholders in the 12 districts covered during this evaluation exercise for their valuable time and support. Many thanks also go to the youth, PLHIV, media organizations, and community members who participated in the consultations. THE i-TEC EVALUATION TEAM USAID/UGANDA commissioned i-TEC (i-Train & Evaluate Center), a recognized performance-based evaluation consulting firm in East & Southern Africa (ESA) Region, based in Kampala, Uganda to design and conduct the external final evaluation of the HCP II project (2007–2012). A multidisciplinary team comprising of experienced evaluators drawn mainly from Uganda conducted the evaluation, with overall coordination and technical guidance provided by Dr. Rosern K. Rwampororo, i-TEC’s President. The rest of the team members included other i-TEC Senior Associates as follows: (1) Mr David Ouma Balikowa, who was the Team Leader for the Eastern Region; (2) Mr John Bosco Asiimwe, as Team Leader for Western Region; (3) Ms. Emilly Kemigisha, as Team Leader for Central; and finally (4) Ms. Pross Kansiime Karangizi, as Team Leader for Northern Regions. The team was supported by a group of seven experienced Research Associates, drawn from the Institute of Statistics, Makerere University. However, i-TEC takes full responsibility of the final product. Page vi LIST OF ACRONYMS: ACP AIDS Control Program ACTs Artemisinin-Based Combination Therapies AIC AIDS Information Center AIDS Acquired Immune Deficiency Syndrome AIM AIDS/HIV Integrated Model District Program AIS AIDS Indicator Survey ART Antiretroviral Therapy ARV Antiretroviral AMREF African Medical Research Foundation ANC Antenatal Care BCC Behavioral Change Communication CBOs Community-Based Organization CDCS Country Development Cooperation Strategy CDFU Communication for Development Foundation Uganda CHCT Couple HIV Testing and Counselling DCC District Coordination Committees DO Development Objective DHE District Health Educators EMHS Essential Medicines and Health Supplies ESA East and Southern Africa FBO Faith-Based Organization FGD Focus Group Discussion FP Family Planning FPRWG Family Planning Revitalization Working Group GBV Gender Based Violence GEM Gender Equitable Male GOLD Generating Opportunities for Leaders and Professional Development GOU Government Of Uganda HCT HIV Counselling and Testing HCP Health Communication Partnership HIPS Health Initiatives for Private Sector HIV Human Immunodeficiency Virus HSSIP Health Sector Strategic & Investment Plan IEC Information, Education, and Communication IDI Infectious Diseases Institute Page vii IPs Implementing Partners IPTP Intermittent Preventive Treatment in Pregnancy IR Intermediate Result IRS Indoor Residual Spraying i-TEC I-Train and Evaluate Center ITN Insecticide Treated Mosquito Net JCRC Joint Clinical Research Center JHU John Hopkins University JHU•CCP John Hopkins Bloomberg School of Public Health Center for Communication Programs KII Key Informant Interviews LCs Local Councils LLINs Long Lasting Insecticide Treated Mosquito Nets M&E Monitoring and Evaluation MJAP Mulago – Mbarara Joint AIDS Program MoH Ministry of Health MHS Management Sciences for Health MUSPH Makerere University School of Public Health NACWOLA National Community of Women Living with HIV/AIDS in Uganda NGOs Non-Governmental Organizations NUMAT Northern Uganda Malaria AIDS and Tuberculosis Program PACE Program for Accessible Health, Communication, and Education PLHIV People Living with HIV PMP Performance Management Plan PMTCT Prevention of Mother to Child Transmission PSFU Private Sector Foundation Uganda RDL Radio Distance Learning RHU Reproductive Health Uganda RLOs Regional Lead Organizations SBCC Social and Behavior Change Communication SCFU Save the Children Fund Uganda SCFUSA Save the Children Fund USA SMC Safe Male Circumcision SMP Stop Malaria Project SO Strategic Objective SOW Scope of Work SPEAR Supporting Public Sector Workplaces to Expand Action and Responses against Page viii HIV/AIDS STARs Strengthening TB and HIV/AIDS Responses STAR-E Strengthening TB and HIV/AIDS Responses in Eastern Uganda STAR-EC Strengthening TB and HIV/AIDS Responses in East Central Uganda STAR-SW Strengthening TB and HIV/AIDS Responses in South Western Uganda STIs Sexually Transmitted Infection SUSTAIN Strengthening Uganda’s Systems for Treating AIDS Nationally TASO The AIDS Support Organization TAT Technical Advisory Team TB Tuberculosis TBCAP TB/HIV Communication Campaign Partners TOT Training of Trainers UAC Uganda AIDS Commission UBOS Uganda Bureau Of Statistics UDHS Uganda Demographic and Health Survey UHCA Uganda Health Communications Alliance UHMG Uganda Health Marketing Group UNICEF United Nations Children’s Fund URC Uganda Red Cross USAID United States Agency for International Development VCT Voluntary Counselling and Testing VHTs Village Health Teams VMMC Voluntary Medical Male Circumcision YAGs Young People Advisory Groups Y.E.A.H Young Empowered And Healthy Page ix EXECUTIVE SUMMARY Background: 1. Health Communication Partnership (HCP II) was a five-year project funded by USAID to develop and implement communication strategies on HIV/AIDS, tuberculosis (TB), family planning, and malaria and strengthen capacity in social and behavior change communication (SBCC) for improved health in Uganda. The HCP II objective on improved abilities of communities, families, and individuals to adopt practices that improve health is in line with the priorities of the Government of Uganda (GoU), which recognize the critical role of Behavior Change Communications (BCC). This $18 million project was implemented and managed by The John Hopkins Bloomberg School of Public Health Center for Communication Programs (JHU•CCP) from 2007-2012. Based on a design rationale to ensure streamlining of BCC activities from the national level to the grassroots and the desire to reach a wider target audience, the project was implemented through a network of partners: government entities such as the Ministry of Health (MoH) and the Uganda AIDS Commission (UAC); institutional partners such as the AIDS Information Center (AIC); the Communication for Development Foundation (CDFU) and a large number of local and international implementing partners including USAID funded partners. Evaluation Purpose, Design and Methodology 2. The purpose of the HCP II project final evaluation was to assess the effectiveness of the major BCC approaches implemented under the project and to identify factors for success. Lessons learned and good practices from this evaluation will be used to inform designs of future health communication interventions by USAID and the Government of Uganda. The evaluation was conducted by i-Train and Evaluate Center (i￾TEC) team from August 6, 2012 to March 31, 2013. i-TEC used a mixed-methods approach that entailed both quantitative and qualitative data collection techniques to arrive at the answers to the evaluation questions. The team triangulated data generated from various methods: (i) document reviews; (ii) key informant interviews with beneficiaries and key implementers such as HCP II staff and partners; (iii) focus group discussions (FGDs) with adults, youth, and people living with HIV; (iv) stakeholder analysis; (v) observations of signage and posters while in the field and/or at health facility sites; (vi) trend analysis of secondary data (HCP 2008 baseline and the 2010 midterm evaluation) plus data generated from the survey conducted by John Hopkins University (JHU) on “Evaluation of Behavior Change Interventions in Uganda,” referred to as “BCC Evaluation, 2012” herein after. 1 Key Findings 3. Extent of achievement of Strategic Objective on Improved abilities of communities, families, and individuals to adopt practices that improve health: - Evidence from the analysis of project monitoring and survey data indicates that the objectives of the HCP II program have been partially met; data on two out of the five selected performance measures at the SO level indicate that there has been an improvement in adoption of some healthy practices like HIV testing, disclosure of results amongst partners and use of modern family planning methods from the baseline in 2008 to 2012. However, there is an increase in risky sexual behavior across the different age groups2 . Findings from youth FGDs indicated that adolescents were beginning to 1 The decision to use data from the BCC Evaluation 2012 resulted in a two months lag in the contract. 2 The two out of five performance measures achieved include: (i) percent of men and women 15–49 years old who took an HIV test and received their results in the last 12 months, from 38 to 76 percent for women against the Life of Project (LOP) target of 50 percent; and for men from 26 to 61 percent against an LOP target of 40 percent; and ii) percent of women aged 15-49 who are currently using any modern family planning method, from 25 to 35 percent against LOP target of 34 percent. The three measures, whose targets were not achieved include: (i) percent of sexually active men and women aged 15–49 years who know the HIV status of their sexual partner(s), from 51 to 53 percent against the LOP target of 59 percent; (ii) percent of sexually active males and females (15–49 years old) who had sex with only one partner in the Page x have sex as early as 15 years of age. Gender differences in the extent of change indicate a need to target women in future interventions. 4. Extent of achievement of Intermediate Result 1 on Effective communication strategies designed and implemented to increase appropriate use of services and/or practices across identified priority programs:- The key implementation approach under the HCP II project was to work jointly and support the MoH as a central government partner, other government institutions, and implementing partners (IPs) to develop and/or revise several communication strategies in HIV/AIDS, HIV/TB, family planning, and malaria. HCP ll rolled out the above stated communication strategies through various programs implemented with the different implementing partners. HCP II promoted a multimedia approach for delivery of messages i.e. interpersonal communication; mass media and community-based activities. Evidence from various studies was used to inform the various campaigns and choice of media. Achievement under each key program area was as follows: HIV/AIDS: Despite the design and implementation of multiple strategies and campaigns to address HIV/AIDS, comprehensive knowledge about its transmission and prevention has stagnated between the baseline in 2008 and in 2012 at about 53 percent. There was evidence of declining trends in urban areas from 60 to 58 percent as compared to 52 to 51 percent in the rural areas. The evaluation discovered no significant relationship between comprehensive knowledge and the HCP II supported Rock Point 256 program and the National Health Hotline. Further analysis indicated that increased knowledge is linked to likelihood of adoption of risk reduction practices such as persistent condom use and reduced number of sexual partners. On the other hand, findings of the Uganda AIDS Indicator Survey (AIS) 2011 indicate that there is still a large proportion of the population (84%) still have more than two sexual partners and are not using condoms, an issue that can be addressed through BCC to contribute to reduction in HIV rates. a) There was an increase in knowledge practice of voluntary safe male circumcision (SMC) from 31 percent in 2008 to 81 percent in 2012. The evaluation revealed a significant and positive relationship between the HCP II supported Stand Proud, Get Circumcised Campaign and SMC practice especially amongst the youth (15-24 years). b) About 60 percent of respondents of the BCC Evaluation 2012 said they had seen or heard messages about the HCP supported campaign on HIV testing “Go Together, Know Together” in the past 12 months. Those who got the “Go Together, Know Together” campaign messages were influenced to take some action on testing and disclosure of status. Initially HIV testing was made easier by a policy adopted by Health Centers to testing every patient who reported with any sickness for HIV but this has been affected by stock-outs of HIV testing kits. c) Though there was no quantitative data to provide a concrete picture on adherence to ART by PLHIV, FGD participants indicated that they try to adhere to the treatment, visit health facilities whenever they fall ill, eat a balanced diet despite the economic hardships, and use condoms to prevent reinfection and infecting others. However, PLHIV also reported that it is difficult for adolescents to adhere to ARVs because of fatigue, while others fear being seen taking ARVs. Family planning: Results indicate that listening to the Nurse Mildred radio drama series was statistically significant (p-Value = 0.002) and was linked to current use of a family planning method. About 40 percent of those who were currently using a family planning method had listened to the radio program, compared to 67 past 12 months, from 83 percent to 72 percent against LOP target of 90 percent; (iii) There is a decline in the percent of unmarried young men and women aged 15–24 years who “have never had sex,” from 52 percent in 2008 at baseline to 47 percent in 2012 against the LOP target of 57 percent. Page xi percent who had not heard the Nurse Mildred message and were currently not using any family planning method. Malaria: Evidence from the FGDs indicated that communities visited were knowledgeable about malaria across all regions. The knowledge has translated into behavioral change in terms of an increasing trend in the ownership and use of mosquito nets from 34 to 74 percent, according to the UDHS as a context at the national level. 5. Though BCC messages were instrumental in stimulating people to adopt practices and seek health services, observations at health facilities and interactions with patients have revealed that these services are not always available. Health service delivery is being impacted by low staffing levels and availability of products at the health centers which limits the full enjoyment of health communication benefits. 6. Extent of achievement of Intermediate Result 2 on Supportive social environments fostered, to enable positive health‐seeking behaviors and result in healthier individuals, families, and communities: - As part of ensuring public support for services and practices that promote youth, HCP II supported MoH and the Uganda AIDS Commission (UAC) who are responsible for supporting policy formulation, providing direction, execution, leadership, and strategy development in the implementation of health communication programs. HCP II supported the roll out of the new policies in support of adoption of healthy behaviors. Increased awareness on the SMC and alcohol abuse policy was registered. However, there is decline in awareness of any law or government of Uganda policy regarding TB from 36 to 27 percent, which was attributed to the short (one year) duration of the TB campaigns. a) HCP II implemented the Y.E.A.H Campaign to raise consciousness and dialogue about social issues targeting mostly the youth. The campaign used media like Rock Point 256 (weekly radio drama series) and comic book series and youth groups such as the Young People Advisory Groups (YAGs). Audience feedback through listenership surveys by radio stations showed that Rock Point 256 had increased station listeners, especially the youth. Though there is an indication that messages from Rock Point 256 have influenced youth to take positive health actions, tests for significance did not fully confirm this association. YAGS and other youth group volunteers trained by HCP ll as peer educators facilitated group discussions among the youths using comic books and videos to trigger discussions. On the other hand, findings from FGDs show that communication between adults and young people has not been strengthened. 7. Extent of achievement of Intermediate Result 3 on Increased capacity for sustained health communication: - MoH has increased its capacity to deliver quality health services through trained staff and health workers, particularly counsellors and VHTs for community mobilization. Practical skills development and partnerships have been created between MoH, AIC and other partners through their direct participation in the development and implementation of health campaigns and IEC materials. The project provided public access to communication strategies and materials during its tenure at the following web link www.k4health.org/toolkits/hcp. HCP II built the skills of health communication journalists to improve reporting on health issues to the target population. A total of 400 journalists from hosts of health talk shows at 21 radio stations around the country received training. The BCC working group, a strategic multi-stakeholder forum for coordinating and harmonizing BCC activities under the leadership of MoH, started with the support of HCPII is not functioning any more. Page xii Effectiveness of BCC approaches 8. HCP II was conceived by USAID as one focal health communication project designed to support communication activities of various partners in the health sector. Government representatives have appreciated this as an effective model for promoting uniformity in health messages. Working together resulted in using single communication strategies for an individual health problem instead of having parallel ones. This model has also limited the duplication of activities and conflicts among the implementing partners as a result of better coordination. 9. To bridge the skills gap on health communication among implementing partners, HCP II through the Generating Opportunities for Leadership and Professional Development, GOLD, program set out to increase the number of young Ugandan professionals with the required job skills. The program trained 200 young professionals, with a 60 percent completion rate. About 93 percent of graduates trained are currently employed as health communications specialists in USAID-funded projects and others in the private and media sector. Out of the GOLD graduates interviewed, 64 percent said they were satisfied with the skills acquired during this internship. Some USAID funded IPs – a target group for absorbing these professionals sighted budgetary constraints to hire and fill communication positions at their organizations. The GOLD program is now run by the Private Sector Foundation of Uganda (PSFU) but its sustainability remains hinged on securing extra funding. 10. Given the high numbers of Village Health Teams (VHTs) country wide, HCP II designed the Radio Distance Learning program (RDL) as a low-cost option for refresher and continuing training. This has improved the VHTs’ knowledge on health issues and their roles. Complaints on the lack of face to face interaction follow up by HCP and inadequate equipment for VHTs have been addressed by World Vision and UNICEF in the roll out of RDL into other districts. Lessons Learned & Implications for Future Designs, Capacity Enhancement, & Partnerships: 11. To achieve its goal, HCP II implemented seven strategic approaches most of which were very effective. However, given the challenges cited in relation to each approach, it is necessary to change and/or improve the way future approaches should be designed. The following are some examples of strategies that will need to be continued and improved: a. Promoting and directing clients to services: Radio is still the most cost-effective channel to bring health messages to mass audiences. This is one communication medium that should be continued along with other multimedia channels like inter-personal communication and print. b. Focusing on underlying social issues that influence behavior: The gap in communication between adults and young people is a hindrance to the creation of a supportive social environment for youth. It is also important to note that some behaviors are challenging to change in the short run because of people’s beliefs and traditions. c. Strengthening strategic communication capacity: Sustainability of this strategic approach is a challenge for MoH and IPs because it is costly to produce IEC materials that are used to promote positive behaviors and use of health services. d. Fostering public supportive social environments: Environments fostered though MoH and UAC for policy formulation, direction, execution, and leadership are still essential for any future design. What may need to change is the nature of the partnerships in terms of both accountability and resource mobilization for sustainability of the BCC programs. Page xiii e. Building coalitions and strengthening partnerships: This approach should be continued with a focus at the community level. HCP II worked well through partners to form coalitions mainly at the national and district levels. However, the failure by HCP II partners to allocate their own resources to support the continuation of HCP ll interventions affected their ability to continue funding the activities after the end of HCP II. Conclusions 12. Behavior change communication programming under HCP II has been effective in improving the abilities of communities, families, and individuals to adopt practices that improve health. Behavior changes in reduction in risky sexual behavior practices, uptake of SMC and HIV testing, and use of family planning methods have been linked to HCP programs. However, the National Health Hotline and Rock Point 256 program were not as effective in improving levels of comprehensive knowledge on HIV and health behaviors like use of family planning amongst its target community i.e. 15–24 year olds. 13. Levels of comprehensive knowledge on HIV/AIDS, risky sexual behavior (not using condoms and having multiple partners) especially among young men in the 15–24 age group, voluntary male circumcision especially among the 25–49 age group and use of contraceptives still remain below the targets set in the Health Sector Strategic Investment Plan and present areas for further investigation and development. 14. While behavioral communication interventions may stimulate healthy behaviors, current staffing and drug shortages at the health facilities are affecting access and utilization of services. Health services like SMC, HIV counseling and testing, and family planning need to be readily available and affordable to enable communities practice the newly acquired behaviors that will lead to better health outcomes. 15. HCP II as a focal organization on communication was deemed an effective model in terms of ensuring uniformity of behavior change messages and communication materials used. This approach also resulted in cost savings by avoiding duplication of efforts and cost savings as a result of working through partnerships. Strong relationships and leadership by the Ministry of Health was instrumental in coordinating multiple partners and enlisting commitment to use of IEC materials developed. 16. The Radio Distance Learning (RDL) designed as a cost effective mechanism for reaching the numerous VHTs across the country was successful in imparting knowledge to these teams who are regarded as the frontline service providers to communities. Efficiency gains are coming at a cost of potential effectiveness. Motivation of VHTs is a key issue that still needs to be investigated and addressed. 17. The capacity of MoH, local implementing partners, and USG implementing partners in behavior change communication has been improved as a result of the practical and targeted training, access to formative research, and joint design, development, and implementation of behavior change communication strategies and campaigns facilitated by HCP. The BCC implementers in Uganda now have access to e-resources on communication strategies and tools that can be reproduced upon demand. 18. HCP II supported communication campaigns, training and other capacity building initiatives like the GOLD program have not been found to be sustainable. Without HCP II and in other cases USG funding that is currently supporting several health projects, the MoH and local partners cannot continue to run health communication programs including re-production of materials due to inadequate financial resources. Page xiv Key Recommendations Recommendations Responsible Entity (ies) Project Design, Strategies, and Approaches 1. Ensure that projects follow the new project design guidance (2011), and have clearly defined performance indicators at all levels that meet the ADS 203 standards for good indicators. USAID 2. Continued use of multimedia channels including interactive radio and drama series as communication mediums for reaching a larger target audience. BCC Partners 3. Future programming should reconsider the target group and thereby the continued use of the national Health Hotline vis-à-vis the costs involved in running it. MoH & BCC partners 4. Explore other appropriate strategies aimed at bridging the communication gap between adults and the young people (15-24 years). MoH 5. Investigation into drivers and barriers affecting improvements in comprehensive knowledge on HIV/AIDS, risky sexual behavior especially among young men in the 15–24 age group, voluntary male circumcision especially among the 25–49 age group and use of contraceptives should be conducted to inform designs of appropriate interventions. MoH, USAID and BCC Partners 6. The Ministry of Health should intensify efforts in improving availability and accessibility of critical health services through better staffing, provision of drugs, and health commodities to absorb the demand created through behavior change communication activities. MoH 7. USAID should continue using a single communication partner with the capacity to coordinate and lead activities and provide technical assistance thus ensuring uniformity, efficiency, and leveraging of resources. A close partnership with the MoH is important to ensure a buy-in from other partners and rolling out of common messages, skills, and products. USAID Implementation & Achievement of Results 8. Strengthen the BCC working group and develop a long-term resource plan for BCC activities to ensure continued implementation of BCC activities. MoH & BCC Partners 9. BCC partners need to mobilize adequate resources to support the continued production of IEC materials in order to sustain the momentum created thus far. BCC Partners 10. Future interventions should widen the age bracket to include adolescents who are below 15 because they are also becoming sexually active. MoH Strengthened Institutional Capacity 11. In order to improve service delivery, future programming should consider improving and rolling out the RDL program nationwide since the program was effective in improving the skills of VHTs that serve community needs directly. MoH Page 1 1.0 INTRODUCTION The United States Agency for International Development (USAID) funded the Health Communication Partnership (HCP) ll project to develop and implement communication strategies and strengthen capacity in social and behavior change communication (SBCC) for improved health in Uganda. This project, funded to the tune of about $18 Million, was implemented and managed by The John Hopkins Bloomberg School of Public Health Center for Communication Programs (JHU•CCP) from 2007 to 2012. This was a follow-on project to expand communication activities initiated between 2004 and 2007under a previous USAID award. a) Social and Behavioral Change Communication Behavioral Change Communication (BCC) activities in health have been on going in Uganda for a long time, disseminated mostly through radio and television, talk shows, features and documentaries, plus newspapers. The onset of HIV/AIDS in the mid-1980s intensified the use of BCC in order to slow down the spread of the pandemic by encouraging safer sexual practices. The MoH AIDS Control Program (ACP) added the use of Information, Education, and Communication (IEC) materials, while UNICEF in the early 1990s supported various BCC activities where children performed plays on HIV and child killer diseases. In addition, USAID-funded projects such as AIDS/HIV Integrated Model District Program (AIM) in 2001-2005, implemented a BCC collaborative partnership with selected local governments and used BCC to increase understanding of HIV and TB link, PMTCT, VCT, male participation, and ABC strategy in the different regions of the country. However, the BCC activities were limited in scale, uncoordinated, and too scattered. The majority of them were not evidence based, not participatory in nature, and not developed following a systematic process. They were also mostly vertical, focusing on knowledge and behaviors at the expense of horizontal aspects such as the social environment. A supportive social environment was essential for influencing the adoption of health practices as well as the capacity of the service providers at the community level. BCC is an important component of the USAID/Uganda strategy as a supportive intervention for service uptake in the health sector. It is on this basis that USAID supported the comprehensive BCC initiative under HCP II to implement a broad communication support to the health sector in Uganda. The underlying rationale behind the design and implementation of the HCP II project was that BCC should be organized and streamlined from the national level down to the grassroots. HCP ll worked with the MoH, as the central government agency responsible for health promotion and education to lead and coordinate the design of evidence-based communication strategies, BCC and IEC materials, and provided capacity building to MoH and other implementing partners including the USAID funded partners3 to design and implement effective BCC interventions. The objective of HCP II was to improve the abilities of communities, families, and individuals to adopt practices that improve health through health communication. The HCP II goal, which focused on behavior change communication, is aligned to the MoH Health Sector Strategic Investment Plans (HSSIP 2005/06–2009/10 and 2010/11–2014/15) objectives. Among these objectives, HSSIP identified strengthening IEC materials used for BCC initiatives in order to bring about changes in health and related behaviors among the people of Uganda as one of the key interventions. However, the plan notes that lack of funds to produce IEC materials was affecting the ability of the MoH to meet the increasing demand for health information in the communities. HCP II was therefore able to fill such a gap and also supported other areas such as the media and VHTs, which are also listed in the plan as some of the key interventions necessary for health service promotion and health-seeking behaviors (HSSIP 20010/11–2014/15; p. 53–56). 3 According to HCP ll Report of 2012, there were 77 HCP ll partners (5 government partners, 6 institutional partners, and 66 IPs. About 22 IPs are USAID-funded partners) Page 2 b) HCP II Project Summary The HCP II project objectives were also aligned with Development Objective 3 (DO3) –Improved Health and Nutrition Status in Focus Areas and Population Groups, under the new Country Development Cooperation Strategy (CDCS). USAID working together with GoU, civil society, and the private sector provide support to address the heavy disease burden, malnutrition, and unmet need for family planning as well as to improve health service delivery systems and literacy rates. BCC interventions therefore provide the link to DO3 outcomes by creating demand for health services and are also crucial for enhancing adoption of health-seeking behaviors. It is against this background that on July 1, 2007, USAID signed a new $6,000,000 Associate Cooperative Agreement with The John Hopkins University through the Health Communications Partnership to implement a three‐year program of broad communications support for the health sector in Uganda. The agreement was later extended two years, through 2012. The purpose of this five‐year program (HCP II) was to bolster the effectiveness of communication programming and implementation under the SO 8 Objective (Improved Human Health), and later DO3. Under this award, John Hopkins was expected to do the following tasks:  Continue ongoing communication initiatives and partnerships such as Y.E.A.H, Be A Man, and TREAT and support malaria and family planning (FP) efforts, while initiating new areas of assistance and support (e.g., for tuberculosis (TB) and HIV counseling and testing).  Contribute to shaping societal norms to enable social and behavioral change to take place.  Contribute to capacity building of the Ministry of Health and local implementing partners (IPs) and increase the professionalism and standards of communication approaches that are most popular in Uganda including media, music, and drama.  Serve as the source of expertise on communications for other IPs including development of communications materials to support their activities’ intermediate results (IR) expected of HCP II, along with key results for each IR. HCP II project was working in collaboration with other partners such as the Ministry of Health, the Uganda AIDS Commission (UAC), institutional partners such as the AIDS Information Center (AIC), and the Communication for Development Foundation (CDFU) as well as a big number of local and international implementing partners. About 22 out of the 66 IPs were funded by USAID. HCP II worked with selected districts to implement BCC activities and disseminate messages that ultimately contribute to changing attitudes and behaviors. The HCP II Conceptual Framework The “Pathways to Community Health” model was the conceptual framework underlying implementation of the HCP ll project. This model identifies four communication domains: engage individuals, stimulate community dialogue and action, create an enabling environment, and make services more user-friendly. According to this model, the underlying theory of change stipulates that strategic communication interventions that address barriers in these domains will result in positive changes in individual behavior and social norms, more supportive environments, and strengthened institutional and service system capacity; these, in turn, will contribute to improved health. To achieve its goal, HCP II used seven strategic approaches: (1) help make services more clients centered; (2) promote and direct clients to services; (3) focus on underlying social issues that influence behavior; (4) improve community mobilization capacity; (5) form strategic partnerships with leaders and media representatives; (6) strengthen strategic communication capacity; and (7) build coalitions and strengthen partnerships. c) Evaluation Purpose The purpose of the HCP II project final evaluation was to assess the effectiveness of the major BCC approaches implemented under the project and identify factors for success. USAID/Uganda identified three key evaluation Page 3 questions (outlined below) to be answered by the evaluation in order to document major achievements in behavior change and successful approaches that contributed to these achievements. In addition, the evaluation was tasked with identifying HCP II’s limitations, challenges, and opportunities for improvement (what could have been done differently to add to project achievements). Finally, the evaluation was also meant to establish lessons learned and good practices that should be adopted in future programs. Information from this evaluation will be used to inform designs of future program work by USAID and the Government of Uganda. The evaluation Scope of Work (SOW)4 questions were as follows: 1. To what extent did HCP II achieve each and all expected outcomes under the three intermediate results? 2. Which project approaches contributed to major project achievements, and which project approaches may have resulted in poor or less-than-expected performance? 3. What is the impact of HCP II on the capacity of local IPs and the MoH to conduct communications activities without continued technical assistance? 2.0 EVALUATION DESIGN AND METHODOLOGY 2.1 Approach and Methodology for Addressing the Key Evaluation Questions i-TEC used an evaluation approach that entailed mixed methods (both quantitative and qualitative), which was necessary to facilitate triangulation of data from all the different sources in order to corroborate findings. This approach ensures that the evaluation fully responds to the key evaluation questions as well as the purpose of the evaluation. The schema on how mixed methods were triangulated is presented in Figure 1 below. Figure 1: Triangulation of Methods 4The detailed SOW is attached as Annex 7.1 Step 1: Document Review -Benchmarks of the various indicators -Performance data - Results mapping -Trend data analysis Step 2: Key Informant Interviews (40) with GoU, IPs, USAID staff, HCP II staff, DHEs, Radio Stations Step 3: Focus Group Discussions (46) attracting 456 participants composed of VHTs, Youth, Adults, & PLHIV. Step 5: Data Analysis from the baseline survey, midterm and BCC Evaluation, 2012 Findings and Analysis:  Progress made by HCP II towards its three Intermediate Results  Lessons learned from implementation of the HCP ll interventions established. Step 4: Observations in the field. This was conducted concurrently with Step 3. Page 4 The detailed description of each of the qualitative and quantitative approaches, plus the major phases of this final evaluation is included under Annex 7.2, Section A. 2.1.1 The Selection of Districts and Target Population For its own data collection, i-TEC evaluation team selected a purposive sample of 12 out of the 145 districts that were covered in both the 2008 and 2010 surveys, all of which are part of the districts that were also covered in the BCC Evaluation, 2012. Collecting qualitative data from districts similar to the survey enhanced comparability of results. The target population was selected from the women ages 15–49 and men ages 15–54, including youth. The list of selected districts is attached in Annex 7.2, Section B. The list of key informants and Focus Group Discussions is attached in Annex 7.2, Section C and D respectively. The list of Health Centers and Hospitals visited is also attached in Annex 7.2, Section E. The tools used for data collection are also attached at end of the report in Annexes 7.10 – 7.12. 2.1.2 Data Analysis Used in Getting to the Answers Data analysis to generate the answers to specific evaluation questions as stipulated in the SOW entailed both quantitative and qualitative data analysis methods. Content analysis was used to analyse and interpret findings from the key informant interviews and focus group discussions. Trend analysis and results mapping were used to analyze the quantitative data. To corroborate findings, the final evaluation also used quantitative survey data drawn from the BCC Evaluation, 2012 conducted by John Hopkins in order take advantage of a much bigger sample. The BCC Evaluation is relevant to the HCP ll final evaluation because it provides the necessary project performance status data in districts that are comparable to both the baseline and midterm surveys conducted by the HCP ll project. Quantitative data analysis was carried out using SPSS on a number of data sets, including the following: 1. The HCP ll baseline data that was undertaken in 2008 with a total of 2,438 respondents by John Hopkins University (JHU) Center for Communication Programs 2. The HCP ll midterm evaluation survey data that was undertaken in 2010 with a total of 2,635 respondents was also undertaken by JHU. 3. The BCC Evaluation, 2012 by JHU with a national representative sample of 7,593 respondents. However, the analysis for this HCP ll final evaluation is based on a sample of 2,246 respondents from the eight districts (Apac, Arua, Bushenyi, Hoima, Kampala, Kamuli, Lira, and Soroti), which matches those covered in the baseline and the 2010 survey. The various data sets were analyzed independently to obtain indicators for trends on knowledge, behavior change, and practices related to HIV/AIDS, sexual behavior, SMC, malaria, and family planning. The datasets mentioned above targeted respondents in a reproductive age category of 15 to 49 years for women and 15 to 54 years among men. In addition, the team used the Uganda Demographic and Health Survey (UDHS) reports of 2006 and 2011 as a reference. Through triangulation, the findings of this final evaluation report can be said to be rigorous and trustworthy. The findings are representative of the target audience reached and their qualitative views and quantified responses on the HCP II project performance is corroborated from the different sources. 2.1.3 Evaluation Data Limitations/Constraints The data presented in this final evaluation report is based on a mixed-methods approach, which is both qualitative and quantitative. Although qualitative data cannot be generalized to the entire target population, it is useful in providing a reflection of the true perceptions of beneficiaries. The inherent limitation is that it does not necessarily 5The two districts that were left out were not covered in the BCC Evaluation, 2012 survey. Page 5 establish causality6 and should therefore be treated as a snapshot (anecdotal) of the beneficiary situations. Data limitations for the quantitative data are those specifically linked to the HCP 2008, 2010, & BCC Evaluation, 2012. Since these surveys were conducted in districts where other partners were conducting similar and additional BCC activities to what HCP was working in, the data limitation is that some of the achievements were not solely a result of HCP interventions. It is therefore important to acknowledge the contributions made by other USG and USAID￾funded partners to the main outcomes of BCC. These include but are not limited to the following: 1) AFFORD (meaning affordable health choices) project supported the Sexual Network Campaign. This promoted several other campaigns such as Genext, Small Families Advocacy Campaign, and the “Good Life" initiative, which encourages people to view health choices as a part of creating the good life they are seeking. 2) Health Initiatives for the Private Sector (HIPS) Project supported the “Good Life at Work” Communication Strategy, which was developed in collaboration with the AFFORD Health Marketing Initiative. This promoted simple steps one can do every day to keep healthy and save money, thus improving the quality of life for employees, their dependants, and community members through daily health practices and decisions. HIPS was also implementing interventions that were geared towards increasing access and use of private sector services in the areas of HIV/AIDS TB, family planning, and malaria. 3) Stop Malaria Project (SMP) supported the Training of VHTs to carry out community sensitization and house-to-house visits providing information about prevention and treatment of malaria. 6 Note: The purpose of this evaluation was not focused on establishing causality but more on determining effectiveness of BCC approaches and lessons learned. Page 6 FY 2012 PERFORMANCE (TARGET VERSUS ACTUAL) # INDICATOR TITLE BASELI NE YEAR BASELINE VALUE 2012 TARGET 2012 ACTUAL Performance over Target OUTCOME INDICATORS STRATEGIC OBJECIVE: COMMUNITIES, FAMILIES AND INDIVIDUALS ADOPT PRACTICES THAT IMPROVE THEIR HEALTH 1 PERCENT OF MEN AND WOMEN 15 – 49 YEARS OLD WHO TOOK AN HIV TEST AND RECEIVED THEIR RESULTS IN THE LAST 12 MONTHS 2008 Male = 25.6% Male = 40% Male = 61% 21% Female = 38.3% Female = 50% Female =76% 26% Overall = 32.1% 45% 68% 36% 2 PERCENT OF SEXUALLY ACTIVE MEN AND WOMEN AGED 15-49 YEARS WHO KNOW THE HIV STATUS OF THEIR SEXUAL PARTNER(S) 2008 Male = 54.6% Male = 62% 60% -2% Female = 47.3% Female = 55% 46% -9% Overall = 50.7% 58.5% 53% -6% 3 PERCENT OF UNMARRIED YOUNG MEN AND WOMEN AGED 15-24 YEARS WHO HAVE NEVER HAD SEX 2008 Male = 43.4% Male - 55% 47% -8% Female = 62.3% Female = 60% 48% 12% Overall = 51.6% 60% 47% -10% 4 PERCENT OF SEXUALLY ACTIVE MALES AND FEMALES (15-49 YEARS OLD) WHO HAD SEX WITH ONLY ONE PARTNER IN THE PAST 12 MONTHS 2008 Male = 71.8 % Male = 80% Male =70% -10% Female = 93.5 % Female = 99% Female =78% -21% Overal = 82.5% 89.5% 72% -18% 5 PERCENT OF WOMEN AGED 15-49 WHO ARE CURRENTLY USING ANY MODERN FAMILY PLANNING METHOD 2008 25% 34% 35% 1% 3.0 HCP II FINAL EVALUATION FINDINGS Evaluation Question 1: To what extent did HCP II achieve each and all expected outcomes under the three intermediate results? 3.1 Extent of Achievement of HCP II Project Results 3.1.1 Mapping Achievements to HCP II Projects Results Framework The HCP II Results Mapping (Figure 2 on P.7 below), is based on the logical framework analysis, which illustrates progress achieved in expected results from specific interventions implemented by HCP ll. The results framework allows for assessing the cause and effect linkages between the inputs/processes and the results achieved under each of the three intermediate results (IRs). Progress achieved towards the IRs and sub-IRs was assessed using data extracted from the baseline, midterm evaluation and BCC Evaluation, 2012 and other project monitoring data against HCP II’s Performance Management Plan (PMP) targets. Findings indicate that performance was achieved for the following PMP results:  Strategic Objective (SO) level: Table 1: FY 2012 Performance (Target Versus Actual) Trend analysis indicates that only two out of the five performance measures originally selected to measure achievement of the objective (Annex 7.3) show a positive trend in adoption of practices by individuals from the baseline in 2008 to 2012. These include: (i) percent of men and women 15–49 years old who took an HIV test and received their results in the last 12 months, from 38 to 76 percent for women against the Life of Project (LOP) target of 50 percent; and for men from 26 to 61 percent against an LOP target of 40 percent; and ii) percent of women aged 15-49 who are currently using any modern family planning method, from 25 to 35 percent against LOP target of 34 percent. The three measures, whose targets were not achieved include: (i) percent of sexually active men and women aged 15–49 years who know the HIV status of their sexual partner(s), from 51 to 53 percent against the LOP target of 59 percent; (ii) percent of sexually active males and females (15–49 years old) who had sex with only one partner in the past 12 months, from 83 to 72 percent against LOP target of 90 percent; (iii) There is a decline in the percent of unmarried young men and women aged 15–24 years who “have never had sex,” from 52 percent in 2008 at baseline to 47 percent in 2012 against the LOP target of 57 percent. More analysis on association of these results with HCP interventions is discussed in subsequent chapters. Table 1, illustrates the FY 2012 performance at the SO level for these same indicators, which also shows that only two out of the five targets were either met or exceeded. There are gender differences for percentage changes for each measure, with the rate of change for females being less than that for the males in most cases and thereby a need for greater targeting of women. Page 7 Figure 2: HCP II Result Framework-Performance Mapping Based on FY 2012 Targets LEGEND: Exceeded 110% Met (90-110% of target) Not Achieved (<90% of target) No Data IR 2.1: Increased public support for services & practices that promote health IR 1.1: Improved attitudes, risk perceptions, normative beliefs & knowledge about services & practices that contribute to health IR 3.2: Increased involvement of communities in designing and implementing health communication programs IR 3.1: Strengthened partnerships and resource sharing among partners in health communication IR 2.2: Strengthened community capacity to address barriers to uptake of health practices IR 1. 2: Increased exposure to information & messages about services, practices, laws, policies & strategies that contribute to health SO: COMMUNITIES, FAMILIES, AND INDIVIDUALS ADOPT PRACTICES THAT IMPROVE THEIR HEALTH IR 2: Supportive Social Environments Fostered to Enable Positive Health Behavior IR 3: Increased Capacity for Sustained Health Communication IR 1: Improved Ability and Motivation to use Services and Practices that Enhance Health Page 8 Intermediate Results (IR) Level: The evaluation team was unable to map and analyse the extent of achievement of project outcomes due to missing data for indicators designed to measure the three Intermediate Results (IR) levels. Absence of such data weakens the analysis of cause and effect logic between the IR level and SO due to lack of credible evidence. The gap also brings into question the quality of performance management at the IP (HCP II) level and overall program level within USAID if there was no data being collected and utilized to track achievement of important project outcomes during the course of the project and guide relevant decision making and course corrections as may have been required. There was room for the HCP II to review and modify key performance indicators in liaison with USAID. Despite this weakness, performance at the sub-IR and output levels can and was used as proxy data by the evaluation team to assess their contribution towards the achievement of the IRs. It is important to note that using lower level data for this analysis in no way replaces the need for the real IR level data that better informs the BCC outcomes.  IR 1 - Effective communication strategies designed and implemented to increase appropriate use of services and/or practices across identified priority programs: five out of six performance indicators that measure IR 1.1 (Annex 7.3) show that the trend except for the indicator: “percent of 15–49 year olds in project areas with comprehensive knowledge about HIV/AIDS transmission and prevention” which has stagnated. There is a slight decline in knowledge about HIV/AIDS from 56 percent in 2008 to 53 percent in 2012. There was no data for IR 1.2  IR 2 - Supportive social environments fostered to enable positive health‐seeking behaviors and result in healthier individuals, families, and communities: IR 2.1 had data for only one out of five indicators, which shows improvement in the “percent of currently married men and women 15–49 years old who discuss family planning with their partners/spouses” increased from 45 to 54 percent between 2008 and 2012. There was no data for IR 2.2  IR 3 - Increased capacity for sustained health communication, IR 3.1 on strengthened partnerships and resource sharing among partners in heath communication had data but was not achieved. IR 3.2 on increased involvement of communities in designing and implementing health communication programs indicates performance exceed targets. Specifically, in terms of the number of communities involved in designing and implementing behavior change communication activities. While the target was to reach 125 communities, 601 communities were involved. The performance at the sub-IRs levels may partially explain the achievement made at the SO level, even with missing data at the IR level itself to assess the real contribution. Nonetheless, based on our mixed-methods design, we will present findings from other data sources such as key informant interviews and FGDs that either support (corroborate) or disprove these project results as self-reported by the project for each IR. According to the evaluation team, the indicators with missing data also happen to be those that were not well formulated to measure the respective results both during the design stage and at the onset of implementation. There is a gap in the performance management of this program that needs to be addressed. Note: IR 1 statement differs in several documents. It is sometimes stated as “improved ability and motivation to use services and practices that enhance health” as in the Results Framework below and the HCP 2007–2012 Final Report, May 2012. For purposes of this report, the evidence presented pertains to the IR 1 statement on “effective communication strategies” as provided in the evaluation SOW. Page 9 3.2 Progress towards Intermediate Result 1 Key Result: IR 1: Effective communication strategies designed and implemented to increase appropriate use of services and/or practices across identified priority programs: The key assumption underlying the achievement of this result is that improving access to information and addressing key knowledge gaps, perceptions, and normative beliefs, among other things will enable individuals and society to adopt practices and behaviors to protect themselves from HIV/AIDS, malaria, and TB as well as reduce high-fertility rates. 3.2.1 Design & Implementation of Effective Communication Strategies The key implementation approach under the HCP II project was to work jointly and support the MoH as a central government partner, other government institutions, and implementing partners (IPs) to develop and/or revise several communication strategies in HIV/AIDS, HIV/TB, family planning, and malaria programs. Instead of designing an overall BCC strategy, MoH as the lead coordinating body with the support of HCP II opted for program specific strategies. The strategies are detailed under Annex 7.4 and these included the following:  HIV/AIDS: (1) Pediatric HIV/AIDS; (2) Treat for Life; (3) Positive Living; (4) Couples HIV Counselling & Testing  TB/HIV: The MoH already had a TB/HIV Collaboration Policy and a Communication Strategy. HCP II supported the Ministry together with TB Control Assistance Program (TBCAP) to review the communication strategy, identify gaps, and come up with a revised message and media plan that guided partners in implementing a multimedia TB/HIV communication campaign.  Family Planning: Male Involvement Campaign and Unmet Needs Campaign  Malaria: Malaria prevention and control The strategies employed a multimedia approach composed of interpersonal communication (peer educators, counselors, National Hotline); mass media (radio, TV, newspapers); and community-based activities (community groups and meetings, village health teams). Various Information and Education and Communication (IEC) materials such as posters, signage, billboards, and comic books as well as job aids were produced. HCP ll rolled out the above stated communication strategies through various programs implemented with different implementing partners. The programs included Y.E.A.H; GOLD; CHCT; Family Planning; SMC; PLHIV; Adult ART/Positive Living; National Health Hotline; Radio Distance Learning program for VHTs; Pediatric ART; Malaria Communication; and TB/HIV Communication. More information on the purpose, scope, and communication channels used under each program is summarized in Annex 7.5. These programs were anticipated to increase the levels of knowledge and practices in HIV/AIDS, family planning, malaria, and TB through campaigns. Desired behavior change outcomes from the campaigns included: (1) HIV/AIDS– consistent condom use, testing for HIV, reduced number of sexual partners, disclosure of HIV status to sexual partners, Safe Male Circumcision, and positive living/antiretroviral therapy (ART) adherence; (2) Family Planning – Use of modern contraceptive methods; (3) Malaria — Use of Insecticide Treated Mosquito Nets (ITNs), among other things. These outcomes and adopted indicators of measurement are consistent with measures defined under the HSSIP Monitoring and Evaluation plan. Specific details on the extent of achievement of results for the expected behavior change outcomes are presented in Section 3.2.2. Page 10 The above health communication program approaches were informed by evidence from various studies, which resulted into the design of tailored campaign strategies targeted at specific health problems, risk factors, age groups, gender, service providers, and partners. Examples of the studies and the campaigns they informed are included in Figure 3. 3.2.2 Increased Knowledge and Behavior Change One of the key expected outcomes from the implementation of campaigns and other communication efforts is increased knowledge of HIV/AIDS, family planning, malaria, and TB prevention and treatment among intended population groups. Knowledge of health behavior and its benefits is a step towards behavior change but is far from being enough to prompt people to act. Quantitatively, the linkage between activities, knowledge, and behavioral change has been tested using cross-tabulation and multiple regression models in order to determine the strength of the relationships. The evaluation findings demonstrate that there was increased knowledge about the various health issues. This section presents findings not only people’s knowledge and behaviors about the different program areas supported by HCP II but also on perceptions of the particular behaviors, misconceptions, and challenges. To the extent possible, the regression model findings will be used to demonstrate whether the achievements made can be associated with the HCP ll interventions. 3.2.2.1 Increased Knowledge Behavior/Practices of HIV/AIDS:  Knowledge HCP ll implemented several programs/interventions that were targeted at improving knowledge, behavior, and practices in HIV/AIDs prevention: Young Empowered and Healthy (Y.E.A.H); Couple HIV Counseling and Testing (CHCT); Medical Safe Male Circumcision for HIV Prevention (SMC); Adult Antiretroviral Therapy (ART); Pediatric ART Family Planning Integrated Communication; and the National Health Hotline. Evaluation findings indicate that comprehensive knowledge7 on HIV/AIDS has declined from 56 percent at the baseline conducted in 2008 and thereafter stagnated at 53 percent in 2012 as illustrated in Figure 4. 7Comprehensive knowledge is defined as consistent use of condoms during sexual intercourse and having just one uninfected faithful partner can reduce the chance of getting the AIDS virus, knowing that a healthy-looking person can have the AIDS virus, and rejecting the two most common local misconceptions (a healthy-looking person can’t have the AIDS virus and that a virus can be transmitted through mosquito or other insect bites, by sharing food with someone who is infected, or by witchcraft or other supernatural means) about transmission or prevention of the AIDS virus (UDHS & AIS). Figure 3: Core Messages from Studies & the Campaigns they Informed Program Area Core Message Study Family Planning Campaign Findings indicated that 41% of women were not using modern family planning methods for fear of side effects. UDHS 2006 The Y.E.A.H Campaign Study found that alcohol consumption before sex increased the risk of acquiring HIV by 67% among men and 40% among women. Study in Rakai in Uganda between 1994 and 2002. The CHCT Campaign Married couples were more at risk of acquiring HIV than the unmarried, accounting for about 65% of new infections. The study also pointed out risk factors such as multiple partners, non-condom use, and non-disclosure of HIV status to sexual partners. SeroBehavioral Survey, 2004-2005 SMC Campaign SMC coupled with less risky sexual behavior could reduce HIV infection by 60% Study on Circumcision in HIV-infected men and its effect on HIV transmission to female partners in Rakai, Uganda, 2007. The choice of radio as a major medium Findings indicated that a high percentage of rural women and men (70% and 87%, respectively) listen to radio UDHS (2006) Page 11 Figure 4: Trend in Comprehensive Knowledge on HIV/AIDS Source: HCP II Baseline (2008), Midterm (2010), & BCC Evaluation, 2012 Comprehensive knowledge is less among women than men as can be seen in Table 2. There is a declining trend in comprehensive knowledge on HIV/AIDS in the urban areas as opposed to an upsurge in the rural areas between 2010 and 2012. Table 2: Change in Comprehensive Knowledge on HIV by Gender and Residence Sex 2008 2010 2012 Male 58.7 52.9 54.1 Female 52.1 46.8 52.8 Type of residence Urban 60.3 62.2 58.1 Rural 52.2 44.8 51.2 Source: HCP II Baseline (2008), Midterm (2010), & BCC Evaluation, 2012 Results from the AIDS Indicator Survey 2011 also indicate that there is stagnation on comprehensive knowledge from 42 to 43 percent for men, and a slight improvement from 31 and 36 percent for women, between 2006 and 2011. Contrary to the quantitative analysis, qualitative data from Focus Group Discussions (FGDs) paints a more positive picture in terms of respondents’ self-reported increase in knowledge of HIV. This was explored through focusing discussions on the following four key HIV interventions:  For Couple HIV Counselling and Testing (CHCT): Findings from the four regions show that both male and female adults knew the importance of couple counselling and testing. Respondents reported that in cases where both husband and wife were found HIV positive, CHCT assisted them with knowledge on how to live positively together. The knowledge gained assisted them to adopt healthy practices that include condom use, ART adherence, reduction in number of sexual partners, and seeking treatment for sexually transmitted infections (STIs). In the cases where both were found to be negative, they said they remind each other about the need to remain faithful. Page 12 Source: BCC Evaluation, 2012; p= 0.020 Source: BCC Evaluation, 2012;  For Safe Male Circumcision (SMC): The results from the FGDs indicated increased knowledge about SMC that was being taken up as a means of preventing HIV among men. The actual practice has been embraced mostly by the youth.  For People Living with HIV (PLHIV): The campaign among adults living with HIV was only implemented from 2011 to 2012. However, the multichannel campaign provided targeted communication to enhance positive living among adults living with HIV. It also helped address challenges faced by adolescents living with HIV and improved the uptake of ARVs among children living with HIV.  For Youth: Through the Y.E.A.H Campaign, young people (15–24 years) were influenced to adopt healthy sexual and reproductive health practices. Findings from youth FGDs indicated that youths know about the importance of key behaviors such as abstinence from sex, sticking to one partner, using condoms consistently at high risk sex, knowing one’s HIV status and that of the partner, drinking alcohol responsibly or not at all, and treating partners non-violently. Table 3: Comprehensive Knowledge on HIV and HCP ll Campaign/Communication Medium Further analysis using cross tabulations by communication activity and its association with comprehensive knowledge on HIV also corroborates the findings from the HCP ll trend data analysis in Figure 4 above. The findings (Table 3) indicate that there is no significant relationship between comprehensive knowledge and the radio drama series, Rock Point 256. Likewise, there was no association between the National Hotline as a channel of communication with comprehensive knowledge. However, findings demonstrate that there is a significant association between the Sexual Network Campaign, with comprehensive knowledge (supported by AFFORD). With regard to the latter, about 49 percent of respondents who listened to the sexual campaigns had more comprehensive knowledge on HIV, compared to 58 percent who did not listen to the program. The significance of the sexual network campaign demonstrates how other programs are also likely to have contributed to BCC outcomes achieved by HCP II. Behavioral Changes Data from the BCC Evaluation, 2012 was tested for the relationship between comprehensive knowledge on HIV and behavioral change. As can be seen from Table 4 on the right, respondents who had comprehensive knowledge were more likely to have one sexual partner, compared to those who lacked comprehensive knowledge. Other data from the AIDS Indicator Survey of 2011 indicate that the percentage of women and men in the 8p-value = probability value helping to measure the level of association between two factors. When p-value is less than 0.05 then there is a positive and statistically significant association between the two factors being investigated. Heard program on the radio called Rock Point 256 for age 15–24 Comprehensive knowledge on HIV p-value8 Yes No Yes 54.8 46.2 0.09 No 45.2 53.8 Heard message in the last 12 months National Health Hotline Yes 25.2 25.1 0.934 No 74.8 74.9 Heard program on sexual network (by AFFORD) Yes 49.1 43.5 0.002 No 51.9 57.5 Number of Sexual partners NOT COMPREHENSIVE COMPREHENSIVE None 9.7 8.6 1 partner 70.2 76.5 2+ partners 18.9 12.9 No response 1.3 2.0 Total percentage 100.0 100.0 Total number of respondents 795 983 Page 13 age range of 15–49 who had two or more sexual partners and were not using a condom during the last sexual intercourse was 84 and 85 percent, respectively. This shows that despite the changes in behaviour due to comprehensive knowledge, there is still a large proportion of the population who still have more than two sexual partners and are not using condoms, an issue that needs to be addressed if HIV rates are to be reduced. There has been a slight decline in the percentage of young (15–24), unmarried individuals who have never had sex in their lifetime from 52 percent at baseline to 47 percent at the end of HCP II, implying that a bit more unmarried youth are having sex. The percentage of young (15–24) sexually active, unmarried individuals who used a condom at last risky sex9 increased for both women and men, from 48 to 64 percent for women and 54 to 58 percent for men (Annex 7.6). The BCC Evaluation, 2012 on condom use at last risky sex for the 25–49 years for both women and men were too few to provide conclusive findings (n=84 for men and n=26 for women). Overall, behavioral change in the population takes time and has not changed much over the HCP II period of implementation (2007–2012).  Knowledge and Practice of Safe Male Circumcision (SMC) Between 2007 and 2011, HCP II supported the MoH to educate the public about SMC and HIV prevention through dissemination of the SMC Policy Guidelines through mass media and public debates. In addition, HCP II designed IEC materials (Posters, Q&A book for health workers, and brochures) for men on SMC, its link to HIV, and the benefits. HCP ll launched the Stand Proud, Get Circumcised Campaign in 2009, which spoke to men and women about SMC. Results indicated that there was a surge in knowledge on SMC as a means of preventing HIV among men, from 31 percent at baseline in 2008 to 80 percent in 2010 after only one year of implementation of SMC interventions (SMC programming started in 2009). There was a slight increase in SMC knowledge from 80 percent in 2010 to 81 percent in 2012, which can partly be explained by the fact that the HCP-supported SMC campaign ended in 2011. Overall, there was an increase over the baseline (Figure 5). This can be attributed to HCP II supported HIV/AIDS campaigns, which were also later supported by the passing of a government policy on Safe Male Circumcision. The national Policy for SMC was launched in September 2010, two years after it was completed, which shortened the period for raising awareness about it. Figure 5: Trends in Knowledge on Benefits of SMC in reducing chances of acquiring HIV/AIDS Source: BCC Evaluation, 2012 9 Risky sex in this survey means: lack of condom use at the last sexual intercourse. Page 14 Source: BCC Evaluation, 2012 Further analysis on the association of campaigns with the practice of SMC indicates that there was significant and a positive relationship between the Stand Proud, Get Circumcised Campaign and SMC practice. 90 percent of youth who were circumcised had obtained information on SMC from the Stand Proud campaign (Table 5). Increasing numbers of SMC were also a result of the scale up the SMC campaign nationwide that was supported by other partners that included Makerere University School of Public Health (MUSPH), STAR E, EC & SW, and IRCU. Interviews with SMC providers at health centers in Western Uganda indicated that the turn up for SMC has increased drastically as a result of massive campaign code named “Circumcision Week/Camp” funded by STAR-SW, another USAID￾funded IP supported SMC. For example, figures from Kyabugimbi HCIV (Bushenyi) show that within just a period of six months from January to June 2012, the number of clients who turned up for HIV testing (5,850) after the campaign were more than what was achieved for the whole of 2011 or 2010, with a total of 3,500 and 3,300, respectively. Figure 6: Male Circumcision by Region However, there were differential outcomes based on age. Male circumcision for the age group 15–24 years had increased slightly from about 20 percent in 2008 to 22 percent in 2012, while there was a decline for the 25–49 years, from 23 to 17 percent for the same period. This is corroborated by observations and exit interviews in Kamuli, which found that demand for SMC is higher among young people than married adults. While the evaluation was not able to establish factors of variations in SMC uptake, AIS findings indicate that there are major regional variations in SMC uptake as can be seen from Figure 6. The highest concentration of male circumcisions was found in Mid-Eastern Region. Further research should be conducted to investigate the drivers for uptake in these regions and understand how these lessons can be used to drive improvements in other regions. Source: Uganda AIDS Indicator Survey 2011  Knowledge and Behavioral Change Among PLHIV The Positive Living campaign was implemented to increase the benefits of positive living and practices among adults living with HIV, address challenges experienced by adolescents living with HIV, and uptake of ARVs among children living with HIV. PLHIV campaign was communicated via several channels, which included the Radio Diary series, My Life My Story and Discussion Guides, Health Workers, the Positive Living Profiling Tool, Peer Educators, and Rock Point 256, among others. A total of 90 PLHIV FGD participants (42 male & 52 female) in 12 FGDs conducted in all the four regions reported living positively. PLHIV said they try to adhere to the treatment, visit health facilities whenever they fall ill, eat a balanced diet despite the economic hardships, and use condoms to prevent reinfection and infecting others. Participants in about two out of the 12 FGDs revealed that they had Stand Proud, Get Circumcised Campaign SMC p-values Yes No Yes 90.0 71.8 0.000 No 10.0 29.2 Heard program on the radio called Rock Point 256 for age 15–24 Yes 79.2 50.1 0.020 No 21.8 49.9 Table 4: SMC and Stand Proud, Get Circumcised Campaign Page 15 Source: BCC Evaluation, 2012 opted to abstain from sex. Participants in about five out of 12 FGDs had HIV negative children at their last birth. However, in all the FGDs, participants reported that it is difficult for adolescents to adhere to ARVs because of fatigue, while others fear to be seen taking ARVs. This was especially the case for those adolescents who had not disclosed their HIV status to their parents. 3.2.2.2 Knowledge and Practices in Family Planning Between 2007 and 2012, HCP ll supported the MoH to implement two campaigns to raise awareness on use and availability of family planning services. These included the Male involvement in Family Planning (Fred and Bernard/ “Neighbors” drama series) and Nurse Mildred drama series. According to the BCC Evaluation, 2012, the percentage of currently married men and women 15–49 years old who discuss family planning with their partners/spouses has increased from 45 percent at baseline to 54 percent in 2012. The evaluation has established that there is a positive relationship between the use of family planning and selected HCP II campaigns. Listening to Nurse Mildred radio drama was statistically significant and linked to current use of a family planning method. About 40 percent of those who were currently using a family planning method had listened to the radio program, compared to 67 percent who had not heard the Nurse Mildred message, were not using any family planning method. These findings are corroborated with findings from the FGDs. Participants in the adult FGDs established that adults knew about the different types of family planning methods (e.g. pills, condoms, implants, and injectables). However, knowledge of female condoms, moon beads, coils, and other permanent forms like tubal ligation was not as common across the various groups. Respondents were also able to cite the benefits of family planning. Those interviewed also identified the benefits of prevention of unplanned pregnancies, reduction of maternal deaths, better feeding, and better education. Findings from the BCC Evaluation, 2012 show that the percentage of young sexually active, unmarried women (15–24 years) who used a condom at last sex increased from about 48 to 64 percent, while that for men in the same age bracket increased slightly from 54 to 58 percent between 2008 and 2012. This is corroborated by data from the UDHS surveys by UBOS between 2006 and 2012, which also shows an increase from 39 to 54 percent for women and from 58 to 63 percent for men. Family size has slightly reduced between 2006 and 2012 from 3.5 to 3.2 for the average number of children according to UDHS data. According to the studies conducted under the HCP program, the percent of currently married men and women (15–49 years) who want to have less than five children also increased from 47 to 63 percent from 2008-2012. In all regions, there was self-reported use of family planning methods among females, and all the married female respondents said they had ever used at least one family planning method. However, some of these women had discontinued the use of family planning due to the associated side effects. For instance, injectaplan was reported to be associated with severe bleeding and pills with disruption of the menstrual cycles, frequent dizziness, back pains, and loss of interest in sex according to a group of eight women in an FGD conducted in Bushenyi. Service Heard Nurse Mildred message in the last 12 months for age 15-54 Currently using family planning method p-values Yes No Yes 40.2 33.4 0.002 No 59.8 66.6 Heard program on the radio called Rock Point 256 for age 15–24 Yes 51.0 52.9 0.949 No 49.0 47.1 Table 5: Association between use of Family Planning Methods and selected HCP Page 16 providers at the health centers visited revealed that these side effects were more pronounced when the users of the different methods were not properly counseled or guided on the appropriate method to use. It was also reported that those who experienced side effects were not in the habit of seeking help from the health centers. 3.2.2.3 Knowledge and practices of Malaria & Tuberculosis Between 2007 and 2008, the HCP ll supported the MoH to increase public awareness of the new malaria treatment policy based on Intermittent Preventive Treatment in Pregnancy (IPTP) and Artemisinin-based Combination Therapies (ACT). MoH designed messages and media plans required to implement the Nation TB/HIV Communication Strategy. The malaria and TB campaigns were implemented over a period of one year.  Tuberculosis (TB) The BCC Survey findings show that about 93 percent of the respondents knew about a disease called tuberculosis (TB) and are knowledgeable on how it is spread as can be seen from Figure 7 below. Figure 7: Knowledge about the spread of TB 76 65 9 7 4 0 10 20 30 40 50 60 70 80 Air when coughing or sneezing Sharing utensils Through food Touching a person with TB Sexual contact How TB Spreads Percentage Respondents were also knowledgeable about the symptoms of TB including cough for two weeks (80%), weight loss (33%), and pain in chest (30%) as illustrated in Figure 8. FGD participants in the communities visited were knowledgeable about the linkage between TB and HIV/AIDS and were aware of some of the symptoms of TB. They revealed that because of their knowledge, when a cough persists for more than three weeks, they normally go for TB Screening. Participants also knew that TB is a curable disease and that the course of treatment is long. TB testing and treatment service is available at the health centers; seeking treatment early helps one not to develop complications like vomiting blood and lung problems. Adults who have ever tested positive for TB knew the importance of adherence to drugs. Figure 8: Knowledge on Symptoms of TB Mentioned by Respondents 80 33 30 17 17 6 5 4 0 10 20 30 40 50 60 70 80 90 COUGH FOR 2 WEEKS OR MORE WEIGHT LOSS PAIN IN THE CHEST COUGHING BLOOD SPUTUM BODY WEAKNESS LOSS OF APETITE EVENING FEVER SWEATING AT NIGHT Symptoms of TB Percentage Page 17  Malaria Evidence presented from the FGDs indicated that communities visited were knowledgeable about malaria. Findings also show that knowledge on the causes of malaria and its prevention was common across all regions. The increased knowledge has translated into behavioral change in terms of an increasing trend in the ownership and use of mosquito nets. FGD participants also knew that sleeping under an Insecticide Treated Mosquito Net (ITN) reduces one’s chances of getting malaria. The use of nets as an adoption of behavioral change practices can be attributed to the increased knowledge about the causes of malaria and its prevention. The BCC Evaluation, 2012 did not explore the questions on use of mosquito nets. Drawing from the national context, results from the UDHS datasets (2006 & 2011) indicate an increasing trend in the use of any mosquito net in a household from 34 percent to 74 percent. For those who own at least one ITN in a household, there is an increase from about 16 percent to 60 percent. Further analysis of data by region indicates that the percentage of households with at least one mosquito net is highest for West Nile (88%), followed by Kampala (82%), Western (78%), Northern (75%), Eastern (73%), South Western (72%), and lastly by Karamoja (68%). 3.3 Progress towards Supportive Social Environments (Intermediate Result 2) The presence of a supportive social environment includes policies, laws, and community norms, which are necessary for families and individuals to adopt positive health practices, while access to services provide the linkage between communication activities and eventual adoption of health practices. HCP II designed and implemented communication approaches (Annex 7.7) that were meant to stimulate recognition of health issues, to debate and dialogue about health issues, and to identify and implement locally owned solutions. This was expected to result in services, policies, and community structures that support adoption and maintenance of positive health behaviors and discourage risky behaviors/practices. 3.3.1 Public Support for Services and Practices that Promote Health A public supportive social environment entailed putting in place policies and structures that would encourage communities, families, and individuals to adopt practices that improve their health. These included community (leaders, adults, youth group roles) and public structures (health facilities, providers, and relevant policies) that were necessary to tackle social issues and barriers such as alcohol abuse, violence against women, discussion among couples about HIV status, SMC, and family planning. HCP ll supported MoH and the Uganda AIDS Commission (UAC), which were responsible for not only supporting policy formulation but also providing direction, execution, leadership, and strategy development to facilitate implementation of health communication programs. The lead implementing partners (AIC & CDFU) mobilized communities and coordinated HCP ll activities that targeted behavior change. AIC focused mostly on couple HIV/AIDS counseling and testing, while CDFU implemented Y.E.A.H programs that included National Health Hotline and Rock Point 256. HCP II supported the roll out of the new policies on which health campaigns were based. For example, the HCP campaigns promoted the awareness of the SMC, alcohol, and TB policies among the public to encourage people adopt healthier practices. Table 7 below shows trend data from the HCP ll surveys from baseline, midterm, and the BCC Evaluation, 2012, which indicates that there is increased awareness about some of the government policies put in place in support of HCP II interventions for SMC and alcohol abuse. However, there is decline in TB awareness, which as mentioned in the previous section may be attributed to the short (one year) duration of the TB campaigns. Page 18 Table 6: Data on Policy-Related SMC, Alcohol, and TB- 2008 – 2012 Indicator HCP II Baseline (2008) percent HCP II Midterm (2010) percent BCC Evaluation, (2012)percent Percentage who heard about Uganda policy to encourage medical male circumcision N/A 72.5 88.2 Percentage aware of any law or local policy regarding alcohol 36.3 N/A 41.4 Percentage aware of any law or government of Uganda policy regarding TB 36.2 N/A 26.8 Source: HCP II Baseline (2008), Midterm (2010), &BCC Evaluation, 2012 3.3.2 Youth supportive structures Young Empowered and Healthy (Y.E.A.H) Campaigns were designed to raise consciousness and promote dialogue about social issues such as alcohol abuse, violence against women, HIV stigma, and smaller family size as well as discussion about HIV status between couples, male circumcision, and family planning. They were implemented between 2007 and early 2012 to encourage young people to abstain from sex, stick to one sexual partner or consistent use a condom, know their own and partner’s HIV status, drink responsibly or not at all, and to treat partners nonviolently. The campaign was a continuation of the initiative started in 2004 and on Rock Point 256 (weekly radio drama series) and comic book series and youth groups such as the Young People Advisory Groups (YAGs). According to the BCC Evaluation, 2012, 51 percent of respondents aged 15–24 said they had listened to Rock Point 256. The latter started in 2004 under HCP and continued in 2007 to 2012 under HCP II. Eighty-two percent read comic books (shortened story versions of the Rock Point 256 drama series) in the past 12 months, compared to 43 percent for the 25–49 years olds and 43 percent for those 54 years and above. The Y.E.A.H initiative was set up under the auspices of the UAC and CDFU as the lead implementing partners targeted the youth as their main audience. Y.E.A.H initiatives were instrumental in setting up youth structures such as YAGs, which worked with Regional Lead Organizations (RLOs) to carry out peer education in schools and communities using IEC materials. YAGs provided input into the work of the Technical Advisory Team (TAT), the Implementation Unit at CDFU as well as acted as the gateway for regional lead partners (RLOs). The latter included Save the Children Fund (SCFU, SCFUSA), Reproductive Health Uganda (RHU), Uganda Red Cross (URC), and AIDS Information Center (AIC) as well as youth clubs and schools in the districts and communities. YAGs delivered messages on HIV/AIDS and family planning. The youth group volunteers trained by HCP ll as peer educators facilitated group discussions among the youths using comic books and videos to trigger discussions. Up to 81 percent of the survey respondents said they had in the past 12 months seen the comic books distributed through the youth structures.  Young people’s sexual and reproductive health knowledge, attitudes, and self-perception of risk improved As a result of the Y.E.A.H campaigns, young people’s sexual and reproductive health knowledge, attitudes, and self-perception of risk have greatly improved. In all the FGDs focused on youth, participants in 10 out of the 12 FGS were knowledgeable about the use condoms, SMC, responsible alcohol consumption, and treating partners nonviolently. Youth reported that they were trying to abstain or stick to one partner but noted that it was challenging. Below are quotes from FGD participants on risky sexual behavior, GBV, and alcohol abuse: Page 19 “When one is drunk, sometimes they don’t listen to their partner’s plea on using protection and when the partner insists, they may end up fighting and breaking the relationship” (youth respondents in Mukono, and Kabarole).“Alcohol puts you in mood for sex and you can’t take precaution” (youth respondents, Arua, Kamuli, and Soroti).  Communication between adults and young people strengthened to provide nurturing environments for the youth Findings from FGDs show that communication between adults and young people has not been strengthened. Adults are not comfortable sharing information with young people about sexual behaviors. On the other hand, some adults who engage with young people sexually are more likely to expose them to risky sexual behaviors such as not using condoms. Adults FGDs mainly supported abstinence, which is not necessarily embraced by active young people. Such insights reveal that there is still a gap in communication between adults and young people. 3.4 Effectiveness of Project Approaches This section looks at the effectiveness of one single broad and focal communication body supporting the communication activities of different implementing partners and the effectiveness of approaches HCP ll used to implement its various programs. 3.4.1 Effectiveness of the One Focal Communication Project Supporting Different Implementing Partners HCP II was conceived by USAID as one focal health communication project designed to support communication activities of various partners in the health sector. The 66 implementing partners included USAID-funded projects, government, institutional, and other implementing NGO partners. Respondents from the government and institutional partners, i.e. UAC, AIC, CDFU, MUSPH, MoH agreed that HCP ll as a focal communication project was an effective model of addressing health issues because it promoted uniformity and a centralized management system. This approach limited the duplication of activities and conflicts between the implementing partners. The model using participatory planning meetings brought together a diversity of government (MoH, UAC) and institutional (AIC, CDFU, MUSPH) partners in health, involved them in formative research and design of materials, and ensured that local partners also got similar training and IEC materials. The approach helped the MoH coordinate communication activities related to HIV and family planning. Working together resulted in single communication strategies for the same health problem instead of having parallel ones. Respondents at UAC, AIC, CDFU, and partners chose activities to implement as well as channels and messages to use based on shared evidence-based research coordinated by HCP II. This, they say, enhanced the uniformity in the messages communicated by different partners as well as a pool of designed material that they can reproduce whenever they need them. USAID & USG implementing partners interviewed during the course of this evaluation reported that HCP II as a focal organization on communication helped coordinate the process of identifying and training service providers in ART, SMC, CHCT, and VHTs along the communication process implemented by various partners. Findings from USAID and USG implementing partner staff indicate that all of them were involved in the design of HCP ll health communication interventions in some way. For instance, RHU was involved in planning meetings, SUSTAIN with partners in the area of Pediatric HIV, and Marie Stopes participated in developing FP posters, flip charts, radio dairies and drama series. TASO was involved in the design of communication materials on positive living, radio dairies, support messages on positive living, supporting PLWHAs. In addition, TASO supported MoH with other partners in developing the National pediatric curriculum on pediatric counseling, developed a pediatric strategy on communication, participated in family planning strategies and making factsheets for positive living, pediatric care Page 20 and SMC. The inherent benefit in this approach was the cost savings made in avoiding duplication of efforts if the trainings, design and dissemination of messages across uniform programs had not been properly coordinated. Working through partnerships also helped reduce the costs of implementing some campaigns in the regions, districts, and communities. For example, CDFU reported that without the regional partners such as the Regional Lead Organizations (RLOs), an initiative funded under HCP II, it would have been difficult and very costly to put in place and maintain Y.E.A.H structures. Partners such as SCU, AIC, and RHU helped provide office space for YAGs, CD/DVD players, and the printing of comic books in four local languages (English, Luganda, Luo, and 4Rs). The distribution of IEC materials was effected through partners who were responsible for cascading them down to health facilities and community level. Project performance data indicates that 2,254 leaders, media workers, and community resource persons received HCP-supported IEC materials exceeding their planned target of 700 people over the course of the project. Several key informants from IPs, MoH, and other institutions confirmed receiving and utilizing the IEC materials. The majority of FGD participants who said they had seen IEC materials saw them at the health facilities. Participants in the Village Health Team FGDs also acknowledged the contribution of IEC materials in terms of knowledge sharing and training of counselors that have brought support and care to the affected families. “IEC materials have assisted a lot because seeing is believing,” as one VHT member put it. A MoH respondent commented that advocacy videos were pretested and relevant. An AIC respondent noted that “people like videos because the testimonies in the video influence couples to come for testing.” For CHCT, empowered testimonies and certificates for tested couples were an effective approach. Evaluation findings also indicate that HCP II did not have a mechanism in place for monitoring the distribution and use of IEC materials by the IPs. Some IPs reported delays in receiving IEC materials, while some district health educators in Soroti said they did not receive adequate materials. The evaluation team saw posters at HCT and ANC service centers at the several health facilities visited around the country during the course of the evaluation. The 18 HCs the evaluation visited had all the signage on CHCT, FP, SMC, and TB developed under the HCP II project. The Ministry and other IPs appreciated that they now have access to the electronic versions, which they can always reproduce. These materials can be found at the following web link www.k4health.org/toolkits/hcp. While this is the case for the USAID implementing partners with programs covering large parts of the country, budget limitations especially in the Ministry and among the local partners inhibit them from producing sufficient quantities of materials for mass reproduction. 3.4.2 Effectiveness of Knowledge and Behavioral Change-Related Approaches As reported under IR1, all the campaigns increased knowledge and to some extent contributed to behavioral changes. Campaigns, especially those that used a strong component of radio and the drama formats plus radio spots, were more popular in their outreach to the target audiences than others as indicated by the findings from the BCC Evaluation, 2012 reflected in the sections below.  Y.E.A.H Campaign The Rock Point 256 Radio program, which was part of the Y.E.A.H campaign, won several awards such as the AfroComNet Award for Excellence in HIV Communication in 2007, Best HIV/AIDS Program at the Africa Edutainment Awards in 2011, and the Best Radio Program by New Vision readers in 2010. Young, urban participants in FGDs cited radio, especially Rock Point 256, as their major source of information on HIV. Key informant interviews at radio stations revealed that although their listenership surveys are based on time segments Page 21 and not individual programs, audience feedback showed that Rock Point 256 had increased station listeners, especially the youth. The reasons given for its effectiveness were the entertainment format, use of young characters that the youth audiences easily identify with, plus drawing on familiar challenges that youth face in the communities. Figure 9: How messages from Rock Point 256 influenced actions According to the BCC Evaluation, 2012, respondents said that messages they got from Rock Point 256 influenced them to take actions such as: abstained from sex; use condoms; stay faithful to one partner; reduced number of sexual partners; test for HIV with partner; and used family planning (Figure 9). It is important to note that analyses in previous sections showed that there was no significant association between Rock 256 and comprehensive knowledge on HIV. However, there was a significant relationship (p=0.02) between comprehensive knowledge on HIV and behavioral practices like reduced number of partners. While there were positive ratings and evidence of changes that can be associated with Rock 256 program, sustainability of such initiatives after the close of the program is weak. Already, some radio stations that had continued to air Rock Point 256 on a voluntary basis have since stopped. This leaves a major gap in the fight against HIV among the young people that needs to be filled by improving and replicating aspects of Y.E.A.H that worked. Youth FGDs mentioned peers and friends as another key source of information on HIV/AIDS and reproductive health. According to BCC Evaluation, 2012, 27 percent listed youth clubs as well as a source of information on family planning in the past 12 months.  Couple HIV Counseling and Testing (CHCT) Between 2008 and 2012, HCP ll supported the MoH and other partners to develop national policy guidelines and training materials on Couple HIV Counseling and Testing (CHCT). Using a multimedia campaign, “Go Together, Know Together,” which run from 2009 until 2012, encouraged couples to talk to partners about going for HIV testing, getting tested together, and then disclosing the results to one another. The rationale behind the “Go together Know together” Campaign was that by getting tested together, couples who did not know their HIV status would recognize their risk of HIV, and this would build their belief that CHCT will benefit their relationships. The CHCT Campaign approach registered significant reach among the target audiences. According to the BCC Evaluation, 2012, up to 59 percent of respondents had in the past 12 months seen or heard of a sign of a couple holding hands, and 58 percent of them said they understood it to mean as testing together for HIV (couple testing). Similarly, up to 60 percent said they had in the past 12 months seen or heard messages about the “Go Together, Know Together” Campaign. The messages reached couples mostly through radio (65%), followed by posters (32%), health workers (13%), and bill boards (10%). Up to 61 percent of the respondents said they were influenced to take an action by messages they got from the “Go Together, Know Together” Campaign (Figure 10). The finding that those who got the “Go Together, Know Together” Campaign messages were influenced to take some action confirms a reasonable level of effectiveness of the approach. Page 22 Figure 10: Actions taken from the Knowledge gained from Go Together, Know Together Campaign 33 24 20 19 19 8 0 5 10 15 20 25 30 35 Talked to my partner about his/her HIV status Talked to my partner about going for couples HIV counseling and testing Talked to my partner about my HIV status Went for HIV counseling and testing on my ow n Went for HIV counseling and testing together w ith my partner Talked to someone else about going for couples HIV counseling and testing Benefits of Go Togther, Know Together Percentage However, there were also cases of misinterpretation of the CHCT poster. When shown the “Go Together, Know Together” poster during fieldwork by the evaluation team, a female respondent in Arua said, “It’s a sign for toilets, which separate female from male toilets.” Survey information presented earlier indicated that HIV testing increased for men and women as well as those who know the HIV status of their sexual partners. This can be attributed to the fact that the promotions for CHCT were good and this led to an increased number of people seeking CHCT services. In addition, the HCs had a policy of testing every patient who reported with any sickness for HIV. While the demand for HIV testing increased as a result of the CHCT Campaign, conducting the HIV tests was affected by the constant stock-outs of HIV testing kits at health centers, which discouraged many service seekers. Survey information indicates that HIV testing increased from 38 to 76 percent for women and from 26 to 61 amongst men. The percent of individuals who know the HIV status of partner increased from 51 percent at baseline (2008) to 53 percent in 2012. It is important to note that while there have been improvements, these percentages are still low and a lot needs to be done in order to increase these levels close to the national targets. For HIV testing for adults 15-49 years who have ever tested, the national target is expected to 80 percent by 2015 as indicated in the National HIV Prevention Strategy for Uganda: 2011-15, Draft Version, February 2011. While women are getting close to achieving this target, men are still lagging behind.  Nurse Mildred and Neighbors Fred and Bernard The BCC Evaluation, 2012 found that 40 percent of the respondents had heard or seen messages about Nurse Mildred in the last 12 months. Findings show that those exposed to the Nurse Mildred Campaign got the key messages. Figure 11 illustrates the actions taken as a result of these key messages, which confirms that behavior change can be influenced with increased knowledge about health issues, in this case family planning. About 40 percent respondents reported that they believe that family planning is safe and effective. Radio was the major source of the information above: radio drama (41%); radio talk show (51%); and radio spots (30%).The radio drama/talk format also proved effective as it involved a nurse to put the drama into context and also respond to issues arising from listeners through calls. A host for a radio talk show, which carries Nurse Mildred at ETOP FM, said people in Soroti town have nicknamed her Nurse Mildred because of the popularity of the program. Page 23 Figure 11: Messages heard by Respondents about the Nurse Mildred Campaign 40 38 29 29 17 16 12 0 5 10 15 20 25 30 35 40 45 They are safe & effective Visit a health facility for more information Avoid rumors Get the facts from a health w orker The w ide range of family planning methods Couple communication is important There are millions of satisfied users in Uganda today Benefits of Family Planning Percentage In comparison to Nurse Mildred, the Fred and Bernard Campaign was less effective. FGD respondents could hardly recall having listened to it. This could be attributed to the fact that the Nurse Mildred Campaign was more recent (2011–2012) than Fred and Bernard (2006–2009). Posters were cited as having been seen on both Nurse Mildred and Fred and Bernard for family planning. However, FGD findings indicated that the level of comprehension among both male and female respondents for the Nurse Mildred poster was higher than the Fred and Bernard one. This is attributed to the strong visual appeal (pictures), which at a glance communicates the message about the burden of having a large family size beyond ones means (Nurse Mildred is the picture on the Left). A lady in Western Region was able to recognize the family planning poster with a picture of a nurse beside a stressed large family. Nurse Mildred Poster Fred & Bernard Poster The Fred & Bernard picture on the right was not as well understood. One respondent in Kabarole who mistook the Fred and Bernard poster to be about nutrition said,” I have seen that picture by the roadside. It is teaching us about a healthy diet and the need to work hard for what one wants in life.” 3.4.3 Behavioral Change Communication Campaigns and Health Service Delivery Behavior change communication messages were disseminated to help educate and encourage audiences to take measures to protect themselves and communities against infections. They also encouraged them to seek health services such as HIV testing and treatment or Safe Male Circumcision from health facilities. While prevention requires that the individuals take personal protection measures, service-seeking behaviors require that the services promoted by communication are available, accessible, and Page 24 affordable at the health facilities people are directed to. Observations in the field indicated that signages for CHCT, family planning, SMC, and TB at all the 18 HCs visited were available and visible. (Signage at Goma HC II, in Mukono in the picture). Respondents in both exit interviews at 18 health centers and FGDs said the signage helped service seekers to know which services were being offered at the health center. The evaluation discovered that signage does not necessarily ensure that the services advertised are actually available. Respondents said that their preferred family planning methods were sometimes not available. The health center staff decried shortage of staff, HIV testing kits, and contraceptives. For instance, the evaluation team found over 15 youths who had come for SMC at Namwendwa HC 1V waiting for several hours without being attended to because the trained staff had long queues of outpatients. In Northern Uganda, it was revealed that health providers are too few compared to the number of males who want to get SMC. In the Eastern and Western Regions, where SMC was matched with steady provision of the service supported by two USAID programs i.e. STAR – South West and STAR-East), there was a much more rapid increase in the number of people seeking the service. Health service providers at various HCs visited noted that the number of women seeking family planning methods increased on the days when Marie Stopes was scheduled to be at the health facilities. This was because Marie Stopes provides a wider choice of family planning methods than the pills HCs often dispense. Health service delivery was therefore being impacted by low staffing levels and availability of products at the health centers. This is a country wide situation. Annual Health Sector Report, 2011/12 reported that only 58 percent of the approved positions are filled by health workers, up from 56 percent in 2011. The Service Availability and Readiness survey in 2012 indicates a general readiness score for health facilities at 66percent, looking at the availability of basic amenities, basic equipment, standard precautions for infection and prevention, diagnostic capacity and essential medicines. However, the report notes a decline in financial investment in the health sector from about 10 percent in 2009/10 to 8 percent in 2011/12. These examples demonstrate that though communication interventions may instill healthy behaviors (e.g. encouraging people to go for testing, use contraceptives, etc.), the health services that will support this behavior change such as SMC, counseling and testing, and family planning need to be readily available and affordable to enable utilization and enjoyment of full health benefits. 3.4.4 Generating Opportunities for Leadership and Professional Development (GOLD) To bridge the skills gap on health communication among implementing partners, HCP II through the GOLD program set out to increase the number of young Ugandan professionals with the required job skills. The GOLD program targeted young graduates who had career interests in health communication and provided training and one-year work-based mentoring. The program trained 200 young professionals, but only 120 completed the one year placement and mentoring sessions. The call for applicants was advertised through universities, in newspapers before resorting to posters, Facebook, and the project website due to the high media advertising costs as reported by HCP ll staff. The GOLD program was run by HCP II with the involvement of IPs on a voluntary basis. Meetings and training seminars were held at HCP II boardrooms but sometimes were rotated and hosted by various IPs. HCP II put emphasis both on skills development and finding placements for those recruited. The evaluation team followed up with 30 graduates of the GOLD program, who could be easily traced to assess effectiveness of the GOLD program, and extent to which GOLD provided them with the necessary health communication skills, career guidance, and opportunities to young graduates. The findings indicate that there were significant (p=0.002) results in a short time and with very minimal resources. 93% (Twenty-eight out of 30 respondents interviewed) of the graduates are currently employed in influential positions with credible organizations mostly in health communications positions. These include communication and training, marketing, strategic communication, and communication research, among others. The organizations that have employed Page 25 them include USAID-funded projects and former implementing partners (IPs) of HCP II such as Marie Stopes, STRIDES, UHMG, CDFU, RHU, MSH, HIPS, and Stop Malaria. A few graduates have moved on to business companies (SHELL, private business) and the media sector (The Monitor and Red Pepper). About 64 percent of the beneficiary respondents said they were very satisfied with the extent to which the GOLD program provided them with the skills needed to secure professional employment and succeed in their new position. In terms of GOLD as an effective approach for beefing up communication skills for implementing partners, the uptake of GOLD graduates by IPs seems to have been achieved. The IP’s ability to hire and retain the young professionals will be a big factor in sustaining the gains of this approach plus their willingness to host future GOLD trainees. A number of them sighted budgetary constraints to hire and fill communication positions at their organizations. In 2012, the program was passed on to the Private Sector Foundation of Uganda (PSFU) which showed more interest in running it. While PSFU has continued to manage it, the prospects for its sustainability remain hinged on securing extra funding. The PSFU says they plan to charge young graduates tuition for the next enrolment in March 2013. The program also faces the challenge of finding placements for those enrolled. Part of the problem is that it puts emphasis on the skills development while leaving the hurdle of placements to the young professionals themselves. This is partly responsible for the high drop-out (15 out of 30) in the 2012 intake. Therefore, the long-term sustainability of the GOLD program is in question. 3.4.5 Radio Distance Learning (RDL) for Village Health Teams The Radio Distance Learning program (RDL) was designed as a low-cost refresher and continuing training to Village Health Teams (VHTs) to enable them understand their roles in providing information and services in the target areas. VHTs are also community-based health structures, through which the communities participate in the management of health services. The VHTs are now referred to as Health Center One (HCIs) because they offer first line health services treating manageable diseases and refer cases to HCIIs, mobilize communities to seek health services, and take preventive measures. The design of Radio Distance Learning program started in 2009 but the actual broadcasts commenced towards the end of October 2010 and ran up to 2012. RDL involves VHT listening groups sitting around radio sets at the home of one of the members using guide books to follow and learn from the weekly radio broadcast sessions about issues such as HIV/AIDS, family planning and how to support communities to deal with these health challenges, among others. The program works with and supplements current VHT training efforts by providing continuing education, refresher training and transfer of knowledge and skills to VHTs on a regular basis and in a cost effective way. RDL was implemented in six districts in Central and three districts in Northern Uganda. According to the Annual Health Sector Performance Report, 2011/12, there is a total of 66,321 VHTs countrywide; out of which there are 39,947 active ones. According to the HCP project PMP data, 12,483 individuals in FY 2011 participated in capacity building activities for VHTs, and another 4,000 in FY 2012. As such, the costs for conducting face-to-face training would have been exorbitant had it not been for using the RDL approach. Overall, all FGDs in the Northern and Central Regions showed that the VHTs were knowledgeable and could comfortably name at least three benefits of using various health practices, in which they were responsible for providing information. They also indicated that they now appreciate better their added role to disseminate health related information to the communities they serve. Lack of batteries for the radio sets to follow the learning and having to share one discussion guide book during the group listening and discussion sessions were some of the complaints raised by the VHTs during RDT (Annex 7.9). World Vision has adopted the Radio Distance Learning (RDL) and has introduced child and maternal health in the training content for the VHTs. The VHTs in Masaka felt that the RDL is performing better under World Vision support. This was attributed to the fact that VHTs preferred face-to-face training provided by World Vision, where they could ask questions and understand better as compared to the RDL offered by HCP II. Face to face Page 26 interactions are necessary for adult learners who are not very illiterate as is the case in Uganda10. In addition, World Vision provided feedback on their performance and also conducted refresher trainings. World Vision supported MoH to produce 13 radio sessions and discussion guides on maternal and child health and increased the number of districts covered. Though efficiency gains through the use of the radio to reach many VHTs at one go is evident, the effectiveness of this approach is questionable due to lack of feedback and follow up that could have been provided for through face to face interactions. This is particularly important in a knowledge and skills transfer set up for adult learners where continuous question and answer, feedback between the teacher and student is required. The VHTs also felt that the sustainability of their activities will depend on the level of motivation they felt is lacking. They complained that HCP II did not provide them with a uniform (T-shirts), which helps them to get easily identified in the community, and also the IEC materials were scarce. For instance, not all VHTs received handbooks. World Vision, provided a full package (bag with gloves, required IEC materials, pens, etc.), which HCP II did not provide. According to HSSIP 2010/11–2014/15 the “attrition rate is quite high among VHTs because of lack of emoluments.” The VHTs are volunteers and the standard service to population ratio is 1:1,000 or 25 households. Thus the need for a long-term strategy on the facilitation and training of VHTs given the important role they play in local service delivery. 3.4.6 National Health Hotline In 2009, HCP II supported the UAC and MoH to start a National Health Hotline (0800-200600/0312-500600), as a supplementary channel to address health information gaps left by the other channels such as radio talk shows and also encourage those who wanted to seek health information in confidence or remain anonymous. The Hotline is managed by CDFU. The BCC Evaluation, 2012 found that only 25 percent respondents had ever heard of the Health Hotline; 58 percent had heard of the National Health Hotline in the last 12 months; 12 percent had called the Hotline in the past 12 months seeking information on HIV testing; and 12 percent had sought information on family planning. The use of mobile phones is still new especially in the rural areas but was reported to be increasing. Therefore, while the number of callers to the hotline is still low, the results demonstrate potential increase in use of this personalized channel through which people can ask questions about their health and that of their families. Currently, CDFU is still managing the Hotline. Because it is a toll free number, its operational costs are very high. It costs CDFU up to $50,000 annually to maintain it. With financial assistance from HCP ll, it was relatively easier to manage the Hotline. However, with the closure of HCP ll activities, CDFU has had to partner with other organizations for the Hotline to continue operating. Marie Stopes has come in to train counselors. Save the Children, Baylor College of Medicine Children’s Foundation, and World Vision have promised to participate and meet some of the operational costs. Given the low usage (12%), future programming may need to reconsider the target audience for the National Hotline vis-à-vis the costs involved in running it. 3.5 Impact of HCP II on the capacity of local IPs and the MoH (Evaluation Question 3)11 3.5.1 Capacity of Government through MoH HCP II provided capacity-building support to the government, primarily the Ministry of Health (MoH) as the lead agent for behavioral change activities in the country. This was in terms of the following interventions: (1) training partners in strategic communication and mentoring them as they implement their programs; (2) sharing 10 The national literacy rate is 73%; 69% in urban areas- Uganda Bureau of Statistics. Statistical Abstracts 2012. 11This section also doubles as a response to achievement towards IR 3 (Improved Communication Capacity). The evidence presented also highlights sustainability issues in terms of skilled personnel and availability of funding to provide continuity of the HCP II communication interventions after the life of the project. Page 27 innovations among partners and improving their ability to coordinate and share information; and (3) establishing and supporting local networks and partnerships. Findings indicate that HCP II had a positive impact on the MoH capacity in three ways: 1) Improved capacity to deliver quality health services: The expected outcomes for HCP ll capacity-building interventions were to increase number of MoH officials, health workers, staff from partner organizations, and ad agencies with knowledge and skills in design, implementation, monitoring, and evaluation of strategic health communication interventions. For example, through training of health workers, particularly counselors and Radio Distance Learning of VHTs for community mobilization, HCP ll increased the capacity of MoH to deliver quality health services. According to the HCP project PMP data, over 19,000 individuals (for both FY 2011 & 2012) participated in HCP-supported capacity building activities, including for VHTs over the planned 14,500 individuals that were to be reached. HCP II also increased the capacity of MoH and AIC directly. MoH benefited from HCP II training in the area of informative research in campaign tools design. According to MoH officials, participation in HCP II activities enabled MoH to appreciate the impact of well-researched and designed programs. MoH is now able to produce own materials subject to availability of financial resources. 2) Ownership of the BCC Strategies & IEC Materials: Study findings reveal that the process of developing IEC materials, which was participatory, created a sense of ownership for the MoH because of their direct involvement in the development, production, and distribution of IEC material as well as networking with HCP II staff. In addition, this participatory approach also improved knowledge and skills of health workers. 3) Strategic management of the BCC activities: HCP supported the MoH to conduct of quarterly BCC Working Group meetings, which structure enabled the MoH to coordinate and harmonize BCC activities in the country among IPs. The group was composed of communication personnel from USAID-supported IPs, local organizations involved in BCC, District Health personnel, etc., who used to meet at the MoH on a quarterly basis. The meetings were initially hosted by the IPs on a rotational basis, but the MoH later decided to host and chair them. The meetings helped the MoH coordinate BCC activities by the various IPs. The quarterly forums were a good feedback mechanism that enabled IPs to share experiences and learn about what others were doing. MoH is supposed to continue coordinating the meetings, but none has been held since the end of HCP ll. The BCC working group would have been an ideal forum for developing joint communication programs and mobilizing / leveraging resources to scale up programs across the country. Absence of such a forum may lead to lack of coordination between partners regarding delivery of common messages on BCC communication. 3.5.2 Capacity of Local Partners Findings presented in this section were gathered from local implementing partners involved in social and behavioral change, integration of health messages, in promotion of family planning, empowering young people, and provision of support to PLWHAs. Capacity building of IPs and institutional partners (CDFU, AIC, AC) and their partners was provided through technical support including numerous training in material development to improve health communication, training of trainers in child pediatric HIV, couple counseling, and development of posters, serial drama development sessions, and use of testimony videos. For the training of VHTs, HCP supported STRIDES, PACE and UHMG to train DHEs and VHTs in family planning. HCP II supported STOP Malaria to train VHTs in Mukono on malaria. HCP II also supported AI and MJAP (Mulago-Mbarara Joint AIDS) to train health workers at HCs in CHCT. HPACE and Baylor were also supported to provide training of health workers in positive living and pediatric counseling using the pediatric and HIV care and treatment guidelines. The two partners later rolled out the training at HCs. Page 28 3.5.3 Capacity for Communication/Media Organizations  Media Staff Capacity Uganda Health Communication Alliance (UHCA): Is one of the key media organizations that benefited from HCP II capacity-building initiatives. UHCA was chosen as a focal body to reach out and impart health reporting skills because it brings together health journalists in both the print and broadcast media across the country, so that that they could improve the low quality of reporting health issues such as family planning SMC, HIV among others. Makerere University School of Public Health (MUSPH) supported and collaborated with UHCA to build the skills of journalists in health communication in order to help them report better and reach out to the target population. A total of 400 journalists at 21 radio stations around the country benefited from the training. The journalists trained included the hosts of health talk shows like the Nurse Mildred drama series. Other key achievements for UHCA include the following: i) the creation of a pool of trainers in health communication who benefited from Training of Trainers (TOT) course under HCP II; ii) developed health communication modules for family planning and HIV/AIDS as a result of the training received from HCP II; and iii) developed health communication materials for journalists. UHCA has received support from the German Foundation for World Population to continue with the in￾house training to radio media houses. They have also secured funding from Tobacco Control to continue with heath communication. The level of radio station staff involvement was high. They got involved in translations of the English Version into their respective vernacular as well as hosting radio talk shows on health. The fact that the media staff were trained in-house exposed many of them to the training, including the programmers who were able to attend the training and be back just in time to change to the next program. This arrangement was better suited for all staff instead of using hotel conference halls as venues, which tends to leave others out. Taking journalists to the field and be exposed to firsthand health rather than being fed with information from press conferences attracted more journalists. This is because journalists realized that they were building their own capacity in health reporting, and since HCP II had no copyright restrictions with their materials, it helped them to disseminate health materials more effectively and fast. For instance, the well-built capacity has attracted other health support programs such as DSW to advocate for tobacco control through the media. 4.0 LESSONS LEARNED 4.1 Implications for Future Designs, Capacity Enhancement, & Partnerships To achieve its goal, HCP II implemented seven strategic approaches12, most of which were very effective as per the evidence presented in this evaluation report. However, given the challenges cited in relation to each approach, it is necessary to change and/or improve the way future approaches should be designed. The following are some examples of strategies that will need to either be continued or shifted in focus and direction: 1. Promoting and directing clients to services: Radio is still the most cost-effective channel to bring health messages to mass audiences. This is one communication medium that should be continued along with other multimedia channels like interpersonal communication and print media. 12 The strategic approaches included: (1) help make services more client centered; (2) promote and direct clients to services; (3) focus on underlying social issues that influence behavior; (4) improve community mobilization capacity; (5) form strategic partnerships with leaders and media representatives; (6) strengthen strategic communication capacity; and (7) build coalitions and strengthen partnerships. Page 29 2. Focus on underlying social issues that influence behavior: The gap in communication between adults and young people is a hindrance to the creation of a supportive social environment for youth. It is also important to note that some behaviors are challenging to change in the short run because of people’s beliefs and traditions. 3. Strengthening strategic communication capacity: Sustainability of this strategic approach is a challenge for MoH and local IPs because it is costly to produce IEC materials that are used to promote health services. 4. Public supportive social environments: Environments fostered though MoH and UAC for policy formulation, direction, execution, and leadership are still essential for any future design. What may need to change is the nature of the partnerships in terms of both accountability and resource mobilization for sustainability of the BCC programs. 5. Building coalitions and strengthening partnerships: This approach should be continued with a focus at the community level, which was a gap. HCP II worked well through partners to form coalitions mainly at the national and district levels. 5.0 CONCLUSIONS Overall, the project performance towards its objective of improving the effectiveness of comprehensive behavior change communication programming and implementation was partially achieved. Judging by the evidence presented, many desired behavior change outcomes in areas including HIV/AIDS, family planning, malaria, and TB were achieved although not to the expected magnitude. Based on the evaluation findings, we conclude that behavior change communication programming under HCP has been effective in several ways towards improving the abilities of communities, families, and individuals to adopt practices that improve health. Behavior changes in reduction in risky sexual behavior practices, uptake of SMC and HIV testing, and use of family planning methods were all found to be statistically significant and positively associated with HCPII-related communication programs. The specific programs included the following: Nurse Mildred Campaign which was associated with use of family planning; Stand Proud, Get Circumcised Campaign with uptake of SMC; and CHCT with increased HIV testing. However, the National Hotline and Rock Point 256 program were not as effective in improving levels of comprehensive knowledge on HIV and health behaviors like use of family planning amongst its target community i.e. 15–24 year olds. Despite sustained communication campaigns, levels of comprehensive knowledge on HIV/AIDS have stagnated, risky sexual behavior (not using condoms and having multiple partners) especially among young men in the 15–24 age group, voluntary male circumcision especially among the 25–49 age group and use of contraceptives still remain below desired levels. Subsequent communication and health service delivery programs need to continue to explore the barriers to behavior changes and respond with tailored interventions. While behavioral communication interventions may instill healthy behaviors (e.g. going for testing and using contraceptives), current staffing and drug shortages at the health facilities are affecting access and utilization of services. Health services like SMC, HIV counseling and testing and family planning need to be readily available and affordable to enable communities practice the newly acquired behaviors that will lead to better health outcomes. Overall, findings indicate that behavioral change in the population takes time and has occurred somewhat but even less over the HCP II period of implementation (2007–2012). Page 30 HCP II as a focal organization on communication was deemed an effective model in terms of ensuring uniformity of behavior change messages and communication materials used. This approach also resulted in cost savings by avoiding duplication of efforts and cost savings as a result of working through partnerships. Strong relationships and leadership by the Ministry of Health was instrumental in coordinating multiple partners and enlisting commitment to use of IEC materials developed. The Radio Distance Learning (RDL) designed as a cost effective mechanism for reaching the numerous VHTs across the country was successful in imparting knowledge to these teams who are regarded as the frontline service providers to communities. Efficiency gains are coming at a cost of potential effectiveness. Motivation of VHTs is a key issue that still needs to be investigated and addressed. The capacity of MoH, local implementing partners, and USG implementing partners in behavior change communication has improved as a result of the practical and targeted training, access to formative research, and joint design, development, and implementation of behavior change communication strategies and campaigns facilitated by HCP. The BCC implementers in Uganda now have access to e-resources on communication strategies and tools that can be reproduced upon need. HCP supported communication campaigns; training and other capacity building initiatives like the GOLD program which has contributed to creating a pool of behavioral change communication experts have not been found to be sustainable. Without HCP and in other cases USG funding that is currently supporting several health projects, the MoH and local partners cannot continue to run health communication programs including re-production of materials due to inadequate financial resources. Page 31 6.0 RECOMMENDATIONS Given the approaches implemented, the recommendations below are geared towards improving not only the challenges presented in the report but also what can be done to improve future project designs and behavioral outcomes from the communication pathways model. 6.1 Project Design, Strategies, and Approaches 1. USAID should ensure that projects follow the new project design guidance (2011), which states that “sound project design requires that M&E be built into the design from the beginning.” Future project designs should have clearly defined performance indicators at all levels that meet the ADS 203 standards for good indicators and thus data quality. 2. Continued use of multi-media channels including interactive radio and drama series as communication mediums for reaching a larger target audience is encouraged by all BCC implementers. 3. Given the low usage of the National Hotline (12%), and based on the finding that it lacks significant association with increasing comprehensive knowledge; future programming by the MoH and USAID may need to reconsider the target audience that should benefit from using a Hotline vis-à-vis the costs involved in running it. 4. Ministry of Health needs to explore other appropriate strategies aimed at bridging the communication gap between adults and the young people (15-24 years). 5. MoH and its development partners should conduct further investigation into drivers and barriers affecting improvements in comprehensive knowledge on HIV/AIDS, risky sexual behavior especially among young men in the 15–24 age group, voluntary male circumcision especially among the 25–49 age group and use of contraceptives to inform designs of appropriate interventions. 6. Ministry of Health should intensity efforts in improving availability and accessibility of critical health services through better staffing, provision of drugs and health commodities to absorb the demand created through behavior change communication activities. 7. USAID should continue implementation using a single communication partner with the capacity to coordinate and lead activities and provide technical assistance thus ensuring uniformity, efficiency and leveraging of resources. A close partnership with the MoH is important to ensure a buy in from other partners and rolling out of common messages, skills, and products. 6.2 Implementation & Achievement of Results 1. Ministry of Health should strengthen the BCC working group and develop a long-term resource plan for BCC activities to ensure continued implementation of BCC activities. 2. BCC partners need to mobilize adequate resources to support the continuation of IEC materials production in order to sustain the momentum created thus far. 3. Future interventions by MoH and its BCC partners should widen the age bracket to include adolescents who are below 15 because they are also becoming sexually active. 4. IPs should put in place mechanisms for monitoring the distribution and use of IEC materials in order to enhance accountability along the distribution chain. 6.3 Strengthened Institutional Capacity 1. In order to improve service delivery, future programming should consider improving and rolling out the RDL program nationwide since the program was effective in improving the skills of VHTs that serve community needs directly. Page 32 7.0 ANNEXES Annex 7.1: Excerpt from the HCP II Final Evaluation Scope of Work RFP: SOL-6IT·t2-OO00111 Attachmenl "A " S l n l .. ",(·,,1 fir " 'O"k rI " " Uh C OIllIllUll k" llo n P " "l nt' ,'sh lp P,'oj",,1 _ Fln ,,1 (:,·"1,,,,1;011 KAIIII,"IM, ~M '''I '' I. BACKGROUNO S,'",e,le communlCI,ion un Ion., . OC;II Ind beh.vio,.1 chIn,., ,hl,..,.;11 imp'ov. IVlil.bill\y. Icuuibill\y. Ind app,oprl.t. 1". 01 cri" col h • • lth •• MC .... ludl" • • o Imp,ov.d h .. lth ".'1" 01 t il. popull';on. The G""e,nm.n, of U,lnd.IGOUJ , .. co.nl ••• ,h .. c,illul '01. of 8ehlvior Cllln,e Communlcl,lon, (acC) .flo .... '" Imp,ovl", til. "".,.1111.01111 01 ,II. 1>01'1'11110", ."d til. M'nhuy of Hul,h (MOH) I. ,hl,lf,,,. IccII.c.t,n, commu"" .. Ii",, IclMtie.loc HIV/AIDS, ml'o,nll. nlonlt.l, .nd child h .. I'h; fom,IV pl.nnln" I"d mllo,lo. Til. Unit.d Sto, •• A,ency fo, In'e,nltion.1 Dev.lopm .... (USAIDI nk.""I.e rellord. communkl,lon I •• " Iml>O,,.nt ,00/'0 IcllfevI Improved hullh IIloblny .• "d h •• InVI •• ed In communlCldon .nd blh.vlor chln," ",o.,.m, 10, morl thln 30 y ..... AI .uch. 8CC I, In Impo,.,.", com pone", of th l USAID/UII.n';. ""'I,V In .he hulth .ecto'. In 2004. USA ID/VIII"dl Int.,.d InIO. "V'" A .. oc'.'1 A"".,d lAA) ""I. h Th. John. Hopk in. Un,v.ro,' V ' h,o .. ,h .he Hul.h Communiution PI"n ... hip,. " ... do, wIt h A.wei • ••• Coop.,.tive Acr ... m .. n. wllhl" tho Globel Hllllh 8u,u ... Thl purpo •• ol thl Anoellt. Aw.rd wl.lo dl.i,,, ."d Impl.ml" t .