1 EVALUATING USAID/UGANDA’S SOCIAL AND BEHAVIOUR CHANGE COMMUNICATION ACTIVITY Evidence-Based Learnings and Strategic Recommendations April 27, 2018 This report was authored by Lynda Bardfield, George Lugalambi, Beth Anne Pratt, Augustine Wandera, and Nadia Shadravan. It was prepared for review by the QED Group, LLC on behalf of the USAID/Uganda Monitoring, Evaluating and Learning Contract No. AID-617-C-13-00007. PHOTO CREDIT: BETH ANNE PRATT, GLOHI 2 ABSTRACT The goal of this evaluation is to assess the progress of CHC, including what has worked well and what has not, and to make recommendations that inform the design of a follow-on project. This evaluation explores how CHC's design and implementation played a role in behavioral outcomes, how CHC coordination structures worked and contributed to increased capacity in SBCC, and how CHC has been able to generate and share evidence of effective SBCC practices with stakeholders. The evaluation followed a mixed method, non-experimental evaluation design, relying predominantly on qualitative data supported by quantitative evidence from both the national Lot Quality Assurance Sampling survey (LQAS) and CHC’s own Phase I/baseline and Phase II/endline surveys to understand what worked with respect to CHC, how and where it worked, and why. Findings showed CHC used “360-degree communication” to saturate the national and local media and leverage community- and facility-based communication channels. This resulted in very high levels of message recall. Obulamu (a national media campaign) can be found everywhere. In terms of coordination, two of CHC’s biggest achievements were: 1) helping to revitalize the SBCC technical working group (TWG) at the Ministry of Health (MoH) making it communication-focused as opposed to content-focused; and 2) streamlining the process by which SBCC materials get approved by the MoH’s Health Promotion and Education (HPE) Department. These achievements were highly valued by the MoH and partners. At the same time, coordination was affected by competing priorities of IPs who also had their own objectives, plans, and targets which sometimes did not segue smoothly with those of CHC. Additionally, goal posts were often moved in response to PEPFAR, many targets for which changed annually. 3 EXECUTIVE SUMMARY 6 ACKNOWLEDGEMENTS 14 INTRODUCTION 15 Background 15 Communication for Health Communities 17 EVALUATION PURPOSE & QUESTIONS 19 Evaluation Purpose 19 Evaluation Questions 19 METHODS 19 Evaluation Design and Approach 19 Sample Selection Criteria 20 Data Collection 20 Data Analysis 21 Ethical Considerations 21 Limitations 21 FINDINGS 22 Evaluation Question: Best Practice Strategies and Behavioral Outcomes 22 Program Strategies 22 Target Audience Segmentation by Life Stage 23 Branding and Positioning 28 “360 Degree” Communication 31 Interactive Campaign Design 36 Evidence on Effectiveness: Health Outcomes 39 Evaluation Question: Misconceptions and Knowledge Gaps in Obulamu Messaging 47 Target Populations’ Misconceptions 48 Target Populations’ Attitudes and Knowledge Gaps 50 Evaluation Question: Converging Messages and Integration 53 Integration: Audience Segmentation 53 Integration: Channel Selection 54 Integration: Message Design 54 Integration: Benefits and Challenges 55 Challenge 1: Integration is in the Eye of the Beholder 55 Challenge 2: Timing is Everything 56 Challenge 3: Creating Room for Improvisation 56 Challenge 4: Dealing with Converging Messages 57 4 Evaluation Question: Coordination and Capacity 57 Streamlined SBCC Coordination and Approval 58 Competing Priorities 60 Systems Bottlenecks to Coordination 61 Overcentralization 63 Evaluation Question: Research and Knowledge Management 64 Evaluation Question: Supply and Demand 67 RECOMMENDATIONS AND CONCLUSIONS 71 Recommendations 71 Conclusions 76 Annex 1: References 78 Annex 2: Team Composition 81 Annex 3: Getting to Answers Matrix 83 Annex 4: Methodology 87 Annex 5: Consultant Scope of Work 98 Annex 6: Data Collection Tools 102 Annex 7: Background to CHC 113 Annex 8: Evidence from CHC Phase I-to-Phase II Data on Attitudes and Knowledge 118 5 LIST OF ACRONYMS ANC Antenatal clinic ART Antiretroviral therapy CHC Communication for Health Communities CQIM Continuous Quality Improvement Monitoring DHE District health educator DHIS-2 Demographic and Health Information Survey DHT District health team FGD Focus group discussion GoU Government of Uganda HBC High burden country HC Health communication HCP Health Communications Partnership HPE Health Promotion and Education Department IP Implementing partners IPC Interpersonal communication KI Key informant KM Knowledge management KP Key populations LQAS Lot Quality Assurance Sampling LS Life Stage MCH Maternal and child health MoH Ministry of Health OSIMS Obulamu Site Improvement Monitoring System PLHIV People living with HIV/AIDS PP Priority populations RHITES (-SW, -EC, - E) Regional Health Integration to Enhance Services (Southwest, East￾Central, East) RHU Reproductive Health Uganda SBCC Social and behavior change communication SMC Safe male circumcision SMGL Saving Mothers Giving Life TB Tuberculosis TWG Technical working group UHMG Uganda Health Marketing Group UPDF Uganda People’s Defense Forces UPHIA Uganda Population-Based HIV Impact Assessment. VHT Village health team member VL Viral load VMMC Voluntary medical male circumcision 6 EXECUTIVE SUMMARY Uganda has made tremendous progress meeting many of its critical health indicators; however, a number of important bottlenecks remain across regions, wealth quintiles, and key population groups. While many of these health problems require technical and clinical interventions targeted at service delivery, their cultural and behavioural foundations are far more difficult to address, rooted as they are in beliefs, attitudes, and perceptions about personal and social well-being. Sustaining and amplifying progress, therefore, requires changing the way Ugandan people behave with respect to their own health and that of their families and, thus, requires significant, continued investment in social and behaviour change communication (SBCC). One such investment is USAID/Uganda’s Communications for Healthy Communities (CHC). The goal of this evaluation is to assess the progress of CHC, including what has worked well and what has not, and to make recommendations that inform the design of a follow-on project. This evaluation explores how CHC's design and implementation played a role in behavioral outcomes, how CHC coordination structures worked and contributed to increased capacity in SBCC, and how CHC has been able to generate and share evidence of effective SBCC practices with stakeholders. Specifically, the objectives of the evaluation are as follows: ● To assess the effectiveness of CHC Uganda toward meeting its various objectives. ● To document how the project has made a difference and where there have been challenges. ● To extract lessons learned to inform future SBCC initiatives in Uganda. Project Background CHC is a five-year (June 2013-2018) activity with the goal of supporting the Government of Uganda and partners in reducing HIV infection, total fertility, maternal and child mortality, malnutrition, malaria and tuberculosis (TB). It uses the Obulamu? (How’s Life?) concept as the organizing principle and campaign platform for an integrated set of health communication interventions targeting these six health areas. Obulamu addresses individual and societal health challenges with an eye on integration through the Life-Cycle Segmentation Model. The Life-Cycle Model targets the population within four unique Life Stages (LS): ● Young adults in relationships (aged 18-30) ● Pregnant women and their partners ● Caregivers of children under the age of five (aged 15-49) ● Adolescent boys and girls (aged 15-19) Planned as a departure from traditional disease-focused approaches that have tended to respond to health problems via technical silos, Obulamu campaign objectives and key behavioral actions are tailored to each LS and revolve around key overarching themes that reflect the Obulamu campaign’s behavior change priorities for each target audience. These themes are ● “How’s your love life?” (LS 1) ● “How’s your pregnant life?” (LS 2) ● “How’s your family life?” (LS 3) ● “How cool is your life?” (LS 4) CHC works with and through other implementing partners (IPs), in particular USG regional health IPs and direct service delivery partners. Although Obulamu is a national campaign - adopted as the national health communications strategy by the Ministry of Health (MoH) - downstream, CHC does not work independently of existing IPs. CHC provides support to district government and IPs for an overall SBCC/demand generation approach in partnership with other community-based 7 organizations, while regional health and service delivery IPs support district government in health services delivery. This is important because traditional evaluations usually seek to attribute health or behavioral outcomes to specific programmatic objectives or activities. This evaluation, however, looks at CHC in the context of relationships. The evaluation assumes that CHC was one important piece of a puzzle, the ultimate picture of which depended to a large extent on the functionalities, efficiencies, and synergies of all the pieces working together. It is concerned with what CHC did in the context of how CHC worked with others. This enables us to better understand what worked (and what did not) with respect to Obulamu, where it worked, and why. Evaluation Questions and Methodology The evaluation followed a mixed method, non-experimental evaluation design, relying predominantly on qualitative data supported by quantitative evidence of effectiveness extracted from both the national Lot Quality Assurance Sampling survey (LQAS) and CHC’s own Phase I/baseline and Phase II/endline surveys1 to understand what worked with respect to CHC, how and where it worked, and why.2 The evaluation drew upon an array of primary and secondary data collection and sources. The six evaluation questions as provided to the evaluation team by USAID/Uganda are: Q1: What did the project learn about the target populations’ misconceptions around intervention messaging and how did they adapt? Q2: What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? Q3: How did the implementer address converging messages and integration during implementation and what lessons can be learned? Q4: How have CHC-relevant coordination structures contributed to improved capacity of Government of Uganda (GoU) and other partners to design, coordinate, and implement integrated SBCC? Q5: How did information generated through research and knowledge management boost and improve health communication in the country? Q6: What are the main barriers in the delivery of the six services promoted by CHC? What changes should be made to SBCC programming in the future to improve linkages between health communication and integrated health service delivery partners? Qualitative data consisted of key informant (KI) interviews of 103 stakeholders from central government, district government, regional health and service delivery IPs, USG agencies, creative partners (advertising agencies and regional radio), and supply-side SBCC stakeholders such as village health team members (VHT), health facility workers, champions, and peer educators. Additionally, multiple interviews were conducted with central and regional staff at CHC’s prime implementer, FHI 360, along with staff from the Uganda Health Marketing Group (UHMG), CHC’s sub-prime partner. Additionally, 47 focus group discussions (FGDs) involving 527 participants were conducted in six target districts: Mukono, Iganga, Amuru, Lira, Kyenjojo, Ibanda. Findings Best Practice Strategies and Behavioral Outcomes 1 A member of the evaluation team was present at the CHC Phase II/Endline Data Interpretation Workshop held in Seeta on December 6-7, 2017 after which the team was provided by CHC with a zip file of all Phase I and Phase II data presented in tabular form. Data from this CHC-provided zip file is used in the evaluation report. 2 Note: although CHC has referred to these surveys as “baseline” and “endline” in numerous reports, emails, and data workshops, the evaluation team has been asked by CHC to refer to these as Phase I and Phase II surveys for their own adminsitrative purposes. 8 Campaign design was driven by international SBCC best practice, including branding, positioning, and target audience segmentation. Additionally, the campaign employed “360-degree communication” and interactive interpersonal communication (IPC). Life Stage Segmentation ● Obulamu used a LS segmentation strategy based on age and the unique health needs of each particular stage of life. ● LS segmentation took a while for some partners to understand; however, once understood, LS segmentation was seen as a valuable way to target communications. ● The four LS, however, were also seen as insufficient. Additional segmentation (e.g. based on regions, key populations, rural versus urban) is desired going forward. ● There is clear evidence that the message of “a good life” that LS segmentation strategy was meant to facilitate has been understood and embraced by beneficiaries. ● At the same time, LS segments overlapped considerably. For campaign design, this was seen as a value add by KIs since individual identity is multi-faceted and the boundaries between categories are often blurred. ● LS segments were also harnessed for monitoring and evaluation purposes. For some forms of research (Action Media, special studies), they were valuable; however, when used to disaggregate data for analyzing SBCC outcomes, LS can be very problematic. Branding and Positioning ● In Obulamu, CHC has succeeded in creating a strong, universally-recognized brand that has helped shift the focus away from disease silos toward life and well-being. ● Some beneficiaries – especially those in the North – had issues with the choice of a Luganda word as a national brand name, although there is some indication that objections are being overcome. ● That said, brand equity may still require reinforcement in some parts of the country due to the politicization of language in Uganda. 360 Degree Communication ● Obulamu used “360-degree communication” to saturate the national and local media and leverage community- and facility-based communication channels, thus creating “surround sound” for its messages. This resulted in very high levels of message recall. Obulamu can be found everywhere. ● Efficient roll out, however, encountered bottlenecks due to a) the reliance on IPs – not all of whom had the same priorities and motivations - to deliver materials and engage district-level stakeholders; b) disorganization with respect to which materials went where; c) lack of regional tailoring and representation in the national campaign and in seed materials. ● While Obulamu prided itself on its use of local languages, many health facilities and other up￾country sites had materials that did not reflect the language of the community. Campaign photos also did not represent the diversity of Ugandans. Interactive Communication ● CHC focused on an interactive - occasionally provocative - campaign design that prioritized dialogue, personalization, and reflection. IPC was a key focus. ● The interactivity was highly valued by KIs and beneficiaries, especially mobilization activities such as Kadankes (youth bashes), community dialogues, and radio talk shows. ● Interactive communication – and IPC more generally – is expensive. Sustainability without partner support is an issue; however, without this component of health communication, KIs note that demand generation will remain sub-optimal and health outcome targets will not be achieved (i.e. demand-side SBCC and supply-side technical/clinical service delivery interventions need to be packaged together). 9 ● The effectiveness of this package depends on the relationship between various demand- and supply-side partners: in the case of Obulamu, this means close coordination between CHC, regional health and service delivery IPs, and district health teams (DHTs). Effectiveness: Behavioural Outcomes ● Best practices adopted by CHC are not mutually exclusive of one another (i.e. they are part of a unified SBCC approach). In addition, CHC’s relationship to health outcomes is indirect (i.e. they are dependent upon the efficiencies and effectiveness generated by the regional health IPs that provide clinical and technical support to service delivery). ● Thus, the evaluation did not seek to assess the effectiveness of CHC on a strategy-by￾strategy, best practice-by-best practice basis. ● The evaluation instead looked at quantitative data in three Obulamu health areas - reproductive health, maternal health, and nutrition - to understand what story health outcome numbers might tell about how CHC works with partners. ● It compared CHC national Phase I data (taken as a baseline) to Phase II data (collected in 2018/endline) data with regionally disaggregated (Eastern, Southwestern, Northern) LQAS trend data from 2012-2016. ● CHC and LQAS reproductive health data showed significant change from Phase I data to Phase II data as to contraceptive use among LS 3 (i.e. mothers of under-5 children aged 15- 49 years). Reproductive health data changed at the same pace in all three regions, regardless of whether CHC was operating in the context of a stable regional health IP (RHITES￾SW/single prime partner), a semi-stable regional health IP (RHITES-E/changeover in prime partner), or an unstable regional health IP (North with multiple, consecutive IPs and prime partners). ● Reproductive health, however, is not necessarily dependent on regional health IPs (e.g. RHITES, etc), relying instead on outreach activities and not-for-profit private sector providers (e.g. Marie Stopes, Reproductive Health Uganda) to create access to services. That synergies between these sorts of direct service delivery IPs and CHC might be expected to bear fruit is, therefore, not surprising. This finding is born out in qualitative interviews. ● CHC maternal health data showed significant change with respect to antenatal clinic (ANC4) attendance among women in LS 2 and 3. LQAS disaggregated trend data also observed significant difference across all three regions; however, rates of change for the Southwest (15.5 percent) were much stronger than for Eastern (10.4 percent), and both were stronger than Northern (6.5 percent). As noted in the previous bullet point, Southwest had a stable regional health IP, Eastern had a semi-stable IP, and Northern had unstable IPs. ● Since maternal health relies on health systems and primary health care, it is likewise not surprising that synergies between CHC and a stable IP context resulted in a greater rate of change than semi-stable or unstable IP contexts. This finding is also born out in qualitative interviews. ● Finally, for nutrition, both CHC national data and LQAS regional data showed either no change or negative change in exclusive breast-feeding from 2012-2016. This finding is not surprising either since nutrition requires a multisectoral response that SBCC and service delivery partners (i.e. the health sector) alone cannot achieve. Additionally, regional health IPs, until recently, have had little to no focus on either breast-feeding or nutrition more generally. This finding is also born out in qualitative interviews. Misconceptions and Knowledge Gaps in Obulamu Messaging ● CHC had a number of on-going monitoring and research activities to identify and address target population misconceptions, in particular Action Media and special studies. ● Examples include creative, dynamic adaptation with respect to findings on teenage pregnancy, infertility, viral load (VL), adolescents living with HIV/AIDS, and ANC attendance. ● At the same time, there are numerous red flags in CHC’s own Phase I/baseline to Phase II/endline data on attitudes and knowledge across variables in many of the six Obulamu health 10 areas. In many instances, outcomes either stayed the same from Phase I to Phase II, or declined markedly. ● These findings may be due to a number reasons: survey design, survey implementation, partnership bottlenecks, message fatigue, etc. Note to: CHC’s predecessor – HCP – experienced similar trends from baseline to endline. ● The larger issue with respect to any follow on SBCC activity is why such red flags were not identified earlier. That is: there was no midline to catch and address/explain negative outcome data and routine monitoring to follow trends was not at a large enough scale. ● As a consequence, it is difficult to know how - or even if - CHC went wrong. ● That said, one detectable pattern in the data is that outcomes in attitudes and knowledge about contraceptives and safe male circumcision (SMC) appear not to exhibit the same problematic outcomes that some of the other health areas demonstrate. ● As mentioned earlier, this might be because CHC may add more value to the highly-targeted campaign-, outreach-, or camp-based service delivery platforms through which family planning and SMC are often rolled out, compared to the sort of routine, facility-dependent service delivery on which maternal and child health (MCH), malaria, HIV care and treatment, and TB traditionally depend. ● There may be a gap, then, between the “good life” holistic model that Obulamu promotes and the reality of the Ugandan health sector which often relies upon targeted, campaign￾based service delivery to bypass a broken health system and meet externally-imposed targets. Converging Messages and Integration ● CHC addressed converging messages and integration through its audience segmentation strategy, channel selection, and message design. ● Integration, while appreciated by service delivery IPs, was often at odds with SBCC specialists who are generally taught to be single-minded. There were a number of challenges to promoting multiple products and behaviors at the same time. ● Timing was also an issue. Roll out of a phased, integrated LS strategy was often disrupted by competing partner needs. These needs were sometimes highly technical and frequently tied to PEPFAR and other USG agency reporting requirements. ● Because of the lack of direct relationship between CHC and messengers on the ground, IPC agents – VHTs, peer educators, champions, health workers – were sometimes unprepared and untrained in the Obulamu approach (i.e. dialogue-based, improvisation-dependent, life￾focused), and message convergence was sometimes dealt with ad hoc. Coordination and Capacity ● Two of CHC’s biggest achievements were: 1) helping to revitalize the SBCC technical working group (TWG) at the MoH, making it communication-focused as opposed to content-focused; 2) streamlining the process by which SBCC materials get approved by the MoH’s Health Promotion and Education (HPE) Department. These achievements were highly valued by the MoH and partners. ● CHC also improved health communication coordination across USG IPs by standardizing the development of materials and messages and providing ‘seed copies’ which IPs can adapt. This achievement was highly valued by KIs. ● At the same time, coordination was affected by competing priorities of IPs who also had their own objectives, plans, and targets which sometimes did not segue smoothly with those of CHC. Additionally, goal posts were often moved in response to PEPFAR, many targets for which changed annually. ● Because CHC received PEPFAR funds some PEPFAR IPs had conflicting perspectives as to CHC’s role. For some IPs, CHC was a service provider for IP health communication needs, rather than a unifier of USG health communications. There were also questions over who 11 should then pay for health communications if CHC was, in fact, the PEPFAR “health communication partner.” ● CHC regional offices were not all operating at the same capacity, and did not necessarily have the synergetic relationship to regional health IPs that was envisioned in the original technical proposal. ● Coordination downstream was also an issue, as CHC did not have a direct relationship to their own messengers and brand ambassadors (VHTs, champions, peer educators, health workers) as it was the role of regional health IPs and their sub-prime partners to manage these relationships. Consequently, many of these messengers were only minimally trained on Obulamu. ● Coordination with district health teams (DHTs), too, was mixed. CHC brought District Health Educators (DHEs) numerous and highly-valued opportunities for capacity-building and participation in health education work which otherwise went totally unfunded by district government. However, this raises sustainability and ownership issues, since district SBCC is subsequently entirely reliant on the existence of CHC (or any other external SBCC project). ● Additionally, over-centralization meant that key regional creative partners – in particular, local media – were not optimally engaged. Research and Knowledge Management ● CHC carried out vast amounts of research, including evaluations, Action Media, case studies, feedback studies, listening surveys, special surveys, and exit interviews. ● Much of the research was highly participatory with respect to beneficiaries and fed directly into campaign design and adaptation that helped Obulamu achieve relevance and memorability among beneficiaries. ● At the same time, very little research was disseminated to partners, nor did IPs or DHTs participate in either the development of indicators or in research itself. ● KIs – including IPs and district government - expressed the desire to have data on SBCC indicators to feed into their own planning and implementation processes, especially those for PEPFAR. They also expressed a desire for input into indicators. ● Additionally, as mentioned earlier, there was a disconnect between the kinds of data captured in CHC’s existing routine monitoring activities and the kind of information needed to catch and remedy negative trends between Phase I and Phase II data collection. Supply and Demand ● Although CHC was supposed to work through and with regional health IPs and DHTs to bridge the gap between supply and demand, this gap remained one of the most serious bottlenecks to CHC’s impact. ● Persistent commodity and health human resources shortages at health facilities undermined many of CHC’s efforts (and have also served to undermine USG health communication investments for more than a decade). ● Where gaps were successfully bridged was in the context of highly technical, one-off service delivery activities - family planning outreaches, SMC camps – often driven by not-for-profit private sector service delivery partners which could guarantee commodities and trained medical professionals for a limited period of time. ● Where gaps were not successfully bridged was in the context of the routine service delivery that most Ugandans rely upon for their daily health needs. ● These gaps point to ongoing issues with USG IP/DHT coordination and with dysfunctional health systems and are not something that a health communication mechanism alone can solve. Recommendations Build on brand equity 12 Build on and leverage Obulamu in the follow-on. The follow on should continue the more memorable and well-liked elements, but also include new content and design features to attract attention, maintain interest, and freshen up the campaign. In some parts of the country (e.g. Acholi), work still needs to be done to improve brand equity as the choice of the brand name Obulamu is easily politicized. Document and share the most compelling success stories from the point of view of people who have been impacted by the project. Segment the audience even further Together with partners - USG agencies, IPs, DHTs - work out where to add additional relevant segmentation, including adding new segments to both primary and secondary audiences. This could include rural versus urban segments, students, faith-based audiences, uniformed services, and other PEPFAR key and priority populations (KP/PP). Make sure that these segments are also tied to relevant campaign messengers (e.g. parents/family members, teachers, religious leaders, uniformed services officers, etc.). Be a catalyst for more creative collaboration Continue to break up the creative pie by segment of population or disease area, but allow only one creative partner to lead the design and implement all communication channels. Additional support can come from subcontractors/consultants, but accountability for work should not be divided among multiple companies. To avoid counterproductive competition and overcentralization, develop a creative coalition to bring together a wide range of creative partners. Add more local and consumer-generated content Improve inclusivity and representation in images and content. Make sure that the people used in photographs reflect the regional diversity of the population. Add more consumer-generated content which can then be used in social media and knowledge management. Ensure that creative partners at regional level have input into the design and delivery of materials. Engage local radio and media personalities as creative partners and incentivize or challenge them to come up with innovative Obulamu ideas. Orient more IPC agents to deliver population-contextualized campaign messages and facilitate population-appropriate dialogue. Rethink and increase capacity-building activities Include a strong downstream training focus that includes not only conveying content of messages, but also focuses on the Obulamu approach to communication: interactive, question-based, life- and wellness-focused, and relevant to where an individual is at in his or her particular point in life. Training requires monitoring, correction, and retraining and IPC is expensive. Yet evidence shows that IPC is also critical, so consider how to optimally, cost-effectively, and collaboratively implement this element of the follow on. Build in mechanisms for improved coordination with DHTs and regional health IPs, as it is they who have the most direct interaction with health workers, VHTs, and champions. Continue to partner on improving coordination As CHC continues in its role of setting the pace at national level while the IPs support localization, it also needs to boost its footprint on the ground. A mechanism for progressively tracking and disseminating national policy changes and decisions that affect the implementation of SBCC will help to ensure quick adaptation and that all partners are reading from the same page. Decentralize and get the districts more involved Delegate more core decision-making authority to CHC regional offices and regional health IPs with respect to planning and implementation so as to enable those with deep knowledge of local communication landscapes to exploit local networks and facilitate adaptation. Institutionalize coordination mechanisms that will ensure regular and meaningful participation of and engagement with DHTs in planning and executing Obulamu communication activities. 13 Keep up the action media With action media, do not confuse the target audience with creative professionals. Allow creative professionals to translate action media into effective SBCC. Make sure these roles are delineated but also synergized while planning future action media activities. Create more SBCC indicators Work with partners to create more SBCC indicators tied to district and partner planning objectives. Make sure that SBCC indicators meet the needs of PEPFAR partners and DHTs. Build into the health communication planning process a means by which to harmonize SBCC targets, indicators, and measurement systems. Include advocacy to ensure that SBCC indicators are integrated into other mechanisms, programs, and initiatives. Disentangle health communication design from monitoring and evaluation. Monitor indicators far more frequently to be able to better detect and respond to red flags in data. Better utilize and disseminate evidence Treat knowledge management (KM) consumers in the same way as LS beneficiaries. Understand their KM needs and wants, including what would be most helpful and in what format. Pretest KM products. Rethink the existing KM strategy for the follow on. Make sure one focus of KM involves supporting the data/research needs of DHTs. Involve CHC research users in decision making about the research agenda and planning process 14 ACKNOWLEDGEMENTS The evaluation team would like to acknowledge the many CHC partners in Kampala and in Lira, Amuru, Kyenjojo, Ibanda, Iganga, and Mukono who shared their thoughts with us. Additionally, we are grateful for the openness, cooperation, and logistical support of the CHC central office and its regional field staff. If it were not for them, we would not have been able to hear from as many beneficiaries, nor understand as much as we did about the project. The team would also like to thank Julian K. Bagyendera of Provide and Equip (P&E) Uganda, and P&E’s team of intrepid research assistants who were instrumental in helping us carry out up-country key informant interviews and focus groups. In particular, we thank Stephen Ancellam, James Acaye, Eglance Namugaya, Fred Mwijuka, Hope Bamuheire, and Helen Rwomushana who served as our district-level field supervisors. Finally, the time that community- and facility-level partners generously spent sharing their observations and insights with us was invaluable. We would especially like to acknowledge the young men and women who took time out of their busy days to speak with us, some with small children and babies in tow. This report is dedicated to them in the hope that projects like Obulamu, along with the implementing partners who bring these projects to life, will continue to create a healthier future for people across their life course. Only then can the next generation see the kind of change that allows them to make their own healthy choices and reach their full potential. 15 “Everything has a beginning. Obulamu has set the pace.” (KII_KLA_CG_4) INTRODUCTION Over the last decade, Uganda has been making considerable progress improving many of its critical national health indicators. Intensified scale-up of HIV prevention, care and treatment services has meant that the country is moving toward meeting its UNAIDS 90-90-90 targets for 2020.3 The Uganda Population-Based HIV Impact Assessment (UPHIA) suggests that almost 60 percent of Ugandans living with HIV/AIDS (PLHIV) have achieved viral suppression,4 while the most recent HIV and AIDS Country Progress Report (2016) found testing coverage of persons estimated to be infected with HIV to be at 69 percent, and antiretroviral therapy (ART) coverage to be at 60 percent.5 The country was removed from the WHO list of High Burden Countries (HBC) for TB in 2016.6 Meanwhile, in spite of one of the highest rates of transmission in the world, Uganda has also made progress on coverage of key malaria interventions, such as vector control, malaria in pregnancy, case management, and surveillance.7 Similarly, upward trends in many maternal health (e.g. ANC visits, skilled birth attendance), child health (e.g. first dose vaccinations), and nutrition (e.g. stunting, wasting, underweight) indicators also suggest steady improvement over the last decade.8 Nevertheless, evidence also indicates that much of this progress has not been equitably distributed across regions, wealth quintiles, and key population groups.9 A number of bottlenecks still prevent Uganda’s health sector from achieving its strategic mission: “To facilitate the attainment of a good standard of health by all people of Uganda in order to promote a healthy and productive life.”10 While many of these health problems require essential technical and clinical interventions directed at facility- and community-based delivery of services, their cultural and behavioral foundations are more difficult to address, rooted as they are in deeply-held and intimately-felt beliefs, attitudes, and perceptions about personal and social well-being, the locus of disease, and one’s own and one’s children’s bodies. Sustaining and amplifying progress, then, also requires changing the way people behave with respect to their own health and that of their families. It thus depends, to a great extent, on continued investment in social and behaviour change communication (SBCC). Background Annex 7 describes the health communication (HC) environment in Uganda prior to the launch of CHC. This context is important as CHC was not only designed to address existing gaps within this environment, but – as the evaluation will make evident – has encountered many of the same challenges as its predecessors. For example, in 2014, CHC/FHI 360 conducted an audit of the HC landscape in Uganda.11 The study argued that, although high quality HC interventions had been undertaken in Uganda in the preceding decade, “their overall effectiveness appeared unclear and not well translated into health outcomes” (p.7).12 3 Government of Uganda, Uganda HIV and AIDS country progress report 2016 (Kampala: Uganda AIDS Commission, 2016). Note: in the last year, PEPFAR – in partnership with the Government of Uganda - has raised its benchmarks to 95- 95-95. This citation, however, is for progress towards the UNAIDS 90-90-90 goals which co-exist alongside 95-95-95. 4 Government of Uganda, Summary sheet: Preliminary findings - The Uganda population-based HIV impact assessment (UPHIA) (New York, ICAP/Columbia University, 2017a). 5 Government of Uganda, 2016. 6 World Health Organization, Use of high burden country lists for TB by WHO in the post-2015 era: Summary (Geneva, WHO, 2016). 7 USAID. President’s Malaria Initiative Uganda: Malaria operational plan FY 2018. (PMI, Kampala, 2017). 8 Government of Uganda, Uganda demographic and health survey 2016: Preliminary findings (Kampala, UBOS, 2017b). 9 Government of Uganda, 2017b. 10 Government of Uganda, Health sector development plan II, 2015/16 - 2019/20 (Kampala, Ministry of Health, 2015), p. 48. 11 FHI360/CHC, Health communication in Uganda: Findings of an audit of strategies, strategies/materials, and implementing partners in 2013/2014. (Kampala, FHI360/CHC, 2014). 12 FHI360/CHC, 2014. 16 The audit found that existing health governance structures provided an overall framework for the health sector. There was clear audience segmentation and message focus, along with avenues for linking demand and supply and for community engagement. Gender and social norms were key factors in health programming, and health-seeking behavior program indicators for monitoring and evaluation existed. The GoU had a number of mechanisms to coordinate HC initiatives including dedicated policies, strategies, working groups, and task forces. There had also been a number of dedicated HC initiatives.13 Additionally, the audit noted that IPs recognized HC as “a specialized technical area.” 14 At the same time, the audit also found a number of inefficiencies. HC strategies and programs were fragmented and siloed. Strategies long remained in draft form. Messages were often inconsistent. Formative research was rarely done. Monitoring and evaluation of results was not systematic, and rarely used for planning and continuous program review. Media coverage was sporadic, one-size-fits￾all, and centered on commemorative events. Additionally, there were major gaps in coordination, including conflicting priorities and timelines as a result of programming dictated by funding mechanisms, along with red tape and delays in the process of clearing materials and messages. An overall lack of coordination in design led to dissemination of competing health messages, and implementation that was geared toward bureaucratic reporting rather than advancing a learning agenda. HC activities tended to overlook best practice SBCC interventions, and made limited use of theory and data in planning and decision-making. There was also limited bridging of supply and demand at community level, alongside high amounts of duplication of effort. Finally, different thematic health areas exhibited different capacity, with HIV and family planning demonstrating much higher competencies than other program areas.15 The audit concluded “These [bottlenecks] have implications for finding an effective tipping point for change, either addressing knowledge, skills and motivation needed, desired modification for gender and other social norms, or what would constitute an enabling environment for change.” 16 USG agencies have a long history of promoting HC initiatives in Uganda and CHC was designed to build on, strengthen, and sustain the results from previous USAID- and, more broadly, PEPFAR￾funded initiatives. One of the most important of these programs was the Health Communications Partnership II (HCP II) implemented from 2007 to 2012 by the Johns Hopkins University Bloomberg School of Public Health Center for Communication Programs (JHU-CCP). HCP II was rolled out in the footsteps of the AFFORD health marketing initiative (2005-2010) which evolved into the current CHC sub-prime partner, UHMG. AFFORD introduced the Good Life platform: an early version of the the life stage, whole life-focused approach that CHC subsequently expanded upon and deepened.17 When HCP II closed out in 2012, USAID/Uganda commissioned an evaluation of HCP.18 The evaluation assessed whether 1) effective communication strategies were designed and implemented to increase appropriate use of services and/or practices across identified priority programs; 2) supportive social environments were fostered to enable positive health‐seeking behaviors and result in healthier individuals, families, and communities; and 3) capacity for sustained HC was increased.19 The evaluation found that HCP was successful in increasing the capacity of the MoH to carry out campaigns and train health workers - including village health teams (VHTs) - in HC and community 13 e.g. AFFORD, Health Communications Partnership (HCP), along with several other existing organizations with HC specialization, in particular, Communication for Development Foundation Uganda (CDFU) and CHC’s partner, the Uganda Health Marketing Group (UHMG). See FHI360/CHC, 2014. 14 FHI360/CHC, 2014, p.15. 15 FHI360/CHC, 2014, p.15. 16 FHI360/CHC, 2014, p. 15. 17 See interview KI_KLA_SUB_1. Also Storey JD, Saffitz GB, & Rimón JG, “Social marketing,” in Health behaviour and health education: Theory, research, and practice. 4th Edition, eds. K Glanz, B Rimer, & K Viswanath (San Francisco, Jossey-Bass, 2008), pp. 435-464. 18 USAID/Uganda, Uganda joint behavior change communication survey. (Kampala, USAID/Uganda, 2012a) 19 USAID/Uganda, Final evaluation of the Health Communications Partnership (HCP II) project (Kampala, USAID/Uganda, 2013). 17 mobilization, and supported the MoH in the roll out of new policies for the adoption of healthy behaviors. However, the evaluation also found that, in spite of increased uptake of some healthy practices (e.g. HIV testing, use of modern family planning methods), other measures of healthy behaviour actually regressed (e.g. risky sexual behaviour among different age groups). 20 Moreover, there remained major gaps between demand generation and supply-side delivery of services. While people wanted to adopt healthy behaviours, they often found the services enabling them to do so were unavailable.21 We highlight these findings from the HCP evaluation because they led directly to the recommendation that the follow on (i.e. CHC) should: ● Promote and direct clients to services ● Focus on underlying social issues that influence behavior ● Strengthen strategic communication capacity ● Foster public supportive social environments ● Build coalitions and strengthen partnerships22 The foundations of CHC, therefore, originate directly from the HCP evaluation’s strategic recommendations, with CHC marking an important pivot in HC and SBCC programming in Uganda. Due to the experiences of its predecessors, CHC had a massive body of locally-generated evidence to draw upon and leverage following a decade of experimentation and intensive learning about what worked, how, and why. At the same time, five years on, many of the same barriers that were found to impede implementation of HCP continue to remain bottlenecks to CHC’s own achievement of its strategic objectives and intermediate results. Much of the story we tell in this evaluation with respect to the challenges, successes, and lessons of CHC and the Obulamu campaign, then, are not unique to this particular project. There are historical antecedents to our findings and future follow-on SBCC mechanisms should expect to encounter and be made to address these ongoing issues. Communication for Health Communities A five-year (June 2013-June 2018) activity, CHC is predominantly funded through USAID, with inputs from CDC and Department of Defense (DoD), with the goal of supporting the GoU and partners in reducing HIV infection, total fertility, maternal and child mortality, malnutrition, malaria, and tuberculosis (TB). It uses the Obulamu? (How’s Life?) concept as the organizing principle, campaign platform, and overarching brand for an integrated set of HC interventions targeting these six health areas. A Luganda greeting, Obulamu is a conversation starter and means of exchanging pleasantries across the Buganda region of central Uganda. In its role as a campaign brand, the word Obulamu has also begun to be used in some non-Luganda-speaking parts of the country with reference to personal health and well-being.23 Planned as a departure from traditional disease-focused approaches that have tended to respond to health problems via technical silos,24 the Obulamu campaign was conceived in 2013/14 as a response to the broader consensus, described above, that Uganda’s HC landscape was characterized by limited implementation and scale-up, fragmented implementation of interventions, disease-focused messaging with many competing messages, audience fatigue with instructive messages, and a 20 USAID/Uganda, 2013, pp. ix-x. 21 USAID/Uganda, 2013, pp. xi 22 USAID/Uganda, 2013. 23 However, how the word Obulamu is used also depends greatly on the region, the local language of its population, the language family to which the language belongs (e.g. Bantu versus Nilotic) and the region’s history and political, economic, and social relationship to central Uganda. Again, we will return to this issue at later points in the evaluation. 24 FHI360/CHC, 2014. 18 disconnect between mass media and interpersonal communication (IPC). 25 As one senior health educator said: If I look back in the area of health communication, we’re always criticized for being vertical, for looking at the disease and not the human being. Coming up with vertical programs and not integrated, having bias or going for programs that have money, ignoring others...There was always that story of integration, but nobody had ever come up with a strategy or a model [KI_KLA_CG_4] The national Obulamu campaign was designed to address these concerns by ensuring messages were integrated and delivered at scale, life-focused (not disease-oriented), adaptive to audience feedback, and linked to interpersonal engagement with beneficiaries. CHC thus supports GoU and other partners in rolling out Obulamu across Uganda. CHC’s strategic objective is to increase the adoption of healthy behaviors (including uptake of critical health services) through strengthened health communication, in order to contribute to these improved health outcomes. 26 It works via three intermediate results (I.R.s): IR1: High quality health communication interventions designed and implemented CHC supports GoU, IPs, and other partners in the design and implementation of state-of-the￾art, theory-grounded, evidence-based, appropriately-targeted, multi-channel communication strategies and messaging linked to service delivery. CHC supports Uganda’s national Obulamu strategy for integrated HC with respect SBCC for six intersecting health issues (HIV, tuberculosis (TB), malaria, maternal and child health (MCH), nutrition, and family planning) targeted to four identified Life Stage (LS) groups (couples, pregnant women, parents of under-5s, and adolescents). CHC links to efforts by central and district government, regional health IPs, and facility and community-based health workers to improve integrated health service delivery. IR2: Improved coordination of health communication interventions CHC strengthens capacity of relevant GoU structures, programs, and coordination mechanisms, as well as USG-supported regional health implementing partners (IPs), to play a stronger leadership role in coordination of HC and to ensure central and local government ownership of these initiatives, as well as to improve intra- and inter-agency coordination through mechanisms such as technical working groups (TWG) and committees. IR3: Increased research and knowledge management to enhance health communication CHC carries out operational research and robust impact evaluation to improve the evidence￾base as to the linkages between HC initiatives, behaviour change, and improved health outcomes, as well as to identify and disseminate the most effective and cost-effective approaches/models and to respond adaptively to new insights.27 CHC is designed specifically to work with and through the MoH and other USG regional health IPs engaged in technical assistance for integrated health service delivery across USG agencies (e.g. USAID, CDC, DoD).28 It is important to note that many of these regional health IPs, especially those managed by USAID, have changed over the duration of CHC. CHC also works with not-for-profit direct service delivery IPs,29 non-USG development partners, international and local non￾governmental organizations (NGOs), and civil society organizations (CSOs), as well as creative partners in advertising and mass media. 25 Kayongo E, Lessons from the integrated life-cycle based health campaign in Uganda. (Kampala, FHI360, 2017). 26 USAID/Uganda, Request for Application (RFA) Number RFA-617-13-000001: “USAID/Uganda Communication for Healthy Communities (CHC) (Kampala, USAID/Uganda, 2012b). 27 USAID/Uganda, 2012b. 28 Relevant regional health IPs which provide technical assistance currently include USAID/RHITES-SW, USAID/RHITES￾EC, USAID/RHITES-E, USAID/SUSTAIN Northern Uganda, CDC/IDI, CDC/Baylor, CDC/TASO, and CDC/Mildmay. Additionally, northern Uganda will very soon receive RHITES-Acholi and RHITES-Lango. 29 These might also include IDI, TASO, Mildmay and other IPs that run health facilities and directly deliver health services. 19 CHC, therefore, does not exist in a bubble, but builds upon and leverages existing efforts by the GoU, its partners, and previous USG communication activities which have focused on developing tools and strategies for social and behavior change to motivate individuals, households, and communities to adopt better health practices linked closely to service delivery. EVALUATION PURPOSE & QUESTIONS Evaluation Purpose The purpose of this evaluation is to establish the extent to which CHC achieved its project objectives. However, as CHC’s IR3 already involves ongoing internal impact evaluation, this external evaluation focuses less on measuring the extent to which CHC achieved outcomes in relation to objectives, and more on using these measurable achievements to extract learnings so as to inform future USAID/Uganda SBCC initiatives. What evidence - both qualitative and quantitative - do we have of CHC’s successes and challenges? What does this evidence tell us about how SBCC in Uganda has been/should be designed and implemented? What do these findings mean for USAID/Uganda SBCC programming going forward? What might future SBCC programs, both in Uganda and elsewhere, learn from CHC’s experience? The evaluation thus assesses the intervention (i.e. “In what way was CHC effective? In what way was it not? What evidence do we have of this? Why?”) as opposed to judges its performance (i.e. “Did CHC succeed or not succeed?”). It seeks to establish an evidence-based set of lessons, best practices, and promising new approaches to guide future national HC investments (i.e. “What can the CHC example teach us about how to design, coordinate, implement, generate evidence, and adapt with respect to SBCC interventions?”). With this in mind, this evaluation focuses on “how,” “why,” and “with what implications” as much as on “what” and “how much.” Evaluation Questions Question 1: What did USG and the implementer learn about the target populations’ misconceptions around intervention messaging and how did they adapt? Question 2: What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? Question 3: How did the implementer address converging messages and integration during implementation and what lessons can be learned? Question 4: How have CHC-relevant coordination structures (intra-agency coordination, national structures, partner platforms) contributed to improved capacity of Government of Uganda and other partners to design, coordinate and implement integrated SBCC? Question 5: How did information generated through research and knowledge management boost and improve HC in the country? Question 6: Which are the main barriers that exist as to the delivery of the six services promoted by CHC? What changes should be made to SBCC programming in the future to improve linkages between HC and integrated health service delivery partners? These questions were provided to the evaluation team by USAID/Uganda. The operationalization and conceptualization of these questions - including sub-questions - can be found in Annex 3: Getting to Answers Matrix. Some of these sub-questions proved to be repetitive once qualitative data was collected; however, the evaluation sought to ensure that all were answered throughout the report. METHODS Evaluation Design and Approach 20 The full methodology can be found in Appendix 4. Briefly, the evaluation followed a mixed method, non-experimental evaluation design, relying predominantly on qualitative data supported by quantitative evidence of effectiveness in order to understand what worked with respect to CHC, how it worked, and why. The evaluation drew upon an array of primary and secondary data sources including: 1) interviews with key informants (KIs); 2) focus group discussions (FGDs) with target audiences; 3) CHC and partner program documentation (quarterly reports, annual reports, work plans, research, communication materials); 4) secondary datasets including Lot Quality Assessment Sampling (LQAS) surveys, and Phase I/baseline data and Phase II/endline data from CHC’s ongoing evaluation/studies and other research activities. As CHC worked predominantly through and with a diverse range of health IPs, the question of attribution of success was not the focus of the evaluation and, thus, was not built into the methodology. Rather, the evaluation sought to explain and explore the reasons for measurable achievements (or lack thereof) and to translate these findings into a set of learnings for USG SBCC planning and program development going forward. Sample Selection Criteria The evaluation relied predominantly on criteria-based purposive sampling across multiple stages (district, sub-county, health facility) since a) the population of technical experts with respect to CHC is relatively small and bounded; b) the fact that maximum variation in opinion and insight was deemed more important for learning than achieving a random sample; c) there was limited time available for the evaluation. Respondents were chosen from among stakeholders at national-, district-, facility-, and community-level. This method led to the selection of the following units at sub￾national level (Table 1): Table 1: CHC Evaluation Target Districts, Sub-Counties, Facilities District Mukono Iganga Ibanda Kyenjojo Acholi Lira Sub-County Kyampisi Bulamagi Nyabuhikye Kyarusozi Atiak Amach Health Facility Kyampisi HC III Bulamagi HC III Nyabuhikye Prison HC II Kyembogo Holy Cross HC III Atiak HC IV Amach HC IV Participants for FGDs within target districts were identified based on a) the four Life Stages (LS) of Obulamu and b) gender. The goal, then, was to carry out eight gender-specific FGDs per sub￾county/district with between eight and twelve paticipants per FGD.30 Except in Ibanda where only seven FGDs were conducted, this led to a total of 47 FGDs involving 527 participants. This methodology was imperfect in that, after getting underway, it became clear that many FGD participants actually fell under more than one LS group (indeed, a number of participants fell under all four). While this was problematic from a sampling perspective, it also is an important finding for the evaluation. For KI interviews, the evaluation used purposive/non-probability sampling, following the principles of expert sampling and maximum variation sampling. KI selection critieria available in Annex 4. In all instances, the categories were limited in scope and selection was opportunistic, depending on which stakeholders were available on the day of the interview. Although a number of stakeholders originally contacted did not turn up for interviews, the evaluation still ultimately spoke to 103 KIs across 10 categories. Data Collection 30 [4 LS x 2 (i.e. 1 male + 1 female) x 6 districts] 21 For both FGDs and KI interviews, category-specific, semi-structured interview guides were developed to prompt stakeholder discussion on relevant key issues. These are available in Annex 4 These guides were not meant as closed-ended surveys but rather as a series of prompts to ensure that FGD participants and KIs touched upon all themes relevant to that particular category of stakeholder. For KIs, the focus of the interviews was on technical expertise while FGDs focused on message clarity/comprehension, memorability, and relevance, as well as perceptions as to the channels of communication and strength of implementation. FGDs were conducted in the relevant local languages - Runyankore, Rutooro, Lusoga, Luganda, Acholi, and Langi. KI interviews were conducted in English except in the case of discussions with supply side messaging experts (health workers, VHTs, champions) which were conducted in local languages. All interview data has been transcribed from recording and translated into English. Data Analysis Transcripts from interviews and FGDs, as well as existing CHC documentation, were analyzed thematically, using codes assigned a priori based on the evaluation questions and, via grounded theory, through emerging themes developed collectively over the duration of the fieldwork. Purposive searches to aid writing were used using Nvivo qualitative software. Drawing on secondary data, the quantitative analysis looked to compare results from the Obulamu campaign Phase I/baseline data (March 2015) and Phase II/endline data (September 2017) evaluations with the overall behavioral outcomes observed through the LQAS trend data from 2012-2016.31 Phase I/baseline and Phase II/endline data was initially presented at the CHC Data Interpretation Workshop in Seeta (December 6th-7th, 2017) and subsquently provided to the evaluation team by CHC on December 14th in a compressed zip file. This quantitative data was analyzed using both trends and timelines to show program performance over time, and to detect the effect of context and program events on program performance indicators. Analysis of variance was used to determine difference in performance indicators across the following variables: a) rural/urban; b) USG implementing agencies; c) implementation regions; d) implementation context; e) emerging issues. A description of the quanitative analysis is described in Annex 4. Ethical Considerations Ethical considerations are described in Annex 4. Here we note only that, because of limited time and resources available, this evaluation was not passed through separate IRB channels. The primary review mechanism was the review of the inception report by the USAID/Uganda health team. However, CHC passed through IRB both in Uganda and internationally, with IRB approvals covering November 2018 when the evaluation took place. The evaluation team ensured that its work fit under existing research ethics agreements. Additionally, the assessment team ensured confidentiality of all transcripts and separate documents linking transcript codes to KI and FGD participants’ identifying information (subjects' names, place of residence or work, contact information) were created. All quotes or pieces of information linked to KIs and FGD participants have been anonymized in this report. Limitations Given the limited time available for this evaluation, there are a number of limitations to the evaluation findings. Certain evaluation questions lacked both breadth of data collection and depth of data analysis, and representative sampling was not possible, especially with respect to KIs and client- 31 Note: although CHC has referred to these surveys as “baseline” and “endline” in numerous reports, emails, and data workshops, the evaluation team has been asked by CHC to refer to these as Phase I and Phase II surveys for their own adminsitrative purposes. 22 level FGDs. There is a degree of bias as some KIs - especially those from CHC - were simultaneously engaged for data analysis, and for input into the content and packaging of the final product. Additionally, the availability of adequate service delivery data is problematic and completeness and accuracy of data sets are not guaranteed. Finally, as with any evaluation of a particular activity being rolled out in a project-rich environment, there are questions of attribution. The purpose of this evaluation, however, is to gain an understanding of stakeholder perspectives on how CHC in Uganda has (or has not) been working. The methods and activities have therefore been selected to achieve maximum variation of stakeholder opinion and to facilitate joint debriefing and learning, as opposed to only measuring outcomes. With this in mind, the evaluation sought less to assess performance than to explore context, highlight alternative views, and illuminate questions that might require further investigation FINDINGS Before introducing the findings, it is important to again reiterate the context of CHC’s HC and SBCC interventions in Uganda. As mentioned before, CHC is a PEPFAR funded above site mechanism. This means that, although Obulamu is a national campaign, downstream CHC does not work independently of existing IPs. This is important, as evaluation often is concerned with the question of attribution, i.e. can outcomes be attributed directly to the activities undertaken by a project? In the case of CHC, Obulamu reached districts, facilities and communities via existing PEPFAR-financed health infrastructure, implemented by regional health IPs and overseen, depending on the region, by different USG agencies (USAID, CDC, DoD). CHC was, therefore, the HC facilitator for these IPs, enabling them to present a common set of HC materials and messages and to focus more fully on the things they do best: that is, providing technical or direct support for health service delivery and capacity-building for sub-national health sector management. Regional health IPs, however, themselves represent partnerships between prime and subprime organizations, all of which have specific roles, responsibilities, and contributions. Attributing improvement - or lack thereof - in health outcomes purely to CHC (or, indeed, to any partner) is not what we seek to do here in these findings. Rather, the evaluation assumes that CHC was one important piece of a puzzle, the ultimate picture of which depended on the functionalities, efficiencies, and synergies of all the pieces working together. The evaluation, then, tells a story of what CHC did in the context of how CHC worked with others. Understanding this context is essential in order to understand what worked with respect to Obulamu, where it worked and why, so as to support a logic model for an evidence-based set of recommendations. NOTE: In this section we address the findings by evaluation question. However, for the purpose of narrative flow, we do not address the questions in chronological order as in the scope of work. Evaluation Question: Best Practice Strategies and Behavioral Outcomes Q2: What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? Program Strategies The project adopted best practices/principles that have long been applied within the private commercial marketing sector, as well as within the field of behavioral science. Key best practices adopted by CHC for Obulamu included: ● Life Stage segmentation 23 ● Positioning and branding ● 360 degree communication ● Interactive campaign design ● Use of formative research/Adaptation to findings It is important to note that past HC projects (e.g. HCP II) and campaigns (e.g. “Good Life”) in Uganda have employed similar principles; however, Obulamu was innovative in the way it deepened, expanded upon, learned and adapted to, and greatly scaled up these practices. In this section, we review the best practices mentioned above, with the exception of use of formative research/adaptation to findings as these issues are discussed at length in subsequent sections of the report. We then examine qualitative and quantitative data on effectiveness, keeping in mind that behavioral and other health outcomes are dependent on a confluence of partners and programs all working toward similar goals, not CHC working alone. The evaluation also does not seek to attribute effectiveness on a strategy-by-strategy/practice-by-practice basis. This is because the Obulamu campaign relied upon the synergy of such practices, the success of which was, to a large extent, contingent upon the effectiveness of the regional health IPs with which CHC worked in tandem. Initial findings on effectiveness or performance, then, seek to assess the whole of this approach to SBCC in Uganda, not each of its constituent parts. This issue of how a PEPFAR funded mechanism like CHC might achieve a balance between centripetal forces (centralizing and unifying under a national or PEPFAR umbrella) versus centrifugal forces (decentralizing and devolving to meet USG agency, IP, or district partner needs) is one that we return to repeatedly throughout this evaluation, especially in the context of Q4: Coordination and Capacity, and one which any follow-on PEPFAR HC central mechanism will need to consider. --*-- Target Audience Segmentation by Life Stage SUMMARY ● Obulamu used a Life Stage (LS) segmentation strategy based on age and the unique health needs at each stage of life. ● LS segments took awhile for some partners to understand, but once understood, were seen as a valuable way to target communication. ● The four LS, however, were seen as insufficient and additional segmentation – based on regions, key populations, rural versus urban, etc. – is desired going forward. ● There is evidence that the message of “a good life” that the segmentation strategy was meant to facilitate has been understood and embraced by beneficiaries. ● LS segments overlapped considerably. For campaign design, this was seen as a value add by KIs since individual identity is multi-faceted and the boundaries between categories are often blurred. ● LS were also harnessed for monitoring and evaluation purposes. For some forms of research (Action Media, special studies), they were valuable; however, when used as a means of disaggregating data for systematically studying SBCC outcomes, they were problematic. 24 Target audience segmentation has been an integral best practice in marketing since the 1920s.32 In the case of CHC, the need for rethinking segmentation became clear in the earliest stages of project design when the technical proposal recommended adapting and updating the Family Life Cycle approach, a segmentation strategy based on LS that FHI360 had successfully supported elsewhere.33 Using an LS approach, Obulamu’s target audience was divided into discrete segments based on age and the unique health needs of each stage [Exhibit 2]. The rationale of Life Stage segmentation is that transitions over the life cycle generate renewed interest and motivation to acquire new information, embrace new actions, confront new realities, and manage new expectations. Examples of key transitions include adolescence, adulthood, pregnancy, relationships, marriage, and parenthood. While the model may appear linear, it also assumes that individuals’ lives and health concerns are different not only across, but also within, each stage. This enabled Obulamu to bring into its fold audience segments that were both related, but also distinct. Obulamu operationalized this approach through a shared, overarching framework connecting the HC needs of individuals, households, and communities (this will be discussed further. The model was translated into four LS groups34 segmented as per Table 2:35 Table 2: CHC/Obulamu Life Stage Group Segments LS1 (Couples) Young adults in relationships, aged 20-30 LS2 (Pregnant Couples) Pregnant women and partners, aged 15-49 LS3 (Caretakers of Under 5s) Female and male caregivers of children under five, aged 15-49 LS4 (Adolescents) Sexually active girls and boys, aged 15-19 The segmentation strategy thus enabled the Obulamu campaign to appreciate, incorporate, and adapt to findings from formative research.36 CHC then was able to come up with sets of tailored core messages that were incorporated in the Obulamu LS campaign strategy. The core messaging plan for different life stages is elaborated below: Table 3: Obulamu Core Messages by Life Stage Lovers Pregnant Couples Caretakers Adolescents 32 Schwartzkopf S, “In search of the consumer: The history of market research, 1890-1960,” in The Routledge companion to marketing history, eds. DGB Jones & M Tadajewski M (Oxon, Routledge, 2016). pp. 61-84. 33 FHI360, Technical proposal: USAID/Uganda Communication for Healthy Communities (Durham, NC, FHI360, 2013a). 34 Each LS includes key populations as appropriate e.g. commercial sex workers, fisherfolk, uniformed forces, and truckers. 35 FHI360. Obulamu national integrated health communication platform, June 2013-June 2018). Kampala, FHI360, 2013b). 36 For example, findings that “recently married men show[ed] more openness to information about family health as they explore new responsibilities as heads of households” or “adolescents in school [had] different levels of interest in information about sexual health and need to be approached differently than single men or women of college age and young adults in the workforce”. FHI360, 2013b, p. 14. 25 Use condoms Reduce sexual partners Test for HIV, receive results Men – get SMC Women – support SMC Prevent unplanned pregnancies via a contraceptive method of your choice Maintain discordant status through adherence to positive prevention and treatment Go for TB screening and testing if cough lasts 2+ weeks Seek correct info on sexual health/reproduction plans Recognize danger signs of pregnancy Make birth preparedness plan (attend ANC, save for birth items/transport) and talk to your partner about it Begin ANC early, attend 4x When attending ANC, demand IPTp 1-2, test for HIV, enroll into eMTCT if positive, receive/sleep under net Learn recommended newborn care practices e.g. skin-to-skin contact, early initiation of breastfeeding Deliver at a health facility Adhere to ART and breastfeeding guidelines Seek info on postpartum care (i.e. FP, healthy timing and spacing of children) Practice good nutrition for pregnancy and breastfeeding Put baby to the breast within one hour after birth Give baby ONLY breast milk during the first six months Feed child 6-24 months mixed diet, small portions, often Breastfeeding women feed on a mixed diet, try eating enough and well Ensure all children under 5 sleep under a treated net Recognize signs of childhood diseases (diarrhea, pneumonia) and seek advice or treatment Give ORS and Zinc for child with diarrhea Go to health center regularly and follow the full course of immunization for the child Wash hands with soap and water before serving food, after using a toilet, and after changing diapers Space next pregnancy at least 24 months to ensure about three years between births Adhere to ART for mothers and children (pediatric ART) Mother and child return to the health center for regular check-up and ART refills Seek correct info on how your body grows and about your new fertility Learn how to make good decisions and negotiate what you want to happen to you and your body (Life skills) Learn how to prevent unplanned pregnancy and HIV and other STIs It’s harmful to have sex too early and to have children too early Use a condom if sexually active Boys - get circumcised to help protect yourself from HIV Test for HIV and get the results Source: FHI360, Obulamu national integrated health communication platform, June 2013-2018 (Kampala, FHI360, 2013b) Table 4: Obulamu Core Messages by Key/Priority Population Female Sex Workers (FSW) Fisherfolk Truckers Use condoms Test for HIV and receive results Men: Get SMC / Women: Support SMC Prevent unplanned pregnancies by using a contraceptive method of your choice Maintain discordance status through adherence to positive prevention and treatment w/regular partner Go for TB screening and testing if your cough lasts two or more weeks Seek correct information on your sexual health Use condoms Practice mutual fidelity – reduce sexual partners Test for HIV and receive results Men: Get SMC Prevent unplanned pregnancies by using a contraceptive method of your choice Maintain discordant status through adherence to positive prevention and treatment Go for TB screening and testing if your cough lasts two or more weeks Seek correct information on your sexual health and reproduction plans Use condoms Practice mutual fidelity – reduce sexual partners Test for HIV and receive results Men: Get SMC Prevent unplanned pregnancies by using a contraceptive method of your choice Maintain discordant status through adherence to positive prevention & treatment Go for TB screening and testing if your cough lasts two or more weeks eek correct information on your sexual health and reproduction plans Source: FHI360, Obulamu national integrated health communication platform, June 2013-2018 (Kampala, FHI360, 2013b) A whole-life, LS-based segmentation strategy took some time for stakeholders to understand, especially at district level, since - as described earlier - SBCC, as well as earlier approaches to HC in Uganda, had previously been primarily disease-specific (See Appendix 7). Moreover, while USG regional IPs have, over the last five years or so, been moving toward integrated health service delivery, for a number of reasons this technical assistance often falls back into disease silos.37 As one former government health educator put it: Initially, the districts took time to understand that life stage approach because they were used to the disease approach and they also had on-going programs still on diseases. [KI_KLA_CG_4] Additionally, planning for LS roll out was meant to go sequentially, with campaign materials and activities delivered for LS 1 first, followed by subsequent stages.38 IPs, however, already had their plans and funding mechanism requirements and, by the time CHC started, it was impossible to align every partner to a sequential roll out, especially if LS messaging did not include a focus on the health area for which the partner had funding.39 There was also recognition that concentrating on sequential roll out was neglecting critical parts of the population, in particular those that were already PEPFAR targets (for example, PEPFAR/DREAMS adolescent targets): 37 e.g. Due to stringent USG reporting requirements, USG funding recipients often end up redirecting efforts towards meeting annual PEPFAR HIV/AIDS or TB targets or PMI malaria targets, thus the focus of IPs necessarily reverts to silos. This was mentioned by a number of district and IP stakeholders in the interviews, and has been well-documented in the public health literature. For example, see Lohman N, Hagopian A, Luboga SA, Stover B, Lim T, Makumbi F, et al., “District health officer perceptions of PEPFAR's influence on the health system in Uganda, 2005-2011,” International Journal of Health Policy Management 6, no. 2 (2016): 83-95. 38 FHI360, 2013b. 39 Personal communication KI_KLA_PRIME_8_8.2.18 26 At the start of the project, we thought we would go systematically, but when you reach Life Stage 1, adolescents are saying: “No. You’ve not addressed us yet.” [KI_KLA_SUB_1] Sequential roll out, then, did not go as planned, and several LS were rolled out concurrently. Where relevant, the evaluation team asked KIs about their perception of the value of LS segmentation. No KI spoke negatively about the approach. This was an important finding, as the evaluation team quickly found utilization of LS segments as an evaluation tool - i.e. as a means by which to analyze data from FGDs - to be problematic since most FGD participants actually represented multiple, overlapping life stages. For example, adolescent girls were simultaneously pregnant, and caregivers to under-five children, and one half of a young couple. The integrity of the evaluation focus groups - and the evaluation team’s ability to treat information coming from these groups as the unique perspective of a particular, bounded population - was thus compromised from the first. Therefore, one immediate question the evaluation team had was: What is the value of LS segments when life stages are not mutually exclusive? This value was perhaps best articulated in two KI interviews, one with a district health educator (DHE) and one with representatives of a regional IP: We can talk of mothers, and you find that there are mothers of 40 and mothers of 15 or 16. But their understanding differs. Because this young mother is more of a child than an adult. So if you generalize that “These are all mothers” then you think their understanding is the same. And yet they have different experiences that give them different ways of understanding. [KI_LIR_DG_1] [E]ven if I fall in different age bands or different categories or sub-populations, and the message is structured for those different sub-populations, they are messages that appeal to me at this particular time. If I am a teenage mother and if I am an adolescent …[and] I am HIV positive... [KI_MUK_RHIP_1_2] …I still want to go and dance. [KI_MUK_RHIP_1_1] Eh hee! “I still want to go and dance”. So the message on “Mother taking your child to immunization and having him tested early” will appeal to me [as a mother]. It appeals to me who goes across different age bands....Then the other message of “Get yourself treatment. Test and Start” will appeal to me because I am in that category of people who are HIV positive. And will also appeal to other people...who cut across. So if you want to mix up, and not categorize, you lose the message. You lose the key message you are trying to pass on. [KI_MUK_RHIP_1_2] The overlap within the LS approach was seen by a number of KIs as a value add, not a detriment, precisely because individual identity is multi-faceted and the boundaries between the categories within which human beings classify themselves are always blurred. Obulamu’s LS approach enabled a pregnant teenage mother to be communicated to as a teenager and as a mother and as a woman expecting a baby. If she had simply been targeted under the umbrella of “pregnant woman,” much of her overall life experience - and, thus, her critical health experience - would have been missed. Many Obulamu beneficiaries themselves spoke at length about the inclusiveness of the messages, and recognized the appeal of the campaign as being in the sense it conveyed of holistic good health for everyone. The following exhibit is a tally of responses among focus groups by LS in all districts but Mukono.40 In all instances, the response that Obulamu is for “everybody”, “everyone,” or some version of “for all” was easily the most frequent response. This 40 As explained in Annex 4: Methodology, Mukono transcripts came in a form that was difficult to analyze, although we do extract quotes from them. 27 is the case even in groups - such as pregnant women and adolescents - who were convened specifically and explicitly due to their age or pregnancy status and, thus, might have been expected to exhibit a clear selection bias (i.e. given a response based upon the shared identity for which they were purposely convened) (Exhibit 3). This finding is important as one of the criticisms of the use of segmentation in HC - especially when applied to high risk groups - is that it can cause members of those groups to feel stigmatized or unfairly targeted.41 That so many FGD participants saw Obulamu - even when providing messages on specific health problems (HIV, malaria, pregnancy) - as nevertheless being for “everyone” suggests that the LS approach, far from “singling out” certain at-risk populations, instead enabled individuals to understand the broad relevance and applicability of messages, at the same time as understanding the specific relevance and applicability to themselves. For example, Obulamu campaign targets the entire people. [R8, LS4_M_KYE] I think Obulamu is for everyone because everyone needs a good health. [LS3_M_MUK] Obulamu came for all Ugandans. [R7, LS4_M_IGA] Obulamu has come for everybody. Even a baby who is just born...even older persons. Obulamu thought the newborn might be born with some problems so they thought: “This young generation, we should lift them up through giving support and advice.” This has improved their life too. Obulamu is supporting our children and us. [R6, LS1_M_LIR] I think it targets all of us: women, men and children. [R3, LS3_F_IGA] “Obulamu is for everyone. That is, if I have a wife, we are suppose be healthy and if I test positive I should know the way to ART medication. If I have a baby, the mothers should know how to take care of the baby and what do after the baby clocks six months. Therefore Obulamu is for everyone” [LS2_M_AMU]42 At the same time, it is also important to note that pregnant women and male and female adolescents were more likely to view Obulamu as intended for their particular LS (i.e. “pregnant women” or “youth” or “young women”). Additionally, many men in LS 1 (young couples) also strongly felt that Obulamu was for “youth” (likely because youth and “men in young couples” are overlapping categories). The ability of participants in these FGDs to strongly identify Obulamu as being meant for them - i.e. for youth and pregnant women - may, again, be because of selection bias in which they assumed the facilitators wanted them to talk specifically about Obulamu’s relationship to this particular aspect of their life. That said, since these segments were able to clearly identify and articulate the groups of messages targeting LS 2 and 4, it suggests that Obulamu’s segmentation strategy - and the ability of segments to identify strongly with targeted sets of messages - was particularly relevant to the youth and pregnant women. While more extensive research is needed to fully understand the balance CHC achieved between “Obulamu is for everyone” and “Obulamu is for people like me,” focus group responses indicate a degree of success. 41 e.g. Gruber J, Caffrey M, “HIV/AIDS and community conflict in Nigeria: implications and challenges,” Social science and medicine, 60, no. 6 (2005):1209-18. doi:10.1016/j.socscimed.2004.06.053 42 Note: Throughout this evaluation we include quotes from FGDs. Due to length requirements, we cannot provide quotes of every instance in which a theme appears. However, to minimize bias and misrepresentation with respect to qualitative evidence – especially quoted evidence - we do not use any quotes as evidence unless we have identified these themes across transcripts representing all districts and most LS groups. Nor do we use multiple quotes as evidence of a theme from the same FGD unless we can clearly distinguish that these quotes came from different people. Thus, if a theme is only found in three quotes stemming from a single FGD, among a single LS group, in Kyenjojo district, we do not use this as evidence. An earlier, more lengthy, draft clearly shows the geographical, LS, and gender spread of quotes. 28 Challenges with Life Stage Segmentation Per Exhibit 2, Life Stage segmentation was also meant to be bisected by other segments, in particular PEPFAR key and priority populations (KP/PP) that face increased risk of HIV (e.g. female sex workers, fishing populations, uniformed services). As a central mechanism, CHC had to be responsive to health IPs’ needs, as partners were simultaneously required to meet PEPFAR targets in these areas. One such area was PEPFAR/DoD support to the Uganda People’s Defense Forces (UPDF) which, as a mobile population with its own hierarchy, vocabulary, and set of challenges, required its own set of targeted communication.43 Both UPDF and DoD stakeholders felt that Obulamu and CHC had been somewhat responsive to these very specific needs. But they were hoping to see far more engagement in the future: We are really wanting something that speaks our language. The language of soldiers. They are a specific population. They feel that what is for the general population is not for them. So for soldiers, special tailoring is vital. With soldiers, if you come to them talking about HIV they say, ‘I’m on the battlefield. HIV is not my main worry.’ So the message can’t just be about HIV or health. It has to also be about sexual satisfaction or feeling like a man. [KI_KLA_USG_1] No, they are not specific. We had actually addressed that challenge to CHC and they were talking of maybe looking for specific materials...targeting maybe officers, then men [and] women in uniform, so that we come up with messages that are actually appropriate for our people. [KI_KLA_CG_2] Following on this, two of the biggest frustrations by KIs, especially at regional level, was the fact that a) the LS strategy did not target all the relevant KP/PP required to meet PEPFAR targets and b) KP/PP potentially varied from region to region, as did local priorities and concerns, thus necessitating improved regional engagement and tailoring. This was particularly felt by KIs in Acholi and Lango regions: While I still believe the messages are supposed to be consistent…but also we need to put into consideration the different context of each of the regions…we should not centralize everything. We need to have a mechanism in place where the region has a lot to contribute in terms of how we can always roll out any health communication project. [KI_GUL_PRIME_1] I see no reason why Obulamu should bring someone from Kampala! We have a lot of people in our sub-region! And even the local people listen to them and talk to them. So if you could give that feedback to them that that is very wrong. When these guys go to the local community, some of these people don’t even speak any English. [KI_LIR_CP_1_2] Thus, there was a disconnect between having a central mechanism responsible for a nationally-rolled out, integrated LS-based campaign (in which the primary regional differentiation was via the language used in communication channels) and the realities of reaching PEPFAR targets that include a clear focus on KP/PP above and apart from designated LS and require a broader and more complex understanding of local priorities and perceptions with respect to illness, well-being, sexuality, and appropriate communication. --*-- Branding and Positioning SUMMARY ● In Obulamu, CHC has succeeded in creating a strong, universally-recognized brand that helped shift the focus away from disease silos toward life and well-being. 43 The UPDF is also considered a separate district by the MoH. 29 ● Some beneficiaries – especially those in the North – had issues with the choice of a Luganda word as a national brand name, although there is some indication that objections are being overcome. That said, brand equity may still need reinforcement in some parts of the country due to the politicization of language in Uganda. The use of positioning was made popular by thought leaders Ries and Trout in the late 1960s.44 One of the most important elements of the CHC approach was the repositioning of health under the Obulamu brand as an integral part of life and living well. By creating a strong, universally recognized national brand, CHC and its partners hoped to shift the focus away from disease, illness, and infirmity (i.e. what should a sick person do?) toward well-being and living a good life (i.e. what should a healthy person do?): We have this perception that you only go to a health center when you are sick. And now we are two healthy young adults who should go and check before we become pregnant. [KI_KLA_CP_5] Most people wait to fall sick and seek health services, but Obulamu has awakened people [KI_MUK_SSCMP_1_3] It came to show people their lives are theirs so that they can own them. [KI_IGA_SSVHT_1_4] KIs consistently highlighted CHC’s ability to create strong national brand as one of its key strengths, and Obulamu’s powerful brand equity as one of its key benefits. KIs did not always use the same language to describe this so, to a certain extent, an analysis of Obulamu’s value as a brand is a subjective one. Still, KIs across regions and professional categories talked extensively about how Obulamu created “top of the mind awareness” and made statements along the line of “everyone knows Obulamu,” “Obulamu is known,” “Obulamu is trusted,” or “Obulamu is everywhere”. 45 When you go upcountry, almost everywhere the word Obulamu is known. The posters [are] in almost every part of the country [KI_KLA_CG_4] They know the Obulamu campaign...at least the brand is out there...of course, when you go ahead to probe further, there is some who know the message…[and there] are some who are not so sure, but I can say the branding is quite high. Lots of them know the Obulamu campaign [KI_IGA_RHIP_1] It was clear that KIs saw the brand, Obulamu, as recognizable, memorable, and attached to exceptionally high quality products, materials, and messages. FGDs also indicated that there was extremely high brand recognition and that the message of a “good life,” “better life,” or “healthy life” (as opposed to a disease-specific focus) had been understood by many beneficiaries, with the phrase “good life” specifically referred to in 15 out of 47 focus groups, and a more general discussion of “handling one’s life,” “living one’s life,” and “living a healthy life” discussed throughout the transcripts.46 Obulamu we hear from the radio posters pinned on trees and through the village health teams who also talk about Obulamu in the villages on how we should live the good life [R10, LS1_M_LIR_1] Obulamu campaign helps all the population from children to adults because everybody deserves good life [R9, LS1_M_KYE_1] 44 Ries A, Trout J, Positioning: The battle for your mind, 20th edition (New York, McGraw-Hill, 2000). 45 See, for example, KI_KLA_CG_4, KI_KLA_CG_2, KI_IGA_RHIP_1, KI_KLA_RHIP_1, KI_GUL_CP_1_1. 46 “Good life” was used in at least one focus groups in every focus district except Amuru; however, even transcripts from Amuru have reference to living a “healthy life” in the general sense (as opposed to specifically living a good life while being HIV positive). 30 Obulamu is meant for me, it reminds me of my responsibility as a parent to make sure that my family lives a desirable life by following and doing what Obulamu campaign encourages me to do. [R1, LS3_F_KYE_1] According to me and what I have heard, Obulamu means how is your live or how you live your life or how you handle challenges that come with life. [LS4_M_AMU] One important lesson to be derived from such quotes is that none were specific to living a good life in the context of being HIV positive. That is not to say that there were not many instances in the transcripts in which Obulamu and a good life was discussed from the point of view of a person living with HIV/AIDS (PLWHA). However, as many, if not more, participants related the good life concept of the Obulamu brand to a sense of general well-being for everyone, not simply PLWHAs, suggesting that CHC’s attempt to shift away from a siloed, disease-focused approach has made some headway. Challenges to Branding and Positioning Ultimately, the evaluation found that Obulamu as a national brand was widely recognized, well understood, and highly valued by KIs and most participants in FGDs, with Obulamu’s strong brand equity considered one of CHC’s lead achievements. That said, there were some challenges branding the national campaign, ones which link to issues discussed earlier about overcentralization of segmentation and which will emerge again throughout this evaluation. For example, the politics of language in Uganda meant that the selection of a Luganda word as the brand name of the national campaign was not unproblematic. The politicization of Luganda as a potential “national” or “unifying” language has been well documented elsewhere.47 It is a testimony to CHC’s tremendous efforts - as well as the power of Obulamu’s promise to beneficiaries of a good life through healthy living - that a Luganda phrase has been able to resonate so effectively nationwide. Additionally, between 40 and 60 other languages are spoken in Uganda, falling into three language groups, all of which are, in their own way, problematic.48 That said, some KIs did discuss at length the difficulties of utilizing a Luganda phrase as the brand for a national health campaign and how there is still work needing to be done. This is, not surprisingly, particularly an issue in the north, especially post-war Acholi sub-region, where all KI’s mentioned the issue of the choice of a Luganda word as the brand name as a hurdle that has had to be overcome: The misunderstanding is “Why does it have to be Obulamu? Why does it have to be in a language we don’t understand?” Then I am supposed to explain and explain. And still everybody says “No! No! No! No!” … If you are going to do a national campaign in the country, you need to find a brand that everybody in the country can identify with.” [KI_GUL_PRIME_1] The word Obulamu to people this side, they found it was not friendly because it’s a Bantu word. And so they were saying “Why didn’t they have regional names?”…At some point it was confusing people. So people were a little bit hesitant. But when it came to pushing Obulamu, yes, it was 47 E.g. Chibita M, “The politics of broadcasting, language policy and democracy in Uganda,” Journal of African media studies, 1, no. 2 (2009): 295-307. 48 Mukama, R. "Theory and practice in language policy: The case of Uganda." Kiswahili, 72, no. 1 (2009); Namyalo S, Nakayiza J, "Dilemmas in implementing language rights in multilingual Uganda," Current Issues in Language Planning, 16, no. 4 (2015): 409-424. 31 received but with some resistance. Because they felt it was not something that they owned. [KI_GUL_CP_1_1] I don’t think I have come across. But there was a time when maybe …okay…this was in the villages. When people were asking about Obulamu, its in Luganda. “What is the meaning of that?” Yet its like a program. So people were like “Maybe these Bagandas are trying to impose something on us. So what is it?” [KI_LIR_RHIP_1] We as community leaders have been trying to interpret such words into our Langi dialect for our people to understand. We should, in the future, translate the word Obulamu into one word in local Langi dialect. [KI_LIR_SSCMP_1_1] While overcentralization of the Obulamu campaign is a theme that this evaluation will continually return to, by and large, those KIs who expressed difficulty with the use of a Luganda phrase as a brand name, nevertheless noted that many of the objections have gradually been overcome. [I explain] the genesis of the word Obulamu is just a project name…[a] greeting which inquires about one’s social life in respect to the behavior ... around sexuality and associated diseases, health seeking behavior, to mention but a few…In addition, the explanation about the situation is not in the dialect of Obulamu, but the local dialect. Just as much some will continue to argue, but they get into the understanding [KI_AMU_DG_1] You know that word Obulamu, it has become like a slang.…In fact, if you ask a few guys around here, every morning somebody comes and asks you “How is your Obulamu!”…in our local language they say “Obulamu kwo tye ning’o?” Like “How is your life? [KI_LIR_CP_1_2] Even in the Southwest of the country, KIs noted initial issues with the selection of Obulamu but also felt that brand equity had reached the point where it was important to continue to build on it, irrespective of the language difficulties: Now...Obulamu in phase 2 you want to continue working with your audience. You should not change that brand to something else. You’d just affect the relevance of the message you give to them. [KI_KAB_RHIP_1] --*-- “360 Degree” Communication SUMMARY ● Obulamu used “360 degree” communication to saturate the national and local media and leverage community- and facility-based communication channels, thus creating “surround sound” for its messages. This resulted in very high levels of message recall. Obulamu can be found everywhere. ● Efficient roll out, however, encountered bottlenecks due to a) the reliance on IPs – not all of whom had the same priorities and motivations - to deliver materials and engage district-level stakeholders; b) disorganization with respect to which materials went where; c) lack of regional tailoring and representation in the national campaign and in seed materials. ● While Obulamu prided itself on its use of local languages, many places had materials that did not reflect the language of the community. Campaign photos also did not represent the diversity of Ugandans. In addition to brand recognition, Obulamu set out to saturate national and local media and leverage community and facility communication channels at scale to create a “360 degree” model creating sufficient enough ‘surround sound’ to ensure high levels 32 of message recall and Obulamu brand recognition (Exhibit 5). This approach to SBCC has been championed by FHI360 globally for some time and is not unique to Obulamu, although CHC is frequently highlighted by FHI360 as a key example.49 These channels of communication will be discussed in detail under evaluation question Q3 (Integration and Converging Messages). In brief, however, the model included messaging through a wide variety of print media (e.g. posters, billboards, job aids), electronic media (e.g. radio spots and talk shows, TV commercials, short videos), and interpersonal communication (e.g. youth bashes, community theater, utilization of peers and champions. Using the 360 degree approach, CHC itself has chronicled extremely high levels of message recall and Obulamu brand recognition through waves of listening surveys (Table 5). Table 5: Obulamu Brand Recognition Have you heard or seen Obulamu campaign messages? Frequency Percent Total (n) 15-19 years 172 95.6 180 20-30 years 935 97.5 959 31-49 years 587 95.6 614 50 years and above 277 90.8 305 Overall 1971 95.8 2058 Source: FHI360, Third listening survey: Preliminary findings, (Kampala, CHC, 2017) Indeed, one of the key findings of the evaluation was that there was virtually no focus group participant who had failed to hear about Obulamu: participants in every focus group around the country had heard about Obulamu and could discuss at length about the channels in which they received information.50 Obulamu means good life. I first heard about it from our village VHT, and then after the Obulamu people were in the trading center with a public awareness system vehicle asking people questions on Obulamu and giving them t-shirts. Even at the health center, nurses tell us about Obulamu. [LS1_F_IBA] I know Obulamu as caring about your life. For example, like us boda boda riders, we have to care about ourselves against HIV/AIDS because we meet many women. I have seen messages on radios and posters on trees. [LS1_M_KYE] It was over the radio and these mobile cars moving around the village. [R2, LS2_M_IGA] Their programs are always in the health centers, they also go to the markets where they move with their vehicle calling people to come for testing of their blood for HIV and other services. [R1, LS3_M_LIR] 49 FHI360, A 360 degree approach to social & behavior change communication, https://www.fhi360.org/sites/default/files/media/documents/sbcc-brochure-final.pdf 50 There were several FGD participants in remote Ibanda who had never heard of Obulamu, one in rural Lira, and one in Atiak (who may, in fact, have actually been a visitor from Eastern Congo). However, these were a drop in the ocean. For the most part, everyone knew of Obulamu. 33 Much of the country was blanketed by intersecting, mutually-reinforcing Obulamu-branded HC, at a scale that many had not seen before in Uganda. As a consequence, one of the most notable successes of the Obulamu campaign has been its ability to achieve extremely high levels of brand exposure. During the evaluation fieldwork, every team that went out - no matter how far from the main road - encountered Obulamu media, usually in the form of print media, but also, indirectly, via beneficiaries’ accounts of interacting with Obulamu mobile units and with Obulamu programs on the radio. Most importantly, we encountered Obulamu media in a myriad of places other than health facilities all over the country (See Exhibit 6). Our findings were echoed by one KI who said: Actually that is where I find uniqueness of Obulamu. You know when Obulamu decides to go for a campaign, they will go full swing. You feel it! Like right now if we pass around here, you will see some of these flyers, some of these boards, at least somebody is there hanging out…So you find Obulamu everywhere. Via coverage, via reaching communities, Obulamu is far far far. [KI_LIR_CP_1_2] FGDs suggest that high levels of exposure and engagement motivated many beneficiaries to pass on messages and discuss products, services, and behaviours with one another, creating word of mouth as an additional communication channel. Obulamu has taught me the importance of sharing information with friends on HIV and SMC which has made me come for SMC and encourage my friends as well. [LS1_M_AMU] I have talked about Obulamu with many people, especially those who are like me to start ARVs and stop spreading it to other people. [R3, LS2_F_KYE] Obulamu talks about sharing ideas such as if a woman gets pregnant, she should seek ANC services. After learning such an important message, I had to use this opportunity to advice a friend on how to take care of his pregnant wife as he was not caring. [LS4_M_AMU] I have talked about Obulamu with my friends, about abstinence and family members because obulamu encourages pregnant couples who are positive to deliver from the health centres so that the baby is protected from getting HIV. [LS3_F_KYE] 34 Exhibit 6: (Clockwise from top left): downtown Lira Town; bar in Atiak Trading Center (Amuru District); Kiyunga Islamic Primary School (Kyampisi, Mukono District); Bulamagi Trading Center (Iganga District); Amach Trading Center (Lira District); Uganda Christian University, Mukono Town. 35 The value of 360-degree communication was also reiterated by those KIs who were charged with delivering Obulamu messages at facility and community levels. Many of those interviewed noted high levels of brand recognition based on message saturation coming from multiple channels, and that this translated to a willingness to engage service providers. Obulamu simplifies our work in communication because the clients come in when they are aware of what they want. For instance one could come in and tell you I have taken three months without testing for HIV, therefore I need to test again. [KI_AMU_SSHF_1_1] Now with the Obulamu campaign, the community trusts and embraces health provider messages since the same messages are also being sent by Obulamu. [KI_GUL_RHIP_1] More people are coming to us than before when a person would see you and hide thinking that you have come to arrest them for not having a toilet...But now since Obulamu, they welcome us and do as we advise them. [FGD_SSVHT_MUK] Challenges to 360 Degree Communication The main challenges to attempting to roll out at scale a massive, multi-faceted, multi-channel campaign such as Obulamu are interrelated: 1) the reliance of CHC on their IPs to deliver various HC materials/activities, not all of whom share the same capacity and interests nor the ability to respond to adaptations; 2) disorganization and inefficiency with respect to which materials go to which regions; 3) issues related, yet again, to overcentralization and lack of regional tailoring/representation in the national campaign and seed materials delivered to partners. For example, materials - especially posters and job aids - were delivered in massive quantities all over the country. Theoretically, these materials were all supposed to be in the relevant regional languages. The evaluation found, however, that many of the posters in Acholi (Atiak Health Center) were actually in Langi not Acholi. Many of the posters in Lira (Amach Health Center and Lira Town) were in Acholi, Luo, or English, not Langi. Some of the posters in the Southwest and West were delivered in English. Some of the materials in Iganga were delivered in English or Luganda, while the UPDF received posters mainly in Luganda even though the ethnic composition of the military is diverse and the language of communication is typically Kiswahili.51 In some instances, the IP immediately corrected the mistake and sent materials back to be replaced. In other instances, posters in non-local languages remained in place for months (e.g. see Exhibit 7). Many KIs either mentioned this problem, or mentioned how important it is that HC is produced in local languages:52 Then IEC materials, those we now have, we didn’t have them before. We could get from the Ministry at times some which were in languages which not appropriate…and at times they were not enough...But now it is easier for us. You can contact Obulamu, just a call and ... they drop them off. 51 e.g. KI_KLA_CG_2 52 e.g. KI_GUL_CP_1_1, KI_LIR_RHIP_1 36 Now things concerning TB, if there is anything wrong in the translations, we say “No.” They say, “Oh sorry. Let us take this one back.” [KI_KYE_DG_1] As you enter the health facility, you can see the posters of Obulamu written in Luganda or Luyankole. It’s very difficult to interpret. People have been criticizing it because there is no communication if the language used is not understandable by the community. [KI_LIR_SSCMP_1_1] I will tell you honestly that we, generally, if someone speaks the Lango language…it shouldn’t really matter. [KI_GUL_PRIME_1] It shouldn’t really matter. But does it matter? [Interviewer] Yeah it really matters. Because now there is a lot of politics, there is a lot of tribal identity. And the people who make a lot of noise about these things is not usually the ordinary people themselves, it’s the politicians. [KI_GUL_PRIME_1] I think it has made a difference. And why I say so is because even these materials are right up to the grassroots and they are even in the local languages, whereby even the community can read and get meaning out of them. Often when you go to the community you find them reading: “Okobaki? Nabayire n’omuvubuka.”[“How are things? I had a relationship” (i.e. sexual relationship) the message on the poster]. Like that. In the local language! It ... really messages. [KI_IGA_DG_1] The issue of regional differences in the context of a national campaign was also noted with respect to the photos used in campaign materials, as most were created from images of people who were identifiable as coming from the Central Region of Uganda. Northerners, in particular, noted that the pictures used in the campaign did not look like them. 53 Like with the branding and segmentation strategies, then, overcentralization coupled with the lack of HC tailored to regional differences served as important bottlenecks to the effective implementation of the “360 degree” communications best practice. So too did regional context with respect to coordination with IPs of material delivery, in particular differences among IPs in terms of capacity to keep track of (as well as potentially recall, if mistakes were detected) the massive amounts of materials being distributed. --*-- Interactive Campaign Design SUMMARY ● CHC focused on an interactive - occasionally provocative - campaign design that prioritized dialogue, personalization, and reflection. Interpersonal communication (IPC) was a key focus. ● The interactivity was highly valued by KIs and beneficiaries, in particular the mobilization activities such as Kadankes (youth bashes), community dialogues, and radio talk shows. ● Interactive communication – and IPC more generally – is expensive. Sustainability without partner support is an issue; however, without this component of health communications, KIs note that demand generation will be sub-optimal and health outcome targets will not be achieved (i.e. demand-side SBCC and supply-side technical/clinical service delivery interventions need to be packaged together). ● The effectiveness of this package depends the relationships between various demand- and supply-side partners: in the case of Obulamu, this means close coordination between CHC, regional health and service delivery IPs and district government. One of the most valued best practice - and the one perceived as most effective, albeit expensive - was CHC’s attention to interactivity across multiple channels, one that prioritized dialogue, 53 See KI_GUL_RHIP_1, KI_GUL_PRIME_1 37 personalization, and reflection. We discuss IPC elsewhere in the evaluation (see, for example, Integration and Converging Messages (Q3)). Here we highlight the role of interactivity as a CHC best practice/key strategy. The interactive nature of the campaign promoted discussion, debate, and questions. Instead of telling people what to do, CHC instead posed the popular question Obulamu? How is life? While other HC campaigns and projects in Uganda have done IPC in the past, sometimes messages have reverted to telling/lecturing beneficiaries as to what they should do, rather than helping them define for themselves the life they wished to live.54 Obulamu engaged target audiences to first reflect on health issues and then discuss them with peers, health providers, community champions and family members. This provided an opportunity for beneficiaries to personalize issues based on their own unique needs and realities. As one of CHC’s advertising partners describes: It’s been very exciting among the target audience. It’s been a different kind of approach where you’re not telling people: “Do this! Do that!” You’re creating a conversation with them. You’re hearing from them. [KI_KLA_CP_1] Interactivity took many forms: radio talk shows and dialogues, Obulamu moments, community dramas, youth bashes (Kadankes) with music and skits, job aids that prioritized IPC, and - most obviously - the question-based design of many of the campaign materials: “How’s Life?”, “How is your baby?”, “How’s your love life?”, “How’s your pregnancy?”, “What’s my choice?”, “What would a pregnancy mean to you now?”, “What’s up?”, “What’s your stand on HIV testing?” These messages were put in question form in order to spur beneficiaries to generate their own answers. Additionally, there were intentionally provocative materials to generate debate. For example, one advert featured an adolescent girl asking: “What would pregnancy mean to me? Trouble! My parents would kill me!” This advert generated a lot of controversy, but was also widely remembered by both KIs and FGD participants.55 I think the only issue that I have heard people express has been about the recent advert on…is it prevention?....Where one of the girls says: “Oh. I wouldn’t have sex because my mother would kill me!” So the question has been: “Is she abstaining because she understands the value of abstinence?” The message is clear, but where does she come from? What drives that girl into abstinence? Is it because she understands the value of abstinence? Or is it because she is afraid of her serious mother! [KI_KLA_RHIP_1] You know different people are giving their different ideas and issues and then there is this lady who…this girl….who is abstaining. And she says “Trouble!! My parents will kill me! That’s why I am abstaining.” So it gives the impression that, with this particular campaign, the people we want to target – this particular target audience – we are assuming that all of them have parents. [KI_IGA_RHIP_1] 54 Kayongo E, 2017. 55 LS2_F_MUK, P6_LS4_F_KYE, P1_LS2_LS3_F_IBA, KI_VHT_IGA 38 [Y]ou remember the Obulamu mentions?…That lady said “If I get pregnant, my parents will kill me!” You see, first of all, it means that she is not afraid of getting pregnant. She is only afraid of who? Parents! So we think it’s the right time to move away from parents killing their daughters ...What if parents pass away? [KI_LIR_CP_1_2] Additionally, there were numerous community-based events around the country, in particular Kadankes. Often involving a mobile stage with musicians, drama groups, community dialogue sessions, and product giveaways, these were hugely popular among FGD participants and seen as extremely valuable by many KIs as vehicles for demand generation.56 We call them the Kadankes in Luganda. The youth bashes. And they would support the mobilization. [CHC] would do amazing work. Because where you are struggling to get people, you do one youth bash, and you are getting a huge number of turn out. You will not be able to get, if you are doing your one-one…because it’s a whole process. [KI_KLA_RHIP_1] So when we shared in one of the meetings with Obulamu, they were like “We could do community dialogues with the men and also home visits”...[W]e did it in Zigoti [Mitiyana District]...It was a youth community event. They call it Kadanke…this young married girl said: “Let me take my husband to the health worker.” So they came to me and we managed to talk about it: “What is your plan as a young couple?” [KI_KLA_RHIP_3] They have means of transport. A VHT who is just on foot really can just visit like ten houses and it ends there. Sometimes you can’t even tell if the VHT has reached the community members. But [CHC] are people who have even instruments. They have the system. They have vehicles. They move to very far places inviting people to come to the venue…Sometimes the music bit of it…these young people, they really love that. When you stage a bash, they really come. And when they come they end up taking up services. [KI_IGA_RHIP_2] In rural communities, these sorts of large community celebrations were so few and far between that people remembered them in great detail sometimes years later. In Atiak, for example, the bash took place almost two years previously. FGD participants and KIs could recall vividly what happened that day, what sorts of entertainments were shown, what kind of promotional materials were given away (e.g. basins), the topics of the community dialogues and, most importantly, the messages that were delivered. Challenges with Interactive Communication The main issue with these sorts of interactive events and materials - and IPC more generally - is that they are expensive and, thus, their sustainability in the absence of external assistance is jeopardized. Without sustainability, moreover, credibility is threatened. In the example from Atiak, beneficiaries made statements along the lines of “Obulamu came once, and never came back.” Of course, this wasn’t true. Obulamu was on posters blanketing the health center and hanging on the walls of bars and drug shops in the trading center. CHC played a role in partner mobilization efforts for VMMC and for family planning. It provided job aids to the health workers in Atiak clinic. FGD participants could recount hearing Obulamu spots and talk shows on the radio, and could clearly identify a number of key Obulamu messages. But the brand itself had become partly associated with the Kadanke. Numerous partners noted this predicament: IPC is expensive, yet if IPC is not done, there is no demand for services, meaning the financing invested in service delivery interventions is wasted.57 Therefore, SBCC and service delivery go hand-in-hand. Both need to take place at the same time. 56 Kadankes or mobile vehicle mobilization were mentioned in 17 out of 47 FGD transcripts, and 13 KI transcripts. 57 KI_KLA_RHIP_1, KI_LIR_RHIP_1, KI_KLA_USG_2, KI_MUK_DG_1, KI_MUK_RHIP_1_2, KI_GUL_PRIME_1 39 Both require significant attention and commitment. Both are, to a large extent, dependent on the context of collaboration. When this “package” comes together, we can see improved health outcomes. When it doesn’t, no partner - whether CHC, regional health IPs, DHTs, USG agencies, facilities, or communities - will achieve its goals and health outcomes will suffer. Below we explore the ways in which these partnerships work - or don’t - across three different health areas: family planning, maternal health, and nutrition. Evidence on Effectiveness: Health Outcomes SUMMARY ● Best practices adopted by CHC were not mutually exclusive (i.e. they were part of a unified SBCC approach). In addition, CHC’s relationship to health outcomes is indirect (i.e. dependent upon the efficiencies and effectiveness generated by health IPs that provide clinical and technical support to service delivery). ● The evaluation did not seek to assess the effectiveness of CHC on a strategy-by-strategy, best practice-by-best practice basis. ● The evaluation looked at quantitative data in three Obulamu health areas - reproductive health, maternal health, and nutrition – and compared this evidence to regional IP contexts to understand what story health outcomes might tell about how CHC works with partners. ● It compared CHC national Phase I/baseline data to Phase II/endline data with regionally disaggregated (Eastern, Southwestern, Northern) LQAS trend data from 2012-2016. ● CHC and LQAS reproductive health data showed significant change from Phase I/baseline to Phase II/endline as to contraceptive use among LS 3 (mothers of under-5 children aged 15- 49 years). Reproductive health data changed at the same pace in all three regions, regardless of the IP context. Reproductive health, however, is not necessarily reliant on regional health IPs, depending instead on outreach activities and not-for-profit private sector providers to provide services. That synergies between these sorts of IPs and CHC might be expected to bear fruit is not surprising. This finding is born out in qualitative interviews. ● CHC maternal health data showed significant change with respect to ANC4 attendance among women in LS 2 and 3. LQAS disaggregated trend data observed significant difference across all three regions with rates of change for the Southwest much stronger than for Eastern, and both stronger than Northern. Maternal health relies on health systems and primary health care. As such, it is not surprising that regional health IP contexts will result in in differing rates of change. This finding is also born out in qualitative interviews. ● Finally, CHC national data and LQAS regional data on nutrition showed either no change, or negative change, in exclusive breast-feeding. This is not surprising since nutrition requires a multisectoral response that SBCC and health service delivery partners alone cannot achieve. Additionally, regional health IPs, until recently, have had little to no focus on either breast￾feeding or nutrition more generally. This finding is also born out in qualitative interviews. Here we evaluate evidence of “effectiveness” of the CHC approach, both quantitative evidence in terms of LQAS and CHC Phase I/baseline to Phase II/endline data on health outcomes and qualitative evidence in terms of KI and FGD participant perceptions. Before doing so, we highlight two important caveats. First, per the Scope of Work (Annex 5), this evaluation was not meant to be a quantitative performance evaluation, but rather a qualitative process evaluation. The evaluation team was not tasked with either a) collecting primary quantitative data or b) analyzing data via existing data sets. We examine quantitative data here only with respect to how it informs the overall story we are telling about CHC and the Obulamu campaign, as well as their relationship to partners in order to inform follow on design and implementation. The quantitative data we present here illuminates some 40 interesting and important issues with respect to the context/s in which CHC is meant to be implemented, and the various bottlenecks such contexts might create. Second, as we mentioned in the introduction to this section, we cannot assess effectiveness of CHC on a strategy-by-strategy, best practice-by-best practice basis. This is because all of the best practices in HC that CHC adopted, innovated upon, and rolled out were not mutually exclusive of one another, but were part of an overall approach (both in terms of the Obulamu national campaign and in terms of the conceptual foundation upon which CHC commenced its work). Additionally, CHC’s relationship to health outcomes has always only been indirect: CHC was responsible for generating demand which would, in turn, be met through the supply of health services. Responsibility for supply of health services, however, has never rested with CHC, but with DHTs and regional health IPs. If bottlenecks to service delivery exist at district or facility level, then CHC’s demand generation efforts do not translate into health outcomes. We deal with this issue directly in Q6 of this evaluation (Supply and Demand). We bring it up here, however, because the evaluation question - “Which strategies were effective?” - requires us to describe effectiveness. However, where we see positive, negative, or no change with respect to health outcomes, we do not claim these are due to CHC alone, but rather due to regional partnerships of which CHC adds value via its HC approach. Where we detect “effectiveness” with respect to health outcomes, what story does this tell us about implementation? What story does it tell us about relations between partners? What story does it tell us about how USG mechanisms work in a practical sense in the various regional contexts in which they are rolled out? In this section, we highlight only one or two variables from three of the six health areas targeted by the integrated messages of the Obulamu campaign: family planning, maternal and child health, and nutrition.58 These have been selected based on the way they contribute to the overall story we are telling about Obulamu’s achievements and the bottlenecks it encountered. Please note: CHC Phase I/baseline and Phase II data surveys focus on three age bands made up of LS segments: a) LS 1 (couples aged 20-30); b) LS 3 (parents of children under-5, aged 15-49); c) LS 4 (adolescents aged 15-19). However, LS 3 (aged 15-49) actually includes LS 1 and LS 4 within. We have already talked about the problems with using LS segmentation as a basis of sampling for monitoring and evaluation purposes due to overlap. Additionally, LQAS data looks only at an age band of people aged 15-49. We therefore only look at the central bar in the CHC survey data bar graphs (below). Reproductive Health The evaluation compared CHC national Phase I/baseline to Phase II/endline data on women aged 15- 49 years who used modern contraceptive methods with LQAS trend data on the same population, disaggregated by region (See Exhibits 8 and 9). The LQAS data is disaggregated based on the evaluation sampling rationale of a) stable health implementing partnerships (Southwest); b) semi￾stable health implementing partnerships (Eastern); c) unstable health implementing partnerships (North). Please see footnote of definition of “stable,” “semi-stable,” and “unstable.”59 58 We do not include nutrition/WASH data in this analysis because of the problematic nature of LQAS and national WASH data. See Pratt B, Wandera A, Owor D, Shadravan N, Uganda nutrition assessment: National and sub-regional profiles and strategic opportunities (Kampala, USAID/Uganda, 2017). 59 By “stable” we mean “same mechanism, same prime over the last 5 years” (i.e. STAR-SW/RHITES-SW, managed by EGPAF), by “semi-stable” we mean “same mechanism, different prime over the last five years” (i.e. STAR-E/RHITES-E, managed by first MSH then IntraHealth), by “unstable” we mean “different prime, different prime over the last five years” (Nu-Hites/SDS/ASSIST/ SUSTAIN/RHITES-Acholi, managed by Plan, Cardno, URC-CHS respectively). 41 The CHC results show significant change as to contraceptive use among LS3 (mothers of children under-5 aged 15-49).60 This concurs with disaggregated LQAS data on women aged 15-49 which also shows contraceptive usage rates changing significantly from 29.9 percent to 34.9 percent in Northern Uganda (which started at the lowest rate), from 32.1 percent to 39.4 percent in Eastern Uganda, and from 38.3 percent to 41.1 percent in Southwestern Uganda. The important points to take home from this comparison are that a) both CHC results and data from the LQAS found significant change in uptake of modern contraceptive methods among the same population band over the duration in which CHC was an active HC partner for regional health IPs not only nationally, but in all three regions; b) the ANOVA test was not significant (p-value 0.0638) and there were no observed differences in women using any form of modern family planning methods in all the three regions from 2012 and 2016. Usage rates generally rose steadily every year except in Southwestern Uganda where usage started declining after peaking in 2015, but the rate of change was the same regardless of the region. What does this mean? First, LQAS confirms CHC’s findings of significant change in modern contraceptive uptake. The partnerships in place to improve contraceptive use among women aged 15-49 in Uganda - of which CHC was a part - were clearly effective. Second, CHC operates as an HC partner - and a mobilization partner - via regional health IPs. The context of regional health implementation differed greatly in Eastern, Northern, and Southwestern regions, yet the difference between those regions in terms of overall rate of change in prevalence of family planning was insignificant, i.e. Northern started at a much lower prevalence than Eastern and Southwestern, but the variance from 2012 to 2016 was on par with the other regions. Thus, family planning uptake improved significantly everywhere, but at about the same pace, regardless of whether CHC was operating in the context of a stable, semi-stable, or unstable primary regional health implementing partner. It is important, then, to look at who exactly was providing reproductive health service delivery in these regions? Who were CHC’s reproductive health partners? Here we see that the main reproductive health partner for these regions was likely not USG-funded regional health mechanisms, but rather private sector reproductive health service providers - either Marie Stopes or Reproductive Health Uganda (the IPPF equivalent) - who via private clinics, mobile outreach, or direct provision of services embedded within a public sector clinic - mobilized mass outreach campaigns with clear, consistent supply of reproductive health commodities and human resources. 60 No significant change was found from baseline to endline among LS 1 (women in young couples) and LS 4 (adolescents). 42 KI interviews with Marie Stopes and with IPs in the North - where the prevalence of family planning was lowest among the three regions in 2012 - confirm that CHC’s awareness raising, mobilization, and outreach work in partnership with Marie Stopes was critical here.61 Indeed, mentions of Obulamu’s role in promoting family planning within focus group transcripts, moreover, were much higher in FGDs from the north compared to FGDs in Ibanda which falls under Southwestern region. Table 6: Transcript Mentions Region District RHIP Status Transcript Mentions: family planning (all LS groups)62 North Lira USAID/Multiple 60 Amuru 65 Southwest Ibanda USAID/RHITES-SW 38 West Kyenjojo CDC/Baylor 55 East-Central Iganga RHITES-EC 31 Central Mukono DoD/MUWRP 46 Obviously, the evaluation is not seeking to attribute success in improving family planning prevalence to either CHC or Obulamu. Rather, we wish to draw attention to what can happen to health outcomes when all the pieces - HC, health commodities, health provider coverage - come together in a synergetic way to deliver essential services. In the case of family planning in the North, health outcomes were able to improve - at a rate on par with other regions - in spite of a health system that had no consistent support from a USG partner. Other regions with consistent support from a USG partner also improved significantly. The common link between these regions? The health area in question - family planning - has a private sector partner delivering highly specific services tied to a clear set of commodities and feasible monthly targets, delivered usually in the context of outreach campaigns (i.e. as opposed to via routine, general health services delivered by the public sector). The argument we make here is that a mechanism like CHC may be most able to influence health outcomes when it works with/through service delivery partners who are able to actually deliver services. In the case of family planning, these partners transcend the regional development/regional rationalization model and - while having intermittent access to USG funding - are not themselves USG mechanisms. Maternal Health Data from maternal health also support this story. Here we look at ANC4 visits using national CHC Phase I/baseline to Phase II/endline data and LQAS 2012-2016 data disaggregated by region. National CHC data (see Exhibit 10) shows significant change both among LS 2 and LS 3 mothers in terms of ANC4 attendance. Percentage of women attending ANC4 among LS 2 (pregnant women) increased from 22 percent at Phase I data to 31.9 percent at Phase II data, and among LS 3 (mothers of children under age 1) increased from 67.1 percent at to 80.6 percent at endline. This suggests that, nationally, the synergy of efforts by health partners in Uganda - of which CHC played the key HC role via its support for the Obulamu national campaign - resulted in significant improvement in ANC4 61 e.g. KI_KLA_RHIP_5, KI_GUL_PRIME_1, KI_GUL_CP_1_1, KI_LIR_DG_1, KI_LIR_CP_1_2, KI_LIR_CP_1_1. Marie Stopes, indeed, when talking about working with Obulamu highlighted only Eastern and Northern regions as being areas in which they worked closely with CHC, e.g. KI_KLA_RHIP_5 62 The Nvivo search string on this was “family planning OR number+of+children OR contraceptive OR birth+control.” Frequency of mentions was confirmed by looking at transcript. 43 attendance by both LS 2 and LS 3 women (please see the footnote, however, in terms of interpreting this data for LS 2).63 LQAS trend analysis for mothers of children 0-11 months who attended ANC4 during their last pregnancy appears to show an increase from 2012-2016 from 46.3 percent to 56.7 percent in Eastern Uganda, 52.1 percent to 67.6 percent in Southwestern Uganda, and 52.0 percent to 58.5 percent in the North (see Exhibit 11). The ANOVA test showed significant difference (p-value 0.0129) across the three regions; however, we were only able to observe similar trends in change between 2012-2016 in the East and North, but not for the Southwest. The change in the Southwest, moreover, was stronger (16 percentage points) than in the East (10 percentage points). Thus, the partnerships supporting increased uptake ANC4 in the Southwestern part of the country worked extremely well, those in the Eastern part of the country worked somewhat well, and those in the North did not work well (i.e. in spite of the fact that percentage of ANC4 attendance in the North was higher in the East - 58.5 percent compared to 56.7 percent - in the East there was significant change over time, while in the North there was not). As with family planning, we again ask: who were CHC’s partners for ANC? Which IPs were meant to support the delivery of maternal health services to link with CHC’s demand generation efforts? Here the answer is, in contrast to family planning: USG￾supported regional interated health IPs: STAR-SW/RHITES-SW (EGPAF prime partner) in the Southwest, STAR-E/RHITES-E (MSH and then IntraHealth the prime partners) in the East, and the confusion of Nu-Hites transitioning to SDS to ASSIST to SUSTAIN (Plan and then Cardno and then URC as the prime partners) in the North. Like with family planning, mentions of Obulamu’s role in promoting ANC within focus group transcripts, moreover, were much higher in FGDs from the Southwest and West, than in other parts of the country. Again, this is not surprising as STAR-SW/RHITES-SW has been underway in Ibanda since 2010 under a single prime (EGPAF), and Kyenjojo in the West has been a Saving Mothers Giving Life (SMGL) district since 2012 supported by CDC/Baylor. 64 A high number of mentions was also recorded in Mukono FGDs. DoD/MUWRP is the key IP there and, like CDC/Baylor, has been a stable presence in the district for many years (i.e. CDC and DoD fund these as partner organizations, not as 5-year programs) with Mukono also being an urban district adjacent to Kampala with potentially more access to maternal health services. Table 7: Transcript Mentions by Partner and Region 63 Note: It is not surprising LS 2 (pregnant women) have lower percentage of ANC4 visits than LS 3 (mothers of under-5s), as the pregnant women in the survey would likely be at different stages of pregnancy and, thus, could not all be expected to be at the same point in their ANC attendance schedule. Thus, the baseline-to-endline data on ANC4 visits by LS 2 may simply reflect that the pregnant women interviewed were not far enough along in their pregnancy to have had a chance to complete 4 visits. 64 Of the six evaluation target districts, Lira is also a SMGL district, but only since Phase 2 which started in 2015. As can be seen, moreover, ANC was only mentioned in the context of Obulamu 13 times in FGD transcripts in Lira which might also support our story with respect to the importance of stable IPs in ensuring HC translates into improved health outcomes. 44 Region District RHIP Status Transcript Mentions: family planning (all LS groups)65 North Lira USAID/Multiple 13 Amuru 18 Southwest Ibanda USAID/RHITES-SW 24 West Kyenjojo CDC/Baylor 43 East-Central Iganga RHITES-EC 9 Central Mukono DoD/MUWRP 29 In this case, then, our story of regional difference yet again rings true: in instances where CHC must rely on USG regional health IPs for an Obulamu health focus area, the delivery of those services which require strong routine, public sector service delivery and robust health systems, CHC’s demand generation efforts are only as good as the service delivery efforts of its partners. Nutrition Lastly, the evaluation focuses on the Obulamu health area of nutrition. The comparison between CHC data and LQAS trend data is a bit more difficult to make, due to the fact that CHC data looks at percentage of caretakers of children under 5 who exclusively breastfed, as well as who fed children a diverse diet, while the evaluation was only able to look at LQAS data with respect to percentage of children under 6 months who were exclusively breastfed. These variables are obviously not equivalent. However, as we are attempting to illuminate what this data means with respect to the story we are telling as to CHC and its relationship to partners, we nevertheless include tables showing outcomes here as these findings feed into our story. CHC not only found a low percentage of caretakers who exclusively breastfed children, but this percentage actually declined nationally from 32.4 percent to 27.0 percent from baseline to endline. The percentage of caretakers of children under 5 who provided them with a balanced diet was also unchanged (47.7 percent to 48.0 percent). LQAS data disaggregated by regions found slightly higher percentages, 66 with 54.2 percent of children in Eastern Region, 64.4 percent of children in Southwestern region, and 64.7 percent in Northern region exclusively breastfed. However, between these regions, the ANOVA test showed significant differences (p-value 0.0404), with the Eastern region (an increase from 46.1 percent to 54.4 percent from 2012-2016) being different from the Southwest (62.1 percent to 64.4 percent) and North (61.7 percent to 64.7 percent), both of which exhibited no significant change from 2012-2016. 65 The Nvivo search string on this was “ANC OR ante OR pregnant” including stemmed words. Then frequency of mentions was confirmed by looking at transcript mentions with respect to “pregnant” to see the context in which pregnancy was referred. 66 Again we are looking at slightly different populations in the LQAS than the CHC baseline-to-endline surveys. 45 This finding is obviously troubling, but the larger question is: what does this evidence of stagnation with respect to nutrition variables say about the programmatic environment in which Obulamu messages and materials related to exclusive breastfeeding (and nutrition more generally) are supposed to be rolled out? Are the health outcomes related in any way to an absence of awareness as to best practices in nutrition (i.e. variables that HC actually might directly influence)? USAID/Uganda has already commissioned an assessment of Uganda’s national, sub-regional, and district nutrition landscapes, the predominant finding of which was that USAID/Uganda nutrition mechanisms - as well as nutrition programs by other partners - both within and outside the the health sector faced numerous bottlenecks with respect to sustained progress on malnutrition in the country.67 There was limited multisectoral collaboration and no clear way to identity at district and community level the constellation of health and non-health drivers of malnutrition that led to health outcomes. Breastfeeding was not a focus, but the assessment pointed out that it is currently difficult to measure most nutrition indicators at district level beyond the LQAS (the data for which is often confusing) and special one-off studies (that do not account for trends across time and tend to be short on analysis of how different drivers mutually reinforce each other). Additionally, the assessment found that USG targeting of its various nutrition initiatives was sub-optimal. Based on these findings, we compare lack of progress seen in the CHC and LQAS quantitative data to the frequency of mentions of breastfeeding and other nutrition-related terms in the FGD transcripts. As is evident below, there were very few mentions of breastfeeding in the context of Obulamu messaging in any of the FGDs, regardless of where in the country they took place. Additionally, and most surprisingly, the districts where breastfeeding was mentioned the most were districts where there was either a) no USAID/Uganda health mechanism with a nutrition component during the duration of CHC (i.e. Kyenjojo, of which CDC/Baylor is the regional health IP with access to nutrition interventions through SMGL) or b) only food security-related interventions (Mukono, a USAID/HarvestPlus district, which received health assistance via DoD/MUWRP, and through MUWRP, via the PEPFAR/DREAMS Program). Table 8: Transcript Mentions of Nutrition Region District RHIP Status Transcript Mentions: Nutrition, Breastfeeding, Handwashing68 (all LS groups) Transcript Mentions: Breastfeeding + Stemmed (all LS groups) USG Programs with some* Nutrition Focus** Partner Nutrition Programs* North Lira USAID/Multiple 32 3 CC, HP, FANTA, SMGL, USAID/DREAMS MFNSP, PforP Amuru 15 4 FANTA MFNSP, ACDP, PforP Southwest Ibanda USAID/RHITES-SW 14 0 CC, SPRING, RHITES BRAC West Kyenjojo CDC/Baylor 38 1 SMGL MFNSP, ACDP East￾Central Iganga RHITES-EC 37 0 HP, RHITES MFNSP, ACDP, Central Mukono DoD/MUWRP 39 2 HP, DoD/DREAMS - * Some programs listed are integrated nutrition programs (CC, FANTA, SPRING, MFNSP, BRAC). Others are support for health￾sector related nutrition interventions (RHITES, SMGL). Others are specific either nutrition via improved food security (HP, PforP, 67 Pratt BA, Wandera A, Owor D, Shadravan N, Uganda nutrition assessment: National and sub-regional profiles and strategic opportunities (Kampala, USAID/Uganda, 2017). 68 The Nvivo search string on this was “nutrition OR breast OR wash OR food OR feed OR fed” 46 ACDP) or via OVC health and livelihoods support (DREAMS). **Here we list high profile programs that had explicit MCH nutrition element embedded within program documentation. This list is not comprehensive as we do not focus on UN agency programs (except PforP) nor emergency food aid nor NGO or FBO assistance.69 In Lira and Iganga, there is some evidence of a critical mass of programs supporting CHC in raising awareness. In Amuru, by contrast, we see a vacuum of USG nutrition IPs, an absence of a stable integrated regional health IP, and an overall lack of recognition of the role Obulamu is supposed to play in nutrition. In Ibanda, we see a similar lack of recognition, in spite of it being a focus district of two flagship USAID nutrition mechanisms - Community Connector and SPRING - and home to a stable regional health IP in the form of RHITES-SW. This is interesting because the main creative partner for CHC does highlight the work it has done in nutrition on behalf of Obulamu. However, it notes that there has been little input from IPs and, moreover, that nutrition has been treated as being subsumed under ANC: Nutrition has been there, advising on how to feed but we did not focus much on, for example, how dangerous it can be if you don’t get folic acid and those things that help the baby develop the spinal cord and all those. So that is an adaptation we are making. [KI_KLA_CP_5] Is that because the technical working group or the medical people aren’t giving you their content? Do they forget about it? [Interviewer] At that moment it might have not felt important, but I think they are realizing that if we highlight and position it as one of the things that could go wrong, if you don’t go for ANC, for example giving birth to a baby who has a deformed spinal cord or something...That’s how we’ve been doing it, under ANC. We do ANC nutrition, malaria, delivering at the health center, but it’s all under Obulamu [KI_KLA_CP_5] The creative partner then - without significant guidance from IPs - assumed that nutrition messages, if integrated with ANC messages, would not get lost. Evidence from CHC, the LQAS, and the FGDs suggests that, in fact, in some parts of the country this assumption may not be true. Thus, treating nutrition as a subset of integrated maternal and child health messages, rather than as a health area in itself, may have diluted the memorability of nutrition messages especially in the absence of no other reinforcement from regional health IPs and other nutrition partners, including in contexts where there is a high volume of food and nutrition partners in a district. Other interviews with KIs reiterate the point that nutrition either has been subsumed under other health areas or, alternately, has been left out altogether because the focus of the IP at the time is a different (usually diseased-focused) issue:70 The difference is…[CHC] responds to your needs...there has been little focus – not for CHC, but for programs- to get into nutrition when you are doing an outreach activity, although we actively identify such children and link them to services. [KI_MUK_RHIP_1_3] [P]eople’s priorities differ. For me, who is a hungry man, I want to hear about nutrition. Which sometimes is not talked about. It is difficult to design nutritional programs when you may not provide the food. You talk about a balanced diet and they start challenging you “Where should we get food from?” So it is tricky. But in reality still there is a focus if you ask about Obulamu, immediately people would say those are talking about Family Planning. [KI_MUK_DG_1] 69 Key: CC = USAID/Community Connector, HP = USAID/HarvestPlus, CDC/SMGL = Saving Mothers, Giving Life, USAID/FANTA = Food and Nutrition Technical Assistance, USAID/SPRING = Strengthening Partnerships, Results, and Innovations in Nutrition Globally, PforP = Purchase for Progress (WFP), ACDP = Agricultural Cluster Development Program (World Bank), MFNSP = Multisectoral Food and Nutrition Security Project (World Bank) 70 See also KI_KLA_RHIP_3, KI_LIR_DG_1 47 …Obulamu was supposed to work across 6 different health issues. So it wasn’t just family planning. It was also HIV. It was also malaria and TB and MCH and nutrition. [Interviewer] CHC was supposed to? [KI_KLA_RHIP_5] Yeah. CHC and the Obulamu campaign were supposed to actually integrate six different issues. [Interviewer] So that is family planning, HIV, malaria? [KI_KLA_RHIP_5] TB, MCH and nutrition. Have you had any encounters with these other messages? Because this [the poster on the wall] is clearly an HIV message. And the other [the poster behind us] is a family planning message. What about the others? Have you encountered any of the SBCC materials…for these other campaigns. [Interviewer] We have seen them around. But, since it is not our core area, we have not given it so much notice and all that.[KI_KLA_RHIP_5] The issue of integration will be discussed later in this report in the section Integration and Converging Messages (Q3). For now, we emphasize with respect to the data on nutrition what we discussed earlier with respect to CHC’s ability to play a role in health outcomes in Uganda: CHC - and the Obulamu campaign - is only as strong, as focused, and as effective as the IPs with whom it works, and the IP context in which it works can vary dramatically from region-to region and from health area-to-health area. ---*--- Evaluation Question: Misconceptions and Knowledge Gaps in Obulamu Messaging Q1: What did USG and the implementer learn about the target populations’ misconceptions around intervention messaging and how did they adapt? SUMMARY ● CHC had numerous on-going activities to identify and address target population misconceptions, in particular Action Media and special studies. ● Examples include creative, dynamic adaptation with respect to materials on teenage pregnancy, viral load, adolescent PLHIV, and ANC attendance. ● At the same time there are numerous red flags in CHC’s own Phase I/baseline to Phase II/endline data surveys on attitudes and knowledge across variables in many of the six Obulamu health areas.71 In many instances, outcomes either stayed the same or declined markedly. ● These findings may be due to a number reasons: survey design, survey implementation, partnership bottlenecks, message fatigue, etc. ● It should be noted too: CHC’s predecessor, HCP, also showed decline or stagnation in many SBCC outcomes over time. ● The larger issue with respect to the follow on, however, is why such red flags were not identified until endline, i.e. there was no midline to catch and address/explain negative outcome data and no large enough scale routine monitoring to follow trends (nor was a midline required by USAID/Uganda). ● As a consequence, it is difficult to know how, or even if, CHC went wrong. ● One pattern in the attitudes and knowledge data, however, is that contraceptive and safe male circumcision (SMC) attitude and knowledge outcomes appear not to exhibit the same problematic outcomes that other health areas. ● This might be because CHC may add more value to highly-targeted campaign-, outreach- or camp-based service delivery activities compared to routine, facility-dependent, service delivery that maternal and child health, malaria, HIV care and treatment, and TB depend. 71 Here we note again: although CHC has referred to these surveys as “baseline” and “endline” elsewhere, the evaluation team has been asked by CHC to refer to these as Phase I and Phase II surveys. 48 ● There may be a gap, then, between the “good life” holistic model that Obulamu promotes and the reality of the Ugandan health sector which often relies upon targeted, campaign￾based service delivery to bypass a broken health system and meet externally-imposed targets. The evaluation sought to address target populations’ misconceptions and knowledge gaps that were identified vis-a-vis SBCC intervention messaging and whether or not these were limited to particular populations, messages, and channels or were generalized. To address these questions, the evaluation draws on primary evidence gathered from KIs and FGDs and on secondary data and information from CHC’s body of research.72 Target Populations’ Misconceptions Ongoing contact with the target audience helped to identify and address misperceptions. Close collaboration and feedback between community-, facility-, and media-based partners, helped CHC to identify and respond to several misperceptions which surfaced as a result of campaign messaging: Each time we do home visits, we want to document some of the misconceptions, the perceptions people have. [...] It’s always a continuous process of trying to understand your audience [KI_GUL_PRIME_1] If you go to Iganga on Sunday and then there are issues that come up about nutrition, they [the IP] capture that information and work with CHC. We repackage it and then we throw it back to them [KI_KLA_CP_5] One example was a confusion among LS 4 (female adolescents) about family planning. The long￾established term “family planning” was found not to be relevant to young women who, although sexually active and at risk for unplanned pregnancies, were not currently planning for a family. As one young woman in Lira District shared: I didn’t understand if family planning is for students or for the married only. [R8, LS4_F_LIR] Once CHC identified this misperception, it addressed the confusion by discussing family planning in a way more relevant to young, unmarried women, referring instead to unwanted pregnancy or teen pregnancy. In another example, CHC carried out a rapid, qualitative assessment looking at contextual barriers and coping strategies related to the uptake of modern contraception among men and women aged 18-49 in Luweero District. The study concluded that one of the major misconceptions about family planning was its association with the traditional Baganda concept of kizaala gumba which has a connotation of barrenness.73 CHC found that this concept was being used to refer to women experiencing infertility issues after using contraception, having unhealthy children after delaying conception through family planning, or failing to function sexually because of the perceived side effects of contraceptives. Based on this learning, CHC adapted by pursuing a strategy that involved a) providing more accurate information; b) spotlighting the economic and personal benefits of birth planning and spacing; c) bringing men on board to improve communication between partners; and d) reorienting SBCC to deal with social determinants of contraceptive adoption. 72 CHC conducted several qualitative rapid assessments/case studies focusing on specific health areas, KP/PP, regions, and health-seeking behaviors. 73 FHI360/CHC, Contextual barriers and coping strategies with the uptake of modern contraceptive services and commodities in a selected community in Uganda: Highlights from a rapid qualitative assessment, July 2014. (Kampala, FHI360/CHC, 2014) 49 Another example of CHC adapation to misconception was in communication materials related to viral load. In order to meet its UNAID’s 90-90-90 target which includes viral suppression and concomitant viral load testing, CHC was tasked with educating PLHIV about the importance of viral load. Despite positive results from pre-testing, the original poster confused many PLHIV into believing that HIV was curable, as the image related to “Low Viral Load” showed no virus in the body. Exhibit 15 shows before and after posters that demonstrate how CHC addressed this misperception and simplified the concept. The “Low Viral Load” body now represents the virus as being present but minimal. In another example, discussions with the target audience during Action Media sessions found that the benefits of ART that were relevant to older people living with HIV - i.e. those who were living a longer and had better quality life – but were not as meaningful to younger PLHIV, who tend to live in the moment and seek immediate gratification. Based on this finding, the project designed communication with an immediate benefit to younger PLHIV. Exhibit 16 shows a poster targeting young women living with HIV, promoting the attractiveness benefit. It was considered a somewhat controversial approach by others not in this segment of the population, yet to LS 4 (adolescents), it was meaningful. In the area of maternal and child health (MCH), evidence from 2013 indicated that, while ANC attendance rates appeared high, most referred to a single ANC visit, one that often occurred only later in women’s pregnancies. The failure by many mothers to complete ANC4 had grave implications not only for their own welfare and that of their children, but also for Elimination in Mother to Child Transmission (eMTCT) of HIV outcomes. CHC carried out a qualitative rapid assessment to understand the factors that affected the uptake of maternal and child health and ANC services.74 One focus of the assessment was the characteristics of spouses who attended MCH services as recommended. 75 The study concluded that, for such male “adopters” of MCH, “antenatal care was an ‘insurance’ against avoidable complications with the pregnancy or delivery of the baby, and subsequently, avoidance of undue financial costs and emotional strain that may come with such emergencies.” 76 Similarly, Action Media that CHC undertook with pregnant women in Mityana District to address barriers to ANC uptake found that non-problemetic pregnancy and successful previous births were some of the biggest barriers to uptake of ANC. 77 As an output of this Action Media, a group of pregnant women was taken through a hands-on process of developing targeted messages and 74 "Barriers and coping strategies with the uptake of MCH/ANC-linked services in a selected health facility catchment area in Uganda. Findings of a rapid qualitative assessment," FHI360/CHC, 2014a. 75 FHI360/CHC, 2014a, p.2. 76 FHI360/CHC, 2014a, p.2. 77 FHI360/CHC, Findings of Action Media with pregnant women, Mitiyana, Uganda, (FHI360/CHC, 2014d). 50 materials (poster prototypes and logos) on ANC that would then appeal to their peers in various occupations (e.g. hairdressers, market vendors, housewives, teachers, waitresses, etc.). 78 Other qualitative rapid assessments/case studies have offered equally valuable lessons in the design of SBCC interventions, underscored the importance of SBCC messaging strategies that focused on information, motivation, ability to act, and norms, and enabled adaptation. 79 Target Populations’ Attitudes and Knowledge Gaps As indicated earlier, the Obulamu campaign was evaluated through two surveys: the Phase I data in March 2015 and the Phase II data in September 2017. To further explore target populations’ misconceptions, we look at variables related to “attitudes” and “knowledge” in CHC’s thematic health areas. In all instances, the focus was on variables where the findings revealed notable declines in positive attitudes between Phase I/baseline data and Phase II/endline data. For purposes of this analysis, a “notable” decline is when the observed change was five or more percentage points between baseline and endline data. Tables presenting this data are shown in Appendix 8. The tables show examples of overall change for different health areas, as well as examples from disaggregated data where the CHC Phase II data uncovered notable declines in positive attitudes from Phase I data. It should be noted that in many cases baseline data were already high to begin with, so the observed declines in positive attitudes are not necessarily causes for alarm in the short term. However, they are indicative of the potential medium- to long-term trajectory of Obulamu messaging and serve as red flags for either the itself campaign or, as we will discuss later, how the campaign – and CHC more generally – was monitored. Some examples of these red flags? Whereas the trend demonstrates an increase in positive attitudes toward family planning (82.1 percent at baseline to 84.5 percent at endline) significant declines were noted among LS 4/adolescents (84.4 percent to 77.1 percent) and beneficiaries aged 15-19 (83.5 percent to 76.9 percent), as well as declines in Eastern (86.9 percent to 80.0 percent) and Karamoja (50.8 percent to 42.4 percent) regions (see Appendix 8, Table A1).80 Likewise, positive attitudes toward condoms were also quite low at Phase I/baaseline data (15.5 percent) and the trend did not improve over time (14.0 percent), with notable declines among social media/SMS users (24.4 percent to 18.3 percent), urban residents (14.5 percent to 7.5 percent), residents of Southwestern region (18.3 percent to 4.4 percent), and the widowed (18.8 percent to 8.7 percent) (see Appendix 8, Table A2). While there was an increase in overall positive attitudes toward reducing the number of sexual partners at Phase I/baseline (71.7 percent to 77.9 percent), there were also notable declines among TV audiences (76.0 percent to 71.6 percent) and residents of the Central (81.9 percent to 72.0 percent) and Eastern (75.4 percent to 69.7 percent) regions (see Appendix 8, Table A3). Similarly, while there was a high percentage of respondents who were confident and comfortable talking about reducing the number of sexual partners, this indicator did not change over time (78.7 percent to 78.6 percent). There were also sharp declines among audiences for outreach activities (87.3 percent to 78.2 percent), most especially among LS 4 (77.0 percent to 67.6 percent), Central (85.3 percent to 73.0 percent) and East Central (75.4 percent to 58.9 percent) regions, 15-19-year￾olds (76.9 percent to 69.0 percent), and those who were never married/single (80.2 percent to 70.7 percent) (see Appendix 8, Table A4). 78 FHI360/CHC, 2014d, p.23. 79 FHI360/CHC, Contextual barriers, motivations, and coping strategies. HTS update and condom use in a fishing landing site in Busia District. A qualitative rapid assessment focused on positive deviance, (Kampala, FHI360/CHC, 2014b); FHI360/CHC, Contextual barriers and motivations for the uptake of HTS and condoms among female sex workers and truckers in Busia township, Uganda. A qualitative rapid assessment, (Kampala, FHI360/CHC, 2014c); FHI360/CHC, Malaria upsurge in northern Uganda: An exploratory rapid assessment with Village Health Teams in Amuru District, (Kampala, FHI360/CHC, 2016). 80 Note: CHC’s evaluation treated LS 4 and 15-19 year olds as separate, even though LS 4 theoretically is adolescents aged 15-19. Again, this raises a number of questions/problems with respect to the utilization of LS (a campaign segmentation strategy) in monitoring and evaluation. 51 What should we conclude from these examples? As we mentioned before, the lack of/negative change over time suggests that – in spite of the many instances of CHC demonstrating a high degree of responsiveness to findings from qualitative research - there nevetheless may have been other areas which CHC should/could have adapted to, but did not, due to the five year gap between Phase I/baseline data and Phase II/endline data findings. Thus, while some research was useful for continuous responsiveness and adaptation, other research - most notably that most directly concerned with CHC’s SBCC outcomes - was too intermittent to detect serious bottlenecks in performance. Indeed, some of CHC’s key service delivery IPs implemented routine performance monitoring and quality improvement (CQI) with respect to their own SBCC and health outcome data, and they were able to adapt immediately to such findings in the way CHC could not [KI_KLA_RHIP_5]. Some of the issues highlighted in the Appendix 8 tables might have been caught had a midline survey been required as part of the monitoring, evaluation, and learning plan (MELP) or had the plan been required to include a subset of indicators that could be monitored annually. Looking futher at CHC data, we examined variables related to “[basic] knowledge” and “comprehensive knowledge.” In all instances, we concentrated on variables where the findings revealed notable gaps in knowledge between Phase I/baseline data and Phase II/endline data. For purposes of this analysis, a “notable” gap is when the observed change was five or more percentage points between Phase I/baseline data and Phase II/endline data. The tables below tell a similar story as those in the section on misperceptions. Overall basic knowledge across many indicators is quite high; however, in many instances no change or minimal change was detected from baseline to endline. In many other instances, knowledge actually declined over time. Negative trends provide insights into what might turn out to be slippery patches on the campaign journey and which could be addressed if follow on activities build in midlines or improved routine monitoring. They also point to potential knowledge gaps that could widen and should be watched in ongoing and future SBCC intervention messaging. For example, the overall level of basic knowledge of risks of having multiple sexual partners was high, but did not change much between Phase I/baseline data and Phase II/endline data (86.2 percent to 88.1 percent). Among audiences reached through video halls/dens, basic knowledge levels that were close to saturation point at baseline (98.1 percent) dropped to 87.4 percent at endline. Among residents of the North, they fell even further: from 93.7 percent to 84.8 percent (see Appendix 8, Table A5). Similar decline or stagnation was found with basic knowledge of TB and malaria, including across all communication channels through which Obulamu was supposed to be conducted (see Appendix 8, Tables A6 and A7). With respect to malaria, this is particularly worrying as a decline in knowledge was seen in regions with some of the highest prevalence (e.g. the North with a decline of 48.1 percent to 28.6 percent from baseline to endline data) (see Appendix 8, Table A7). Even comprehensive knowledge about HIV - PEPFAR’s primary brief - declined. The overall incidence of comprehensive knowledge about the spread of HIV dropped from an already low level of 33.8 percent at baseline to 27.6 percent at endline. There were notable declines in virtually every dimension on which the data was disaggregated: extent of exposure to Obulamu messages, familiarity with Obulamu, channel, life stage, gender, residence, region, education, age, and marital status (see Appendix 8, Table A8). 81 81 Again, all Phase I/baseline to Phase II/endline data described in this section comes from data tables provided to the evaluation team from CHC as presented in the CHC Data Interpretation Workshop held in Seeta, December 6-7, 2017. This data was sent to the evaluation team in tabular form on December 14th, 2017 in the form of a compressed zip file. 52 There were two notable contrasts. First, comprehensive knowledge of SMC increased generally from 72.3 percent to 76.5 percent, except for a notable decline in Northern Uganda from 72.1 percent to 65.0 percent (see Appendix 8, Table A9). CHC’s work on male circumcision corresponded with a PEPFAR voluntary medical male circumcision (VMMC) push across 2016-2017. Most KIs perceived that Obulamu messaging contributed to an overall reduction in misconceptions about SMC and, 31 out of 47 FGDs mentioned SMC as a key message of Obulamu. As one VHT interviewee in Mukono District said: Misconceptions about SMC were there before Obulamu campaign where people were told that when they circumcise they take their skins [...] to South Africa to make Vaseline out of them. And at that time people could not allow to go for SMC. But since Obulamu came, now many people want to go for SMC [KI_MUK_SSVHT_1] Similarly, the overall level of comprehensive general knowledge about contraceptive use showed marked improvement, increasing from 51.7 percent at Phase I/basline data to 71.1 percent at Phase II/endline data, although notable declines were observed among social media/SMS users (74.8 percent to 58.2 percent) and in the Karamoja region (27.0 percent to 17.2 percent) (see Appendix 8, Table A10). Why would contraceptive choice and male circumcision be success stories here, among all health areas? We return to this issue in Q6 on Supply and Demand. Here, however, we will note that family planning and VMMC in Uganda is often delivered in the context of outreach campaigns and camps, i.e. there is a specific, targeted service offered with guaranteed access to both commodities and specialized health professionals. As discussed in the previous section on best practices and effectiveness (Q2), it is possible that one of CHC’s greatest value adds has been in demand generation for the sorts of services that do not rely on routine health service delivery and strong health systems (e.g. TB, malaria). As the foregoing analysis has highlighted, evidence from the CHC Phase I/baseline data-to-Phase II/endline data surveys does point to potentially worrying trends and raises concerns about the overall impact of Obulamu on attitudes and knowledge, as well as its ability to continuously monitor and respond to findings. It also raises issues about regional targeting since the Central, Eastern, Northern, and Karamoja regions appear to have experienced more negative trends both in attitudes and knowledge than did other regions. Even in cases where the overall performance of Obulamu remains robust, other interesting questions emerge: Could the declining incidence of positive attitudes and emerging gaps in knowledge suggest that message fatigue is creeping up on the campaign? Could it be that audiences are beginning to grow complacent? Could it be that the content of the campaign is in some respects losing its appeal? Could it be that there is more that Obulamu needs to learn about the interactions among different audience dimensions targeted by the campaign? As CHC cannot yet routinely engage with large scale, disaggregated SBCC outcome data, these questions cannot be easily answered. Indeed, the declines may have nothing to do with the integrity of the Obulamu campaign and, instead, have to do with how SBCC outcomes are measured. Additionally, as highlighted in the introduction to this evaluation its important to remember that CHC’s predecessor, HCP, also showed stagnation and decline in knowledge and attitudes outcomes from baseline to endline data. 82 ---*--- 82 FHI360/CHC, 2014. 53 Evaluation Question: Converging Messages and Integration Q3: How did the implementer address converging messages and integration during implementation and what lessons can be learned? SUMMARY ● CHC addressed converging messages and integration through its audience segmentation strategy, channel selection, and message design. ● Integration, while appreciated by service delivery IPs, was often at odds with SBCC specialists who are taught to be single-minded. There were a number of challenges of promoting multiple products and behaviors at the same time. ● Timing was also an issue, with roll out of a phased, integrated Life Stage strategy disrupted by competing partner needs which were often highly technical, and tied to PEPFAR and other reporting. ● Because of the lack of direct relationship between CHC and messengers on the ground, IPC agents – VHTs, peer educators, champions, health workers – were sometimes unprepared and untrained in the Obulamu approach (i.e. dialogue, improvisation, life-focused), and message convergence was sometimes dealt with ad hoc. As a starting point, it is important to understand the context of integrated SBCC programs. Obulamu, although one of the first, is not the only integrated SBCC program. Other USG-funded integrated SBCC projects over the past five years include Malawi’s Health Communication for Life, Tanzania’s Tulonge Afya, Pakistan’s Bright Star Movement, and Guatemala’s Together We Prosper. This new generation of SBCC programs has been designed to deliver integrated interventions that cut across multiple technical areas. The rationale for this shift in approach has been to optimize SBCC funding, to address audiences’ health needs in a more holistic way, and to align integrated SBCC activities with integrated service delivery programs to achieve greater synergies. Obulamu has worked at bundling its six health interventions under one common theme, implementing a mix of mass media, interpersonal communication, and other community-based channels to achieve measurable outcomes. In addition to the original six health areas, the project is at the very early stages of adding others, including GBV and WASH and there are plans to expand the portfolio even further. Here we explore CHC’s approach to integration and the convergence of messages. Integration: Audience Segmentation As described earlier, Obulamu revolved around four broad target populations called Life Stages (LS). CHC’s theory of change opened up a way for the campaign to zero in on specific audiences. Two broad strategies were employed: demand creation and linkages between supply and demand side communication. CHC’s approach to audience segmentation treated the target populations for demand creation as primary audiences and those for linking supply and demand side communication as secondary audiences, as illustrated in Table 19. Table 19: Primary and secondary audiences DEMAND CREATION PRIMARY AUDIENCES LINKAGES BETWEEN SUPPLY￾AND DEMAND-SIDE COMMUNICATION SECONDARY AUDIENCES ● Adolescents (boys and girls 10-19) ● Young adults (20-24) ● Single adults (men and women) ● Newly married/cohabiting couples ● Pregnant women ● Caretakers ● MARPs* (commercial female sex workers, men who have sex with men, fisherfolk) ● PLHIVs ● Health service providers ● Product providers ● VHT members ● Community-based volunteers ● Community health workers ● Local leaders ● Cultural/traditional leaders ● Religious leaders ● Peer groups e.g. women groups *Most at risk populations, also known as key populations (KP). 54 Integration: Channel Selection As described in the section on best practices, the concept of 360 degree communication83 was at the heart of CHC’s approach to integration (see Exhibit 17). As an integrated campaign, the foundation of Obulamu was built on a multi￾media communication model comprising a mix of channels. IPC was the nucleus of the model and its delivery entailed a combination of household, community, and facility- or service￾based IPC. Experiential activities and community dialogues were vital parts of the mix. IPC was complemented and reinforced by mass media (radio, TV, video), social/mobile media (Twitter, Facebook, WhatsApp, SMS), and outdoor media (targeted placements). Household- and community-based IPC were designed to link health seekers at those levels to services/facilities via two routes. First, IPC was channeled through the primary audience’s key influencers – i.e. peer leaders, VHTs, teachers, and a host of local champions – whose role was “to engage audiences in conversations that trigger self￾reflection, action, and referral to available services”.84 Second, IPC was delivered by trained campaign champions – i.e. health workers, VHTs, expert clients, linkage facilitators, peer leaders, mentor mothers, teachers, local leaders, pharmacists, nurses, journalists, religious leaders – who were “provided with tools on how to use the Obulamu approach to trigger dialogue and engage audiences on a variety of health issues.”85 The critical role of these champions was to address people’s fears, myths, and misconceptions in whatever health situations they were in and to refer them for services at facilities. The selection of channels took into account the audience profile of each LS as determined by formative research and Action Media. In aligning the various channels with relevant audiences, the unique properties of each channel were harnessed and applied in defining its “specific role in addressing barriers related to knowledge, motivation, skills, or norms.”86 In the Obulamu campaign strategy, CHC laid out in detail the generic functions of each type of channel.87 Experiential activities like Kadankes (youth bashes) and community shows were popular and effective as vehicles for community and social mobilization to rally people for mass participation in programs like male circumcision, HIV testing, bed net distribution, etc. Integration: Message Design CHC worked out a dynamic message development process through which Obulamu concepts, themes, and branding were created. Part of the process was a workshop that involved creative and media partners and key stakeholders transforming UHMG’s former “Good Life” brand into a new Obulamu platform with an associated positioning and branding strategy. Message concepts, campaign themes, and branding propositions were then tested with audiences. Next, CHC worked with MoH to plan the campaign and design and produce communication materials, toolkits, and job aids (June 2013-June 2014). As discussed earlier in relation to sequential 83 FHI360, 2013b, p.43 84 FHI360, 2013b, p.43 85 FHI360, 2013b, p.43 86 FHI360, 2013b, p.43 87 FHI 360, 2013b, p.44 Exhibit 17: CHC Communication Strategy 55 implementation, the plan was to roll out the campaign in phases, one LS at a time. On paper, LS 1 was scheduled for December 2014-May 2015, LS 2 for June-August 2015, LS 3 from September￾December 2015, and LS 4 from December 2015- March 2016. The same process would then be repeated through the rest of 2016 to the end of the project in 2018. However, the reality was different. The IPs upon whom CHC relied to implement activities had their own work plans that did not necessarily dovetail with the Obulamu phased roll-out plan. IPs were constrained by program commitments they already had in place and largely operated under different funding mechanisms and cycles that were not flexible. Moreover, there was often pressure from below for CHC and IPs to respond to the emergent needs of beneficiaries under whatever LS group and health area that a given IP or district was interested in. Integration: Benefits and Challenges Challenge 1: Integration is in the Eye of the Beholder Although integration makes complete sense from the point of view of service delivery, it is not without its challenges when it comes to communication and influencing behavior change. Different KIs had differnt points of view. Looking through a service delivery lens, many KIs believed strongly in integration: I believe integration is really good. You cannot just offer a single kind of service because people in the community have different issues. So as you go with an integration approach, you find every person will be targeted, will be served, and will end up getting access to services...You find that the community has been served holistically. [KI_IGA_RHIP_2] It is good we integrate. Because all health issues are important. If we concentrate on maybe HIV, then we leave other areas aside, it will not be good. Because we are looking at health holistically. [KI_IGA_DG_1] Integration works well because of the holistic needs of our people. I think they should maintain that. [KI_KLA_RHIP_3] Although it is hard to dispute that integration is critical from a public health and service delivery perspective, it is less apparent when 'delivering' social and behavior change. From a communication perspective, SBCC specialists are taught to be single-minded, as the human brain can only retain so much information. In addition, each health behavior comes with its own set of obstacles and barriers, which can seldom be addressed through communication and education alone. From a communication perspective, therefore, there are challenges when promoting multiple products and behaviors at the same time. Integrated messages would be advantageous in terms of economizing resources. But people sometimes lose their track, because they get very many messages at once. [KI_MUK_DG_1] A creative professional at one of CHC's advertising partners came up with a potential solution for packaging integrated messages under one overarching question. He calls this creation 'talking billboards' where all messages targeting, in this case, LS 2 (young lovers) are held together with the question: “How is your love life?” (Exhibit 18). Exhibit 18: Integrating SBCC Messages 56 Challenge 2: Timing is Everything Although the CHC project began with an ambitious phasing and campaign roll-out strategy (Exhibit 19), reality set in when it came to implementation and CHC had to learn to adapt quickly. This meant that Obulamu had to be responsive to partners and roll out materials as needed, as opposed to as planned. Integration, then, took a back seat to partners immediate priorities and CHC had to be flexible with its timing. Exhibit 19: CHC Phased Roll Out Strategy We always had a challenge where you would go to a family and want to talk about nutrition for children under 5. You talk about nutrition, the child has malaria. The mother is only concerned about this child who is shivering with high temperature and you’re talking about nutrition. But if you talked about nutrition and talk about malaria at the same time, maybe make malaria as an entry point for nutrition, it becomes relevant. [KI_KLA_CG_4] One of the challenges we’ve encountered is you get to the community and you’re talking about ANC visits, but their issue is teenage pregnancy. So we have had to adapt. That has been a key challenge for us, ensuring that you give people enough messaging, the messaging that they require at that particular time. [KI_KLA_SUB_1] CHC is a five-year program. But PEPFAR targets change every year. It's good to have generic, big picture themes, but these can’t keep up with the change required. The contract has to make allowance to shift objectives year by year and region by region as priorities change. I feel they [CHC] have learned a lot because they had to meet demands they hadn’t planned for. Challenge 3: Creating Room for Improvisation Some health areas lend themselves to integration more naturally. For example, there is a clear link between HIV and TB. Maternal and child health include a bundle of products, services, and behaviors. HIV includes prevention as well as care and treatment along with its own 'integrated' set of behaviors and services. However, the integrated approach is not as clear when combining other health areas and puts more pressure on the person delivering IPC. As questions and confusion arise, frontline 'brand ambassadors' need to be prepared to facilitate a conversation and provide answers. Obulamu's multi-channel communication involves mass media posing questions (e.g. “How's your baby?”) without necessarily answering the question. Instead, Obulamu suggests that the reader/viewer speak to a community champion or visit the health facility to access products and services. This question format created a bit of a strain on IPC champions and others in the position of authority at the district and facility level. Although talking points were provided to IPs and, in turn, IPC agents and champions, the need and the ability to improvise presented a challenge: 57 Sometimes you ask the questions and the health workers shy away from them. This is especially common with the VHTs [LS1_F_MUK] Being the LC1 of the area, youth called me and were asking me about Obulamu. I didn’t have the appropriate response, but I referred them to the facility and told them to read the poster in the markets and roads. [KI_AMU_SSVHT_1_1] Challenge 4: Dealing with Converging Messages Convergence in messaging was a logical outcome of integration in line with the principles of 360 degree communication. We have already noted that integration in practice was more complex than on paper and its challenges have been discussed at length in the earlier sections. This evaluation probed the issue of converging messages in interviews with many frontline health educators. The intention was to gain a deep understanding of how convergence in messaging factored into the way they approached SBCC. In response to whether there were instances, such as outreach and mobilization activities, when messages on several health issues were addressed simultaneously, one interviewee said: When they come for a bash, it’s like one issue at a time. [...] They don’t mix the information. [...] You know, sometimes when you focus on very very many things, people have interests. They can select the point of their interest and leave the others. So it’s better at a time to focus on one, then maybe you move to the next. When, at one event, you talk of various things, then there is [selective perception] where some other people may take [some messages] and then they leave [others] depending on the facilitators. If there are different facilitators doing different topics, then they would choose. Maybe one would choose the one [facilitator] that is so eloquent and leave the other one. So it’s better to focus at least. At one event, focus on one issue. [KI_LR_DG_1] ---*--- Evaluation Question: Coordination and Capacity Q4: How have CHC-relevant coordination structures (intra-agency coordination, national structures, partner platforms) contributed to improved capacity of Government of Uganda and other partners to design, coordinate and implement integrated SBCC? SUMMARY ● Two of CHC’s biggest achievements were: 1) helping to revitalize the SBCC technical working group (TWG) at the MoH, making it communication-focused as opposed to content-focused; 2) streamlining the process by which SBCC materials get approved by the MoH’s Health Promotion and Education (HPE) Department. These achievements were highly valued by the MoH. ● CHC also improved health communication coordination across USG IPs by standardizing the development of materials and messages and providing ‘seed copies’ which IPs can adapt. This achievement was highly valued by KIs. ● At the same time, coordination was affected by competing priorities of IPs who also had their own objectives, plans, and targets that sometimes did not segue with CHC. Additionally, goal posts were often moved in response to PEPFAR, many targets for which changed annually. 58 ● Because CHC was a PEPFAR “central mechanism,” some PEPFAR IPs had conflicting perspectives as to CHC’s role. For some, CHC was a service provider for IPs health communication needs, rather than a unifier of USG health communications. There were also questions over who should then pay for health communication, if CHC was, in fact, the PEPFAR “health communication partner.” ● CHC regional offices were not all operating at the same capacity, and did not necessarily have the synergetic relationship to regional health IPs that was envisioned in the original technical proposal. ● Coordination downstream was also an issue, as CHC did not have a direct relationship to their own messengers and brand ambassadors (VHTs, champions, peer educators, health workers), as it was the role of regional health IPs and their sub-prime partners to manage these relationships. As a consequence, many of these messengers were only minimally trained on Obulamu. ● District government coordination, too, was mixed. CHC brought District Health Educators (DHEs) numerous and highly-valued opportunities for capacity-building and participation in health education work which otherwise went unfunded by district government. However, this raises sustainability and ownership issues, since district SBCC is subsequently entirely reliant on the existence of CHC (or any other external SBCC project). ● Additionally, over-centralization meant that key regional creative partners – in particular, local media – were not optimally engaged. Per CHC’s IR2: Improved coordination and collaboration among partners and standardization of health communication interventions, CHC was meant to partner with: ● The Government of Uganda to improve its capacity to roll out the national health communications strategy ● District government to strengthen capacity for health promotion and education and improve district health outcomes ● Regional health and other IPs to roll out coordinated, integrated demand generation tied to IP supply-side support ● Primary health workers, VHTs, peer educators, and champions at community level to improve capacity to communicate with Obulamu beneficiaries and generate demand for services ● USG PEPFAR agencies to ensure a uniform PEPFAR SBCC message ● Local creative partners to boost capacity to design and deliver high quality HC materials CHC had mixed success in doing so, although many of the bottlenecks to improved coordination exist beyond the control of CHC and relate to dysfunctionalities in IP coordination in Uganda more generally. Here we focus on five themes with respect to CHC’s impact on coordination of HC in Uganda. ● Streamlined SBCC coordination and approval ● Improved USG IP coordination of health Materials ● Competing priorities ● Systems bottlenecks ● Overcentralization As will become apparent, many of the issues we have touched upon before - e.g. high quality of campaign, high value placed by partners on CHC materials, problems with overcentralization, large regional discrepancies in capacity/motivation/interests both with respect to district government and IPs - come up again here, and serve as facilitators/barriers to CHC’s achievement of its IR2. Streamlined SBCC Coordination and Approval 59 The BCC Technical Working Group (TWG) existed for a number of years before CHC (and previous to this, BCC was subsumed under a general Health Promotion TWG). However, prior to CHC, its responsibilities were confused and its composition was not matched to its objectives. The members tended to be higher level technical people and meetings - which were infrequent - often ended up stalling over objections to the clinical/technical content of the messages as opposed to BCC design and implementation.88 CHC advocacy played a fundamental role in the redesign of the objectives and composition of the BCC TWG. The BCC TWG is now only composed of professional BCC experts, and the focus of the meetings is only on SBCC itself (message design, communication channels, how to bridge supply and demand). The technical and/or clinical content, meanwhile, is dealt with under health area￾specific working groups and SBCC sub-committees within. The BCC TWG, comprised of 30 partners - with representatives from the Ministry of Health’s Department of Health Promotion and Education (HPE) and SBCC specialists from the national disease control programs (HIV/AIDS, TB, Malaria) and partner organizations - meets quarterly and provides a platform for sharing, strategizing, and standardizing SBCC. CHC’s contribution to the revitalized TWG was highly valued by central government.89 In addition to this, CHC has helped streamline the process of getting SBCC materials approved through HPE by working closely with the MoH to create a set of SBCC material approval guidelines. This work has also highly valued by partners. We were always meeting and updating each other, reviewing almost on a monthly basis to see how things are moving. We were fully involved with what was happening: work planning, reviews, all these issues of health development, everything would come through our office. [KI_KLA_CG_4] With the central nature, the concentrated specialization in communication in one place, it can be very difficult to get approval from MoH. But now they (CHC) have close collaboration with the Ministry so it's easy to get messages approved. [KI_MUK_RHIP_1_1] We always have meetings, like we’d have weekly meetings to touch base on what has happened and we provide updates and those meetings were like comprehensive technical meetings. You’d provide an update and if there were any issues, we’d have another meeting to iron them out and decide on a way forward. And then we were given a work plan that clearly spelled out the direction we needed to take. [KI_KLA_SUB_1] Close collaboration with government has resulted in the MoH having a high sense of ownership and in the Government of Uganda and its partners better understanding the value of designing an integrated campaign under one unifying brand (Obulamu). It has also contributed to MoH adoption of the Life Stages segmentation approach in its new National SBCC strategy. It’s a credit to this project. Actually, out of it we came up with a national strategy, a national SBCC strategy, an integrated SBCC strategy. The CHC has also set the pace for integration. [KI_KLA_CG_4] Improved USG IP Coordination of Health Materials Additionally, since 2013, CHC has provided technical assistance to the MoH and USG IPs to standardize the development of health communication materials and targeted messages on HIV/AIDS, MCH, malaria, family planning, TB, and nutrition. To date, over 2 million 'seed copies' of HC materials in 19 languages have been developed and disseminated to IPs across the country. In 88 KI_KLA_CG_1 89 KI_KLA_CG_4, KI_KLA_CG_1 60 addition, CHC developed implementation guides for each of the campaign’s LS segments with IPC talking points in order to standardize messages delivered by multiple IPC agents. Nine regional CHC officers (Exhibit 20) are supposed to work in close collaboration with regional IPs doing planning and providing technical assistance. Seed copies were highly valued by IPs, and helped the unify their messages. Additionally, seed copies helped harmonized communication across PEPFAR programs, most notably DREAMS. They would actually recommend for us printers. There were printers they would have used and we would get in touch with them and print extra. So even for DREAMS, they would give us seed. They would give you a particular number per district at the beginning, and then you would do more for yourself. Because we had budgeted for this in our program. [KI_KLA_RHIP_1] As [name of RHIP], we haven’t developed the messages or branded our messages. But the deal was we can own them and brand. For example, if it’s a poster with all those logos, we can also put our logo somewhere. But we haven’t got that far. So we get…I should call them maybe messages. And of course if they are messages, yes, they are generic messages from CHC. [KI_IGA_RHIP_1] Before every partner would do its own thing and deliver. CHC however worked with all partners and beneficiaries to design materials. Uniformed forces were brought in as well and consulted...It's an interagency mechanism so the “the whole idea was for us to speak the same language” and to help MoH lead the effort. [KI_KLA_USG_1] Competing Priorities At the same time, coordination was affected by regional health IPs relationship to both USAID/Uganda and PEPFAR. IPs have their own contracts, IRs, and work plans to fulfill. They also have their own targets (both those spelled out in their monitoring and evaluation plans and those that change annually with PEPFAR). These competing - and sometimes rapidly changing - priorities, motivations, and obligations have sometimes stood in the way of coordination and collaboration. Now the problem is, how are we going to marry those targets with what CHC is supposed to be doing? [KI_GUL_PRIME_1] Sometimes it got a little confusing because there would be so many priorities and we had an obligation to deliver our part of the work plan. [KI_KLA_SUB_1] The project also represents a partnership between USG agencies which brings its own set of challenges. As one of the USG partners explained: Part of the limitations of coordination between agencies is that each agency has its own contractual responsibilities and organizational restrictions [KI_KLA_USG_1] These competing priorities were exacerbated by the fact that, as CHC was designated a central PEPFAR mechanism, there were uncertainties as to whether or not PEPFAR-funded IPs should now have to pay for HC out of their own budgets. There were expectations by a few partners that, if CHC was “doing SBCC” for PEPFAR, then CHC should assume the costs. Additionally, while partners appreciated seed copies being distributed to them, many actually didn’t have budget lines to develop, reproduce and deliver these materials. Building capacity for HC, in these instances, did not translate into implementation. [We] just want them to not only give us seed copies. Why don’t they take full charge so that we know that they have documentation materials? And because now they tell them maybe it’s USAID that does it, we shall give this partner money for materials so Obulamu will do only seed copies. But 61 as you know for us, we are programmers. All the money will be spent on the program, not on materials development…They gave us seed copies and knew that we were going to reproduce. We’ve not reproduced any. [KI_KLA_RHIP_3] And even then the fact that mechanisms have not received money for communication. It was sort of done automatically. You don’t have money for communication. Communication has been given to a central mechanism. “Now why are you going to an area that is not yours?” … I wouldn’t go into the expense now of trying to develop a message, trying to get it to the Ministry to get it approved, when I don’t have…no one pays for that time. That time should be paid for from the CHC budget. So they have done that for us. [KI_MUK_RHIP_1_2] Now as CHC we are supposed to only produce seed copies. Seed copies of materials. And time and again I’d get communication from CHC headquarters, the country office, saying that while we disseminate and do placements for these seed copies, the IPs are supposed to be printing additional copies. But it wasn’t really clear. [KI_GUL_PRIME_1] Thus, while CHC has developed capacity among IPs, the ability of IPs to actually act on that capacity is constrained by financing and by confusion of roles vis-a-vis PEPFAR. Additionally, while CHC ideally has nine regional offices, not all of them are operating at the same capacity. Regional CHC offices were meant to be based at the office of the regional health IP (i.e. RHITES, Baylor, Nu-Hites in the north). However, in many instances, the CHC regional coordinator was not based with the IP. In Iganga, for example, the regional coordinator did not have clear office space and had to proceed through the same security checks and leave identification at the compound gate at RHITES-EC as an external visitor would. In Mukono, the CHC regional coordinator was based out of the Kampala office and was not part of MUWRP, while in Western, the regional coordinator had his own office that was separate from Baylor. In the North, there was a single stand￾alone office for all of Acholi and Lango sub-regions. This office used to be shared with Nu-Hites, but Nu-Hites closed and the subsequent string of implementing partners were based elsewhere. Additionally, most regional health IPs had other communication and community mobilization partners as well, contracted directly as sub-primes (e.g. CDFU). Therefore, the model of CHC and regional health IPs as essentially being a single vehicle for demand generation/supply delivery was not necessarily born out in practice. Systems Bottlenecks to Coordination Additionally, CHC’s ability to strengthen capacity and improve coordination was impeded by ongoing systems bottlenecks. Ultimately, Obulamu requires messages to be delivered at the lowest most levels of the health system, i.e. in households and communities. The primary means by which these messages are delivered - through which IPC, community mobilization, and brand ambassadorship takes place - are via Village Health Teams (VHTs), peer educators, and community champions. VHTs are, essentially, community health workers, a voluntary cadre of health workers who are being increasingly institutionalized into the Ugandan health system. It should be noted that, in rural communities, VHTs, as well as Obulamu-identified champions and peer educators, are not infrequently the same person, i.e. a VHT is also a peer educator is also a champion, and that the 62 formal recruitment, training, monitoring, and incentivization of these cadres of volunteers are not yet fully agreed upon, let alone streamlined and standardized. As mentioned before, CHC does not work directly with VHTs, but instead works with and through regional health IPs, as well as through regional health IPs sub-prime community-level communication and mobilization partners. For this reason, VHTs/peer educators/champions are not always entirely clear as to their relationship to Obulamu or CHC, or when/how/if they should be trained/incentivized by CHC itself. This has created problems in term of IPC and community-level messaging. Ideally, Obulamu relies upon a cascade training approach where CHC orient IPs in IPC, then IPs orient community and facility-based IPC agents. CHC regional officers should then do site monitoring, supervisory visits, and client exit interviews. However, not all CHC regional offices have enough staff to do this. Many VHTs and health providers we spoke with told us that they had not been trained on Obulamu at all and those delivering Obulamu's interpersonal communication (IPC) were sometimes stretched and not as prepared as they would have liked. Some did not understand messages. On a few occasions, the evaluation team found that VHTs were unaware of Obulamu's current media products, including billboards and radio spots. This kind of gap could easily result in a loss of credibility for VHTs and other Obulamu messengers. A number of KIs felt this gap had to be addressed in the follow on mechanism. The next project should think of investing more at the community level and trying to link the media with the community. That is a gap. [KI_KLA_CG_4] We are looking at mass media as a complement to IPC. For example, a radio spot can become a catalyst for discussion with communities. But I think we have not made deliberate effort to bring mass media into close contact with IPC. We normally say our IPC stands out but that disconnect is there. It’s something we need to address. [KI_KAB_PRIME_1] The biggest challenge we have in this region are counselors. Counselors would be better at consistently communicating to our target population, but we have few counselors because they are not in the Ministry of Health structure. So we’ve gone ahead to train nurses and midwives in counseling. [KI_KAB_RHIP_1] If they can provide us with trainings, we would really appreciate that. Trainings especially on this kind of program of Obulamu and what they need from us as health workers. What should we do for them so that we can work hand in hand? [KI_SSHW_IBA] We need more training, especially on how to interact with people with different characteristics, especially those who are HIV positive. [KI_SSCHAMP_IGA] District capacity was also an issue. Here CHC’s performance was mixed. On one hand, District Health Educators (DHEs) greatly appreciated the capacity building and training brought by CHC, including training in SBCC and participation in Communities of Practice. Several DHE’s quoted directly from the CHC training modules as to various SBCC approaches and principles, and saw these opportunities as extremely valuable.90 Additionally, several partners noted that - as SBCC is notoriously underfunded (or not funded at all) at district level - one of CHC’s most important contributions was to finally give them something to do.91 CHC came in at the time when we really needed the support. [KI_MUK_DG_1] 90 KI_LIR_DG_1, KI_AMU_DG_1 91 63 When we are doing an Obulamu event, we are integrating first of all the multi-disciplinary teams. We are bringing in our community leaders because they have to sustain this. Before they begin an event, CHC does an introductory meeting with the community leaders, the VHTs, and the DHO. [KI_KLA_RHIP_3] The uniqueness of Obulamu is that it would motivate us to come up with (SBCC) plans for a quarter. Templates would be provided and the expected areas suggested. [KI_GUL_RHIP_1] That said, there were also concerns that CHC’s approach - in the absence of financial commitment by district government - was also unsustainable, and that responsibility for SBCC, and HC more generally, now could just be diverted by DHTs to partners. Government... give up their responsibility. Whereas it would have been an encouraged if I had been assisted in a certain way. I think I have to put more effort. But they just give up and they say “Ah. CHC is there! You think you can talk to CHC? They will link you to the radio station.” While it’s the role of the government to sustain these efforts. [KI_MUK_DG_1] [W]e need to advocate for districts to begin to have at least some money set aside for health education. They shouldn’t rely on partners only. I think there should be a budget. They don’t have the budget I think because they don’t see the significance. … So whenever you sit in the meeting they say “Health education? CHC!” And then I tell them “Look. CHC alone cannot do health education. Its our education. [KI_GUL_PRIME_1] Overcentralization Many of the bottlenecks with coordination were perceived as being due to the overcentralization of the project; that is, in spite of Action Media, attention to local languages, and key role played in community mobilizations, Obulamu and CHC were nevertheless perceived as being somewhat “top down.” This perception was especially evident in the north: A big disappointment where people from central [Kampala] come to the district and begin fixing campaign materials by themselves without involvement of district authority. Yet, initially, the Gulu￾based (Obulamu) staff would work with the District officials... One of the DHE’s roles is to monitor people’s perceptions of health messages disseminated to the community through any media. The challenge was that we did not know the frequency or duration of (Obulamu) spots / DJ messages on TV and radio which originated from the Ministry of Health. We were just passive listeners, and only convinced to be part of it because the Ministry of Health is mentioned. [KI_AMU_DG_1] At the district level…that is where the gap is. There is lack of coordination and you find that there are some duplications where you can’t distinguish who does what and the success goes to who. I think most of the coordination is done at the highest level rather than at the district level. [KI_LIR_DG_1] However, even outside of Northern Uganda there was a desire for more autonomy, involvement, and capacity building, with local creative partners identifying clear instances in which they were underutilized and could, if engaged, add value to CHC’s efforts. In particular, regional radio had existing platforms - community correspondents, fan clubs, specific regional radio programs, vox pop - that could have been leveraged to provide regionally-tailored communications. A number of stakeholders felt that CHC had missed opportunities to do so. There are 5 radio stations engaged in our Obulamu campaign and there is an agency in Kampala that directly deals with them. At the same time, I need to build a relationship with those radio 64 stations so that I am in a position to negotiate for free air time. But I don’t have that leverage. Everything is too centralized. The region can only do so much. And yet we understand the context here better than anyone else. [KI_GUL_PRIME_1] Usually CHC will organize Action Media research, but we are not involved. They will just come and give us results, but it would be great if we also went out with them. [KI_KLA_CP_1] We should get to know each other. Share ideas. Because we are part of the team. Now we only receive materials. But even if the material has already been designed, they should get us on board, be part of the decision, when deciding on the content. Because there is some content which is easily not applicable to my listeners. [KI_LIR_CP_1_2] ---*--- Evaluation Question: Research and Knowledge Management Question 5: How did information generated through research and knowledge management boost and improve HC in the country? SUMMARY ● CHC carried out vast amounts of research, including evaluations, Action Media, case studies, feedback studies, listening surveys, special surveys, and exit interviews. ● Much of the research was highly participatory with respect to beneficiaries and fed directly into campaign design and adaptation that helped Obulamu achieve relevance and memorability among beneficiaries. ● At the same time, very little research was disseminated to partners, nor did IPs or DHTs participate in either the development of indicators or in research itself. ● KIs – including IPs and DHTs - expressed the desire to have data on SBCC indicators to feed into their own planning and implementation processes, especially those for PEPFAR. They also expressed a desire for input into indicators. ● Additionally, as mentioned earlier, there was a disconnect between the kinds of data captured in CHC’s existing routine monitoring activities and the kind of information needed to catch and remedy negative trends between Phase I/baseline data and Phase II/enline data. This section reflects CHC’s IR3: Increased research and knowledge management to enhance health communication. The Obulamu campaign followed the principle of data-driven design from the very beginning. As a consequence, CHC generated numerous KM products out of the wide body of research it accumulated. These included: ● Evaluative studies ● Action media/Participatory research ● Audit ● Case studies ● Rapid qualitative assessments ● Listening surveys ● Qualitative studies ● Special studies ● Exit interviews ● IP feedback studies Even before the first partners’ meeting was convened, CHC conducted an HC audit reviewing over 50 national HC strategies and policies related to the six health areas. Phase I/basline data and formative assessments served as an evidence-based foundation for campaign strategy, revealing that health was not a priority for many people. As described previously, that insight drove the creation of the popular Obulamu concept which positions health in the wider context of life and well-being. The brand was conceived at a national design workshop with multiple stakeholders, including creative partners, who all worked together to translate this research into the Obulamu brand. 65 Consultative meetings and interactions with the MoH, regional IPs, district health authorities, IPC agents, community champions, and target audience members were planned to ensure that partners stayed involved and up-to-date on research and monitoring activities and results. As demonstrated earlier, Action Media, a participatory method of developing messages and materials alongside the target audience, became one way that CHC integrated the normally separate steps of research and creative development, designing targeted materials with the participation of the people they were meant to reach (see Exhibit 21). This helped to increase relevance and created a sense of ownership. Several of the people we spoke with commented on CHC's use of Action Media, including CHC staff and advertising agency partners. “It's always a continuous process, trying to understand your audience” [KI_GUL_PRIME_1] CHC uses Action Media. You get the input of the community and when it comes out, it’s something they appreciate, something they understand. [KI_KLA_CG_4] The most notable [research activity] is our Action Media workshops where we conduct research as a process of designing and developing these materials. As far as health communication is concerned, it’s the most important research that you can do because it’s out of that that you can actually design and focus your messages. [...] It’s always a continuous process of trying to understand your audience. [KI_GUL_PRIME_1] To improve HC and uptake of services in selected priority districts, the project also conducted on￾going Continuous Quality Improvement Monitoring (CQIM). CQIM was conducted at the facility and community levels and through IP feedback. Additionally, to determine the effect of health messages and a client's ability to take action, and to assess provider-client interactions and utilization of HC materials, the Obulamu Site Improvement Monitoring System (OSIMS) was created. All this fed into an attempt to carry out continuous monitoring and adaptation. Once you get evidence, use that evidence you’ve generated to guide you in planning and also documentation. You can talk of some success stories of CHC, because there is that kind of tracking. [KI_KLA_CG_4]. We do exit interviews. We want to find out the perception people have about, for example, a health camp, a youth bash, a community show. Even if it is a mini-community show, we want to conduct exit interviews to understand our audience better. [KI_GUL_PRIME_1] Following from this research, multiple KM products were created including Action Media reports, case studies, survey results, research briefs, and newsletters. (Exhibit 22). Learning events and Communities of Practice were held with districts and regional IPs almost every quarter. In addition, data for decision-making sessions were conducted for DHEs. Exhibits 21 and 22: Examples of CHC Research and KM Products 66 We try to put health communication on the district's agenda. We have organized, for example, meetings with DHEs. We had a Communities of Practice meeting where we invited DHEs and biostatisticians. And we tasked one of them to start a WhatsApp group to share some of the new developments in health communication. [KI_GUL_PRIME_1] In most health facilities, Obulamu team together with our community linkage team have done what we call CMEs, Continuous Medical Education, on how to respond to the needs of clients that have been mobilized maybe during community activation shows. [KI_KAB_RHIP_1] Challenges to Research and KM In spite of all of CHC’s research, evaluation, and KM work, most KIs interviewed said they had not received (or could not recall receiving) any of the project's KM products. This includes DHEs, who as members of the DHTs would have needed such information for district planning, as well as the other USG agency PEPFAR partners. Many had not heard of CHC’s research or, if they had, only heard of a one-off study.92 Many did not know exactly what CHC was measuring. None described measuring indicators jointly with CHC. Only a few could recount being asked to be involved in research, but described the context of this engagement as being asked to comment on research design.93 Additionally, while learning events promoted KM products, little institutional memory was left behind. Nothing was disseminated to me. The project officer based in Gulu informed me about the research work, and that marked the end of it all. [KI_AMU_DG_1] We would be interested in knowing the level of knowledge that people have in the community and also people’s perceptions of the services. That would help us build strategies to address those perceptions. Whether it’s a negative or a positive one, promote it. That is where you find a way of addressing it. [KI_LIR_DG_1] There should be participatory monitoring of programs, especially health communication to determine good indicators on health. There should be exchange visits to enhance learning especially to areas where Obulamu had positive outcomes. [KI_LIR_SSCMP_1_2] A lot of research it seems has been done, but it's not proportionate to what has been made available. [KI_KLA_USG_1] The other main challenge - described more thoroughly in Q2: Best Practice Strategies and Behavioral Outcomes - was that when it came to evaluation data (i.e. research on SBCC indicators as opposed to formative, special, or small scale listener studies), this data was collected only twice: at Phase I/baseline data and at Phase II/endline data. As the section on the extent to which Obulamu may have influenced behavioral outcome clearly shows, the endline data gives the imprssion that in many instances there was either a) no change or b) negative change. Here we do not presume - based on this data - that CHC had no impact. Many factors - including study design - play contributing role to such data on SBCC indicators. That said, because there was such long duration between the Phase I/baseline data and the Phase II/endline data, and no midline in between, it is impossible to determine what exactly happened along the way. Additionally, since baseline to endline data sits disconnected from regional implementation contexts (that is, even when disaggregated by region, the data itself is decontextualized), CHC alone ends up judged on behavior change indicators for which multiple partners were actually responsible. As Q1 on Misperceptions and Adaptation shows, CHC did an amazing job at adapting to learning. However, as the Phase I/baseline and Phase II/endline data surveys show, routine monitoring clearly 92 e.g. KI_MUK_DG_1, KI_KLA_RHIP_5 93 KI_RHIP_MUK_1_1 67 did not capture certain red flags in the data. As a consequence, quite a lot of the endline data - if taken at face value - paints a portrait of CHC performance that, we believe, is not fair, since so many other categories of CHC research, as well as our own findings, highlight CHC’s numerous achievements and innovations. Additionally, the Phase I/baseline data and Phase II/endline data surveys made the same mistake that our evaluation did: they sometimes used the audience segmentation strategy - i.e. Life Stages - as a means of disaggregating data for monitoring and evaluation. Yet LS overlapped, so baseline data and endline data were extremely difficult to interpret and compare. This is not to say that using LS as a route into research and learning is not useful. For Action Media and special studies, using LS as an analytical frame seems to have worked well. But for tracking SBCC indicators over time - i.e. for analyzing change in behaviour - they are problematic. Lastly, several KIs talked about the disconnect between what CHC was measuring and what KIs actually need to know to meet their own targets, especially in the context of PEPFAR. They expressed a need for more diverse, and more jointly-developed SBCC indicators, so that SBCC findings could feed into and inform across mechanisms and across USG agencies. ---*--- Evaluation Question: Supply and Demand Q6: Which are the main barriers that exist as to the delivery of the six services promoted by CHC? What changes should be made to SBCC programming in the future to improve linkages between HC and integrated health service delivery partners? SUMMARY ● Although CHC was supposed to work through and with regional health IPs and district governments to bridge the gap between supply and demand, this gap remained one of the most serious bottlenecks to CHC’s impact. ● Persistent commodity and health human resources shortages at health facilities undermined many of CHC’s efforts (and have also served to undermine USG HC investments for more than a decade). ● Where gaps were successfully bridged was in the context of highly technical, one-off service delivery activities - family planning outreaches, safe male circumcision camps – often driven by not-for-profit private sector service delivery organizations which could guarantee commodities and trained medical professionals for a limited period of time. ● Where gaps were not successfully bridged was in the context of the routine service delivery that most Ugandans rely upon for the daily health needs. ● These gaps point to ongoing issues with USG IP/district government, coordination and dysfunctional health systems. While delivering health services and health system strengthening is not included in CHC's IRs, the lack of linkages between demand generation and supply of health services was a key bottleneck identified in the 2012 Joint Behavior Change Survey, the 2013 evaluation of CHC’s predecessor, HCP, as well as in the 2014 CHC HC audit activity.94 Therefore, while CHC was not explicitly tasked with bridging this gap, there was an underlying acknowledgement that efforts to strengthen linkages between CHC and its partners were critical, since failure to deliver these services has 94 USAID/Uganda, Uganda joint behavior change communication survey. (Kampala, USAID/Uganda, 2012a); USAID/Uganda, Final evaluation of the Health Communications Partnership (HCP II) project. Kampala, USAID/Uganda, 2013); FHI360, Health communication in Uganda: Findings of an audit of strategies, strategies/materials, and implementing partners in 2013/2014, (Kampala, FHI360, 2014). 68 persistently undermined SBCC and demand generation in the past. If products and services are not available, clients lose motivation to change behavior, lose interest in seeking health care, and lose faith in the messages, messengers, and institutions that promote them.95 This loss of trust was reiterated by many FGD participants in over a quarter of all FGDs we carried out:96 There are times when you come to give birth and mama kits are not enough, so if you are fortunate you get and, if you are unlucky, it’s your friend who gets. Then sometimes one mama kit can be used by three mothers and a situation like that is not good at all. [P4, LS2_F_IGA] Obulamu tells us of use the net but they don’t distribute the nets. If the information is given, the product should also be given. [LS4_M_AMU] There are few doctors and midwives in the health facilities....[There is] [l]ittle medical supplies (little medicine) in the health facilities and so people are given Panadol only [P6, LS2_M_IBA] No, though the awareness has been raised, sometimes these services are not available. For example, you go to a health centre and drugs are not there which demoralises other people who would love to go for these services. [P7, LS1_F_KYE] Health systems assessments, along with numerous studies referencing the supply-demand gap, have long pointed to persistent commodity and human resource shortages within Uganda’s public sector health facilities, alongside demotivated and imperfectly trained health workers, high levels of staff turnover, poor information systems and patient tracking, and a myriad of other bottlenecks that prevent services from being delivered on demand, thus leading to disillusionment and distrust of the health system by beneficiaries.97 District health educators and facility- and community-based health workers and champions emphasized this gap:98 The biggest challenge is creating demand when there is no service. [KI_MUK_DG_1] They’ve been let down! It's like we’ve been selling what we don’t have. ...When we are creating awareness…then the service delivery also needs to be improved. [KI_LIR_DG_1] Most of the time we create demand, but then the supply side is weak. I remember one time, we were with Obulamu and we wanted to test, but then there were no testing kits. Yet we had already created demand. And the clients were there ready to be tested for HIV. We ran up and down but test kits were nowhere to be found. [KI_IGA_DG_1] 95 Altaras R, Montague M, Graham K, Strachan CE, Senyonjo L, King R, et al, “Integrated community case management in a peri-urban setting: a qualitative evaluation in Wakiso District, Uganda,” BMC health services research, 17, no.1:785 (2017): Anastasi E, Borchert M, Campbell OM, Sondorp E, Kaducu F, Hill O, et al, “Losing women along the path to safe motherhood: why is there such a gap between women's use of antenatal care and skilled birth attendance? A mixed methods study in northern Uganda,” BMC pregnancy and childbirth, 15, no. 287 (2015); Babirye JN, Engebretsen IM, Rutebemberwa E, Kiguli J, Nuwaha F, “Urban settings do not ensure access to services: findings from the immunisation programme in Kampala, Uganda,” BMC health services research ,14, no.111, (2014); Byabagambi J, Marks P, Megere H, Karamagi E, Byakika S, Opio A, et al, “Improving the quality of voluntary medical male circumcision through use of the continuous quality improvement approach: A pilot in 30 PEPFAR-supported sites in Uganda.” PLoS one, 10, no.7:e0133369, (2014); Kananura RM, Kiwanuka SN, Ekirapa-Kiracho E, Waiswa P. “Persisting demand and supply gap for maternal and newborn care in eastern Uganda: a mixed-method cross-sectional study,” Reproductive health, 14, no.1:136, (2017). 96 See also P9_LS4_M_IBA, P4_P6_LS1_M_LIR, P8_LS2_M_KYE, P2_P3_P6_LS3_F_LIR, P10_ LS1_F_IBA, P2_LS2_M_IBA, LS_2_AMU 97 Baine SO, Basaza R, Pratt BA, Health system pre-assessment report, 2016, (Kampala, USAID/Uganda, 2016); Madinah N, Challenges and barriers to the health service delivery system in Uganda, IOSR journal of nursing and health science, 5 no.2:30-38 (2016); Mukasa N, “Uganda healthcare system profile: Background, organization, policies and challenges,” Journal of sustainable regional health systems, 1 no.1:2-10, (2012). 98 See also KI_SSCHAMP_IGA, KI_VHT_IGA, KI_KYE_DG_1 69 Of course, support for district health service delivery is the responsibility of the regional health IPs through which CHC is supposed to partner, all of which are funded by USG agencies (USAID, CDC, DoD) as well as, through their respective agencies, by PEPFAR. However, these IPs also note the problems with linking supply and demand: If you look at the service providers themselves, service provision, it's also still lacking. So even in instances whereby I assume we have the products – the kits are there – we might have a problem with the health workers. [KI_IGA_RHIP_1] Health workers in most government health facilities were overwhelmed with our mobilization efforts. [There is a] lack of commodities, stock outs due to the push [for VMMC]. Health facilities make projections, but less of what is projected is delivered and other deliveries do not come in time. [KI_GUL_RHIP_1] The demand usually outweighs the supply side. Because on many occasions, we have run short of supplies. Then even when efforts are made to bridge the gaps, we still have issues on the supply side. [KI_IGA_RHIP_1] Thus, health commodity stock outs and health human resources shortfalls are well-documented, on￾going bottlenecks in Uganda. These problems are neither new nor surprising to anyone familiar with the Ugandan health system. What lessons, however, can we derive from the CHC/Obulamu experience in navigating such gaps? One issue that emerged in the evaluation was the frequency with which IPs highlighted the contribution CHC made with respect to “outreach” activities (31 KI interviews and 15 FGDs referred to some form of service delivery outreach).99 By outreach, we are referring to special events - campaigns, camps, mobilizations - (over the course of a day or a week) that occur for specific, typically siloed, service delivery (e.g. VMMC, family planning, HIV counselling and testing (HCT) and STI screening, immunization, etc.). We contrast outreach with “routine” service delivery such as antenatal care, case management of childhood illnesses, case finding and case management of malaria, routine nutrition screening, etc.. While outreaches can identify and refer, for example, malnourished children or HIV/TB/malaria patients, they do not provide long term care. For this, patients must rely on the health facilities in their communities (in Uganda, these are Health Center (HC) IIs, IIIs, and IVs). 99 The Nvivo search string was “outreach OR camp OR mobilization” 70 The evaluation was unable to directly measure the extent to which CHC/Obulamu adds value to an outreach activity versus routine service delivery, but it was clearly a theme that recurred throughout KI interviews and FGDs. This is not surprising since outreaches - as intermittent, one-off (as opposed to daily) activities - can coordinate with partners and marshal HC, commodities, and human resources in the short term far more easily than ensuring their consistent availability over the course of months or years. Therefore, the MoH and its partners - in particular not-for-profit private health service delivery IPs - often rely on outreach models in order to 1) deliver these targeted health services at scale; 2) match supply and demand; 3) rapidly register outcomes toward meeting targets: The mobile services we have are for circumcision and maybe when we do targeted outreaches and usually these are at our bases because the capacity we have is limited. We do not get the funding to go everywhere. We assume Ministry of Health is also covering out there, so that has been the biggest challenge. [KI_KLA_CG_3] But issues of supply, we’ve never even…even head office does not allow that]...Because we have a store at the branch and ever we have to request for things…you request when you know you have supplies for like 3 months ahead. You push in another request to head office and supplies do come very fast. So our supply chain is very clear, very flexible. [KI_IGA_RHIP_2] Like the event I gave you an example of […] in Gomba sub-county. It’s very far from the health centre. So you know, at times, people are not funded to do outreaches. But now when Obulamu comes in and does an event and talks about the importance of men also seeking health services and those ones that can’t get services here [in the facility], these health workers whom we have brought in [to] this outreach are right there. So at least [we can say] “Please go for follow up visits to the health centre.” [KI_KLA_RHIP_3] Even right now I’ve told you we’ve gone to Nambule for an outreach, but you find the population, the turn out, is not all that good. But if the Obulamu was to be there, their mobilization strategy is really good … The music. You know people love such things. People come… They attract people. And as they come, they start their sensitization issues. People take in the message. [KI_IGA_RHIP_2] Problems with this model occur when those who have received services via an outreach attempt to follow up at the clinic, or when a patient misses an outreach and has to rely upon routine service delivery. At this moment, the patient has to rely on a functional health system, and it is here where Obulamu’s demand generation activities so often run into a brick wall. There have been at least two recent BCC TWG meetings on the issue of gaps between supply and demand. Follow on mechanisms and their partners, however, must explicitly address this question via building in improved coordination between facilities, DHTs, and regional health IPs tasked with service delivery and health communication partners. Additionally, they must also address the outreach versus routine health service divide, because many of the integrated messages within Obulamu actually relate to health issues and health-seeking 71 behaviour that must be resolved at a health facility (i.e. that cannot be effectively managed by outreach). Moreover, even in instances in which such health issues and health-seeking behaviour can be resolved at camps and through outreach, follow up is often required (e.g. when there are complications related to circumcision or contraceptive choice, or when HIV counselling and testing transitions to care and treatment). In addition, following from earlier discussions of the need for further regional decentralization, as well as the importance of interactive communication, follow on mechanisms should more directly address health workers as valued campaign 'messengers' and brand ambassadors, and attempt to bring in more innovative facility-based and community-based IPC: I was actually able to go to remote areas and see what is happening and appreciate what these health workers go through. And if they are left out of such programs, then they don’t link the service to messages that are being promoted. That is something very important. [KI_KLA_CG_4] We need to design messages that target the attitudes of the heath workers offering the services. As we are targeting messages to the community to create demand, we need to target the health workers as well to offer the service. [KI_LIR_DG_1] ---*--- RECOMMENDATIONS AND CONCLUSIONS Recommendations Based on the evidence presented in the evaluation, we make the following recommendations for the CHC follow-on: 1. Build on brand equity Why can’t we create another platform that people who have benefited from this communication testify and say “This has changed my life”? [KI_LIR_CP_1_1] What the evidence says: ● High levels of exposure and engaging materials and activities have created a memorable brand that is credible, well-known, and liked nationwide. ● There are still issues with language - in particular, the choice of a Luganda word as a national brand - but there are many indications that misgivings in many parts of the country are being overcome. What we recommend: Build on and leverage Obulamu in the follow-on. Phase 2 should continue the more memorable and well-liked elements (e.g. interactive and interpersonal communication, 360 degree messaging, real life experiences), but also include enough new content and design to attract attention, maintain interest, and freshen up the campaign. At the same time, in some parts of the country (e.g. Acholi), work needs to be done improving brand equity as the choice of the name Obulamu is easily politicized. Document and share the most compelling success stories, from the point of view of the people who were impacted by the project. 2. Segment the audience even further 72 It was important to come up with something generic. And LS [life stage segmentation] did the job. But a follow on must build from this. We can now split it and break it down according to PEPFAR’s key populations and tailor messages to program strategies. [KI_KLA_USG_1] What the evidence says: ● LS segmentation is a highly-valued best practice. It creates more targeted and meaningful communication and is well understood by most partners. ● Overlap between stages is seen as a value add not a problem (except when LS are deployed for monitoring and evaluation purposes). ● LS segmentation, however, is insufficient and leaves out differences among, for example, regions and key populations. What we recommend: Together with partners - USG agencies, IPs, districts - work out where to add additional relevant segmentation, including adding new segments to both primary and secondary audiences. This could include rural versus urban segments, students, faith-based audiences, uniformed services, and PEPFAR KP/PP. Make sure that these segments are tied to relevant campaign messengers (e.g. parents/family members, teachers, religious leaders). 3. Be a catalyst for more creative collaboration We should get to know each other, you know? That would bridge the gap. Sharing ideas. Because we are part of the team. [KI_KLA_USG_1] What the evidence says: ● CHC and Obulamu have been highly creative and collaborative via the use of SBCC best practices and Action Media. ● The relationship between CHC and the creative/advertising agencies and media houses has been inefficient and confusing in some cases and often resulted in bottlenecks. What we recommend: Continue to break up the creative pie by segment of population and by disease area, but allow one creative partner to lead the design and implement all communication channels, including mass media, IPC, print, facility-based signage, and social media. Additional support may come from sub-contractors or consultants, but accountability for the work should not be divided among multiple companies. To avoid counterproductive competition and overcentralization, develop a creative coalition to bring together the range of creative partners from commercial and non-profit sectors, academia and research institutions. 4. Add more local and consumer-generated content I suggest that Obulamu could form groups in the villages like youth groups, farmers groups [etc]. It would make it easy for Obulamu messages to pass through such groups. [P4_S3_M_KYE] We as a radio station have built…a Fans’ Club. Active listeners who are really listening. Who are calling in to request their music and send their greetings to their loved ones. So we organize games between us the presenters and the radio personalities and them. So this really is a good platform to use to spread messages [KI_MUK_CP_1] What the evidence says: ● Not everyone in the target audience felt represented. They wanted to hear their own language and see more local people telling their own stories. They wanted their region more represented in images and voices. 73 ● Not all creative partners have been optimally used, in particular, radio partners. While Action Media has been extensive, local creative input into design has been minimal. ● Media organizations (commercial and non-profit) are highly motivated and possess skills and substantial experience in content development. They know what works for and appeals to their audiences. What we recommend: Improve inclusivity and representation in images and content. Make sure the people used in photographs reflect the regional diversity of the population. Add more consumer-generated content, which can, in turn, be used in social media and as knowledge management. Ensure that creative partners at regional level have input into the design and delivery of materials as they have a more intimate understanding of their audiences. Engage local radio as creative partners and incentivize them to come up with innovative Obulamu ideas e.g. through competitive proposals for Obulamu creative grants. Mobilize regional musicians to document local stories of change in song. Orient more IPC agents to deliver population-contextualized campaign messages and facilitate population-appropriate dialogue. 5. Rethink and increase capacity-building activities We’ve had success with younger peer educators. How do we recreate the success in other districts? How do we interest the younger generation to become VHTs? [KI_KAB_PRIME_1] What the evidence says: ● Lower cadres of messengers - VHTs, champions, peer educators (many of which actually were the same individual), and health workers - are often insufficiently trained and incentivized to facilitate discussion, answer questions, and act as brand ambassadors for Obulamu. These messengers were nevertheless identified by KIs and FGD participants as key change agents for SBCC. Poor training and understanding of Obulamu messages and the campaign more generally, as well as lack of motivation due to under-incentivization, were identified as key barriers to Obulamu and CHC more generally. ● CHC capacity-building activities tend to target upstream implementers (government, IPs) rather than the individuals most directly engaging beneficiaries at facility, community and household levels. Obulamu is an interactive campaign that relies, to a great extent, on posing questions. Not having enough trained peers or health workers on the ground to help answer questions is problematic. ● This issue is compounded by the fact that the relationship between CHC and lower level messengers is mediated by regional health IPs and, often, mediated again by those IPs sub￾prime communication/mobilization partners (e.g. CDFU). ● Capacity building for IPs wasn’t always appreciated as such. Many PEPFAR IPs already believed that they had capacity for SBCC or, alternately, understood CHC as a service provider to whom they outsourced their HC (albeit on CHC’s own dime, since CHC had PEPFAR HC financing). ● There is need for clarity about what is expected of various stakeholders at different levels. The project should set realistic and functional goals for stakeholders - e.g. training the government to be critical evaluators of evidence and good consumers of communication and creative services by understanding the process and valuing the outputs. What we recommend: Include a strong downstream training focus that includes not only conveying content of messages, but also focuses on the Obulamu approach to communication: interactive, question-based, life- and wellness-focused, relevant to where an individual is at his or her particular point in life. Training necessarily requires monitoring, correction, and retraining and IPC can be expensive. Yet evidence shows that this element of SBCC is critical, so thought should be given about how to optimally, cost-effectively, and collaboratively 74 implement this element in Phase 2. Build in mechanisms for improved coordination with DHTs and regional health IPs as it is they who have the most direct interaction with health workers, VHTs, champions, etc. on a day-to-day basis. 6. Continue to partner on improving coordination The biggest challenge is that all these IPs have their own plans and...the district, they also have directives [KI_KAB_PRIME_1] One thing in Uganda is that everywhere you go they are developing new guidelines. Before you finish rolling out these ones, there are already new ones coming out, so it’s a very big challenge. [KI_KLA_CG_3] What the evidence says: ● CHC’s contribution to the revitalization of the BCC TWG was highly valued. ● Coordination with DHTs and IPs, however, varied from region to region, the result of which was major gaps between a) demand and supply of services; b) CHC’s ability to influence health outcomes; c) the context of outreach activities versus routine service delivery. ● CHC, as a PEPFAR central (and primarily indirect) mechanism, is only as effective as its regional health and service delivery IPs. CHC’s mandate is frequently constrained by individual IP priorities and PEPFAR programming. ● CHC frequently has found itself in a reactive role vis-a-vis individual partners, responding to requests by partners to mobilize or message on their behalf. ● Some partners felt overwhelmed having to keep abreast of changes in policies and guidelines that have implications for the way they communicate about health issues. What we recommend: CHC needs to boost its footprint on the ground. A mechanism for progressively tracking and disseminating national policy changes and decisions that affect the implementation of SBCC will help to ensure quick adaptation. More consideration needs to be given in the follow on as to how PEPFAR IPs coordinate with a central HC mechanism. 7. Decentralize and get the districts more involved Participating also motivates us. If we participate, we can own the information, the data, and utilize it. [KI_IGA_DG_1] What the evidence says: ● Obulamu reached most corners of the country with materials produced and disseminated in the major languages. ● A centrally-managed creative and media buying process allowed for coherence in all aspects of communication planning. However, it left limited room, if any, for some of CHC’s regional offices to leverage their relationships with local radio stations for asks like free broadcast airtime and for local radios to harness their relationships with and understanding of audiences in pursuit of Obulamu goals. ● There were recurrent suggestions for more involvement of the DHTs, with calls for CHC to involve all stakeholders, including the politicians, in communication planning and coordination activities. Districts will be more motivated to own and utilize the information and data they have contributed to generating. What we recommend: Delegate more core decision-making authority to CHC regional offices and IPs with regard to communication planning and implementation. This will enable offices and partners with deep knowledge of local communication landscapes and needs to exploit local networks and to facilitate adaptation. Institutionalize coordination mechanisms help ensure regular and 75 meaningful participation of and engagement with DHTs in planning and executing Obulamu communication activities. 8. Keep up the Action Media I participated in Action Media and I think that was really good to get the insights directly from the people and then market them and then take them back to them. [KI_KLA_SUB_1] What the evidence says: ● Action Media was highly valued by KIs. However, it was felt that sometimes there was a blurring of roles whereby target audiences via Action Media usurped the work of creative partners and were allowed to produce their own messages and materials. ● While the participatory attributes of Action Media were valued, executing it parallel to the mainstream work of creative partners opened an information gap in planning and denied creative partners vital opportunities for more first-hand learning about audiences to adapt accordingly. ● Action Media should be responsive to the project’s needs for real-time learning and adaptation, such as opportunities to explore more experiential learning tailored to literacy levels and media accessibility. Messages channeled through drama, youth, farmers, women, etc. can bring more people of low literacy and those who cannot access radio messages into the Obulamu catchment. What we recommend: With Action Media, do not confuse the target audience with creative professionals. Allow creative professionals to translate Action Media into effective SBCC. Make sure these roles are delineated but also synergized when planning future Action Media activities. 9. Create more SBCC indicators There is need to improve on the monitoring and evaluation component, integrating SBCC indicators....we want to start seeing things measured like how many men can confidently demonstrate condom use to their fellow peers. That level of confidence. I want to feel that that indicator measures behavior. [KI_KAB_RHIP_1] If we mobilize so much and we are working with an IP and then with a district...and these people [are] able to come as a result of our mobilization effort … and they received a service, we want to know how many? Much as the IP also collects the same data, we also want to collect the same data, so we understand how effective our mobilization is. [KI_GUL_PRIME_1] What the evidence says: ● While IPC in all its domains - household, community, facility - is recognized as a key driver of change in health-seeking behavior, there is less frequent and systematically-generated evidence on the processes, results, and outcomes of IPC interventions/activities. ● CHC and IPs have an interest in measuring results and outcomes, and typically gather data in line with their own programming goals and SBCC targets which are often identical. ● Using LS as a means of disaggregating monitoring and evaluation data creates major problems in measurement since LS overlap, making it difficult to interpret data. What we recommend: Work with partners to create more SBCC indicators tied to district or partner planning objectives. As a PEPFAR funded mechanism, make sure that SBCC indicators meet the needs of PEPFAR partners, including DHTs, regional health and service delivery IPs and on-going programs (e.g. DREAMS). Build into the HC planning process a mechanism to harmonize SBCC targets, indicators, and overall measurement systems. Include advocacy to ensure that 76 SBCC indicators are integrated into other mechanisms, programs, and initiatives. Disentangle the HC design approach from monitoring and evaluation. Monitor indicators far more frequently to be able to detect and respond to red flags in data. 10. Better utilize and disseminate of evidence Once you get evidence, use that evidence you’ve generated to guide you in planning and also documentation. [KI_KLA_CG_4] That [knowledge management] is the area of improvement for Obulamu. The people in the previous project were very quick. Whenever you did something with them, you were able to find it on the net. So that is where they need to improve. I’ve not found the materials. [KI_KLA_RHIP_3] What the evidence says: ● CHC accumulated massive amounts of compelling data on its target audiences, campaign design elements, and behaviour change outcomes. The amount of effort its monitoring, evaluation, and KM team invested in data collection and analysis was truly remarkable. ● There were many instances of CHC learning and adapting to the findings of its various research streams. ● Evidence, however, was not being systematically disseminated to, or used by, partners. Many had never heard of CHC’s research. Knowledge was accumulated, but not managed. ● Evidence never reached district levels. DHTs were not routinely involved in data collection. They often did not know that research was happening, and had no access to CHC data for decision making. Opportunities for DHTs and IPs to adapt to CHC learning were limited. What we recommend: Treat KM consumers in the same way as LS beneficiaries. Understand their KM needs and wants, including what would be most helpful and in what format. Pretest KM products. Rethink the existing KM strategy for the follow on. Make sure one focus of KM involves supporting the data/research needs of DHTs. Involve CHC research users in decision￾making about the research agenda and in the planning processes. Conclusions CHC has created a popular, integrated SBCC campaign that is well-loved by beneficiaries and partners, many of whom report feeling pride and a sense of ownership. In the process, they have shared international best practices in strategic communication, marketing, and behavioral science. CHC has strengthened the SBCC capacity of GoU and IPs through its leadership and collaboration. CHC Uganda has integrated messaging in six different health areas and targeted four discrete segments of the population under the same thematic umbrella, with enough exposure to create massive recall, brand recognition, and word of mouth. More importantly, they have repositioned health in the minds of many Ugandans who now understand that, more than treating illness, health is the foundation of life. Through this innovative and interactive project, CHC has bridged gaps between regional IPs, facilities and communities, USG agencies, and the supply and demand sides of health and well-being, with the goal of addressing people’s needs holistically. While gaps still exist and more work needs to be done, CHC has helped to unify programs and, in many key areas, led to increases in knowledge, shifts in attitudes, and improved uptake of a number of essential services and healthy behaviors...and it all began with the simple question, Obulamu? (How's life?). 77 78 Annex 1: References Altaras R, Montague M, Graham K, Strachan CE, Senyonjo L, King R, et al. (2017). Integrated community case management in a peri-urban setting: a qualitative evaluation in Wakiso District, Uganda. BMC health services research. 17(1):785. doi: 10.1186/s12913-017-2723-0. Anastasi E, Borchert M, Campbell OM, Sondorp E, Kaducu F, Hill O, et al. (2015). Losing women along the path to safe motherhood: why is there such a gap between women's use of antenatal care and skilled birth attendance? A mixed methods study in northern Uganda. BMC pregnancy and childbirth.15:287. doi: 10.1186/s12884-015-0695-9. Babirye JN, Engebretsen IM, Rutebemberwa E, Kiguli J, Nuwaha F. (2014). Urban settings do not ensure access to services: findings from the immunisation programme in Kampala Uganda. BMC health services research 14:111. doi: 10.1186/1472-6963-14-111 Baine SO, Basaza R, Pratt BA. (2016). Health system pre-assessment report, 2016. August 2016. Kampala: USAID/Uganda. http://pdf.usaid.gov/pdf_docs/PA00MKXW.pdf Byabagambi J, Marks P, Megere H, Karamagi E, Byakika S, Opio A, et al. (2015). Improving the quality of voluntary medical male circumcision through use of the continuous quality improvement approach: A pilot in 30 PEPFAR-supported sites in Uganda. PLoS one. 10(7): e0133369. doi: 10.1371/journal.pone.0133369 Chibita M. (2009). The politics of broadcasting, language policy and democracy in Uganda. Journal of African media studies,. 1(2): 295-307. doi.org/10.1386/jams.1.2.295_1 FHI360. (2013a).Technical proposal: USAID/Uganda Communication for Healthy Communities. Durham, NC: FHI360. FHI360. (2013b). Obulamu national integrated health communication platform, June 2013-June 2018. Kampala: FHI360. FHI360. (2014). Health communication in Uganda: Findings of an audit of strategies, strategies/materials, and implementing partners in 2013/2014. October 2014. Kampala: FHI360. FHI360. (2014b).Contextual barriers and motivations for the uptake of HTS and condoms among female sex workers and truckers in Busia township, Uganda. A qualitative rapid assessment. Kampala: FHI360. FHI360. (2014b). Contextual barriers, motivations, and coping strategies. HTS update and condom use in a fishing landing site in Busia District. A qualitative rapid assessment focused on positive deviance. Kampala: FHI360. FHI360. (2014c). Malaria upsurge in northern Uganda: An exploratory rapid assessment with Village Health Teams in Amuru District. Kampala: FHI360. FHI360. (2014d). Findings of action media with pregnant women - Mityana, Uganda. Kampala: FHI360. FHI360. (2014e). Contextual barriers and coping strategies with the uptake of modern contraceptive services and commodities in a selected community in Uganda: Highlights from a rapid qualitative assessment. July 2014. Kampala: FHI360. FHI360. (2017). CHC/Ipsos third listening survey, June 2017. Kampala: FHI360. 79 Government of Uganda. (2010a). The second national health policy. Kampala: Ministry of Health. Retrieved from https://www.unicef.org/uganda/The_Second_National_Health_Policy.pdf Government of Uganda. (2010b). Health sector strategic investment plan 2010/11-2014/15: Promoting people’s health to enhance socio-economic development. Kampala: Ministry of Health. Retrieved from https://www.unicef.org/uganda/HSSIP_Final.pdf Government of Uganda. (2015). Health sector development plan II, 2015/16 - 2019/20, p. 48. Kampala: Ministry of Health. Retrieved from http://health.go.ug/download/file/fid/834 Government of Uganda. (2016).Uganda HIV and AIDS country progress report 2016. November 2016. Kampala: Uganda AIDS Commission. Retrieved from http://library.health.go.ug/publications/service￾delivery-diseases-control-prevention-communicable-diseases/hivaids/uganda-hiv Government of Uganda. (2017a). Summary sheet: Preliminary findings - The Uganda population-based HIV impact assessment (UPHIA). August 2017. New York: ICAP/Columbia University. Retrieved from http://www.afro.who.int/sites/default/files/2017-08/UPHIA%20Uganda%20factsheet.pdf Government of Uganda. (2017b). Uganda demographic and health survey 2016: Preliminary findings. March 2017. Kampala: UBOS. Retrieved from https://dhsprogram.com/pubs/pdf/PR80/PR80.pdf Gruber J, Caffrey M. (2005). “HIV/AIDS and community conflict in Nigeria: implications and challenges.” Social science and medicine. 60(6):1209-18. doi:10.1016/j.socscimed.2004.06.053 Kananura RM, Kiwanuka SN, Ekirapa-Kiracho E, Waiswa P. (2017). Persisting demand and supply gap for maternal and newborn care in eastern Uganda: a mixed-method cross-sectional study. Reproductive health. 14(1):136. doi: 10.1186/s12978-017-0402-6. Kayongo E. (2017). Lessons from the integrated life-cycle based health campaign in Uganda. Kampala: FHI360. Lohman N, Hagopian A, Luboga SA, Stover B, Lim T, Makumbi F, et al. (2016). District health officer perceptions of PEPFAR's influence on the health system in Uganda, 2005-2011. International journal of health policy management. 6(2):83-95. doi: 10.15171/ijhpm.2016.9 Madinah, N. (2016). Challenges and barriers to the health service delivery system in Uganda. IOSR journal of nursing and health science. 5(2): 30-38. doi:10.9790/1959-0502053038 McKee N, Manoncourt E, Chin SY, and Carnegie R (eds.). (2000). Involving people, evolving behavior. New York: UNICEF. Mukama R. (2009). Theory and practice in language policy: The case of Uganda. Kiswahili. 72(1). Mukasa N. (2012). Uganda healthcare system profile: Background, organization, policies and challenges. Journal of sustainable regional health systems. 1(1):2-10 Namyalo S, Nakayiza J. (2015). Dilemmas in implementing language rights in multilingual Uganda. Current issues in language planning. 16(4): 409-424. doi: 10.1080/14664208.2014.987425 Schwartzkopf S. (2016) “In search of the consumer: The history of market research, 1890-1960. In The Routledge companion to marketing history. Jones DGB and Tadajewski M (eds).Oxon: Routledge. pp. 61-84. 80 Storey JD, Saffitz GB, Rimón JG. (2008). Social marketing. In Health behaviour and health education: Theory, research, and practice. 4th Edition. Glanz K, Rimer B, and Viswanath K. (eds). San Francisco: Jossey-Bass. pp. 435–464. Ries A & Trout J. (2000)Positioning: The battle for your mind. 20th Edition. New York: McGraw-Hill. USAID/Uganda. (2012a). Uganda joint behavior change communication survey. (October 2012). Kampala: USAID. USAID/Uganda. (2012b). Request for application (RFA) number RFA-617-13-000001: USAID/Uganda Communication for Healthy Communities (CHC). November 28, 2012. Kampala, Uganda. USAID/Uganda. (2013). Final evaluation of the Health Communications Partnership (HCP II) project. April 2013. Kampala: USAID Uganda. USAID. (2017). President’s Malaria Initiative Uganda: Malaria operational plan FY 2018. PMI: Kampala. https://reliefweb.int/sites/reliefweb.int/files/resources/fy-2018-uganda-malaria-operational-plan.pdf World Health Organization. (2016). Use of high burden country lists for TB by WHO in the post-2015 era: Summary. Retrieved from: http://www.who.int/tb/publications/global_report/high_tb_burdencountrylists2016- 2020summary.pdf?ua=1 81 Annex 2: Team Composition Name Position Lynda Bardfield Evaluation Team Lead George Lugalambi Senior Health Communication Expert Beth Anne Pratt Public Health Specialist Lynda Bardfield ● General evaluation management: Lead the evaluation team (including finalization of work plan and schedule) and initial review of background documents ○ Participate virtually in planning meetings with CHC evaluation team and partner staff ○ Review and suggest revisions to improve draft evaluation design/plan. ○ Draft data collection tools and revise after USAID/Uganda and partner feedback ● Prepare for fieldwork, including: ○ In-briefing with USAID Uganda ○ Draft data collection tools and revise after USAID/Uganda and partner feedback ○ Develop field guide (in collaboration with CHC evaluation team) ○ Provide input in the selection of respondents ○ Support scheduling of data collection activities (in collaboration CHC evaluation team) ● Fieldwork in Uganda: ○ Ensure data collection is completed as planned ○ Ensure consistency and quality of data collection effort among field team ○ Travel to and from field sites ○ Contribute and manage analysis of qualitative data (in collaboration CHC evaluation team ○ Develop initial findings, conclusions, and recommendations – to be presented to Mission staff before departure ● Analysis, and development of draft findings ● Develop draft report (with input from CHC evaluation team) ● Develop final report, incorporating feedback from USAID/Uganda George Lugalambi ● Prepare for fieldwork, including: ○ In-briefing with USAID Uganda ○ Input to Inception report ○ Participate in meetings to design sampling frame and data collection approach ○ Participate and supervise data collection activities (in collaboration CHC evaluation team) ● Fieldwork in Uganda: ○ Participate in data collection as planned ○ Supervise field visits ○ Travel to and from field sites ○ Contribute to analysis of qualitative and quantitative data (w/CHC evaluation team) ○ Develop initial findings, conclusions, and recommendations – to be presented to Mission staff before departure ● Support Analysis, and development of draft findings 82 Beth Anne Pratt ● Prepare for fieldwork, including: ○ In-briefing with USAID Uganda ○ Develop the Inception report ○ Participate in meetings to design sampling frame and data collection approach ○ Participate in data collection activities (in collaboration CHC evaluation team) ● Fieldwork in Uganda: ○ Participate in national level and sub national fieldwork ○ Supervise select field visits ○ Travel to and from field sites ○ Contribute to qualitative data analysis 83 Annex 3: Getting to Answers Matrix Evaluation Questions Sub-Questions Type of Answer Needed [Descriptive, Comparative/ Normative, Cause-and-Effect] Data Collection Methods & Tools Data Sources Sampling or Selection Criteria Data Analysis Methods Q1: What USG and the implementer learn about the target populations’ misconceptions around intervention messaging? How did they adapt? LEARNING ABOUT THE TARGET POPULATIONS 1.1: What were the target populations? *1.2: What misconceptions and knowledge gaps were identified or observed vis-à-vis the SBCC intervention messaging? 1.3: Were the misconceptions and knowledge gaps limited to or concentrated in particular populations, messages, and channels or were they generalized? 1.4: What was the intervention messaging composed of with respect to the health issues CHC focuses on and the target populations? 1.5 Did USG, CHC and other key players (GoU/MoH, IPs, and development partners) learn anything about the misconceptions and knowledge gaps? Descriptive Key informant/ stakeholder Interviews (interview guide) Desk research (program document review) Key informants/ stakeholders Program documents Purposive (expert and maximum variation) Thematic Analysis ADAPTATION TO LEARNING ABOUT THE TARGET POPULATIONS 1.6: Did USG, CHC and other key players (GoU/MoH, IPs, and development partners) act upon and adapt to evidence of target populations’ misconceptions and knowledge gaps? If so, how? If not, why? 1.7: Where is the evidence of adaptation to what was learned? 1.8: What was the process of and experience with adaptation? Descriptive Key informant/ stakeholder Interviews (interview guide) Desk research (program document review) Key informants/ stakeholders Program documents Purposive/ sampling (expert and maximum variation) Thematic analysis Q2: What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? PROGRAM APPROACHES AND STRATEGIES 2.1: What approaches/strategies were used? 2.2: How were these approaches/strategies selected? [Was the selection criteria determined a priori or was it developed inductively? Who were involved in determining the selection criteria and how were they involved? Who were involved in the actual selection of the approaches/strategies used and how were they involved?] 2.3: Where is the evidence of the selection process used to determine the approaches/strategies used? Descriptive Key informant/ stakeholder Interviews (interview guide) Document review Secondary data analysis Program documents Purposive/ sampling (expert and maximum variation) Trend analysis Timelining Analysis of variance 84 EFFECTIVENESS OF PROGRAM APPROACHES AND STRATEGIES *2.4: Which approaches/strategies were most/least valued or viewed as most/least effective at different levels? [By primary audiences, service providers, implementation partners, USG, GoU/MoH, CHC?] 2.5: What made these approaches/strategies effective or ineffective and why? [What were the criteria used to determine effectiveness?] *2.6: Where is the evidence of their effectiveness? [Service utilization, audience feedback, adoption by campaign champions, health outcomes?] Comparative (based on KAP/behavioral change targets or expected outcomes) Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Purposive/ sampling (expert and maximum variation) Secondary data analysis BEHAVIORAL OUTCOMES 2.7: What were the expected behavioral outcomes? *2.8: Which outcomes were achieved and which were not? How do the achievements compare with the targets? 2.9: Where is the evidence of achievement? 2.10: What factors influenced the achievement of outcomes? Descriptive Focus group discussions (discussion guide) Key informant/ stakeholder interviews (interview guide) Audience members & beneficiaries Key informants Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis ADAPTATION TO PROGRAM EFFECTIVENESS AND BEHAVIORAL OUTCOMES *2.11: Did the implementers adapt to the evidence on program effectiveness and behavioral outcomes? If so, how? If not, why? 2.12: What was the process of and/or experience with adaptation? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis Q3: How did the implementer address converging messages and integration during implementation and what lessons can be learned AUDIENCE SEGMENTATION 3.1: Among the target populations, what specific audiences were SBCC interventions directed to? 3.2: What criteria were used to segment the audiences? [Demographic, geographic, psychographic, attitudinal, cultural, behavioral, message preferences, channel preferences, theory?] Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis CHANNEL SELECTION 3.3: Which channels were used to reach the audiences? 3.4: What criteria were used to select the channels? [Desired communication properties?] 3.5: What was known about the channel preferences of the intended audiences per their segmentation? [Evidence?] Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis MESSAGE DESIGN 3.6: What were the objectives of the SBCC campaign? 3.7: Which behavioral theory and/or theory of change, if any, informed the SBCC campaign? [Evidence?] 3.8: What was the process of message development? 3.9: What were the messages of the SBCC campaign? [What were audience members being asked to think/do?] Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis 85 3.10: What message design theory or principles, if any, was/were followed in the development of messages and to ensure that the messages were effective and to the expected standard/quality? [Convergence, look and feel, credibility, etc.?] 3.11: Did target audiences experience the messages as anticipated? If so, how? If not, why? 3.12: What benchmarks were used to determine message effectiveness/quality? INTEGRATION BENEFITS AND CHALLENGES 3.13: Was there integration in messaging? Where is the evidence? *3.14: What benefits were anticipated from integration in health messaging? 3.15: What were the challenges faced by implementing partners in regard to integration? How did they address these challenges? *3.16: What lessons can be learned from this experience and the way that CHC addressed these issues? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis Q4 How have CHC￾relevant coordination structures (intra-agency coordination, national structures, partner platforms) contributed to improved capacity of Government of Uganda and other partners to design, coordinate and implement integrated SBCC? COORDINATION MECHANISMS 4.1: What coordination structures were created and/or strengthened by CHC? 4.2: How were these structures meant to improve capacity by the GoU/MoH and implementing partners to design, coordinate and implement integrated SBCC? *4.3: Have these coordination mechanisms improved the capacity of GoU/MoH and implementing partners to design, coordinate, and implement integrated SBCC? 4.4: How, if at all, has increased capacity led to improved coordination of health communication interventions? Where is the evidence of these improvements? *4.5: What has CHC, GoU/MoH and implementing partners learned about coordination? Has there been any adaptation to this learning? Where is the evidence? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis Q5: How did information generated through research and knowledge management boost and improve health communication in the country? RESEARCH 5.1: What research has CHC undertaken and where is the evidence? *5.2: What knowledge did CHC generate through its research? *5.3: How and where is the research documented? How was it disseminated? Descriptive Document review Program documents N/A Secondary data analysis STRATEGIC ACTION 5.4: How did CHC translate the evidence and knowledge generated from research into strategy? 5.5: Was there a deliberate link between research and strategy design/strategic planning? Where is the evidence? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis PROJECT IMPROVEMENTS *5.6: How did research and evaluation lead to project improvement? *5.7: Who (actors) and what (factors) were the key drivers of change? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis 86 RESEARCH DISSEMINATION AND KNOWLEDGE MANAGEMENT *5.8: How did CHC disseminate research findings to partners? *5.9: Was this part of an overarching knowledge management (KM) strategy? 5.10: What feedback has CHC received about the research output/findings it has shared with partners? How and where is it documented? 5.11: Which KM activities triggered action? What actors/factors were critical in bringing about or hindering change/improvement? How? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis Q6: Which are the main barriers that exist as to the delivery of the six services promoted by CHC? What changes should be made to SBCC programming in the future to improve linkages between health communication and integrated health service delivery? *6.1: What was the role of communication in facilitating the link between demand and supply? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis 6.2: What barriers did CHC encounter? How, if at all, did CHC adapt and/or overcome these barriers? *6.3: What should SBCC programming do in the future to strengthen the link between demand generation and service delivery? Descriptive Key informant/ stakeholder interviews (interview guide) Document review Key informants Program documents Convenience sampling Purposive sampling (expert and maximum variation) Thematic analysis 87 Annex 4: Methodology Evaluation Design The evaluation was a mixed-method, non-experimental learning evaluation predominantly relying on qualitative data supported with quantitative evidence of effectiveness to understand what worked with respect to CHC, how it worked, and why. As CHC was implemented predominantly with and through other health implementing partners (IPs), CHC’s effectiveness has depended, to a large extent, on other IPs’ successful implementation of their own programs and activities. As such, the question of attribution - i.e. what behavioral or health successes or failures can be specifically credited to CHC alone – was not the focus of this evaluation. Rather, the focus was on explaining and exploring the reasons for such measurable achievements (or lack thereof), and translating these findings into a set of learnings for USG SBCC planning and program development looking forward. Data Sources This evaluation drew upon an array of primary and secondary data sources. In particular: 1. Interviews with key informants representing critical areas of implementation 2. Focus group discussions (FGDs) with target message beneficiaries (Life Stage groups) 3. CHC and partner program documentation (quarterly reports, annual reports, work plans, research, communication materials) 4. Secondary datasets including DHIS-2, Lot Quality Assurance Surveys (LQAS), and Phase I data and Phase II data from CHC’s ongoing evaluation/studies and other research activities Sample Selection Criteria The evaluation was limited in scope for several reasons: 1) the population of technical experts with respect to CHC is relatively small and bounded; 2) maximum variation in opinion and insight was deemed most important for learning; 3) the rapid nature of the evaluation meant that the time allocated for fieldwork was quite short (3 weeks). Respondents were chosen from among national level stakeholders, district level stakeholders, facility level stakeholders (including health facility workers, VHTs, peer educators and champions), and community level stakeholders who, as part of Life Stage groups - were recipients of SBCC messaging. Selection of Respondents The evaluation relied predominantly on a) key informant interviews at national level and b) key informant (KI) interviews and focus group discussions (FGDs) at sub-national level. This section describes the sampling methodology for sub-national fieldwork. Sample selection for sub-national interviews and FGDs involved five (5) levels: districts, sub-counties, health facilities, supply-side technical experts, demand-side message beneficiaries. These levels are represented schematically in Figure 1 below. 88 Purposive Sub-Counties no. 6 Semi-Random Purposive Health Facilities no. 6 Supply-Side VHTs, HWs, Champions Demand-Side Purposive LS 1 LS 2 LS 3 LS 4 Purposive Level 1 Level 2 Level 3 Level 4 Level 5 2 groups (1 male & 1 female) x 4 LS x 6 districts = 48 FGDs x 6-8 people = no. 288 - 384 people 1-2 from each category (3-6) x 6 districts = no. 18-36 Figure 1: Evaluation Sampling Methodology Level 1: District Selection Level 1 involved selection of the districts for sub-national fieldwork. CHC works throughout Uganda, but has focused research on 16 districts which were selected based on rural versus urban, performance with respect to certain health and behaviour indicators, and density of USG IPs. These 16 focal districts are: Urban Lira Ntungam o Mbale Iganga Arua Moroto Mukono Kasese Rural Amuru Ibanda Kumi Buyende Nebbi Kaabong Sembabule Kyenjoj o Source: FHI360, 2016 Additionally, in four of these districts - Lira, Iganga, Mukono, and Kyenjojo - CHC is carrying out a qualitative outcome evaluation study. The purpose of this evaluation was to extract as many learnings as possible, by looking at implementation of CHC across as many different contexts as possible, both to identify common lessons learned and to understand the ways in which context makes a difference to program design, implementation, coordination, and adaptation. Additionally, the evaluation sought to avoid duplication of existing CHC research while simultaneously contributing to and amplifying CHC’s 89 already extensive monitoring, evaluation and learning efforts. Thus, the evaluation used the existing CHC focal districts as a frame to purposively select 6 districts based on the following criteria: 1. Achieves an equal balance of “rural” versus “urban/peri-urban” districts 2. Achieves maximum variation of USG agencies/implementing partners (USAID, CDC, DoD) 3. Achieves maximum variation of regional representation 4. Achieves maximum variation when looking at the differences between implementation in the context of stability of IP/service delivery partners versus high turnover of partners 5. Achieves logistical feasibility in the context of the duration of time and resources assigned for field work (i.e. 3 weeks) Based on this, the evaluation focused on the following 6 districts: Region Districts Criteria North Lira 1) Urban 2) USG Agency: USAID 3) Stability of IP: Rapid turnover of implementing service delivery partners (NU￾HITES, SDS, ASSIST, SUSTAIN, TBD RHITES-Lango) 4) Other: 1 out of 4 CHC’s Qualitative Outcome Evaluation Study districts Amuru 1) Rural 2) USG Agency: USAID 3) Stability of IP: Rapid turnover of implementing service delivery partners implementing service delivery partners (NU-HITES, SDS, ASSIST, SUSTAIN, TBD RHITES-Acholi) East￾Central Iganga 1) Urban 2) USG Agency: USAID 3) Stability of IP: Semi-stable transition from STAR-EC to RHITES-EC, with turnover of management from JSI to IntraHealth 4) Other: 2 out of 4 CHC’s Qualitative Outcome Evaluation Study districts Central /North Central Mukono 1) Urban 2) USG Agency: DoD 3) Stability of IP: Long-term stable management by MUWRP (Walter Reed) 4) Other: 3 out of 4 CHC’s Qualitative Outcome Evaluation Study districts Southwest Ibanda 1) Rural 2) USG Agency: USAID 3) Stability of IP: Stable transition from STAR-SW to RHITES-SW (one of the longest running integrated service delivery partners, consistently managed by EGPAF) West Kyenjojo 1) Rural 2) USG Agency: CDC 3) Stability of IP: Long-term stable management by Baylor Uganda 4) Other: 4 out of 4 CHC’s Qualitative Outcome Evaluation Study districts Colors represent logistics: each pair of districts can be covered by a single team with the capacity to speak related language groups (i.e. Runyankore/Rutooro, Acholi/Langi, Luganda/Lusoga), on a single looped trip. Thus, the maximum variation represented is: Urban/Rural Geographic Spread USG Agency Stability of RHIP Relationship to other CHC studies 90 3 Urban 3 Rural 2 West/Southwest 2 North 1 East Central 1 Central 1/North Central 4 USAID 1 CDC 1 DoD 2 highly stable 2 semi-stable 2 highly unstable All focus districts represented in CHC impact evaluation 4 part of QOES 2 not part of QOES Within this, several parts of the country were left out of the evaluation: ● Karamoja because of the patchwork of implementing partners and donors, its special status in the country as an outlier, as well as time and financial resource constraints ● West Nile because of time and financial resource constraints ● Central 2/South Central because the patchwork of regional health implementing IPs there made it difficult to look for patterns. Note: health implementing partners who work in these regions were included in national level interviews (i.e. Mildmay, TASO) Finally, Kumi and Mbale - in Eastern Uganda and part of the CHC 16 focus districts - were initially considered for inclusion. Kumi is rural and Mbale is urban. They sit on a logistical loop (Pallisa-Kumi￾Mbale road). They are both represented by USAID/RHITES-E, which transitioned from STAR-E and, with it, in management from MSH to IntraHealth. However, a decision was ultimately made to leave these districts out for the following reasons: ● Kumi and Mbale represent different language groups (Iteso and Lugisu/Lumasaaba). During the research planning stages when we were unclear about the budget available for fieldwork, the team believed that this potentially necessitated completely different FGD facilitation teams to be sent instead of conducting a logistical loop based on teams being able to speak two related languages (i.e. there are likely FGD facilitators who can speak both Runyankore/Rutooro (Southwest/West loop), Acholi/Langi (Northern loop) and Luganda/Lusoga (Central/East Central loop). The team also felt it might be more difficult to find fluent dual Iteso/Lugisu speakers, thus the number of research teams would actually have had to increase from three to five). Ultimately, six single language teams went out anyway. Yet the cost implications of another two teams remain valid. ● Limited time for up-country fieldwork and limited personnel (the evaluation team is made up of only 3 people) meant the evaluation team could realistically supervise no more than two up-country research teams a week. ● Beyond adding Eastern Uganda to the sample, it was difficult to see how these districts would contribute to maximum variation. Level 2: Sub-County Selection Following selection of the 6 districts, all sub-counties within a single district were put in a hat, and one sub-county per district was chosen. The number of sub-counties was thus equal to the number of districts: 6. The sample selection was not entirely random, as once a sub-county was selected the following criteria were applied: a) CHC/Obulamu had to be clearly implemented in these sub￾counties; b) there had to be at least one health facility present within the sub-county that had been documented as having received training/materials/messages; c) the sub-county had to be accessible by road; d) all US agencies had to be represented based on the principle of maximum variation. Thus, the selection was reviewed by the evaluation team and discussed with both CHC Monitoring and Evaluation (M&E) staff centrally and at the CHC regional office. If there was limited CHC presence in a sub-county, lack of a health facility, lack of road network, or lack of maximum variation, the name of the sub-county was to be discarded, and another drawn until another sub￾county that met the criteria was chosen. Ultimately, only one randomly selected sub-county was excluded from the sample - Kichuzi sub-county in Ibanda – due to inaccessibility given the time frame available. The sub-counties chosen were as follows: Mukono Iganga Ibanda Kyenjojo Acholi Lira 91 Kyampisi S/C Bulamagi S/C Nyabuhikye S/C Kyarusozi S/C Atiak S/C Amach S/C Level 3: Health Facility Selection Following selection of the sub-counties, all health facilities in a single sub-county were listed. Then, per the description of Level 2 selection, the evaluation again confirmed with CHC M&E staff centrally and at their regional offices that a) the staff at the facility had received training and/or materials to participate in CHC messaging and/or Obulamu; b) the facility was a recipient of RHIP assistance; c) using the principle of maximum variation, the RHIP providing the support was not a duplicate of one already interviewed. Ultimately, two randomly selected facilities were excluded from the sample, one in Kyenjojo because it was a private not-for-profit facility that received no support from an RHIP and one in Ibanda because it was a unique CDC-supported site in a region that was predominantly supported by USAID/RHITES-SW. Additionally, it should be noted that Gulu and Lira districts have been previously lumped together by USAID under broad health assistance to the north, although will be split into RHITES-Acholi and RHITES-Langi moving forward. The number of health facilities is thus equal to the number of districts (i.e. 6) and the facilities chosen were as follows: Mukono Iganga Ibanda Kyenjojo Acholi Lira Kyampisi S/C Bulamagi S/C Nyabuhikye S/C Kyarusozi S/C Atiak S/C Amach S/C Kyampisi HC III Bulamagi HC III Nyabuhikye Prison HC II Kyembogo Holy Cross HC III Atiak HC IV Amach HC IV Level 4: Health Workers, VHTs, and Champions At the facility: a) two to three (2-3) health facility workers; b) two to four (3-4) VHTs; and 3) two to three (2-3) champions or peer educators in the catchment area of the facility were convened to participate in interviews that focused specifically on the technical implementation of messages. That is, the focus was on these individuals as technical, supply-side messaging experts able to comment with authority on implementation (as opposed to measuring effectiveness or quality of performance). The number of supply-side participants per district was as follows: Supply-Side Messaging Experts Category Mukono Iganga Ibanda Kyenjojo Amuru Lira Total Health Workers 3 2 2 4 3 2 16 VHTs 4 4 2 4 3 3 20 Champions 3 0 0 0 0 2 5 Peer Educators 0 3 3 2 2 0 10 Total 10 9 7 10 8 7 51 It is important to note, however, that some of these categories were not mutually exclusive. In some instances, notably in Kyenjojo and Amuru, there was overlap between VHTs, peer educators, and champions. Level 5: Life Stage Groups Obulamu targeted four Life Stage (LS) groups with integrated messaging: couples, pregnant women, parents of children under the age of five, and adolescents. CHC has already identified existing 92 populations for different research and learning activities targeting LS groups, including formative research, evaluative research, special studies, and routine monitoring. Moreover, within IR3 of CHC, formative research - in particular design research and action media - and routine monitoring activities exist for the purpose of contributing to project design and implementation. This is important, as FGDs can be perceived as human subjects research if such discussions target information about participants’ health, health-related decisions, and behaviour (particularly if such research targets minors or vulnerable populations). Because this evaluation exists primarily as a learning exercise to feed into the design of the CHC follow on, evaluation FGDs fell within CHC’s existing formative and routine monitoring work. For this reason, IRB was not sought, although basic human subjects protections such as informed consent and anonymization of transcripts was followed, and the evaluation ensured that most FGD districts (with the exception of Ibanda and Amuru) overlapped with CHC evaluation districts - in particular the qualitative outcome evaluation study - and that questions fell within relevant domains already covered by existing CHC IRB approval. The evaluation relied on existing CHC networks - in particular, CHC regional staff and their sub￾county and village-level partners - to identify participants for FGDs within target districts. As this evaluation is - again - about extracting learnings and achieving maximum variation, teams sought to carry out gender-specific focus groups for each of the four LS group in each of the 6 sub￾counties/districts, leading to target of 8 FGDs per sub-county/district. This was achieved in all districts except Ibanda, in which only 7 FGDs were carried out due to issues with mobilization. The evaluation sought FGDs of 8-12 participants each; however, a few FGDs fell short or exceeded this target. Ultimately, the evaluation team carried out a total of 47 FGDs covering 527 participants, the distribution of which is available in the table below. Focus Group Discussions District LIFE STAGE 1 Young Couples LIFE STAGE 2 Pregnant Women & Partners LIFE STAGE 3 Parents with Children under-5 LIFE STAGE 4 Adolescents Total Male Female Male Female Male Female Male Female Mukono # FGDs 1 1 1 1 1 1 1 1 9 # Participant s n = 12 n = 12 n = 12 n = 12 n = 11 n = 12 n = 10 n = 12 93 Iganga # FGDs 1 1 1 1 1 1 1 1 8 # Participant s n = 8 n = 8 n = 5 n = 8 n = 9 n = 8 n = 8 n = 8 62 Lira # FGDs 1 1 1 1 1 0 1 2 8 # Participant s n = 12 n = 12 n = 12 n = 12 n = 0 n = 12 n = 12 n = 24 96 Amuru # FGDs 1 1 1 1 1 1 1 1 8 # n = 12 n = 12 n = 12 n = 12 n =12 n = 12 n = 12 n = 12 96 93 Participant s Kyenjojo # FGDs 1 1 1 1 1 1 1 1 8 # Participant s n = 12 n = 12 n = 11 n = 12 n = 12 n = 12 n = 12 n =9 92 Ibanda # FGDs 1 1 1 1 1 0 1 1 7 # Participant s n = 12 n = 13 n = 12 n = 12 n = 14 n = 0 n = 13 n = 12 88 TOTAL FGDs 12 12 11 12 47 TOTAL Participants 137 132 114 144 527 Selection of Key Informants The evaluation used purposive/non-probability sampling, following the principles of expert sampling and maximum variation sampling. The criteria for selection of key informants were: ● The potential informant had technical expertise in implementing the CHC activity ● The potential informant was clearly cognizant of having been a stakeholder in CHC activities ● The potential informant was willing to provide informed oral consent following a short briefing of the nature and purpose of the assessment The reasons for these criteria are explained in the section on Ethical Considerations below. There were several broad categories of key informant: ● Those who were involved directly in CHC implementation (CHC itself and sub-prime UHMG) ● Those USG agencies involved in the financing and oversight of CHC (USAID, CDC, DoD) ● Those who were CHC service delivery partners (RHIPs, NGOs, CSOs) ● Those who were CHC creative partners (advertising agencies, radio) ● Those who were CHC/Obulamu stakeholders at central government level (MoH, UPDF) ● Those who were CHC/Obulamu stakeholders at local government level (district government) ● Supply-side SBCC stakeholders (health facility workers, VHTs, champions, peer educators) In all instances, the categories were limited in scope and selection was opportunistic, depending on which stakeholders were available on the day of the interview. Although a number of stakeholders originally contacted failed to turn up for interview, the evaluation ultimately spoke to 103 key informants across 10 categories. Some organizations preferred interviews with technical staff to be conducted in a group setting (in some instances, adding additional people to an interview without the evaluation team knowing in advance, thus introducing snowball sampling). While, technically, interviews involving more than one person would be better conducted as FGDs, uncertainties with respect to organizational preference, as well as the fact that - regardless of whether interviews are held with one person or a group of people - the ultimate focus of the interview was on technical expertise, opinion, and perspective, the evaluation classified all such sessions as “key informant”. Key Informant Interviews District Prime Sub CG USG DG HIP CP SSH SSVHT SSCMP Total 94 F * * Mukono 0 0 0 0 1 5 1 3 4 3 17 Iganga (Jinja) 0 0 0 0 1 2 1 2 4 3 13 Ibanda (Mbarara) 0 0 0 0 1 2 1 2 2 3 11 Amuru (Gulu) 1 0 0 0 1 1 1 3 3 2 11 Kyenjojo (Kabarole) 1 0 0 0 1 2 0 4 4 2 14 Lira 0 0 0 0 1 1 2 2 3 2 11 Kampala 8 1 4 2 0 5 5 0 0 0 25 Total 10 1 4 2 6 18 11 16 20 15 103 *There were instances where there was overlap between a) central government and champions (2 individuals could have been classified in both categories); and b) VHTs and champions/peer educators (4 individuals could have been classified in both categories). Key Prime CHC HIP Health Implementing Partner Sub UHMG CP Creative Partner CG Central Government SSHF Health Facility Worker USG US Government Agencies SSVHT VHT DG District Government SSCMP Champion/Peer Educator Document Review Documents under consideration were selected purposively based on CHC providing full access to their Knowledge Management files. The KM files are a comprehensive library of CHC’s annual reports, quarterly reports, work plans, protocols, documentation, and research (across the various categories). Additional documents included key government documents such as the National Health Sector Development Plan, the National HIV/AIDS Strategic Plan, the National Development Plan II, and assorted other plans, strategies, policy documents, manuals and guidelines. Data Collection Key Informant Interviews Category-specific, semi-structured, interview guides were developed to prompt stakeholder discussion as to relevant key issues: design, integration, coordination, implementation, evaluation, dissemination, adaptation. These guides were not meant as closed-ended surveys but rather as a series of prompts to ensure that key informants touched upon all themes relevant to that particular category of stakeholder. The goals of the guides were to a) ensure consistency across interviews should team members end up responsible for different interviews and b) allow key informants to elaborate on the issues they felt were most pertinent to learning. It was assumed that key informants might have very different opinions and perspectives and so the evaluation not only sought out patterns of consensus, but also was interested in divergent points of view. The interview guides aimed to ensure that interviews lasted no longer than 60 minutes, although if an organization preferred to convene groups of key informants rather than carry out one-on-one discussions, interviews lasted a bit longer. In the case of some key informants - in particular, those working with CHC - repeat interviews were necessary for clarification and gap filling. The focus of the interviews was on technical expertise with respect to perceptions of effective strategies, behavioural/social/technical/systemic bottlenecks to implementation encountered, perspectives on coordination, change and adaptation and other issues related to the six evaluation questions. Key informant interviews were conducted by 1-2 interviewers. Interviews were conducted in English except in the case of discussions with facility workers, VHTs, peer educators and champions which were conducted in local languages. All interview data has been transcribed from recording and translated into English by a qualified translation/transcription service. 95 Focus Group Discussions Focus group discussions were conducted by externally-contracted, qualified focus group discussion facilitators in the relevant local languages - Runyankore, Rutooro, Lusoga, Luganda, Acholi and Langi. There were gender-distinct focus groups of 5-14 individuals - split equally between male and female - covering the four Obulamu Life Stages: adolescence, couples, pregnant women and parents of children under the age of 5. FGDs focused on message clarity/comprehension, memorability and relevance, as well as perceptions as to the channels of communication and strength of implementation. In order to avoid blurring boundaries with human subjects research, questions about participant health, behaviour, relationships, and other invasive topics were avoided. Tools for open-ended FGD discussion were created and these, along with samples of Obulamu communication materials to prompt discussion, guided the FGDs. Discussions were recorded, then transcribed and translated into English by qualified transcribers and translators. Transcriptions were randomly selected for a third party quality-check against the recordings to ensure integrity of data capture. Focus groups of women were led by a female facilitator, backed up by an assistant who managed the recorder and note-taking. Focus groups of men were likewise led by a male facilitator, backed up by an assistant. All fieldwork was supervised by a member of the evaluation team, with facilitators and assistants debriefed at the end of the day, and preliminary focus groups themes identified, discussed and shared with teams working elsewhere in the country. The team carried out an initial round of fieldwork in Mukono and Iganga, in which both field supervisors participated. This enabled field supervisors to test and troubleshoot tools, harmonize expectations and ways of working, and assess and troubleshoot relationships with the externally contracted company. This also ensured that - when overlapping up-country field visits (North and Southwest/West) occurred two weeks later in different parts of the country - that the field supervisors proceeded with a unified approach. Secondary Data Data was obtained from CHC performance records, DHIS-2, Lot Quality Assurance Surveys (LQAS), and Phase I data and Phase II data from CHC on-going impact evaluation and other research activities. Data Analysis Qualitative Data Transcripts from interviews and FGDs, as well as existing CHC documentation, was analyzed thematically, using codes assigned a priori based on the evaluation questions and, via grounded theory, through emerging themes developed collectively over the duration of the fieldwork. Secondary Data Quantitative data was analyzed using both trends and timelines to show program performance over time, and to detect the effect of context and program events on program performance indicators. Analysis of variance was used to determine difference in performance indicators across the following variables: a) rural/urban; b) USG implementing agencies; c) implementation regions; d) implementation context; e) emerging issues. This quantitative aspect of the evaluation looked to compare the results of the OBULAMU campaign with the broader behavioral outcomes observed through the Lot Quality Assessment Survey (LQAS) 96 data. The LQAS analyzed the national trends in behavioral changes by region from 2012 to 2016 with regard to the following health areas: (a) HIV (i.e. percentage of youth who had sexual intercourse before the age of 15). (b) Malaria (i.e. percentage of mothers of children 0-23 months who received two or more doses of IPT2 during their last pregnancy; percentage of children 0-59 months who slept under an ITN the night preceding the survey; percentage of mothers of children 0-59 months who always slept under an ITN during the last pregnancy). (c) Reproductive health (i.e. percentage of sexually active women age 15-49 years who are using any modern method of family planning; percentage of mothers of children 0-11 months who attended ANC at least four times during the last pregnancy; percentage of mothers of children 0-11 months who delivered their last baby in a health facility). (d) Vaccination (i.e. percentage of children 12-23 months who are fully vaccinated). (e) Breastfeeding (i.e. percentage of children under 6 months of age who are exclusively breastfed). (f) Hygiene (i.e. percentage of individuals who wash their hands with soap after visiting the toilet). Data for the CHC evaluation100 was generated from two repeated evaluative studies that were conducted in March 2015 (the Phase I data) and in September 2017 (the Phase II data) on the following dimensions: ● The five communication effects/behavioral determinants under the social ecological model i.e. Exposure, Comprehensive Knowledge, Approval, Intention to Act, Action; ● The four OBULAMU life stages (LS) i.e. LS1 Young adults in relationships (20-30-year-olds), LS2 Pregnant couples (women 15-49-year-olds), LS3 Mothers of children under one year old, LS4 Adolescents (15-19-year-olds); and ● The six health areas targeted by the OBULAMU campaign i.e. HIV, Family Planning, Nutrition, Maternal Child Health, TB, and Malaria). Comparing the patterns observed in the LQAS with the results of the OBULAMU Phase I data and Phase II data evaluations is the basis for establishing the relationship, if any, between CHC’s SBCC interventions and some of the overall regional level changes measured through the LQAS. But whereas some variables in the CHC evaluation were applicable to some life stage groups and not to others, two notable differences between the LQAS and the CHC study need to be noted. First, whereas in the LQAS the country was clustered into three regions (Eastern, South-Western, and Northern), the OBULAMU survey had eight regions (Central, East Central, Eastern, Karamoja, Northern, South Western, West Nile, and Western). Nonetheless, both surveys were based on nationally representative samples, and the aggregate results are indicative of the national picture irrespective of the approach to regionalization. Second, the scope of the CHC evaluation was far more comprehensive than the LQAS in terms of the behavioral domains investigated. TB which is one of the health areas targeted by CHC was not included in the LQAS. Under HIV, while the LQAS measured only one variable i.e. “youth who had sexual intercourse before the age of 15,” CHC measured six variables i.e. “HIV testing,” “condom use,” “ART,” “EMTCT,” “SMC,” and “partner reduction.” Under malaria, the LQAS measured three variables i.e. “mothers of children 0-23 months who received two or more doses of IPT2 during their last pregnancy,” “children 0-59 months who slept 100 The eight regions covered and respective districts sampled were: Central (Mukono, Sembabule); East Central (Buyende, Iganga); Eastern (Kumi, Mbale); Karamoja (Moroto, Kaabong); Northern (Lira, Amuru); South Western (Ntungamo, Ibanda); West Nile (Arua, Nebbi); Western (Kyenjojo, Kasese). 97 under an ITN the night preceding the survey,” and “mothers of children 0-59 months who always slept under an ITN during the last pregnancy,” while CHC measured four variables i.e. “LLIN use for children under 5,” “malaria prevention and treatment,” “pregnant women sleeping under LLIN,” and “IPTp.” Under reproductive health/family planning, the LQAS measured three variables i.e. “sexually active women age 15-49 years who are using any modern method of family planning,” “mothers of children 0-11 months who attended ANC at least four times during the last pregnancy,” and “mothers of children 0-11 months who delivered their last baby in a health facility,” while CHC measured four variables i.e. “family planning methods,” “delay of sexual debut,” “child spacing,” and “teenage pregnancy.” Under maternal and child health, the LQAS measured only two variables i.e. “children 12-23 months who are fully vaccinated” and “individuals who wash their hands with soap after visiting the toilet,” while CHC measured five variables i.e. “completion of 4 ANC visits,” “delivery in a health facility,” “fever treatment,” “immunization of under-5-year-olds,” and “hand washing.” Under nutrition, the LQAS measured only one variable i.e. “children under 6 months of age who are exclusively breastfed,” while CHC measured i.e. “feeding young children” and “breastfeeding.” For purposes of this analysis, the comparisons of interest relate to the ultimate behavioral outcomes – individuals taking action – of the audiences or populations that the OBULAMU campaign targeted across the various life stages. This applies at the national and regional levels and within individual life stage groups. The analysis therefore focuses on actions – the ultimate communication effect – as informed by the social ecological model from which the OBULAMU campaign’s theory of change is derived. Trend Analysis was carried out to explore the pattern of key behavioral indicators over time using LQAS data from Eastern, South-Western, and Northern Uganda. The thematic areas covered by the LQAS included HIV, malaria, reproductive health, child vaccination, breastfeeding, and hand-washing after use of the toilet. This analysis allowed for comparison of trends across the Eastern, South Western, and Northern regions. Analysis of Variance (ANOVA) technique was used to assess potential differences in the indicator (dependent variables) by the region (a nominal-level variable) with three categories i.e. Eastern, South-Western and Northern. Thus, the ANOVA enabled us to examine potential differences in indicator levels by region. ANOVA uses F-tests to statistically test the equality of means (indicator) in the regions and the t-test to test for the differences in means (indicator level). 98 Annex 5: Consultant Scope of Work USAID/Uganda Monitoring, Evaluating and Learning Contract: AID-617-C-13-00007 Evaluating USAID/Uganda’s Communication for Healthy Communities (CHC) Period of Performance: September 18 to November 10, 2017 A. Introduction Communication for Healthy Communities (CHC) is a 5-year (June 2013- 2018), USAID funded project whose goal is to support Government of Uganda and partners to design and implement quality health communication interventions that contribute to reduction in HIV Infections, total fertility, maternal & child mortality, malnutrition, malaria and tuberculosis. This activity builds upon efforts by the Government of Uganda, previous communication activities which focused on developing tools and strategies for social and behavior change, motivating individuals, households and communities to adopt better health practices. The Strategic Objective of this program is to increase adoption of healthy behaviors (including uptake of critical health services) through strengthened health communication, in order to contribute to improved health outcomes. To achieve this objective, the program aims to achieve the following three Intermediate Results (IRs): IR1: High quality health communication interventions designed and implemented; IR2: Improved coordination of health communication interventions; and IR3: Increased research and knowledge management to enhance health communication. Details on indicators and targets of the abovementioned results are outlined in the project proposal, work plans and Activity Monitoring Evaluation and Learning Plan. B. Purpose of the Evaluation The purpose of the evaluation is to establish the extent to which CHC achieved its project objectives. The evaluation will assess the performance and effectiveness of interventions in order to provide lessons learned and guide future national health communication investments. It will evaluate the technical assistance, and identify best practices and promising new approaches, including innovative means to reach people. C. Audience The primary user of the evaluation findings is USAID/Uganda Mission staff, other United States Government agencies; USAID funded implementing partners (IPs), Ministry of Health and other national and international stakeholders with interest in health communication programing. D. Evaluation Questions The specific questions to be addressed by this evaluation are: Q1. What did USG and the implementer learn about the target populations’ misconceptions around intervention messaging and how did they adapt? Q2. What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? Q3. How did the implementer address converging messages and integration during implementation and what lessons can be learned? Q4. How have CHC-relevant coordination structures (intra-agency coordination, national structures, partner platforms) contributed to improved capacity of Government of Uganda and other partners to design, coordinate and implement integrated SBCC? Q5. How did information generated through research and knowledge management boost and improve health communication in the country? Q6. Which are the main barriers that exist as to the delivery of the six services promoted by CHC? What changes should be made to SBCC programming in the future to improve linkages between health communication and integrated health service delivery partner? E. Methodology 99 The evaluation will apply cross-sectional design using a mixed method approach i.e. using both quantitative and qualitative methods. The evaluation team will be able to propose an appropriate methodological approach to be signed off on by USAID Uganda. The evaluation team is expected to propose and use sound sampling techniques to determine districts, health facilities and their respective staff to be visited from which data will be collected. Suggested data collection methods include: ● Review of Program Documents and Related Literature: The evaluation team will be expected to review several program related documents that have been produced by CHC project. These documents may include solicitation documents, annual work plans, Activity Monitoring, Evaluation and Learning Plans (AMELPs), progress reports (e.g. quarterly, semi-annual and annual reports), district and health sector plans as well as strategy policy and performance documents. In addition to these documents, the evaluation team may access service delivery data through DHIS-2. The evaluation team will decide on the type of analysis that can be conducted using these secondary data sources in order to answer the evaluation questions. ● Key Informant Interviews (KII): Using a structured questionnaire comprising primarily of open ended questions, the evaluation team will conduct in-depth interviews with respondents from USAID, MOH, local government, non-government stakeholders, health facility managers and service providers in CHC project’s implementation area. ● Focus Group Discussions: Focus group discussions (FGD) will be another method that could be used to collect data from CHC’s stakeholders. These may include staff from local governments, partner organizations, CSOs and the target population. ● Survey of Key Stakeholders: It is also suggested that the evaluation team consider conducting a survey of a sample of clients/beneficiaries in selected districts that were served by the CHC project. This survey CHC’s approach was effective in delivering the health communication. This data collection may be complemented with data from available sources such as LQAS, HMIS, Demographic Health Survey, AIDS Indicator Survey and other and other relevant population based surveys. The evaluation team will develop tools and detailed guidance for data collection and work closely with CHC, MoH and district staff as well as USAID/Uganda to identify appropriate respondents. All data collection instruments and guides will be approved by USAID/Uganda prior to the beginning of fieldwork. The evaluation team will propose data analysis strategies and tools for both the qualitative and quantitative data. The team will be expected to conduct any other comparisons that could appraise the achievement of positive effects of the CHC project. Data disaggregation and analysis by sex and age will be used to establish the differential effects of the project on men, women and different age groups. The team will propose other analysis approaches. The evaluation team shall describe the type of software they propose to use for quantitative and qualitative data analysis. F. Deliverables ● Inception report showing the evaluation design, a detailed evaluation plan with timelines and data collection tools. A filled evaluation design matrix following the template included in section L “Evaluation Design” should be included in the Inception Report. The report should also provide an overview of the methodology that will be used to select areas to be visited and respondents/participants. ● Oral Presentation: Power Point presentation (including hand-outs) to: i. USAID alone ii. USAID and CHC project iii. USAID, CHC project and other selected stakeholders. The main presentation will normally take 45 minutes covering the major findings, conclusions and lesson learned, allowing for approximately 45 minutes of discussion and feedback. ● First Draft Evaluation Report: The first draft should be at most 20-25 pages excluding annexes. The content should cover all the main elements of the report including major findings, conclusions, lessons learned, and relevant annexes. The input from the oral presentation sessions should also be incorporated in the report. ● Final Draft Evaluation Report: A complete report presented in the agreed-upon format and incorporating comments from USAID and other stakeholders; including annexes. ● Cleaned labeled and ready to use electronic copies of datasets collected through fieldwork and cleaned ready to use electronic copies of FGD analyses if any. In addition, copies of all instruments used in data collection must be separately delivered to USAID. ● Final Report: The team leader will submit a final report within one week of receiving final comments from USAID including those from other stakeholders. The core report should be 20-25 pages excluding annexes. G. Team Composition The evaluation will be conducted by an external evaluator and a team that may include international and Ugandan team members. The evaluation team must include one team leader, two senior experts and may include research assistants/middle level evaluators as needed. The evaluation team will have complementary skills in: 1) Monitoring and evaluation of health programs particularly in evaluating SBCC programs. 2) Demonstrated expertise in evaluation techniques involving advocacy and partnership components. 100 3) Extensive experience in implementing SBCC strategies. 4) Excellent knowledge of Uganda’s health care system Health Evaluation Expert/ Team Leader shall coordinate the activities, assign tasks to team members and supervise performance. S/he will be the main point of contact between the evaluation Activity Manager at USAID/Uganda and the evaluation team. The Team Leader will review all plans and outputs and be responsible for submitting quality products to USAID on a timely basis. The team leader shall have: ● A minimum of a Master’s degree in health management, public health, applied research or related fields. PhD is preferred ● Significant expertise in conducting evaluations with a minimum of ten years’ experience in health ● evaluation management ● Played significant role designing, managing or executing a minimum of ten evaluations, at least three for health communication activities and at least two in sub Saharan Africa. ● Been team leader in a minimum of five evaluations, with at least one evaluating activities of similar scale ● Skills in quantitative and qualitative evaluation design and methods. Other team members will perform tasks in a professional manner and assist the Team Leader to develop and review deliverables. Senior Health Communication Expert shall be responsible for assessing key communication strategies at national and project level. S/he shall assess the innovative concepts piloted by the project and their effect on health outcomes. S/he shall have: ● A minimum of an advanced degree in Behavior Change, Behavioral Science, Social Sciences, ● Communications or Public Health. S/he should possess demonstrated expertise in designing, planning, and implementing BCC efforts and/or community level activities, including communication campaigns and trainings. ● 7-10 years of experience, in assessing the lessons learned from SBCC activities, demand generation, marketing and generic promotion activities such as campaign development and will provide recommendations for strengthening these interventions, as well as provide suggestions for new directions. ● Experience in working closely with government staff and capacity building ● Participated in a minimum of three evaluations. Public Health Specialist shall be responsible for assessing outcomes of the communication strategies. Specifically, s/he must have a thorough knowledge of successful approaches to institutional strengthening, community engagement and experience working with governments, and various management issues related to such donor assistance. S/he shall have: ● A minimum of an advanced degree in an area related to public health management ● Experience in designing and/or management of health interventions at national and community level ● Over five years’ experience in community organizing and mobilization ● Knowledge of local governance structures in Uganda ● Thorough knowledge and understanding of the roles and responsibilities of CSOs in Uganda H. Duration The task is estimated to begin on or about September 1, 2017 and be completed no later than March 31, 2018. The Contract may provide for protracted timeline outside the performance time to provide for billing and closure. I. Location of Assignment The assignment will cover districts and communities found in the CHC’s zones of influence in Uganda. J. Management Roles and Responsibilities The Evaluation Team will liaise closely with the Agreement Officer’s Representative (AOR) for CHC project, the USAID Senior Strategic Information Advisor (SSIA) for the Health, HIV and Education Team, M&E Specialist for the Health, HIV and Education Team, as well as the Program Office M&E Specialist, on coordination and clarification of USAID requirements and standards for maintaining effective communication on what is to be learned. CHC project will participate in the design and planning of the evaluation, support with logistics for implementation (avail copies of relevant documents, arrange for meetings and interviews, etc.), participate in the oral presentation and review the draft and final reports. GOU/MOH will participate in the oral presentation and review of the draft and final reports. Other USAID Implementing Partners may be invited to presentations. The evaluation contractor will provide own laptop computer(s) and other equipment necessary for performance of this activity; transport hire (within Kampala and upcountry), office supplies, communication costs, field work expenses (including survey, focus groups and dissemination). The evaluation contractor should also anticipate meeting field costs for government staff who may participate in the evaluation upon approval by USAID. K. Evaluation Criteria 101 ● Technical Approach ○ Sub criteria in order of importance ■ Extent to which the proposed technical approach is clear, logical, well-conceived, technically sound and accurately interprets the evaluation questions provided in this Statement of Work ■ Extent to which the proposed technical approach demonstrates an understanding of the implementation context ● Key Personnel ■ Extent to which the proposed key personnel meet the required qualifications demonstrating the Offeror’s ability to conduct the evaluation L. Evaluation Design The evaluation will involve three evaluation tracks and four case studies to explore how CHC implementation influenced activity outcomes. The evaluation also aims to generate learning that will inform current implementation and future programming. Evaluation Tracks The evaluation will involve three study tracks: ● The first track will focus on CHC activities and coordination at national level, including Ministry of Health (MoH), USAID, CDC and other national level stakeholders (for example DFID, Health Marketing Group) engaged in SBCC, including managing the health communication platform (improving the coordination of health communications interventions) ● The second track will focus on CHC activities in the focus districts. These are districts in which CHC implemented its activities directly. ● The third track will focus on CHC activities in the non-focus districts, these are districts in which CHC implemented activities through USAID and CDC implementing partners Case Studies The case study data will be collected mainly using qualitative field-based/observational approaches. Information to produce learning will be generated through interviews and group discussions with key program personnel and teams, review of documentation and reports, site visits and in-field consultation with service providers and beneficiaries. The proposed four case studies are presented in the graphical form below: CHC focus districts • In X district • X # of FGD • Technical focus: Integrated program • What worked/Changed over time • Opportunities for improvement CHC non-focus districts • In X district • X # of FGD • Technical focus: Integrated program • What worked/Changed over time • Opportunities for improvement CHC focus districts • In X district • X # of FGD • Technical focus: HIV • What worked/Changed over time • Opportunities for improvement CHC non-focus districts • In X district • X # of FGD • Technical focus: HIV • What worked/Changed over time • Opportunities for improvement Target Audience Analysis To understand whether the intended target was reached with the relevant message, the evaluation team will interface with a selected sample of target audiences to determine the access and content of the CHC’s SBCC messaging. 102 Annex 6: Data Collection Tools 4.1 Focus Group Beneficiaries (Primary Target Audience Members) Date: Number of Participants: XX female/ XX male Participant Life Stage Group_________________________ District___________________________ Location: _______________________________________________________________ Facilitator: ______________________________________________________________ Note taker: ______________________________________________________________ About this Focus Group Discussion The purpose of this FGD is to understand CHC primary target audience's level of exposure, reaction to, and understanding of the CHC Obulamu campaign messages/materials via mass media, community-based, and facility-based activities. In addition, if participants have heard about Obulamu from a health provider, effectiveness of the counseling will be probed. Objectives By the end of this focus group discussion, data will be collected that will inform: Relevance, comprehension, and motivation of Obulamu campaign: How well have SBCC materials and activities reached adolescents, young couples, pregnant women and their partners, and caregivers of children under age 5? How were they experienced and what was the result? ● Evaluation Question 1: What did the USG and implementer learn about the target populations’ misconceptions around intervention messaging? How did they adapt? ● Evaluation Question 2: What strategies were effective in achieving the desired behavioral outcomes and how did the implementer adapt? ● Evaluation Question 3: How did the implementer address converging messages and integration during implementation and what lessons can be learned? Materials Needed: It is recommended that select Obulamu campaign materials targeted each of the Life Stage groups be on hand in the event that participants are not familiar with the campaign. LIFE STAGE GROUP: LOCATION: LANGUAGE USED: # Name Sex Age Relationship status # Children/# U-5 1 2 3 4 5 6 7 8 9 10 11 12 Introduction: 103 Today we will be talking about Obulamu/health in Uganda. We will also be talking about any materials, advertising, or activities that you are aware of related to this topic. Before I/we begin, I/we want to let you know that any information or examples we discuss during this focus group discussion will not be attributed to any specific person. All quotes used in the report will be attributed to a general group, not by individual, and all identifying information will be removed. You are free not to respond to any of our questions or to leave the group at any time. The discussion will take 60 minutes. If you don’t mind, I would like to record this discussion, solely for the purposes of listening attentively now and taking notes later. Before I/we begin, do you have any questions? Introduction & ice breaker exercise: Ask each participant to introduce themselves (name, age, where they live) and describe one thing about themselves that almost nobody knows. Questions 1. Does the phrase ‘"Obulamu" sound familiar? If so, where have you heard about it? What does it mean to you? (PROBE: What do you think of when you hear "Obulamu"?) 2. Where have you heard about or seen Obulamu? (PROBE: On TV? On the radio? From a health provider/at the health facility/ from VHTs or CHWs? In the community event or at a community events?) NOTE: If participants have not all heard of the Obulamu messages, show select materials and ask their impressions.) 3. What kind of people is the Obulamu campaign for? (PROBE: Have you talked about Obulamu with friends or family members?) 4. What did Obulamu talk about or promote? (PROBE: HIV/AIDS prevention/treatment, malaria, family planning, nutrition, maternal child health, etc.) 5. If you are familiar with the Obulamu messages, what, if anything, did you learn from them? Did it motivate you (or your spouse) to do anything new or different? (PROBE: What was most motivating to you?) 6. Did you find yourself accessing more health services or products (e.g. condoms) as a result of the Obulamu campaign? If not, why not? 7. Which materials, messages, or activities do/did you like the most and why? 8. Was there anything that you disliked? If so, what and why? 9. Do/did you find any of the Obulamu messages hard to understand or confusing? If so, did you seek clarification? If not, why not? What would have made them easier to understand? 10. If you heard about Obulamu from a health provider, how would you describe the way he or she talked to you about it? Did he or she give you any 'educational' materials? If so, what did you think of the materials? (PROBE: Were the materials helpful or not? Why or why not?) 11. What, if anything, would make accessing services or the other things (choices/behaviors) that Obulamu promotes easier for you? 12. What, if anything, would you still like to know about the topics that Obulamu covers? (PROBE: Where do you go/whom do you speak with when you are seeking (health) information?) 13. In closing, I wanted you to know that the Obulamu campaign was created for you. With that in mind, do you have any other suggestions or recommendations for the people who are designing and implementing the Obulamu campaign? THANK YOU ALL VERY MUCH FOR YOUR TIME AND YOUR IDEAS. FACILITATOR NOTES Include any comments, impressions or special information about the participants interviewed or the FGD process 1. What were the most important or memorable statements the respondent made during the discussion? OR What were the most salient points that came out of this group discussion? 2. What was the most compelling story shared? 3. Overall, what do you think was most important to the respondents? 4. What did you learn that should be explored further? 5. What did you already know that this discussion confirmed? -----*----- 104 4.2 CHC Uganda Interview Guide This tool will likely be adapted to have task-specific conversations with relevant groups within CHC Date: Respondent Name: Job Title: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like to spend some time with you to hear your thoughts on the "‘Obulamu’" campaign-- what you think is working and where there is room for improvement. I am not evaluating you nor your organization, but simply looking for your ideas so that this project and projects like this can be even more effective in the future. The data collected will be used to inform this project going forward. Any information or examples we discuss during this interview will be confidential. You are also free not to respond to any of my questions or to stop the interview at any time. May I record this interview? Doing so will allow me to focus on your discussion instead of taking notes. Before I begin, do you have any questions? Can we set aside a maximum of one hour for our discussion? Obulamu? Campaign I would like to start by hearing how you would describe the "‘Obulamu" campaign 1. How would you describe the "‘Obulamu" campaign? a. In your opinion, what about the campaign is working well? b. What strategies were most effective in achieving behavior change? What evidence is there of this effectiveness? c. What have been the biggest challenges? d. What, if any, target audience misperceptions were identified and how did the project adapt? e. If you could change anything in terms of campaign design or implementation, what would it be? Probe: Have you seen any actual changes take place as a result of the campaign? If so, please describe. 2. How did the project address "Obulamu" converging messages and what lessons can be learned‘? a. From your experience, how did one national umbrella brand work across six (6) health areas and different target audiences? b. Is there evidence that one national brand resulted in demand for services? c. What challenges to integrated messaging did CHC encounter? How were the challenges addressed? Probe: How has this campaign addressed the challenge of integrated messaging and what lessons can be learned? 3. How have you been involved in the "‘Obulamu" campaign? a. What are the biggest challenges you face working on (or partnering with) this project? b. What, if any, feedback have you received from stakeholders and implementing partners? 4. To what extent have you been involved in CHC district level implementation? (PROBE: Engagement in planning, monitoring, providing supportive supervision, applying lessons/feedback into the program). Coordination/Integration/Capacity Building Now I would like to focus on coordination and any capacity building activities that you have been involved in. 1. How successful has the project been coordinating SBCC health interventions with multiple partners? 2. What coordinating (intra-agency, etc.) mechanisms were in place and how did they work? What were the challenges and how were they addressed? 3. What, if any, kind of capacity building resulted from this coordination? a. How would you describe this improved capacity? b. How, if at all, has it changed the practices of those involved? c. How is the project monitoring the progress of those involved? d. What, if anything, would you recommend changing about capacity building efforts to make them more effective? Probe: What has CHC and its partners learned about coordination and integration? What are your recommendations based upon lessons learned? Supply & Demand 1. How, if at all, did CHC link demand and supply side communication? 2. What challenges did the project face coordinating demand generation and service delivery? 105 3. What can be done in the future to improve linkages between demand generation and supply side services and communication? Research & Knowledge Management 1. The project began with a baseline, a review of secondary research, and a formative assessment. How were the results used as a foundation for program/campaign strategy and design? 2. How did on-going research and evaluation contribute to project improvement? What, if any, monitoring data has the project collected that was applied to making adjustments? What were those adjustments? 3. How was research/evidence documented and disseminated to partners? 4. Did CHC have a knowledge management strategy? If so, how effective was it? Did it lead to any observable improvements in Uganda's health communication efforts? Concluding Questions 1. What else would you like to tell me/us? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. INTERVIEWER NOTES Include any comments, impressions or special information about the person or organization interviewed or interview process 1. What were the most important or memorable statements the respondent made during the interview? OR What were the most salient points that came out of this interview? 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed? -----*----- 4.3 MoH Uganda/GoU/LG Interview Guide This tool will be adapted for relevance based on the department/level of government targeted. Date: Respondent Name: Title of Respondent: Organization Affiliation: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like spend some time with you to hear your feedback on the current “Obulamu” campaign-- what you think works, and what could be better. It is important for me to tell you that I am not evaluating you or the people you work with. I am simply looking for your thoughts so that projects like this can be most effective in the future. The data collected will be used to inform this project going forward. Any information or examples we discuss during this interview will be confidential. You are also free not to respond to any of my questions or to stop the interview at any time. May I record this interview? Doing so will allow me to focus on your discussion instead of taking notes. Before I begin, do you have any questions? Can we set aside a maximum of one hour for our discussion? Obulamu? Campaign I would like to start by hearing what you think about the “Obulamu” campaign. 1. What is your involvement in the Obulamu campaign? a. How would you describe it? b. How, if at all, have you heard target audience members (i.e. adolescents, young married couples, lovers, key populations, etc. ) or implementing partners talk about it? c. Have you encountered any misconceptions on the part of the target audience as a result of the campaign? If so, have they been addressed? d. What, if any, feedback have you or your staff received from members of the target audience populations about campaign activities or materials? e. What strategies or approaches do you think have been most effective influencing (health-seeking) behavior? Is there any evidence of this? Probe: Has the “Obulamu” campaign changed the way that you talk or think about health in any way? If so, how? 106 1. What other health communication campaigns or projects in Uganda have you been involved in or are familiar with? How is Obulamu different? What is the added value of this project? 2. How did the project address "Obulamu" converging messages and what lessons can be learned‘? a. From your experience, how did one national umbrella brand work across six (6) health areas and different target audiences? b. Is there evidence that one national brand resulted in demand for services? c. What challenges to integrated messaging did you or your staff encounter? How were those challenges addressed? Probe: How has this campaign met the challenge of integrated messaging and what lessons can be learned? 3. How have you or your organization been involved in the “Obulamu" campaign activities? a. Is your organization's visual identity or name associated with the campaign? If so, how well is that working? b. Have you or your organization been involved in implementing campaign activities or distributing campaign materials? If so, what has worked well? 4. Have all planned activities been implemented and if not, why? What have been some of the challenges? Integration, Coordination, and Capacity Building Now I would like to focus on coordination and any capacity building activities that you have been involved in. 1. How successful has the project been coordinating SBCC health interventions with your organization and multiple partners? 2. What coordinating (intra-agency, etc.) mechanisms are in place and how do they work? What challenges have you faced and how have they been addressed? 3. What, if any, kind of training or capacity building have you or your staff received from the project in the planning, design, and implementation of SBCC interventions? a. Has campaign coordination improved capacity in any way? b. How, if at all, has it changed the way you or your staff do your job? c. What part of capacity building has been most useful? What gaps remain? d. What, if anything, would you recommend changing to improve your and your colleague's capacity? Probe: What have you learned about coordination? What would you recommend in the future based upon lessons learned? Supply & Demand 1. How, if at all, did the project link demand and supply side communication? 2. What challenges were there coordinating demand generation with service delivery? 3. What can be done in the future to improve linkages between demand generation and supply side services and communication? Research & Knowledge Management 1. Were you or your organization involved in collecting any data? If so, how was it used? Did it contribute to project improvement? Can you give an example? 2. How was project research/evidence documented and disseminated to you and other partners? 3. Did CHC's knowledge management efforts improve the way that you or your colleagues do your job? If so, how? Probe: Can you make any recommendations for improving knowledge management or adapting and disseminating research results? Concluding Questions 1. What else would you like to tell me/us? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. INTERVIEWER NOTES Include any comments, impressions or special information about the person or organization interviewed or interview process 1. What were the most important or memorable statements the respondent made during the interview? OR What were the most salient points that came out of this interview? 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed? -----*----- 107 4.4 IPs Interview Guide Date: Respondent Name: Title of Respondent: Organization Affiliation: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like to spend some time with you to hear your feedback on the current “Obulamu” campaign-- what you think works, and what could be better. It is important for me to tell you that I am not evaluating you or the people you work with. I am simply looking for your thoughts so that projects like this can be most effective in the future. The data collected will be used to inform this project going forward. Any information or examples we discuss during this interview will be confidential. You are also free not to respond to any of my questions or to stop the interview at any time. May I record this interview? Doing so will allow me to focus on your discussion instead of taking notes. Before I begin, do you have any questions? Can we set aside a maximum of one hour for our discussion? Obulamu? Campaign I would like to start by hearing what you think about the “Obulamu” campaign. 1. What is your involvement in the Obulamu campaign? a. How would you describe it? b. How, if at all, have you heard target audience members (i.e. adolescents, young married couples, lovers, key populations, etc.) talk about it? c. Have you encountered any misconceptions on the part of the target audience as a result of the campaign? If so, how have you addressed them? d. What, if any, feedback have you received from members of the target audience populations about campaign activities or materials? e. What strategies or approaches do you think have been most effective influencing (health-seeking) behavior? Is there any evidence of this? Probe: Has the “Obulamu” campaign changed the way that you talk or think about health in any way? If so, how? 2. What other health communication campaigns or projects in Uganda have you been involved in or are familiar with? How is Obulamu different? What is the added value of this project? 3. How did the project address "Obulamu" converging messages and what lessons can be learned? a. From your experience, how did one national umbrella brand work across six (6) health areas and different target audiences? b. Is there evidence that one national brand resulted in demand for services? c. What challenges to integrated messaging did you encounter? How did you deal with those challenges? Probe: How has this campaign met the challenge of integrated messaging and what lessons can be learned? 4. How have you or your organization been involved in the “Obulamu" campaign activities? a. Is your organization's visual identity or name associated with the campaign? If so, how well is that working? b. Have you or your organization been involved in implementing campaign activities or distributing campaign materials? If so, what has worked well? c. Have all planned activities been implemented and if not, why? What have been some of the challenges? Integration, Coordination, and Capacity Building 1. Now I would like to focus on coordination and any capacity building activities that you have been involved in. 2. How successful has the project been coordinating SBCC health interventions with your organization and multiple partners? 3. What coordinating (intra-agency, etc.) mechanisms are in place and how do they work? What challenges have you faced and how have they been addressed? 4. What, if any, kind of training or capacity building have you received from the project to help you plan, design, and implement SBCC interventions more effectively? a. Has the campaign coordination improved your capacity in any way? b. How, if at all, has it changed the way you view/do your job? c. What part of capacity building has been most useful? What gaps remain? d. What, if anything, would you recommend changing to improve your and your colleague's capacity? Probe: What have you learned about coordination? What would you recommend in the future based upon lessons learned? Supply & Demand 1. How, if at all, did the project link demand and supply side communication? 108 2. What challenges did you face coordinating demand generation with service delivery? 3. What can be done in the future to improve linkages between demand generation and supply side services and communication? Research & Knowledge Management 1. Were you or your organization involved in collecting any data? If so, how was it used? Did it contribute to project improvement? Can you give an example? 2. How was project research/evidence documented and disseminated to you and other partners? 3. Did CHC's knowledge management efforts improve the way that you do your job? If so, how? Probe: Can you make any recommendations for improving knowledge management or adapting and disseminating research results? Concluding Questions 1. What else would you like to tell me/us? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. INTERVIEWER NOTES Include any comments, impressions or special information about the person or organization interviewed or interview process 1. What were the most important or memorable statements the respondent made during the interview? OR What were the most salient points that came out of this interview? 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed? -----*----- 4.5 Providers Interview Guide Date: District/Community: Name of Facility: Respondent Name: Title of Respondent: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like to spend some time with you to hear your feedback on the current “Obulamu” campaign-- what you think works, and what could be better. It is important for me to tell you that I am not evaluating you or the people you work with. I am simply looking for your thoughts so that projects like this can be most effective in the future. The data collected will be used to inform this project going forward. Any information or examples we discuss during this interview will be confidential. You are also free not to respond to any of my questions or to stop the interview at any time. May I record this interview? Doing so will allow me to focus on your discussion instead of taking notes. Before I begin, do you have any questions? Can we set aside a maximum of one hour for our discussion? Obulamu? Campaign I would like to start by hearing what you think about the “Obulamu” campaign. 1. What is your involvement in the Obulamu campaign? a. How would you describe it? b. How, if at all, have you heard target audience members (i.e. adolescents, young married couples, lovers, key populations, etc.) talk about it? c. Have you encountered any misconceptions on the part of the target audience as a result of the campaign? If so, how have you addressed them? d. What, if any, feedback have you received from members of the target audience populations about campaign activities or materials? e. What strategies or approaches do you think have been most effective influencing (health-seeking) behavior? Is there any evidence of this? Probe: Has the “Obulamu” campaign changed the way that you talk or think about health in any way? If so, how? 2. What other health communication campaigns or projects in Uganda have you been involved in or are familiar with? How is Obulamu different? What is the added value of this project? 109 3. How did the project address "Obulamu" converging messages and what lessons can be learned‘? a. From your experience, how did one national umbrella brand work across six (6) health areas and different target audiences? b. Is there evidence that one national brand resulted in increased demand for services? c. What challenges to integrated messaging did you encounter? How did you deal with those challenges? Probe: How has this campaign met the challenge of integrated messaging and what lessons can be learned? 4. How have you or your organization been involved in the “Obulamu" campaign activities? a. How has your organization/facility promoted the Obulamu brand? (PROBE: Obulamu signage displayed in facility? Obulamu posters? Obulamu materials in waiting area?) b. Have you or your organization been involved in implementing campaign activities or distributing campaign materials? If so, what has worked best? c. Have all planned activities been implemented and if not, why? What have been some of the challenges? Integration, Coordination, and Capacity Building Now I would like to focus on coordination and any capacity building activities that you have been involved in. 1. How successful has the project been coordinating SBCC health interventions with your organization/facility and multiple partners? 2. What coordinating (intra-agency, etc.) mechanisms are in place and how do they work? What challenges have you faced and how have they been addressed? 3. What, if any, kind of training or capacity building have you received from the project to help you plan, design, and implement SBCC interventions more effectively? a. Has the campaign coordination improved your capacity in any way? b. How, if at all, has it changed the way you view/do your job? c. What part of capacity building has been most useful? What gaps remain? d. What, if anything, would you recommend changing to improve your and your colleague's capacity? 4. What kind of training, if any, in interpersonal communication (IPC) did you received from the project? How would you describe it and how useful was it? (PROBE: Did you receive orientation in the Obulamu brand?) Probe: What have you learned about coordination? What would you recommend to improve coordination in the future based upon lessons learned? Supply & Demand 1. What challenges did you face coordinating demand generation with service delivery? 2. What can be done in the future to improve linkages between demand generation and supply side services and communication? 3. What else, if anything, can health communication/SBCC projects do to help you do your job? Research & Knowledge Management 1. Were you or your organization involved in collecting any data? If so, how was it used? Did it contribute to project improvement? Can you give an example? 2. How was project research/evidence documented and disseminated to you and other partners? 3. Did CHC's knowledge management efforts improve the way that you do your job? If so, how? Probe: Can you make any recommendations for improving knowledge management or adapting and disseminating research results? Concluding Questions 1. What else would you like to tell me/us? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND CONTRIBUTION. INTERVIEWER NOTES Include any comments, impressions or special information about the person or organization interviewed or interview process 1. What were the most important or memorable statements the respondent made during the interview? OR What were the most salient points that came out of this interview? 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed? -----*----- 110 4.6 Creative Partners Interview Guide Date: Respondent Name: Title of Respondent: Organization Affiliation: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like to spend some time with you to hear your feedback on the current “Obulamu?” campaign-- what you think works, and what could be better. It is important for me to tell you that I am not evaluating you or the people you work with. I am simply looking for your thoughts so that projects like this can be most effective in the future. The data collected will be used to inform this project going forward. Any information or examples we discuss during this interview will be confidential. You are also free not to respond to any of my questions or to stop the interview at any time. May I record this interview? Doing so will allow me to focus on your discussion instead of taking notes. Before I begin, do you have any questions? Can we set aside a maximum of one hour for our discussion? Obulamu? Campaign I would like to start by hearing what you think about the “Obulamu?” campaign. 1. What is your involvement in the Obulamu? campaign? a. How would you describe it? b. How, if at all, have you heard target audience members (i.e. adolescents, young married couples, lovers, key populations, etc.) or implementing partners talk about it? c. Have you encountered any misconceptions on the part of the target audience as a result of the campaign? If so, have they been addressed? d. What, if any, feedback have you heard from members of the target audience populations about campaign activities or materials? e. What strategies or approaches do you think have been most effective influencing (health-seeking) behavior? Is there any evidence of this? Probe: Has the “Obulamu” campaign changed the way that you talk or think about health in any way? If so, how? 2. What other health communication campaigns or projects in Uganda have you been involved in or are familiar with? How is Obulamu different? What is the added value of this project? 3. How did the project address "Obulamu" converging messages and what lessons can be learned? a. From your experience, how did one national umbrella brand work across six (6) health areas and different target audiences? b. Is there evidence that one national brand resulted in demand for services? c. What challenges to integrated messaging did you or your staff encounter? How were those challenges addressed? Probe: How has this campaign met the challenge of integrated messaging and what lessons can be learned? 4. How have you or your organization been involved in the “Obulamu" campaign activities? a. Is your organization's visual identity or name associated with the campaign? If so, how well is that working? b. Have you or your organization been involved in implementing campaign activities or distributing campaign materials? If so, what has worked well? c. Have all planned activities been implemented and if not, why? What have been some of the challenges? Integration, Coordination, and Capacity Building Now I would like to focus on coordination and any capacity building activities that you have been involved in. 1. How successful has the project been coordinating SBCC health interventions with your organization and multiple partners? 2. What coordinating (intra-agency, etc.) mechanisms are in place and how do they work? What challenges have you faced and how have they been addressed? 3. What, if any, kind of training or capacity building have you or your staff received from the project in the planning, design, and implementation of SBCC interventions? a. Has campaign coordination improved capacity in any way? b. What part of capacity building has been most useful? What gaps remain? 4. Do you feel that you or your organization have been utilized by the project in the best way possible? If not, in what other ways could you or your organization contribute? Probe: What have you learned about coordination? What would you recommend in the future based upon lessons learned? Research & Knowledge Management 111 1. How was project research/evidence documented and disseminated to you and other partners? 2. Did CHC's knowledge management efforts improve the way that you or your colleagues do your job? If so, how? Probe: Can you make any recommendations for improving knowledge management or adapting and disseminating research results? Concluding Questions 1. What else would you like to tell me/us? Any other comments/insights/questions you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND INSIGHTS. INTERVIEWER NOTES Include any comments, impressions or special information about the person or organization interviewed or interview process 1. What were the most important or memorable statements the respondent made during the interview? OR What were the most salient points that came out of this interview? 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed?1. -----*----- 4.7 USG Interview Guide Date: Respondent Name: Job Title: INTRODUCTION: Thank you very much for setting aside time to talk with me today. I would like to spend some time with you to hear your thoughts on the "‘Obulamu’" campaign-- what you think is working and where there is room for improvement. I am not evaluating you nor your organization, but simply looking for your ideas so that this project and projects like this can be even more effective in the future. The data collected will be used to inform future SBCC projects. Before I begin, do you have any questions? 1. We're here to talk about the "Obulamu" campaign. Could you please tell me first what role you play or what your relationship is to the Obulamu SBCC campaign? 2. In your mind, is there anything that differentiates it from other previous SBCC or health communication/behavior change campaigns? Please describe. 3. Where did the concept of an integrated campaign come from? Why was it considered necessary? 4. What were your expectations for the Obulamu campaign when it began? 5. Were there any concerns regarding a health communication project that covers six (6) separate health areas and targets four (4) or more different segments of the population? Did these concerns prove to be valid or not? 6. Has Obulamu met (or exceeded) your expectations? If not why not? 7. What is working well? Please give examples. 8. What evidence is there that Obulamu has met its behavioral objectives/influenced behavior? Increased health￾seeking, healthy choices, or uptake of services? (PROBE: What strategies had an effect on behavioral outcomes?) 9. What improvements would you suggest at this point? (PROBE: If you could redesign the project, what would you do differently?) 10. How do you see the (intra-agency etc.) coordination working? Could anything more be done to improve coordination? 11. What, if any, evidence of improved capacity (in the design, implementation, or evaluation of SBCC programs) have you observed among GOU and implementing partners as a result of this project? How could this be improved? 12. Has the CHC Project helped to coordinate demand generation with service delivery? If so, how? What lessons have been learned and what are the areas for improvement? 13. How has the CHC Project used research, monitoring, and evaluation findings to adapt its strategy and make project improvements? Can you identify any gaps? 14. How would you describe the CHC Project's Knowledge Management (KM) strategy? What, if anything, have they done to disseminate research results to the GOU and implementing partners? How, if at all, could the KM component be improved? Concluding Question 112 1. What else would you like to tell me/us? Any other comments/insights you would like to share? THANK YOU VERY MUCH FOR YOUR TIME AND THOUGHTS. INTERVIEWER NOTES Include any comments, impressions or special information about the interview. 1. What were the most important or memorable statements the respondent made during the interview? (What were the most salient points that came out of this interview?) 2. What was the most compelling story the respondent shared? 3. Overall, what do you think was most important to the respondent? 4. What did you learn that should be explored further? 5. What did you already know that this interview confirmed? 113 Annex 7: Background to CHC The State of Health Communication in Uganda Pre-CHC/Obulamu In 2014, CHC/FHI 360 conducted a rapid audit of the health communications (HC) landscape in Uganda.101 The study’s authors argued that although high quality HC interventions had been undertaken in Uganda in the preceding decade, “their overall effectiveness appeared unclear and not well translated into health outcomes,” implying the need to understand the state of the field at the time (p.7).102 The audit sought to shed light on the existing Government of Uganda (GoU) and US Government (USG)-funded implementing partners’ HC initiatives in the context of CHC’s planned thematic health areas. The audit found that existing health governance structures - e.g. the second National Health Policy (NHP II) and the third Health Sector Strategic Investment Plan (HSSIP III)103 - provided an overall framework for the health sector. HC was one of four clusters of the National Minimum Health Care Package. There was clear audience segmentation and message focus, along with specific avenues for linking demand and supply and for community engagement via VHTs and champions. Gender and social norms were key factors in health programming. Health-seeking behavior program indicators for monitoring and evaluation were included. Furthermore, GoU had a number of mechanisms to coordinate HC initiatives including dedicated policies and strategies, the Health Education and Promotion (HEP) Division in the Ministry of Health (MoU), and regional/district health education officers. Platforms for collaboration between GoU and implementing partners (IPs) existed, such as the national Behaviour Change Communication (BCC) Working Group, Uganda AIDS Commission (UAC) BCC Clearing House, and other thematic working groups/task forces. There had also been a number of dedicated HC initiatives - eg. AFFORD, Health Communications Partnership (HCP) - along with several other existing organizations with HC specialization, in particular, Communication for Development Foundation Uganda (CDFU) and CHC’s partner, the Uganda Health Marketing Group (UHMG).104 Additionally, the audit noted that IPs recognized HC as “a specialized technical area, hence their outsourcing this support from CDFU and UHMG, [and] reliance on MoH branded materials produced in collaboration with the former AFFORD and HCP programs.”105 Nevertheless, the audit also found that strategies for and implementation of HC in specific health areas were fragmented and not linked with other sectors. Reviewed or updated strategies remained in draft form or were yet to be launched. There were inconsistencies in messages and formative research was rarely done. There were no systematic plans for monitoring and evaluation of results. Gender sensitive strategies did not translate into message development. Media coverage was sporadic, one-size-fits-all, and centered on commemorative events.106 Additionally, there were also major gaps in coordination, including conflicting priorities and timelines as a result of programming dictated by funding mechanisms, red tape and delays in the process of clearing materials and messages. An overall lack of coordination in design led to dissemination of competing health messages, and implementation that was geared toward bureaucratic reporting rather than advancing a learning agenda. While partners frequently procured external support for HC, the services they received tended to overlook best practice SBCC interventions, and made limited use of theory and data in planning and decision making. Many HC programs failed to take into account the whole range of cognitive and social factors that determine health-seeking behavior and, even when they recognized that health-seeking behavior was influenced by multiple determinants, in practice this idea 101 FHI360/CHC, Health communication in Uganda: Findings of an audit of strategies, strategies/materials, and implementing partners in 2013/2014 (Kampala, FHI360/Communication for Healthy Communities (CHC), 2014). 102 FHI360/CHC, 2014. 103 Government of Uganda, The second national health policy (Kampala, Ministry of Health, 2010a); Government of Uganda, Health sector strategic investment plan 2010/11-2014/15: Promoting people’s health to enhance socio-economic development (Kampala, Ministry of Health, 2010b). 104 FHI360/CHC, 2014. 105 FHI360/CHC, 2014, p.15. 106 FHI360/CHC, 2014. 114 was only given lip service. Additionally, the vital role that VHTs/health champions play within the health system of bridging demand and supply was undermined by fragmentation and duplication of effort, poor reporting skills, lack of job aids, insufficient support supervision, and absence of IPs on the ground in areas where VHTs/health champions operated.107 The audit also found issues with planning and designing effective HC. IPs consistently failed to meet all the standards of recommended communications practice. The use of communication implementation plans was mostly ad hoc with a low level of adherence and marginal attention to quality assurance. Monitoring data was seldom used for program improvement. While routine program monitoring happened, its scope was largely limited to inputs and outputs for the purpose of administrative reporting as opposed to continuous program review and planning. While most partners understood the value of sharing and exchanging knowledge and having systems to identify and fill knowledge gaps, as well as made proactive efforts to manage knowledge, information, and data, the audit also found a “tendency to simply report outputs for accountability purposes e.g. at BCC WGs [working groups], and regional DHTs [District Health Teams] without clear attempts towards data use for decision making”.108 Monitoring and evaluation tended to focus on outputs (e.g. number of health personnel trained, number of commodities supplied) as opposed to evidence of the impact of interventions on service delivery or SBCC indicators. KM resources were underutilized with platforms deployed more for reporting and visibility than for program learning and adaptation.109 Finally, the audit found that the different thematic health areas of interest to CHC diverged greatly with respect to how communication channels were selected, priority messages were identified and aligned, and audiences were targeted. In maternal and child health (MCH), for example, passive modes of communication (e.g. posters) were dominant, whereas TB relied on interpersonal communication strategies, albeit badly under-funded ones. In HIV prevention, by contrast, the National HIV Prevention Strategy 2011-2015, reflected global experience, lessons learned, and best practices with respect to effective HC. Monitoring and evaluation indicators were defined. Barriers to behavior change and health-seeking were outlined. Audiences were segmented by age and appropriate communication channels were selected. Reproductive health and malaria also exhibited more sophisticated planning, along with a more comprehensive mix of communication channels and a robust monitoring and evaluation strategies.110 The audit concluded “These [bottlenecks] have implications for finding an effective tipping point for change, either addressing knowledge, skills and motivation needed, desired modification for gender and other social norms, or what would constitute an enabling environment for change.” 111 Prior USG Investments in Health Communication USG agencies have a long history of promoting HC initiatives in Uganda and CHC was designed to build on, strengthen, and sustain the results from previous USAID- and, more broadly, PEPFAR￾funded initiatives. One of the most important of these programs was the Health Communications Partnership II (HCP II) implemented from 2007 to 2012 by the Johns Hopkins University Bloomberg School of Public Health Center for Communication Programs (JHU-CCP). HCP II was rolled out in the footsteps of the AFFORD health marketing initiative (2005-2010) which subsequently evolved into the current CHC sub-prime partner, UHMG. AFFORD introduced the Good Life platform: an 107 For example, the attention to gender and social norms, for example, was confined to sexual and gender-based violence (SGBV) as well as male involvement, with the study reporting: “Apart from SGBV and male involvement, existing HC implementation almost does not mention what aspects of gender issues are to be addressed, who the audience would be, and how.” FHI360/CHC, 2014, p.16. 108 FHI360/CHC, 2014, p 30. 109 FHI360/CHC, 2014. 110 FHI360/CHC, 2014. 111 FHI360/CHC, 2014, p. 15. 115 earlier version of the sort of the life stage, whole life-focused approach that CHC subsequently expanded upon and deepened.112 When HCP II closed out in 2012, USAID/Uganda commissioned an evaluation of 24 IP-supported BCC interventions, including AFFORD/UHMG and HCP.113 The evaluation focused on the effectiveness of family planning, HIV, malaria, as well as maternal and child HC at national and district levels, evaluating exposure to/reach and associated behavioral effects of the communication efforts directed toward interventions in family planning, HIV, malaria, as well as maternal and child health. The evidence pointed to generally high levels of exposure (71 percent ) across all four areas of intervention. However, behavior change (e.g. adoption of recommended practices, health-seeking, uptake of promoted health products and services) varied widely among the audiences exposed to the different campaigns, and between the exposed and the unexposed. Similarly, the final evaluation of HCP II itself assessed the extent to which the project strengthened the abilities of communities, families, and individuals to adopt practices that improved their health. It measured whether 1) effective communication strategies were designed and implemented to increase appropriate use of services and/or practices across identified priority programs; 2) supportive social environments were fostered to enable positive health‐seeking behaviors and result in healthier individuals, families, and communities; and 3) capacity for sustained HC was increased.114 The evaluation found that HCP was successful in increasing the capacity of the MoH to carry out campaigns and train health workers, including VHTs, in HC and community mobilization, and supported the MoH in the roll out of new policies in support of adoption of healthy behaviors. However, the evaluation also found: 1) In spite of increased uptake of some healthy practices (for example, HIV testing, use of modern family planning methods), other measures of healthy behaviour actually regressed. For example, risky sexual behaviour among different age groups - including adolescents - increased.115 2) There remained major gaps between demand generation and supply-side delivery of services. While people wanted to adopt healthy behaviours, they often found the services enabling the to do so to be unavailable.116 We highlight these two findings from the HCP evaluation because they led directly to the recommendation that future or follow-on HC interventions should ● Promote and direct clients to services ● Focus on underlying social issues that influence behavior ● Strengthen strategic communication capacity ● Foster public supportive social environments ● Build coalitions and strengthen partnerships. The foundations of CHC and the Obulamu campaign, therefore, originate directly from the HCP evaluation’s strategic recommendations, with CHC marking an important pivot in HC and SBCC programming in Uganda. Due to the experiences of its predecessors, CHC had a massive body of locally-generated evidence to draw upon and leverage following a decade of experimentation and intensive learning about what worked, how, and why. 112 See interview KI_KLA_SUB_1. Also Storey JD, Saffitz GB, & Rimón JG, “Social marketing,” in Health behaviour and health education: Theory, research, and practice. 4th Edition, eds. K Glanz, B Rimer, & K Viswanath (San Francisco, Jossey-Bass, 2008), pp. 435-464. 113 USAID/Uganda, Uganda joint behavior change communication survey. (Kampala, USAID/Uganda, 2012a) 114 USAID/Uganda, Final evaluation of the Health Communications Partnership (HCP II) project (Kampala, USAID/Uganda, 2013). 115 USAID/Uganda, 2013, pp. ix-x. 116 USAID/Uganda, 2013, pp. xi 116 At the same time, five years on, many of the same barriers that were found to impede implementation of HCP and other HC initiatives in Uganda continue to remain bottlenecks to own CHC’s achievement of its strategic objectives and intermediates results. Much of the story we tell in this evaluation with respect to the challenges, successes, and lessons of CHC and the Obulamu campaign, then, are not unique to this particular project. There are historical antecedents to our findings and future follow-on SBCC mechanisms should expect to encounter and be made to address these ongoing issues. Communication for Health Communities A five-year (June 2013-June 2018) PEPFAR funded mechanism, CHC is predominantly funded through USAID, with additional inputs from CDC and Department of Defense (DoD), with the goal of supporting the Government of Uganda and partners in reducing HIV infection, total fertility, maternal and child mortality, malnutrition, malaria and tuberculosis (TB). It uses the Obulamu (How’s Life) concept as the organizing principle, campaign platform, and overarching brand for an integrated set of HC interventions targeting these six thematic health areas. A Luganda greeting, Obulamu is a popular conversation starter and means of exchanging pleasantries across the Buganda region of central Uganda. In its role as a campaign brand, the word Obulamu has also begun to be used in some non-Luganda-speaking parts of the country with reference to personal health and well-being as will become evident throughout this evaluation.117 Planned as a departure from traditional disease-focused approaches that tended to respond to health problems via technical silos,118 the Obulamu campaign was conceived in 2013/14 as a response to the broader consensus, described above, that Uganda’s HC landscape was characterized by: ● Limited implementation and scale up ● Fragmented implementation of interventions ● Disease-focused messaging with many competing messages ● Audience fatigue with instructive messages ● Disconnect between mass media and interpersonal communication.119 As one senior health educator said: If I look back in the area of health communication, we’re always criticized for being vertical, for looking at the disease and not the human being. Coming up with vertical programs and not integrated, having bias or going for programs that have money, ignoring others...We were more interested in diseases and programs than looking at an integrated approach. There was always that story of integration, but nobody had ever come up with a strategy or a model [KI_KLA_CG_4] The national Obulamu campaign was designed to address these concerns by ensuring messages were integrated and delivered at scale, life-focused (not disease-oriented), adaptive to audience feedback, and linked to interpersonal engagement with beneficiaries. CHC thus supports GoU and other partners in rolling out Obulamu across Uganda. CHC’s strategic objective is to increase the adoption of healthy behaviors (including uptake of critical health services) through strengthened health communication, in order to contribute to these improved health outcomes. 120 This objective is to be met via three Intermediate Results (IRs): IR1: High quality health communication interventions designed and implemented 117 However, how the word Obulamu is used also depends greatly on the region, the local language of its population, the language family to which the language belongs (e.g. Bantu versus Nilotic) and the region’s history and political, economic and social relationship to central Uganda. Again, we will return to this issue at later points in the evaluation. 118 FHI360/CHC, 2014. 119 Kayongo E, Lessons from the integrated life-cycle based health campaign in Uganda. (Kampala, FHI360, 2017). 120 USAID/Uganda, Request for Application (RFA) Number RFA-617-13-000001: “USAID/Uganda Communication for Healthy Communities (CHC) (Kampala, USAID/Uganda, 2012b). 117 CHC supports GoU, IPs, and other partners in the design and implementation of state-of￾the-art, theory-grounded, evidence-based, appropriately-targeted, multi-channel communications strategies and messaging linked to service delivery. CHC supports Uganda’s national Obulamu strategy for integrated HC with respect SBCC for six intersecting health issues (HIV, tuberculosis (TB), malaria, maternal and child health (MCH), nutrition and family planning (FP)) targeted to four identified Life Stage groups (couples, pregnant women, parents of under-5s, and adolescents). CHC links to efforts by central and district government, regional health IPs, and facility and community-based health workers to improve integrated health service delivery across Uganda. IR2: Improved coordination of health communication interventions CHC strengthens capacity of relevant GoU structures, programs and coordination mechanisms, as well as USG-supported regional health implementing partners, to play a stronger leadership role in coordination of HC and to ensure central and local government ownership of these initiatives, as well as to improve intra- and inter-agency coordination through mechanisms such as technical working groups and committees. IR3: Increased research and knowledge management to enhance health communication CHC carries out operational research and robust impact evaluation to improve the evidence-base as to the linkages between HC initiatives, behaviour change and improved health outcomes, as well as to identify and disseminate the most effective and cost-effective approaches/models and to respond adaptively to new insights.121 CHC is designed specifically to work with and through the MoH and other USG regional health IPs engaged in integrated health service delivery, including across USG agencies (e.g. USAID, CDC, DoD). Relevant regional IPs currently include USAID/RHITES-SW, USAID/RHITES-EC, USAID/RHITES-E, USAID/SUSTAIN Northern Uganda, CDC/IDI, CDC/Baylor, and CDC/Mildmay. It is important to note that many of these regional IPs, especially those managed by USAID, have changed over the duration of CHC, an issue we will return to at later points in this evaluation. CHC also works with non-USG development partners like DfID, international and local non-governmental organizations (NGOs) and civil society organizations (CSOs) engaged in health service provision and/or health advocacy, and creative communications and advocacy partners in advertising and mass media. CHC, therefore, does not exist in a bubble, but builds upon and leverages existing efforts by the GoU, its partners, and previous USG communication activities which have focused on developing tools and strategies for social and behavior change to motivate individuals, households and communities to adopt better health practices linked closely to service delivery. 121 USAID/Uganda, 2012b. 118 Annex 8: Evidence from CHC Phase I data-to-Phase II data on Attitudes and Knowledge Table A1: Evidence of Notable Declines in Positive Attitudes toward Family Planning HEALTH AREA: Family Planning (Methods) VARIABLE: Positive attitude toward family planning TRENDS PHASE I (%) PHASE II (%) Overall Full sample 82.1 84.5 Life stage LS4 84.4 77.1 Region Eastern 86.9 80.0 Karamoja 50.8 42.4 Age 15-19 83.5 76.9 Marital status Widowed 18.8 8.7 Table A2: Evidence of Notable Declines in Positive Attitudes toward Condoms HEALTH AREA: HIV (Condoms) VARIABLE: Positive attitude toward condoms TRENDS PHASE I (%) PHASE II (%) Overall Full sample 15.5 14.0 Channel Social media/SMS 24.4 18.3 Residence Urban 14.5 7.5 Region South Western 18.3 4.4 Marital status Widowed 18.8 8.7 Table A3: Evidence of Notable Declines in Positive Attitudes toward Reducing the Number of Sexual Partners HEALTH AREA: HIV (Multiple Partners) VARIABLE: Positive attitude toward reducing the number of sexual partners TRENDS PHASE I (%) PHASE II (%) Overall Full sample 71.7 77.9 Channel TV 76.0 71.6 Region Central 81.9 72.0 Eastern 75.4 69.7 Age 15-19 76.9 69.0 Marital status Never married/single 80.2 70.7 119 Table A4: Evidence of Notable Declines in Confidence/Comfort Talking about Reducing the Number of Sexual Partners HEALTH AREA: HIV (Multiple Partners) VARIABLE: Confident/comfortable talking about reducing the number of sexual partners TRENDS PHASE I (%) PHASE II (%) Overall Full sample 78.7 78.6 Channel Outreach 87.3 78.2 Life stage LS4 77.0 67.6 Region Central 85.3 73.0 East Central 75.4 58.9 Age 15-19 76.9 69.0 Marital status Never married/single 80.2 70.7 Table A5: Evidence of Notable Gaps in [Basic] Knowledge of Risks of Having Multiple Partners HEALTH AREA: HIV (Multiple Partners) VARIABLE: Knowledge of risks of having multiple partners TRENDS PHASE I (%) PHASE II (%) Overall Full sample 86.2 88.1 Channel Video halls/dens 98.1 87.4 Region Northern 93.7 84.8 Table A6: Evidence of Notable Gaps in [Basic] Knowledge about the Spread, Symptoms, and Treatment of TB among Key Populations HEALTH AREA: TB (General) VARIABLE: Knowledge about the spread, symptoms and treatment of TB (Key populations – Residents of Amuru, Ntungamo, Mukono and Sembabule districts) TRENDS PHASE I (%) PHASE II (%) Overall Full sample 57.2 53.5 Channel Radio 62.5 56.8 TV 66.1 56.6 Printed media 68.3 63.6 120 Outreach 68.9 58.8 Home visits 66.3 61.3 Social media/SMS 75.0 50.0 Community events 76.7 62.6 Residence Urban 65.6 52.5 Region Central 60.3 49.2 Table A7: Evidence of Notable Gaps in Comprehensive Knowledge about Malaria Prevention HEALTH AREA: Malaria (General and Children Under 5) VARIABLE: Comprehensive knowledge about malaria prevention TRENDS PHASE I (%) PHASE II (%) Overall Full sample 22.4 23.7 Residence Urban 30.4 24.8 Region Central 28.8 14.7 Northern 48.1 28.6 Table A8: Evidence of Notable Gaps in Comprehensive Knowledge about the Spread of HIV HEALTH AREA: HIV (Testing) VARIABLE: Comprehensive knowledge about the spread of HIV TRENDS PHASE I (%) PHASE II (%) Overall Full sample 33.8 27.6 Exposed to messages None 16.5 11.2 Two or more times 36.8 31.7 Mentioned Obulamu No 32.0 19.9 Yes 48.0 40.5 Channel Outreach 43.3 32.0 Social media/SMS 58.5 37.7 Life stage Adolescents 31.7 21.8 Gender Female 33.0 27.9 Male 34.8 27.3 121 Residence Rural 32.6 25.8 Region Central 31.2 17.5 East Central 38.3 17.9 Karamoja 22.8 7.5 Northern 50.9 18.1 Western 30.5 25.3 Table A9: Evidence of Notable Gaps in Comprehensive Knowledge of the Shortcomings of SMC HEALTH AREA: HIV (Safe Male Circumcision) VARIABLE: Comprehensive knowledge of the shortcomings of SMC TRENDS PHASE I (%) PHASE II (%) Overall Full sample 72.3 76.5 Region Northern 72.1 65.0 Table A10: Evidence of Notable Gaps in Comprehensive Knowledge about Contraceptive Use HEALTH AREA: Family Planning (Methods) VARIABLE: Comprehensive general knowledge about contraceptive use TRENDS PHASE I (%) PHASE II (%) Overall Full sample 51.7 71.1