The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation Oleksandr Rohozynsky Christie Billingsley Annette Bongiovanni The QED Group, LLC ii Acknowledgements The evaluation team acknowledges the open and willing participation of the staff of AFFORD and its partners, as well as the Uganda Health Marketing Group. We appreciate the efforts of the Ugandan evaluation team members: Paul Bukuluki, Ronard Mukuye, Margaret Banga, Ddumba Nianzy, and all the interviewers who collected data for this study. We also greatly appreciate the guidance and oversight from Deborah Weiss and Sam Kalibala of The Population Council. Cover photo credits: (c) 2007 Jennifer Orkis (all three photos), Courtesy of Photoshare (www.photoshare.org) Top left and right picture: Children in Uganda attend a regional prize giving ceremony for The Good Life Show, an entertainment education quiz show that's part of a campaign under the AFFORD Health Marketing Initiative. The game show features different health topics each week, and shows how simple health practices and lifestyle choices can protect the family's health and lead to socio-economic fulfillment. Bottom left picture: A woman hands out communication materials at a regional prize giving ceremony for The Good Life Show, an entertainment education quiz show that's part of a campaign under the AFFORD Health Marketing Initiative. The game show features different health topics each week, and shows how simple health practices and lifestyle choices can protect the family's health and lead to socio-economic fulfillment. This report was made possible through support provided by the United States Agency for International Development (USAID)/Uganda under the terms of Contract No. GHH-I-02-07-00034-00. The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. The QED Group, LLC is a full-service consulting firm that provides practical solutions to social problems through sound analysis, proven management techniques, and creative implementation. QED stands for the Latin phrase, Quod Erat Demonstrandum, meaning “proven,” or “it is proved.” Sound information and empirical analysis are the foundations of effective decision-making, and the QED Group is a proven performer in helping its clients make a difference. www.qedgroupllc.com The Population Council conducts research worldwide to improve policies, programs, and products in three areas: HIV and AIDS; poverty, gender, and youth; and reproductive health. www.popcouncil.org Published in September 2009. Copyright © 2009. United States Agency for International Development. Suggested citation: Rohozynsky, Oleksandr, Christie Billingsley, and Annette Bongiovanni. 2009. “The AFFORD Health Marketing Initiative in Uganda: Mid-term evaluation,” Final Report. New York: Population Council. This document may be reproduced in whole or in part without permission of the Population Council provided full source citation is given and the reproduction is not for commercial purposes. iii Table of Contents Acronyms v Executive Summary 1 Introduction 8 Background 8 Rationale 15 Objectives and Research Questions 15 Methodology 17 Study Design 17 Selection and Description of Sample/Study Participants 18 Ethical Considerations 20 Data Collection Methods 21 Data Management and Analysis 21 Limitations 22 Results 24 Description of the Survey Respondents 24 Overarching Achievements 25 Progress toward Meeting AFFORD Objectives 26 Discussion 37 Framework for the Discussion 37 Validating Project Results 38 Effectiveness of AFFORD Activities 41 Sustainability of UHMG 43 Conclusions and Recommendations 46 Conclusions 46 Recommendations 48 References 51 Annexes 53 Annex A : AFFORD Activities Timetable 53 Annex B : UHMG Organizational Structure 55 Annex C : Key Informant Interview Methodology 56 Annex D : Retail Store Survey Methodology 57 Annex E : Exit Survey Methodology 59 Annex F : Focus Group Methodology 63 Annex G : Interviewer Training Manual 65 Annex H : Key Informant Interview Guide 81 Annex I : Retail Store Survey Questionnaire 84 Annex J : Exit Survey Questionnaire 97 iv Annex K : Retail Store Survey Informed Consent 103 Annex L : Exit Survey Informed Consent 104 Annex M : Short Summary of Key Informant Interviews 105 Annex N : Statistics for Retail Store Survey 118 Annex O : Statistics for Exit Survey 121 v Acronyms ACT Artemesinin Combination Treatment AFFORD AFFORD Health Marketing Initiative ANC Ante-natal Clinic CBO Community-Based Organization CDFU Communication for Development Foundation Uganda CSW Commercial Sex Worker DADI District Assistant Drug Inspector DHE District Health Educator DHO District Health Officer DHS Demographic and Health Surveys FGD Focus Group Discussion GOU Government of Uganda HCT HIV Counseling and Testing JHU-CCP Johns Hopkins Center for Communications Programs LLIN Long Lasting Insecticide Net MARP Most At-Risk Population MOH Ministry of Health NGO Non-Governmental Organization NICRA Negotiated Indirect Cost Recovery Agreement PLHA People Living with HIV and AIDS POL Popular Opinion Leader QED The QED Group, LLC SSECODA Ssesse Community Development Association STI Sexually Transmitted Infection SWOT Strengths, Weaknesses, Opportunities, and Threats UPDF Uganda People‘s Defense Force UHMG Uganda Health Marketing Group USAID United States Agency for International Development VCT Voluntary Counseling and Testing VHT Village Health Team WAWI Women at Work International The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 1 Executive Summary Introduction AFFORD is a five-year (2005–2010) social marketing initiative financed by the United States Agency for International Development (USAID)/Uganda. It aims to support the Uganda Ministry of Health‘s (MOH) strategic plan to promote positive healthcare-seeking behavior and to reduce the shortage of health products and services related to HIV prevention, care, and treatment; family planning and reproductive health; child health; and malaria prevention and treatment. AFFORD is managed by the Johns Hopkins Center for Communications Programs (JHU-CCP) and partners: Futures Group International, Malaria Consortium, Communication for Development Foundation of Uganda (CDFU), Pulse Communication Limited, and Aclaim Africa Limited. AFFORD seeks to achieve the sustainable marketing of products and services that prevent the transmission of HIV, malaria, and diarrheal diseases; to help couples plan their families and to help people living with HIV enjoy improved quality of life. Target groups include families, young couples, caretakers of children under age five, pregnant women, and people living with HIV. The three objectives of AFFORD are to: 1. Increase the accessibility and affordability of products and services; 2. Enhance knowledge and correct use of products and services to encourage and sustain healthy behaviors and lifestyles within communities and families; 3. Establish and build capacity of an indigenous health marketing organization and distribution systems. USAID/Uganda requested a mid-term evaluation to determine the initiative‘s progress, identify course corrections for the remaining two years of programming, and inform future programming. Through a subcontract from the Population Council, the QED Group, LLC (QED) designed and conducted the evaluation to answer the following questions: 1. What is the effect of AFFORD activities on the volume of sales/distribution of the AFFORD-supported healthcare products and its substitutes? 2. What is the effect of AFFORD activities on the health behavior of the general population and specific at-risk groups, such as sex workers and soldiers? 3. Was AFFORD successful in creating a sustainable indigenous health marketing organization and network of distributors? Methodology The core evaluation team comprised QED staff based in Washington, DC and Ugandan consultants based at Makerere University in Kampala. The team used both qualitative and quantitative methods, including document review, key informant interviews (persons working in private and public sectors), a retail store check survey, client exit surveys, and focus group 2 discussions (FGDs). A team of 23 experienced Demographic and Health Surveys (DHS) interviewers were trained by the evaluation team and participated in a pilot study before conducting field interviews. Data processing and analysis was handled by the core team. The Ugandan staff handled data transcription from narrative accounts and entered survey responses into an SPSS spreadsheet format. A Ugandan data analyst ran initial, basic frequencies; the QED team reviewed the results for quality and followed-up with more analyses. Results AFFORD is widely known in Uganda. It is managed by an effective partnership and has an outstanding record of relations with the Government of Uganda. Respondents, especially in the professional health and donor community, are aware of AFFORD and understand all or part of its work. Its programs and products are well-recognized by target populations and were visible in all regions visited by the evaluation team. The AFFORD partnership is cohesive. All partners play a significant role in implementation, which is fostered by the prime contractor‘s strong commitment to collaboration and development of a recognizable brand for the initiative. AFFORD has also developed strong, transparent relations with the Government of Uganda at both the central and district levels, and engages the government in ongoing dialogue to align its activities with government plans. Key informants, especially at the central level, attribute the good relations of AFFORD with government authorities to the dedication of AFFORD‘s senior management. AFFORD has also made impressive progress toward its three main objectives: 1) Increase accessibility of products and services. AFFORD is distributing 12 diverse, socially-marketed products and will begin distribution of an additional product in the next few months. These products are distributed through a private network created by AFFORD. AFFORD continued the distribution of three products (USAID brands) previously introduced and distributed by a predecessor social marketing project implemented by Population Services International: Pilplan, Injectaplan, and Protector. AFFORD also introduced a line of new products registered by the Uganda Health Marketing Group (UHMG). USAID brands are sold in 85 to 91 percent of private drug stores in the counties surveyed, and other brands are available in at least 21 percent of stores after being on the market for a year. Several of the products have a substantial share of the market (InjectaPlan, PilPlan, and Protector) and in some cases have created new markets (Moonbeads). In addition, AFFORD is introducing standards of services for small, private clinics, and uniting such clinics under the Good Life! clinic brand. In 2007 AFFORD selected and trained 100 private clinics that later were branded as Good Life! clinics. 2) Enhance healthy behaviors and lifestyles. AFFORD conducted several health promotion campaigns during the period October 2005 to October 2008. The health promotion campaigns are highly visible and recognized in the areas where they are available, and respondents perceived that they have resulted in improved healthcare seeking behavior and health status. A number of these activities, such as the Good Life! show on television and radio, the Under the Mango Tree radio show, and HIV/AIDS films, are well-recognized by the target population and watched by The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 3 significantly-sized audiences. Other activities, such as PULSE community activation campaigns that use presentations, discussion, or HIV/AIDS films and the Popular Opinion Leader (POL) program that promotes community health through local leaders, are less recognized by the general population because of their local nature, but are highly known in the communities exposed to them. These activities are regarded as highly effective and innovative by district authorities and communities in the areas where they take place. Though the evaluation could not definitively determine the effect that any of these activities had on health behavior, there is indication that AFFORD activities, such as PULSE activation, the POL program, and long-lasting insecticide net (LLIN) distribution, are linked with increases in demands for the AFFORD health products and positive changes in health behavior, such as use of HIV testing or mosquito nets for pregnant women and children. 3) Establish an indigenous health marketing organization. At the end of the first year of implementation, AFFORD registered and established an indigenous organization, UHMG, to assume its work after the initiative ends. UHMG has an organizational structure, dedicated staff, and has taken over much of AFFORD‘s day-to-day work. AFFORD has worked on capacity building with the UHMG staff and begun building networks of technical and financial support outside UHMG. Respondents were universally favorable regarding the idea of creating UHMG and impressed at its early establishment. However, they also identified specific areas of development and support needed by UHMG if it is to be truly sustainable, such as new business development expertise and financial planning skills. Discussion The evaluation results suggest that the AFFORD initiative made significant progress towards achieving all three objectives. The evaluation team considers AFFORD‘s Objective 1—to increase accessibility of products and services—near completion, but recognized that further actions may be necessary to support and strengthen its private distribution network over the remaining two years of the initiative. Two major challenges should be noted: 1. There is competition between socially-marketed brands of health products distributed by different donor campaigns in Uganda. This problem was noted in previous social marketing studies in Uganda and was also reported by key informants in this study. This competition leads to inefficient use of the resources when donors spend money to protect their market share from other donors instead of spending these resources to scale up distribution, for example, or to provide other services. The competition also drives down the price of socially marketed brands. This is a problem for a social marketing project that intends to establish self-sustained marketing of these products. Competition drives the price of the products below cost of the product distribution which makes sustainability impossible. 4 2. The original USAID brand of condoms (Protector), though technically meeting specifications, experienced issues with quality and consumer properties, such as size and smell, which have negatively affected distribution. Despite these challenges, the outlook for continued distribution and increases in sales volume of AFFORD products looks promising. The initiative created an operational distribution network that does not require significant financial support and therefore should function sustainably after AFFORD ends. Operations management for product development, marketing, and distribution has already been transferred to UHMG. For all brands, except the three USAID-supported brands, operations are conducted on a cost-recovery basis, in which prices set for these products equal production and distributions costs: price paid to manufacturer, cost of repackaging and warehousing, and margin provided for distributors and retailers. The cost-recovery price does not, however, include costs of social marketing campaigns. The initiative introduced and completed a variety of health promotion activities to achieve Objective 2—to enhance health behaviors and lifestyles. Activities supporting Objective 2 are considered almost completed at this point by the project. At this early point in the initiative, the evaluation could not determine the effect that any of these activities had on health behavior. The market response to the social marketing campaigns of AFFORD is reported by some drug shop owners who indicated an increase in the number of customers for AFFORD products after local or national marketing activities. However, the behavior change or stable market response can only be determined more than one year after campaigns end. It also should be noted that the high costs of the campaigns may render them unsustainable without external funding. As mentioned above, current cost-recovery prices for the UHMG-branded products do not include cost of social marketing campaigns. If such costs were included into the price, the products would become unaffordable to the target population, which defeats the goal of the initiative. AFFORD is also on its way to achieving Objective 3—to create a sustainable, indigenous social marketing organization. UHMG was created, registered, and has been functioning for at least two years. AFFORD is gradually transferring staff and responsibilities for social marketing activities to UHMG and is transitioning the international experts (who started AFFORD) into short-term UHMG consultants to facilitate capacity building. According to key informants, AFFORD has already achieved more in creating an indigenous organization than other international projects implemented in Africa in a similar time frame. Yet additional steps are needed to ensure the sustainability of UHMG. UHMG staff was trained and gained experience in social marketing and related activities, but not in fundraising or organizational management and development. As a result, currently UHMG staff, particularly upper management, lack knowledge and experience in these areas, all of which are fundamental to sustainability. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 5 Conclusions and Recommendations The evaluation team formulated the following conclusions based on the evaluation results. 1. AFFORD has made significant progress towards the achievement of its goals and objectives. a. It has nearly achieved Objectives 1 and 2. b. It has made significant progress on Objective 3, but must focus further efforts during the remaining years of the initiative. 2. AFFORD implemented several innovative activities in social marketing and health promotion that can be considered promising practices for the implementation of similar campaigns, and have great potential for further support by USAID. These include: a. Working with private distributors to deliver socially marketed products with no distribution cost to the initiative is highly effective and sustainable. b. Developing social marketing methods, such as television and radio game shows and the POL program, that were innovative, well noticed, and remembered by the populations exposed to them. The POL program can be considered a pilot that demonstrated effectiveness in the areas where it was implemented and was highly regarded by local officials and the communities. 3. The initiative undertook early, significant, steps to ensure sustainability of UHMG. However, UHMG requires additional capacity building and support from USAID. a. UHMG socially-marketed brands are distributed on a cost-recovery basis, which provides a foundation of sustainability. It should be mentioned, however, that USAID brands (Pilplan, Injectaplan, and Protector) are distributed at prices below cost-recovery. Therefore, continuing distribution of these brands by UHMG will need financial support from USAID to cover the cost of acquiring these products, as has been done since the creation of these brands. b. UHMG has capacity in the distribution of socially-marketed health products, but lacks experience in financial management and new business development. c. UHMG needs continuing technical support and expertise from the international partners of the AFFORD initiative in implementation of core activities and capacity building. Recommendations for AFFORD for the near term AFFORD can take some immediate steps to improve its product distribution, which eventually will support the UHMG distribution efforts. The evaluation team offers the following recommendations to address existing weaknesses: 6 Assess the distribution network and provide training to distributors and, if possible, store owners, on best business practices, such as calling distributors to order products when current supplies are running low, confirming the needs of shops by phone before sending large supplies of drugs for distribution, and keeping lists of distributor contacts. Provide simple instructions and a re-order form in each bulk package of drugs that contains contact telephone numbers for all 12 UHMG distributors and a short statement-request for re-supply of the drug that can be sent by mail or by fax. Provide a detailed description of drugs in each bulk package with details about side effects and advantages of each drug to inform drug shop owners about the product. AFFORD can also enhance its work with local/district level authorities in promotional activities: Involve the UHMG managing director and top management in meetings with local authorities and make a clear statement about what should and should not be expected from UHMG compared to current AFFORD activities in order to avoid ambiguity. Extend AFFORD‘s practice of collaboration to local government structures with the ultimate goal of establishing district plans for health promotion activities, capacity building in the districts to carry out these activities, and creating an understanding about sources that could continue funding for these activities after AFFORD has ended. Extend the trainings and other activities conducted under the POL program to local community health structures already in place, such as Village Health Teams (VHT). Continue to identify and engage local partners, including non-governmental organizations, in health outreach activities, such as the PULSE activities. Finally, there are a number of activities that can strengthen UHMG and increase the probability of survival for the organization beyond the project end date: Strengthen UHMG‘s organizational structure through inviting periodic participation of the international organizations that helped to form it. For example, the UHMG could establish an informal network or an Advisory Board comprised of representatives from international organizations. Develop a medium-term financial plan for UHMG that considers several scenarios of funding availability, including a scenario in which AFFORD funding ends according to the initial plan. Continue development of UHMG capacities in the new business development through training and guided proposal development efforts. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 7 Extend UHMG contacts with non-US international donor organizations, such as the World Bank and other government development programs active in Uganda to provide income and growth for UHMG. Long-term recommendations The following recommendations are for consideration during the design and implementation of new USAID social marketing programs in Uganda and elsewhere, rather than for ongoing AFFORD activities: The AFFORD name brand has high visibility and positive recognition in Uganda. Using the same name for an extension of the initiative or a follow-up would capitalize on its positive associations. USAID/Washington and USAID missions have used this example specifically to extend the longevity and influence of a project name; such as the BASICS project and the Benin PISAF project. The AFFORD activities on the LLIN distribution could be implemented as a part of an AFFORD initiative extension or as a separate project. Schools are major mechanisms in Uganda for reaching communities with information that promotes demand and use of health products. USAID should consider an extension of the initiative or a follow-up project that concentrates on health promotion through schools and engages the Ministry of Education and Sports, especially at the district level. For example, education about certain health products could be included in the PIASCY curriculum or its successor program, and implemented in cooperation with UHMG. AFFORD was successful in distributing USAID-branded products, such as Protector, PilPlan, and InjectaPlan. However, if USAID continues to distribute these products through UHMG beyond the end-date of AFFORD, it should support the broader UHMG product distribution network, where current volume is too low for many distributors to earn a profit. The availability of larger volumes of USAID-branded products through this network will boost distributor loyalty and ensure that USAID￾branded products continue to reach consumers without the disruption of a distributor change. Such a step would also ensure the products are distributed by a qualified indigenous partner already experienced in handling these products. Continued financial support to UHMG beyond the remaining two years of AFFORD will ensure a smooth transition to independence and avoid turnover of qualified staff. UHMG needs continued access to the technical experts beyond the remaining two years of the project in order to build capacity in new business development, financial planning, and intermittent health technical areas as it transitions to independence. 8 Introduction Background Social marketing ―is a process that applies marketing principles and techniques to create, communicate, and deliver value in order to influence target audience behaviors that benefit society (public health, safety, the environment, and communities) as well as the target audience" (Kotler et al. 2006). Social marketing is a systems approach that can increase access to life-saving products and services and encourage healthy behaviors and lifestyles. Since the late 1960s the United States Agency for International Development (USAID) has embraced social marketing to address communication gaps in public health education. It introduced this strategy in about 30 countries worldwide though flagship projects, such as the Social Marketing for Change program to address rapid population growth and the Commercial Market Strategies, which broadened this work by partnering with private and commercial sector entities to increase the use of quality family planning and other health products and services. Nearly every country with a USAID presence currently includes social marketing within its portfolio, either as a stand-alone project or as a component of projects, such as those addressing HIV prevention, care, and treatment. USAID is increasingly interested in applying social marketing strategies with direct links to public and private sector partners while promoting sustainability. Social marketing is based on the concept of applying principles and approaches of commercial marketing in order to achieve specific behavioral goals. One of the main differences between social marketing and commercial marketing is their primary goals: social marketing tries to influence a target population in order to achieve a social goal, while commercial marketing usually has a financial goal of maximizing profits. However, both commercial and social marketing also have common goals, such as maximizing sales and consumption of products by target populations. The difference between these two concepts is usually the target population. Social marketing arose in low-income countries because commercial entities enter the market with products that are affordable only to a small proportion of the population. However, there is a large social benefit in some products, especially health products for family planning, HIV prevention, and water purification, which are used by majority of the population. The social benefit is especially large when these preventive products are used by the poorest populations because they avoid high costs associated with unintended pregnancy and infectious disease. Social marketing is typically conducted through multimedia campaigns and often is supplemented by the distribution of the products at affordable (highly subsidized or free) prices. Social marketing programs usually rely on donated products that are repackaged and rebranded for specific markets. Governments in low-income countries usually have limited resources and health care capacity and are especially hard-pressed to provide free or subsidized products. In addition, the introduction of such products requires educational and motivational campaigns to stimulate their use—steps that in developing countries further strain the limited resources and knowledge required to design and execute these types of campaigns. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 9 The social marketing approach tries to influence the behavior of a target population with no previous experience using specific health products. Therefore, contrary to commercial marketing, its social counterpart can achieve its goal even if the market for a specific health product is not growing. For example, prevention of adolescent pregnancies can be achieved through use of hormonal contraceptives, barrier methods, or abstinence. If a social marketing campaign increases abstinence, it achieved its goal, although the sales of contraceptives and barrier methods might decrease. The development of the social marketing concept under the financial constraints of governments and donor funding resulted in development of the total market approach to social marketing (see R. Pollard, 2007). This approach intends to reduce the financial burden of achieving a social goal by increasing targeting for free or subsidized products to the poorest segments of the population, while simultaneously increasing the proportion of the population that buys the products in commercial markets. It introduces specialized, low-cost, mass-produced products specifically targeted to the low-income groups that cannot afford products currently available on the market. At the same time, these commercially-distributed products are intended to be more attractive than the free or subsidized versions. Social marketing is used to create and increase demand for these and other commercially-available products. The success and sustainability of the total market approach depends on several factors, some of which are not necessarily under the control of social marketing projects. For example: 1. There should be sufficiently large unsatisfied demand for low-priced, socially￾marketed products to allow the high-volume of the market to compensate for the low profit margin on these products. 2. The existing market organizations (public and private) must have logistic and management capacities large enough to handle the volume and provide low-cost manufacturing and distribution of these products. The commercial market should also be willing to expand into the low-income population segment with low-priced products. One of the conditions to achieve this goal is that the low￾priced products do not enter the high-profit segments and do not substitute the currently available high-profit products. The AFFORD Health Marketing Initiative (AFFORD) is a five-year project funded by USAID/Uganda that uses social marketing techniques to increase the sustainable marketing of health products and services throughout the country. AFFORD was created to support the Uganda Ministry of Health‘s (MOH) current strategic plan, a plan that aims to promote positive healthcare-seeking behavior and reduce the unmet need for health products and services related to HIV prevention, care, and treatment as well as in other areas such as family planning and reproductive health, child health, and malaria prevention and treatment. AFFORD is intended to engage multiple partners from the private commercial sector, non￾governmental organizations, and the public sector in comprehensive efforts to increase the practice of healthy behaviors and generate/ fulfill demand for an affordable, accessible range of "public health" products and services. AFFORD is one of a new generation of social marketing initiatives whose primary goal is to create an indigenous, stand-alone entity that will outlive the project and carry on its work. 10 AFFORD was awarded as a cooperative agreement to Johns Hopkins Center for Communications Programs (JHU-CCP) and partners (Futures Group International, Malaria Consortium, Communication for Development Foundation of Uganda [CDFU], Pulse Communication Limited, and Aclaim Africa Limited) in 2005. It focuses on a wide variety of products and services to prevent the transmission of HIV, malaria, and diarrheal diseases. It also helps couples plan their families and those living with HIV enjoy a healthier and improved quality of life. AFFORD aims to improve health among key target groups— including families, young couples, caretakers of children under age five, pregnant women, and people living with HIV or AIDS—by promoting positive behavior change through various innovative and proven methods, and to ensure widespread access to affordable, high￾quality products and services. AFFORD has the following objectives: 1. Increase the accessibility and affordability of products and services that address HIV/AIDS, reproductive health, child survival, and malaria; 2. Enhance knowledge and the correct use of these products and services to encourage and sustain healthy behaviors and lifestyles; 3. Strengthen and establish an indigenous organization as well as distribution systems for the sustainable and self-sufficient delivery of key health marketing functions, including management, distribution, and promotion. To achieve these objectives, AFFORD combines targeted product development and distribution with media and other communication campaigns to promote knowledge of and correct use of products. AFFORD has introduced and supported 12 brands of health care products (see Table 1), which it repackages and provides to a network of 12 private distributors. These distributors were recruited from existing large pharmacies and trained by AFFORD. The project encourages the distributors to use their current retail networks to sell AFFORD products and refers pharmacy owners to these distributors for supplies of AFFORD products. It also provided bikes to local small entrepreneurs to support distribution of small quantities of the products to remote villages that do not have their own drug shops. Finally, AFFORD established the UHMG to continue distribution of the products beyond the lifespan of the initiative. Products AFFORD continued the distribution of three products previously introduced and distributed by a predecessor, the USAID-funded social marketing project implemented by Population Services International. These products, Pilplan, Injectaplan, and Protector are referred to as USAID brands in this report. AFFORD also introduced a line of new products registered by UHMG (UHMG brands). Both groups of products are distributed and marketed by AFFORD and collectively referred to as AFFORD brands. AFFORD also conducts different types of malaria prevention activities such as: campaign￾style long-lasting insecticide net (LLIN) distribution, Artemesinin Combination Treatment The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 11 (ACT) training with the private sector, and LLIN ante-natal clinic (ANC) distribution. Most of the net distribution is carried out by the Malaria Consortium. Table 1 USAID and UHMG brands Brand Name Product Strength/ Composition USAID Brands Protector Condoms Lubricated with SK70 PilPlan Low-dose combined oral contraceptive Duofem Injectaplan Injectable contraceptive that prevents pregnancy for three months 3-month vial of Depo-Provera UHMG Brands RESTORS R.O. ORS Low osmolarity WHO formula ZINKID Zinc Zinc sulphate 20mg DT Cotramox Cotrimoxazole Cotrimoxazole tablets 960mg "O" Condoms Ribbed/Studded /Colored AQUASAFE Water purification 70mg NaDCC per tablet Vitaboost Multivitamin Multivitamins with antioxidants Newfem Combined oral contraceptive pill Levonorgestrel 0.15mg / Ethinyl Estradiol 0.03mg + Ferrous Fumarate 75mg tablets Soft-sure Progesterone-only oral contraceptive pill Levonorgestrel 0.03mg Clovirex Acyclovir tablets and cream Acyclovir tablets 200mg, 400mg Programs AFFORD also established several health programs (see Table 2) including outreach events, television shows, radio programs, and newsletters. The foundation of AFFORD‘s programs is the Good Life! campaign, which began as a health-focused game show broadcast in English, with episodes rebroadcast on both television and radio. The Good Life! name and logo subsequently became recognizable and were put on every AFFORD and UHMG product, as well as clinics operating according to the standards of AFFORD. 12 Table 2 AFFORD Programs Activity Description The Good Life! Platform The Good Life! Show Good Life! Show is an interactive, entertainment-education game show broadcasted on TV in English (24 episodes) and on radio in five local languages, and distributed in video halls and during community road shows. During the show, couples are challenged to answer short questions, to see how much they know about their partners, and to act out health behaviors for their partners to guess. The Good Life! Show Question of the Week Listeners and viewers call a toll-free hotline for their chance to win weekly prizes and be entered to win the grand prize, a Toyota pick-up truck, donated by Toyota Uganda and IAA Health Care. The Good Life! Network (Good Life! Clinics) UHMG identifies and trains staff of clinics that meet agreed-upon standards. These clinics are branded as the Good Life! clinics, part of The Good Life! Network, to increase their visibility to clients. The Good Life! Teams UHMG's Good Life! teams are an extension of the Popular Opinion Leaders (POL) program (see below). The teams are trained and supported to utilize opportunities such as church gatherings, social celebrations, local council sessions and informal one-on-one interactions with neighbors, family, and friends to advocate for the simple things communities can do every day to keep healthy and fulfill their family's dreams. The Good Life! Support Media UHMG uses a variety of supporting social and behavior change communication vehicles to reinforce Good Life! messages, including the Everyday Health Matters newsletter and Under the Mango Tree community outreach and radio program. The Good Life! at Work Good Life! at Work is the UHMG activity conducted together with other (non-AFFORD) projects that trains company employees and sensitizes their dependents and communities on prevention and health maintenance behaviors. Other AFFORD marketing and outreach activities Everyday Health Matters Everyday Health Matters is a free, medium literacy newsletter produced quarterly in English and local languages. Each issue of this four-page newsletter focuses on a specific health issue including HIV/AIDS, malaria, child health, or family planning and includes information about related UHMG health products. It acts as support material under the Good Life! Campaign, and for free net distribution activities with Malaria No More and the President’s Malaria Initiative, and at AFFORD/UHMG events. Pulse Activations Pulse Activations are experiential marketing activities for various audiences through clinics, ladies clubs, Under The Mango Tree village meetings (see below), and men’s clubs. Activities include road shows, bar promotions, films, and video shows. Popular Opinion Leaders Network (POL) Implements the idea of community health education advocacy & mobilization by organizing and training volunteers from local communities to advocate positive health behavior and provide information and advice on health products use to the community members. AFFORD works with local partners to identify, train and supervise popular opinion leaders in areas where AFFORD works. A training curriculum and information cards were developed in collaboration with MOH. HIV Prevention Films Strong collaboration with MOH and UPDF, the AFFORD dubbing of entertainment education films (Yellow Card, Center 4, Time to Care), and screens the shows at the community meetings and other occasions. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 13 Under the Mango Tree radio show Under the Mango Tree is an innovative community outreach program and reality radio show. Community members gather under a mango tree to discuss real issues they face on one of AFFORD’s supported topics, with a moderator and health professional. These live sessions are recorded and edited into entertaining 30-minute radio segments. Listeners provide feedback by calling a toll free hotline with questions and comments about the show, and frequently asked questions are selected and answered in 45 second radio spots. Other AFFORD activities (non-comprehensive list) Malaria Prevention and Nets distribution Project distributes and promotes the use of long lasting insecticide nets (LLINs), especially by pregnant women and children under five. LLINs are distributed through antenatal clinics, the Good Life! Campaign, and an NGO LLIN facility Routine distribution of free Nets through 417 ante-natal clinics in 23 districts in Northern Uganda. Free distribution of nets through the NGO LLIN facility (40 NGOs in 20 districts). Campaign style distribution of Nets. MARPS Network Established the MARPS Network of organizations, provides institutional support, technical expertise, and capacity building to the network. Sub grants to WAWI,Uganda Police, UPDF, SSECODA. Training of Peer Educators – messengers of good life. Referral for HTC, STI and other needed services. HIV/AIDS Program Palliative Care Strengthen capacity of private clinics & drugs shops through training and support supervision. Established referral mechanism from VCT to palliative care through POL program, MARPS network, and Good Life! Clinics. Provide support materials and job aids to health workers. Support PLHAs to access palliative care products. Introduced new products for HIV-positives Cotramox, Clovirex. Developing Distribution Network for Socially Marketed (Low-cost) Health Products AFFORD is gradually putting in place a distribution structure which relies on local distributors rather than on its own distribution force. One key element in the distribution strategy is the incorporation of small scale entrepreneurs, based on the model of community distribution successfully used in other countries but with a special emphasis on financial sustainability. Establishing Indigenous NGO for Sustainable Delivery of Health Marketing Functions – UHMG UHMG was incorporated on 13th October 2006, began implementing program activities in April 2007, and was officially inaugurated by AFFORD on 29th May 2008. Services UHMG, the independent entity that AFFORD created to continue its work after the initiative ends, provides a number of services listed below: Consultations UHMG attracts new project funds from donors in exchange for the delivery of high quality programming and deliverables using its skills, efficiencies, and cost containment approaches. 14 Product Facilitator UHMG acts as a broker between frontline health organizations fighting AIDS, malaria and other diseases, and product and commodities suppliers. Sales and Marketing UHMG eases access to a range of quality and affordable health products. One-Stop-Shop Pharmacy Network. A small network of one-stop shop pharmacies provides health solutions for the majority of health issues faced by Ugandans. Research and Market Data UHMG's research and evaluation unit conducts health marketing and consumer behavior studies. Health Conference for Health Marketing Practitioners UHMG holds an annual Network Conference for Health Marketing Practitioners to create a network of health service marketers and providers. The objective of the conference is to facilitate change, share success stories, improve distribution, and identify future opportunities for growth and investment. An example of the integration of AFFORD products and programs can be illustrated by their work in family planning. AFFORD‘s family planning program focuses on couples. The initiative works to market and make accessible an array of seven family planning products including oral contraceptives (Pilplan, NewFem, SoftSure), injectables (Injectaplan), condoms (Protector, "O"); or a natural, rhythm-based method (MoonBeads). Complementary media programs, such as the Good Life! television show, Everyday Matters written materials, and Under the Mango Tree radio show, aim to promote these contraceptives products and their proper use. Simultaneously, popular opinion leaders (POLs)—AFFORD-trained community members who hold respect and trust and whom others may seek out for advice and help—work to make sure people in remote areas know about the products, where to get them, and how to use them. Finally, AFFORD works through the private sector to make sure the products are distributed to clinics and retail stores. For a summary of all AFFORD programs, products and services, please refer to Table 12. AFFORD targets specific population groups for different activities. Some socially-marketed products are targeted at potential consumers. For example, USAID brands are marketed to the poorest populations since they are subsidized and accomplish USAID mandated goals to focus on vulnerable populations. UHMG products such as SoftSure, NewFem, and the ―O‖ condom were introduced on a cost recovery basis (with the intention of recovering a percentage of costs on each sale) and target low to medium-income populations that have money to buy contraceptives, but cannot afford full-priced commercial brands that already exist on the market. Specific products such as Moonbeads were introduced for women who will not use hormonal or barrier methods of contraception for religious or health reasons. Large health promotion campaigns, such as Good Life! programming target the general Ugandan population with access to television and radio. Other activities have more specific audiences, determined either by the Government of Uganda (GOU) or USAID. For example, the distribution of long-lasting insecticide nets (LLINs) and related health promotion The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 15 campaigns targeted families with children under age five according to GOU policy and to correspond with available resources. A specific mandate of AFFORD is HIV prevention, a goal it hopes to accomplish by targeting most-at-risk-populations (MARPs), such as female and male sex workers and their clients (including uniformed servicemen, truckers, and fishermen). AFFORD assisted in establishing a MARPs network of non-governmental organizations that work with these population groups, such as the Women at Work International (WAWI) and the Uganda People‘s Defense Force (UPDF). Working with these organizations also allowed the project to address other health issues such as family planning, malaria control, and child health by reaching families of female sex workers. AFFORD also provided training in HIV testing and counseling (HTC) at private clinics. The Good Life! clinics were supplied HIV testing kits and training. Community leaders participating in the POL program were trained to refer community members to Good Life! clinics for HIV counseling. The POLs were also trained to promote voluntary HIV testing among the population and to provide information about HIV, its prevention, and support services available to the HIV-positive population. Rationale AFFORD has now reached its mid-point with implementation well underway and measurable results realized. USAID/Uganda has requested an evaluation to determine the initiative‘s progress thus far and allow time for course corrections. The evaluation will provide USAID/Uganda with data to help make decisions regarding the continuation of the program. More broadly, it will inform USAID‘s worldwide investments in health marketing strategies. The QED Group, LLC (QED) conducted the mid-term evaluation via a subcontract from the Population Council. Objectives and Research Questions The purpose of the mid-term evaluation of the AFFORD initiative is to determine whether activities implemented during its three-year base period have achieved the desired results. This mid-term appraisal is a part of larger evaluation efforts undertaken by USAID/Uganda with objectives to: 1. Perform process, outcome, and impact evaluations of ongoing HIV/AIDS programs in Uganda; 2. Conduct formative research and assessments to inform HIV/AIDS program design and policy; 3. Synthesize data, findings and other sources of information in order to make strategy, policy, and programming recommendations to the United States Government, and when requested, to the Government of Uganda; and 4. Provide technical assistance to national and project-level monitoring and evaluation systems. 16 Based on the initial assessment of the AFFORD, project, the following specific evaluation objectives were formulated for this mid-term evaluation: 1. What is the effect of AFFORD activities on the volume of sales/distribution of the AFFORD-supported healthcare products and its substitutes? a. Did the market grow or are AFFORD products substitutes for other products? Is there a spill-over effect from retailers working with AFFORD to all retailers? 2. What is the effect of AFFORD-supported activities on the health behavior of the general population and specific at-risk groups, such as sex workers and soldiers? a. Did they change their behavior over the last three years, and, if so, what was the reason for such changes? 3. Was AFFORD successful in creating a sustainable indigenous health marketing organization and network of distributors? a. What tasks still have to be completed before the project ends? This report documents the planning, execution, analysis, and findings of the AFFORD mid￾term evaluation process. It begins with a detailed description of the methodology for the evaluation including the study design, data collection methods, ethical considerations, data analysis and quality, and limitations of the study. Next, it summarizes the evaluation results in terms of the three AFFORD project objectives. Results are followed by a discussion of the broader implications of the evaluation findings. Finally, conclusions and recommendations are offered for AFFORD, partners, and USAID. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 17 Methodology Study Design The evaluation team used both quantitative and qualitative methods to enable cross-validation of information and enhance the contextual analysis. The mixed-methods approach was particularly important since AFFORD implements a variety of distinct interventions and targets several audiences. The team hypothesized that there should be a large spillover effect of the social marketing interventions in the ―intervention‖ and ―comparison‖ groups. The team assumed that within the same locality (e.g., town, village, or district) retailers and other community members would be exposed to the same television and radio programming interventions and would share information. Therefore, there should be only small differences between drug stores that interacted with AFFORD/UHMG distributors directly and those that did not within the same proximity/district. In order to test this hypothesis, we included both types of stores and clinics in the sample. Data collection methods The evaluation team used the following five methods for data collection: Document review: The evaluation team reviewed AFFORD documents, such as the performance monitoring plan, work plans, and annual reports. The team also examined Ministry of Health documents to understand the epidemiological situation in the country and current health priorities. Key informant interviews: The team conducted key informant interviews to provide context for the achievements of AFFORD. Key informants included persons representing public and private sectors, namely: Ministry of Health officials, USAID, NGO representatives, MARP network representatives, AFFORD partners, UHMG founding partners, current members of AFFORD/UHMG staff and partners, all of whom had intimate knowledge of, and experiences with, AFFORD at different points in time and from varying perspectives. For example, AFFORD staff members had broad knowledge of AFFORD activities, while partners often brought knowledge in a specific area, such as HIV prevention. Retail store survey: The retail store survey was designed to assess the effect of AFFORD/UHMG‘s direct work with distributors and retail drug stores. The team collected data to assess the difference in availability of AFFORD/UHMG supported brands in drug stores that work with one of 12 AFFORD/UHMG distributors and, by comparison, those that are not direct partners of such distributors. In addition, the survey gathered data on the influence of AFFORD/UHMG promotion campaigns on the demand of the AFFORD/UHMG-supported products in these stores through retailer assessment of changes in sales volume and their experience with sales of particular brands of a product. Client exit survey: The client exit survey was designed to assess knowledge of and health behavior change among the targeted beneficiaries of AFFORD interventions. For example, questions addressed knowledge and use of mosquito nets, condoms, vitamins, and water 18 filters or purification tablets. Respondents were also asked if they had considered HIV testing and if they considered being potentially exposed to HIV while engaging in sexual activities. They were asked about both their own behaviors and those of their family members in the past six months. Finally, the survey sought to gauge respondents‘ rationale for their individual behavior. Respondents were asked how their behavior changed over the last three years in the use of specific health products, visits to doctors, and HIV testing. If respondents indicated a change in the behavior, they also were asked to indicate the reason for the change. Standard reasons included information received through television, radio, recommendations of doctors, and personal experiences, but respondents were not limited to these choices. Focus group discussions (FGDs): FGDs were held with POLs and community members. They were designed to assess the effect of the POL program. The FGDs also included a discussion of AFFORD products to determine if participants recognized them and knew how to use them. Semi-structured interview guides and structured questionnaires were used to collect data. A team of experienced interviewers associated with Makerere University were trained over a one-week period, which included a pilot study. The purpose of the pilot was three-fold: 1) to assess the quality of interviewers‘ skills; 2) to test the implementation of the study design; 3) to test the survey questionnaires. Additional details on the methodologies, including questionnaires, informed consent forms, and the interviewer training manual are available in the annexes. Selection and Description of Sample/Study Participants The selection of study participants varied depending on the data collection method. Table 3 summarizes the sampling approach and corresponding respondents for each method. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 19 Table 3 Description of samples and study participants Method Sample Study Participants Key informant interviews The team used a snowball sampling approach starting with recommendations from AFFORD/UHMG and USAID/Uganda. Key informants were asked to recommend informants for further discussion of the topic, who were contacted and included in the key informants list by the evaluation team. Overall 76 informants were interviewed. AFFORD staff, AFFORD partners, and Uganda MOH staff at the national and district levels, representatives of MARPs network, experts from NGOs and other international projects, and USAID. Retail store survey Intervention group: In each district, twelve retail stores were randomly selected from a list of retail drug stores working with AFFORD/UHMG distributors (Note: 12 stores were selected as a practical limitation of the survey in correspondence with the number of interviewers). Comparison group: The geographically closest retail store to the intervention store was selected from a list of all licensed drug stores provided by the District Assistant Drug Inspector (DADI). The study design estimated sample size of 240 retailers. Retail store owners and managers. Client exit surveys Clinic exits: Clients exiting health clinics. The Good Life! clinics were chosen as the place for interviews in districts targeted by AFFORD. Clinics recommended by AFFORD or district health officials were chosen in comparison districts. (Note: the evaluation team chose primarily Good Life! clinics, which are typically small, private clinics, but also included public clinics that have an informal relationship with AFFORD). If a district had more than six Good Life! clinics, facilities with the largest expected client volume were chosen. The study design estimated 1920 respondents for the clinic exit surveys. Clients or accompanying persons (one person from a group) exiting clinics who were over 18 years of age and were not employees, were not previously interviewed, and were not a family member of a previously interviewed person. Retail store exit surveys: Clients exiting each store included in the retail store survey were interviewed for the client exit survey. The study design estimated 480 respondents at the retail store exit surveys. Clients entering and exiting retail store clinics who were over 18 years of age and were not employees or previously interviewed. FGD 8 FGDs were conducted in communities in four AFFORD districts with an active POL program and where malaria interventions were implemented. Popular opinion leaders and community members. Each FGD included 5–10 persons of mixed gender between 18–45 years of age. Both the retail store survey and client exit surveys had mixed-sampling strategies. In order to minimize cost and time, and to simplify logistics, both shared the same unified geographical coverage, based on a sampling of district sites selected according to the following criteria: 20 The intervention group covered at least one district within the designated areas of each of the 12 AFFORD/UHMG distributors. Comparison groups were selected from the districts that are not primary activity areas for AFFORD/UHMG, but are physically adjacent to the intervention group districts. Districts chosen were also in areas where AFFORD/UHMG conducted different types of malaria prevention activities. The availability of trained interviewers who spoke the local language was a factor in selection. Similarly, the main languages spoken in these districts were considered in order to be able to recruit an adequate number of fluent interviewers. For a list of districts included in the sample and actual sample sizes please refer to Table 4. Two districts, Kampala and Kiboga, did not have formal comparison group districts because of their uniqueness. Kampala is the capital of the country and is the primary target for several AFFORD/UHMG activities that are not targeted to the rest of the country, such as the English language Good Life! show on television. Kiboga District is in the distribution area for AFFORD/UHMG. However, there were no AFFORD activities conducted in this region except for the distribution of AFFORD/UHMG products. Ethical Considerations The evaluation team made every effort to address ethical considerations during the planning and implementation of the evaluation. All evaluation team members signed QED‘s confidentiality statement, which confirmed their commitment to keep verbal and electronic information collected from respondents safe and confidential. Interviewers‘ contractual letters of agreement explicitly stated that they should safeguard confidentiality of completed questionnaires, should provide all completed questionnaires to the supervisor at the end of each day, and should not discuss completed questionnaires with anybody except for evaluation team members. Informed consent was given before each interview (see ANNEX K and ANNEX L for examples of informed consent forms). During training, interviewers were instructed on how to request informed consent and use the forms. Interviewers ensured that the interview was conducted in a confidential setting, and that it was one-on-one with a respondent so no one else could hear the respondent‘s answers. They also assured respondents that their participation was voluntary and that they could choose to stop the interview at any time. Only evaluation team members had access to the raw data, which were kept in a locked file at QED headquarters. Data were not linked to names or organizations, but rather, used unique identifiers. The survey procedures did not request any information that could lead to the identification of a respondent. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 21 Data Collection Methods The team sought to collect a robust body of complementary qualitative and quantitative data. Document review. The early document review informed the study design and the next steps of the evaluation process such as tool development, selection of respondents, and sample selection. The location of surveys gave the team a sense of the context in which AFFORD was working and provided a thorough, informed foundation before the field work began. Key informant interviews. The key informant interviews were conducted by the team intensively at the beginning of the evaluation in Kampala and continued at a steadier pace throughout the survey and focus group work at similar locations. Team members took notes by hand which were then typed up and readied for coding by the data analyst. Client exit surveys and retail store survey. The client exit survey and retail store survey results were gathered personally by the team of Interviewers. They were then entered into an SPSS data base by the data analyst, cleaned, tabulated and subjected to a series of comparison analysis. FGDs. The focus group discussions were documented with an audio recorder, transcribed, and translated from local Ugandan languages into English. Observers also took notes on the discussions and noted their non-verbal observations in writing (for example; the observer‘s perception of the openness of the group, the flow of conversation, the group‘s acceptance of the facilitator/venue, discomfort/comfort with certain topics, etc.). The notes were analyzed by first extracting key words and phrases and organizing them by guiding topics/questions. Then, the FGD findings were compared with others. The evaluation team took care to note both common and uncommon remarks, opinions, and feelings shared by participants. Data Management and Analysis Key informant interviews: The qualitative data analysis included summaries of the views of key informants by region and by project objective to determine the most typical answers. The answers were grouped by the questions included in the interview guide, which are also linked to the evaluation questions. The responses were also grouped by region to explore regional differences. Standardized typical responses were synthesized based on the answers of the key informants. The most unusual answers were also reviewed to determine if there were any relationships between the informant‘s observable characteristics, such as location, education level, or position and these answers. Retail store survey and client exit surveys: Data analysis included descriptive statistics, cross-tabulations of different variables, and a comparison of intervention and comparison groups. Given the diversity of AFFORD‘s target groups, it was cost prohibitive to select a representative sample. By interviewing a convenience sample of people visiting a clinic or drug shop, the sample was intentionally biased towards those more concerned with health problems. Therefore, it is not appropriate to make statistically significant generalizations extrapolated to AFFORD‘s target populations. 22 FGDs: FGD data were reviewed to anecdotally substantiate the findings from the quantitative data and key informant interviews. The FGD data was grouped by questions and topics from the FGD guide and then compared to the results of the surveys and key informant interviews to verify the results from these data sources. Data quality The evaluation team took great care to ensure the integrity of the evaluation process and data collection. Only experienced interviewers, most of whom had previously been trained as DHS surveyors, where selected. They were given thorough training and guidance by QED staff and were supervised daily. Each survey team had a supervisor who regularly checked data collector performance. Every evening, the team met for about one hour to debrief on the day‘s events, report obstacles, and ask questions about coding. Data was also reviewed for inconsistencies or outliers that required further investigation. Limitations The team had a finite amount of time and funding to conduct the evaluation. Activities needed to be completed rapidly in order for USAID to make course corrections for the remaining two years of the AFFORD initiative. In spite of AFFORD‘s broad reach across Uganda, financial limitations precluded the team from visiting every AFFORD district. As noted previously, the client exit interview sample was inherently biased because respondents were seeking services and potentially more aware of health products and behaviors. The results presented might overestimate the true effectiveness of the AFFORD interventions. Better performing retail outlets in the AFFORD districts could be due to the criteria applied by AFFORD to select target districts. In order to successfully achieve a goal of developing sustainable distribution network, AFFORD selected districts that have larger, more urbanized populations more likely to have a sufficient number of individuals able to pay for the socially marketed products. Therefore, it is possible that even without AFFORD activities, the drug shops in these districts are more likely to be successful enterprises and sell larger quantities and offer a greater variety of health products than similar stores in other districts. As mentioned, the sample for the client exit survey is biased towards people with a greater propensity to care about health and health products. In addition, given that AFFORD selected clinics to participate in the Good Life! program based on their commitment to comply with standards and promote healthy behavior, these clinics may have achieved more positive changes in health behavior than those in comparison districts even without AFFORD interventions. Finally, it is impossible to attribute some study findings—particularly changes in behavior— directly to the AFFORD initiative. Other groups working in Uganda have been implementing social marketing programs since the 1990s, supported by a variety of donors including bilateral donors such as the US government (USG) and multi-lateral donors. Early on, the PilPlan and Injectaplan were introduced by USAID-funded Population Services International. At the same time, international agencies have used different funding sources to The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 23 implement similar activities such as HIV testing and counseling, mosquito net distribution, and distribution of condoms. Many activities have nation-wide coverage, such as distribution of socially marketed condoms by PSI and distribution of free condoms by MOH. By definition, these programs cover some of the same areas where AFFORD distributed Protector condoms. At the local level, most activities are implemented by local community￾based organizations (CBOs). Some of these CBOs have the option to implement AFFORD activities one week and activities for other international organization the next week. For example, the Malaria Consortium could distribute AFFORD-funded mosquito nets in one district, and simultaneously distribute the same nets for other organization in a neighboring district (with all nets coming from government warehouses). The scope of this study did not allow for the collection of comprehensive information for each district to account for the direct influence of the AFFORD interventions. But even with such data, it would be difficult to attribute behavior change to one particular project or intervention. 24 Results This section presents the main findings of the AFFORD mid-term evaluation. First, general characteristics of the study population are presented. This information is followed by evidence of AFFORD‘s overarching achievements to date. The section concludes with specific results for AFFORD‘s three main objectives. Description of the Survey Respondents Based on the selection criteria, the team visited 12 districts with the respective numbers of respondents depicted in Table 4 below. Focus groups with POLs and community members were conducted in the four districts distinguished in bold. Table 4 Sample sizes for selected districts AFFORD Districts Comparison Districts District No. Respondents District No. Respondents Retail Store Exit Survey Retail Store Exit Survey Arua 12 62 Nebbi 12 41 Kabarole 12 77 Kamwenge 11 47 Mbarara 10 53 Ntungamo 13 44 Wakiso 12 59 Mpigi 12 48 Masaka 10 62 Rakai 12 69 Jinja 12 56 Iganga 12 59 Mbale 12 68 Budaka 12 76 Lira 11 74 Pader 11 57 Gulu 11 67 Oyam 12 68 Kampala 12 67 n/a n/a n/a Kiboga 12 61 n/a n/a n/a Total 131 706 Total 108 509 There were few differences between the retail store survey respondents in the intervention and comparison groups (see ANNEX D): 44 percent interviewed were the proprietors; 74 percent indicated that they had received specialized health or health care education without specifying the sources and type of education; and 96 percent managed the drug shop or pharmacy on a regular basis. One notable difference is that the AFFORD-targeted retailers had been in business longer than their comparison counterparts (6.6 and 5.2 years, respectively, p = 0.023). The average respondent in the client exit interview survey was 31 years-old with an average household of six persons, three of whom were children. The male/female ratio was 40:60, most likely because women go to clinics more frequently for services for themselves or children. There were some differences between AFFORD and comparison samples. The intervention group had a more balanced urban and rural mix (53 percent of respondents were urban and 47 percent rural) than the comparison group which was primarily a rural sample The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 25 (29 percent urban and 71 percent rural). As noted above, AFFORD purposely targets urban districts. Also noteworthy is that while the client exit survey comprises two populations— those exiting retail outlets and those exiting clinics—the sample is skewed toward the latter by design and more people were interviewed at clinics than in stores. Seventy-seven percent of all respondents in the client exit survey were found at clinics (78 percent were in clinics for the intervention group and 75 percent in the comparison group). Overarching Achievements The AFFORD initiative and its programs and products were visible in all regions visited and were well-recognized by target population groups. Ninety￾eight percent of drug stores and pharmacies visited sell at least one of the AFFORD/UHMG-marketed products, and 68 percent display advertisements for AFFORD/UHMG products or events. Among the retailers interviewed, 79 percent indicated that they saw, heard, or had participated in at least one of the AFFORD/UHMG shows or events.1 Additionally, 63 percent of client exit survey respondents indicated that they had heard, seen, or participated in at least one AFFORD/UHMG show or activity. This figure is higher for the AFFORD districts (66 percent) than for comparison districts (53 percent) and the difference is statistically significant (p = 0.004). Two overarching achievements were mentioned by most key informants as indicators of AFFORD‘s progress. First, the AFFORD/UHMG partnership, comprised a diverse group of organizations, including several Ugandan partners, that implements its activities seamlessly under the AFFORD brand. Contributions of partner organizations are recognized by local and central authorities first as contributions of the initiative and second as individual achievements. This equal partnership is a result of the prime contractor‘s strong commitment to collaboration and development of a recognizable brand for the initiative. For example, JHU-CCP set an example by substituting any JHU-CCP branding of activities with the AFFORD brand. The partners admitted that this approach contradicts the interests of individual organizations in brand building, but they said they recognized the important role of this approach for the initiative‘s development and results. The unified branding also contributes to the transition from the USAID project to the indigenous organization. However, contributing to the foundation of UHMG, this approach may diminish the advantages for other indigenous organizations, such as PULSE Communications or CDFU. 1 Herein the references to AFFORD/UHMG shows and events does not include viewing of HIV/AIDS films or LLIN distribution; since these AFFORD/UHMG activities can be easily confused with similar activities of other projects or the MOH. AFFORD brings together different partners with different expertise, and draws from the strength of each partner. This was unique from other projects. —AFFORD implementing partner From the inception of the project, MOH was in close collaboration with the project. Ministry participated in the development of the project strategies, in the trainings provided by AFFORD, design and placement of media information programs, etc. —MOH official 26 Second, the AFFORD initiative has developed a strong, transparent relationship with the Government of Uganda at both the central and district levels and is recognized by government officials for staying in contact and coordinating well with local government bodies. AFFORD is working with all relevant MOH divisions: HIV/AIDS, malaria prevention, health promotion, child health, and family planning. AFFORD is recognized by these divisions as an important counterpart and engages them in ongoing dialogue to align their activities with government plans. The project follows MOH policies and, through MOH, coordinates activities with other projects. These include: malaria prevention/nets distribution; the health materials, training guides, and message campaigns in HIV/AIDS and FP areas developed by AFFORD and approved by the MOH. The initiative is known by—and has cultivated relationships with—government officials at every district visited by the evaluation team. AFFORD‘s designated focal persons within district-level government, usually the District Health Educator (DHE), work with other district officials to involve them in major events and activities conducted by the initiative. Key informants, especially at the central level, attribute AFFORD‘s good relationship with government authorities to the strong leadership and excellent performance of the chief of party and his team. Progress toward Meeting AFFORD Objectives AFFORD has achieved significant progress towards all its main objectives during its first three years of implementation. Achievements were validated through results from the retail store survey and client exit survey, and supported by findings from the key informant interviews and FGDs. The following results are presented according to AFFORD‘s three project objectives. Objective 1: Increase the accessibility and affordability of HIV/AIDS, reproductive health, child health, and malaria prevention and treatment products and services for communities and families in Uganda through innovative private sector approaches. Availability of new products Currently, AFFORD is distributing 12 socially-marketed products and will begin distributing an additional product (a new brand of multivitamins that was not included in this evaluation) in the next few months. The 12 products include three already distributed by USAID: the Protector condom and PilPlan and Injectaplan contraceptives. Since September 2006, the initiative introduced nine other products to the market with prior approval of the Government of Uganda. Based on the results of the retail store survey, 97 percent of drug stores and pharmacies sell at least one AFFORD-marketed product; however, outlets in the AFFORD-targeted districts are more likely to sell these brands (see Table 5). No significant differences were found between AFFORD-supported and comparison outlets, [AFFORD] is an innovation for marketing healthcare to people. Previously there was not [any] marketing for healthcare. It is sending out message[s] in very fundamental way[s]. It is not just ―buy condoms‖, but more comprehensive and catchy. The messages were not simple instructions, but they explain [to] people how to live a good life through good health. —MOH staff The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 27 both for Pilplan and Injectaplan which have been on the market for a long time, as well as for the recently introduced products, NewFem and SoftSure. Table 5 Products sold in retail outlets by study samples in retail survey Type of Product Retail Outlets in AFFORD Districts % Retail Outlets in Comparison Districts % Difference % p-value (2-tailed) 95% Confidence Protector condoms 92 78 14 0.004* "O" condoms 24 7 17 0.000* Pilplan 93 88 5 0.181 Injectaplan 90 86 4 0.458 Moonbeads 18 7 11 0.007* NewFem 5 2 3 0.141 SoftSure 7 2 5 0.052 Restors 76 51 25 0.000* Zinkid 57 32 25 0.000* Aquasafe 47 32 15 0.017* Cotramox 29 12 17 0.001* Sold at least one AFFORD product 99 95 4 0.077 AFFORD/UHMG were able to introduce and register products in response to rapidly expanding markets. For example, 63 percent of drug retailers reported an increase in sales of anti-diarrheal drugs for children over the last year. AFFORD/UHMG is working in this fast￾growing market with recently-introduced products such as ZinKid and Restors, which are sold in 45 percent and 64 percent of retail stores visited, respectively. HIV/AIDS-related products are another fast-growing market, with 51 percent of AFFORD retailers reporting an increase in sales of HIV/AIDS-related products (5 percent reported a decrease and the remaining showed no changes). UHMG is present in this market with the Cotramox brand, and 21 percent of retailers reported selling this UHMG-distributed brand for the treatment of opportunistic infections in people living with HIV. Finally, 58 percent of retailers reported an increase in sales of vitamins (4 percent reported a decrease with the remainder unchanged). UHMG is developing a new brand for multivitamins to enter this market. Popularity of products and services PilPlan, Injectaplan, and Protector are among the most recognized and used AFFORD￾marketed products. Key informants indicated that demand is high in all districts visited for the most popular product, Injectaplan. The retail store survey indicated 89 percent of AFFORD drug stores and pharmacies sell Injectaplan. Since drug shops are small commercial enterprises with limited resources, they stock only products that can be easily sold. Therefore, the availability of this and other AFFORD/UHMG brands in such stores indicates a high level of popularity for this product. Key informants and store sales persons suggested that this figure could be even higher if stores were allowed to administer 28 injections.2 Interestingly, key informants and FGD participants claimed PilPlan is a less popular contraceptive than InjectaPlan since it requires discipline to take pills daily and is more visible to family members. Nevertheless it is the most available product—sold in almost 91percent of the drug stores and pharmacies visited. Protector condoms are well known but are not the most popular brand among those socially￾marketed in Uganda. Eighty-five percent of the retail stores sampled sell this brand. However, key informants indicated that Trust brand condoms (not an AFFORD-marketed product) are the most popular condoms and are consistently named most popular by store owners. Trust condoms and those condoms distributed free of charge by the Government of Uganda at government clinics have become more popular among Ugandans, with the latter providing competition to commercially-distributed brands. Only 61percent of surveyed drug retailers reported that sales of Protector condoms increased over the last year, while 77 percent reported an increase in sales of other brands, although not necessarily those of USAID. Issues related to the popularity of Protector demonstrate how complex consumer preferences can be. On one hand Protector, successfully introduced in the Ugandan market earlier than Trust, appears to have become synonymous with ―condom.‖ When purchasing condoms, people request ―a protector‖ when in fact, they want the ―blue ones‖—the Trust brand. Despite its strong brand recognition, however, Protector seems to be haunted by quality issues. Prior to implementation of AFFORD, despite meeting technical specifications, there were complaints of Protector’s unpleasant odor and inconsistent fit. AFFORD worked to ameliorate these problems and procures the Protector condoms from the one supplier who has the best and most consistent quality. It is hoped that this will improve its popularity. AFFORD also implemented activities to increase access to services. The most visible and recognizable has been introducing standards of services for small, private clinics, and uniting such clinics under the Good Life! clinic brand. In 2007, AFFORD selected and trained staff from 100 private clinics in providing HIV treatment and counseling as well as using AFFORD/UHMG products. This training also promoted positive health behavior and common standards of health care services. These clinics later were branded as Good Life! clinics. Staff of an additional 100 clinics covering ten districts were trained in 2008. These clinics continue to collaborate with AFFORD, distributing the AFFORD brands, participating in trainings and providing HIV testing and counseling to people referred by POLs. They are provided with a Good Life! logo and other branding products, and often supplied by UHMG/AFFORD-branded healthcare products for distribution. The FGDs demonstrated that the Good Life! clinic brand is recognized by people in the community who visit these clinics. The branding of these clinics allows AFFORD to achieve its goal of increased access to quality healthcare by the population because the brand provides additional information and assurance that a Good Life! clinic services are at or above the standards of other clinics. By contrast, local MOH authorities are not very familiar with the Good Life! clinics yet. AFFORD progress on Objective 1 is clearly reflected in the Performance Management Plan indicator annual measures compared to the baseline set at the beginning of the project. For 2 Note: The MOH prohibits drug store personnel from administering injections. However, many of them are unofficially performing this service for their clients. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 29 example, the indicator, estimated couple-year protection attributable to UHMG brands of family planning products, increased steadily from 2006–2008, exceeding its baseline target for 2010 of 575,000 by achieving a value of 584,578 in 2008. The percentage of prospective retail outlets carrying AFFORD products came close to meeting its target in 2007 (25 percent baseline; 23 percent actual value). Similarly, the value of the indicator, number of private retail outlets carrying AFFORD products, exceeded its baseline estimate of 25,000 in 2007 with 29,931. And, finally, AFFORD exceeded the baseline of the number of targeted condom service outlets (PEPFAR 5.1), in 2008—baseline 31,000; actual value 47,563. Targeting most-at-risk-populations In order to reach most-at-risk-populations (MARPs), AFFORD supported the establishment of a MARP network comprising 24 Ugandan non￾governmental organizations (NGOs) that work with specific at-risk groups such as commercial sex workers (CSWs), truck drivers, fishermen, and uniformed forces (military and police). Representatives of these organizations, which include Women at Work International (WAWI) and the Uganda AIDS Commission, suggested that their integration into the MARP network, along with training provided by AFFORD, allowed them to work more effectively. They exchange ideas and best practices and are able to collectively lobby their interests within the government. Notably, MOH informants recognized that using the term ―most-at-risk‖ allowed the government to start working with CSWs, which was previously impossible because prostitution is illegal and not recognized at the official level. AFFORD support to the MARP network also includes distribution of condoms to at￾risk populations in bars and video salons. Key informants pointed out that sales of condoms provided by AFFORD allows some CSWs who sell the condoms to generate a significant amount of additional income, which enables them to quit prostitution. In addition, the MARP network established several ―referral‖ clinics, where MARPs can receive HIV counseling and testing discreetly. AFFORD provided test materials and training to the clinics and mobile testing centers operated by WAWI. This approach led to testing of a population that was previously deterred from seeking services. The biggest credit for AFFORD goes that they started working with the groups of population that [were] not covered before. This is the only agency that started to work with commercial sex workers, and this really opened this line of work within ministry, commission, etc. —Representative of AFFORD partner organization In some remote areas, sex workers are difficult to help. Like in Hima which had many people with HIV/AIDS, the sex workers …are not easily identified among population...with support from AFFORD we identified several sex workers one-by￾one and trained/educated them to be a peer-to-peer educators.... As the results, sex workers started using condoms. Thanks to AFFORD the condoms became affordable and sex workers are buying and using them. —Representative of a MARP network organization 30 Objective 2: Enhance knowledge and correct use of HIV/AIDS, reproductive health, child survival, and malaria products and services to encourage and sustain healthy behavior and lifestyles within communities and families. Interventions to enhance awareness of products and services AFFORD conducted several health promotion campaigns during the period October 2005 to October 2008 (see ANNEX A). At the time of data collection, most of these had finished, and the initiative does not plan to renew these campaigns during its remaining years. These campaigns include: Good Life! show on television and radio and related activities such as prize drawings under the brand of the Good Life! show; Under the Mango Tree radio show; PULSE activation campaigns and HIV/AIDS films and other radio spots. AFFORD is continuing its POL program and Everyday’s Health Matters newspaper, but both are planning to phase out soon. Finally, AFFORD and UHMG are planning to start direct advertisement campaigns on television and radio for the products distributed by UHMG. The health promotion campaigns are highly visible and recognized in areas where they were available. For example, the Good Life! show on television was very popular in the Central Region that has high television coverage, and the initial broadcast in 2007 was re-aired in 2008 with great success. About 63 percent of respondents in Kampala and Wakiso Districts indicated that they had seen the Good Life! show on television. However, this show is not widely known outside the Central Region due to the low availability of television and also because the show was not translated into local languages. The proportion of client exit survey respondents in other districts who indicated that they watched the show varied widely from as low as 1 percent to as high as 38 percent. However, these districts more evenly received the radio version of the Good Life! show. In nine districts, over 50 percent of the survey respondents indicated that they heard the Good Life! show on the radio. Under the Mango Tree radio shows are also popular in the districts outside Kampala because they were aired in local languages by local radio stations. In northern districts of Uganda, 59 to 78 percent of respondents indicated that they listened to this show. Some district officials suggested that if district communications specialists received training on how to produce such shows, they would likely continue production of similar shows on their own. Finally, statements by some distributors of UHMG products indicate the PULSE activation and related promotion events greatly increase demand for health products in days following a show—especially UHMG brands. The retail store survey revealed that respondents noticed an increase in demand for AFFORD/UHMG-marketed products after the promotion campaigns. Among those retailers who heard or participated in meetings with community leaders to discuss health-related issues with AFFORD/UHMG, 73 percent said they observed increased demand for the products. The PULSE activation reported increased demand for the products from 44 percent of retailers who heard or participated in the show. However, since these activities had local scope, the number of stores who reported increased demand in the overall sample of the survey was low. UHMG is doing very well in terms of providing messages, and this messages should be kept in local languages, especially the health seeking behavior. They should make sure that this continues on radio and in signs with a reach to rural areas. This is an important service that they are providing. —District MOH official The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 31 Because of the intensive media campaign and sensitization, many youths are now using condoms. Every once in a while adverts are played on radios reminding people to mind their lives, to be responsible—to use condoms. —Focus group participant A larger effect was observed for the nation-wide campaigns. A total of 61 percent of retailers who heard the Good Life! show on radio and 35 percent of those who saw the show on TV reported increased demand for the products after the show. These stores constitute 35 percent and 10 percent of the overall survey sample, respectively. Please refer to ANNEX N for more detailed information. The POL program is one of the most successful AFFORD programs and is in great demand by district officials. In 2007, AFFORD engaged an outside organization to evaluate the POLs performance (Wilsken Agencies, Ltd 2007). The report found that the POLs were accepted and trusted by community members, who noted that their health behavior changed for the better as they increased their purchasing of healthy products. Under the current evaluation, key informants, especially district representatives, provided positive feedback on the POL program and expressed interest in extending it in their districts. The Government of Uganda has a similar structure in place already called Village Health Teams (VHT). The VHT program, which is currently being rolled out in every sub-county in every district, is designed to promote good health in the community through the actions of local representatives. District health officials view the two programs as complementary. During outreach planning, when health officials roll out the VHT structure, they take into account the existence of a POL program. The Government of Uganda has confidence in the AFFORD-trained POLs and chooses to use them for the core of the VHT program. AFFORD staff meet with POLs quarterly and monitor their activity forms. This simple form records the nature of POLs‘ interactions with groups and individuals, noting such things as topics discussed and questions asked and answered. The forms help AFFORD learn more about demand from the community and how to modify future training and job aids. Use of products and services and effect on health behavior The respondents of the retail store survey indicated market growth for the categories of health products distributed by AFFORD—although not necessarily for the AFFORD brands. Survey results indicate that, depending on the category of the product, between 52 percent and 77 percent of stores experienced an increase in sales over the last year. During that same time period between 4 percent to16 percent of stores reported a decrease in sales of the same product categories (see ANNEX N). As discussed in the previous subsection, the same survey suggests that AFFORD health marketing campaigns increased demand for health products, at least temporarily. It should be noted that reported increases were subjective opinions of retailers, and the amounts of the increases (i.e., numbers of extra products sold) could not be quantified from the survey results. There are also cases when POLs helped community members to change negative cultural behaviors. For example, [a] person gave a birth in the village, and according to the culture she supposes to participate in a ritual in four days. However, she had serious complications from the birth, and POL insisted that she got to hospital [and she did]. —Focus group participant 32 Table 6 provides views of clients exiting clinics and retail shops regarding improvements in their health behavior during the past three years. The most improved behavior came in the use of mosquito nets, visits to health clinics, pharmacies and drug shops, and treating water. Among the eight products and services queried, only three were more likely to have improved in the AFFORD districts compared to comparison districts: mosquito nets, water treatment, and the use of condoms to prevent HIV. Table 6 Reported increased use of products and services by clients Question: Over the last three years, did your personal behavior change (increased use) in regard to the use of… Product or Service AFFORD District % Comparison District % Difference % p-value 95% Confidence mosquito nets 61 55 6 0.0373* water treatment for drinking water (through boiling or tablets) 52 42 10 0.0007* vitamins and mineral supplements 20 17 3 0.2343 condoms for family planning 18 14 4 0.0987 pills, injections or other contraceptives 15 12 3 0.1432 condoms or other products to prevent HIV 29 21 8 0.0028* visits to health clinics 55 53 2 0.6564 visits to pharmacies and drug shops 45 45 0 0.9603 Conversations with key informants supported the findings of the survey about increased use of condoms. District MOH officials suggested that people quickly started choosing free condoms from government clinics as soon as they were available. Previously, the same stock would have remained on clinic shelves for months at a time because of low demand. Now, despite the steady supply of condoms from the government, these supplies are gone within days of receipt because of high demand. Although increased demand for the free condoms does not necessarily imply increased usage for family planning or prevention of HIV and other sexually transmitted infections in conjunction with the results of exit survey, it does provide additional evidence for the behavior change with regard to condom use. Besides distribution of socially-marketed drugs, AFFORD participated in distribution of LLINs. More than 577,000 nets were distributed in 118 sub-counties in 2007 and more than 453,000 were distributed in 100 sub-counties in 2008. LLIN distribution is a highly visible activity of AFFORD, which is recognized by district authorities. These authorities report they observed a corresponding reduction of malaria cases in sub-counties where nets were distributed and requested further net distribution to the sub-counties not covered by previous efforts. Several indicators suggest overall improvements in health status and positive health behavior in Uganda that coincide with the three years of AFFORD‘s implementation. While this study cannot definitively determine a causal relationship, evidence suggests that AFFORD activities may have played a part in stimulating positive changes in health behavior. Key The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 33 informants in the districts outside Kampala noted that health conditions had notably improved in the last three years, especially in terms of people suffering from malaria and HIV/AIDS. Childhood illnesses were reduced and the use of family planning increased. District health authorities also provided anecdotal evidence that distribution of LLINs changed health behavior in families with children. In a significant number of cases, nets are used to protect the youngest children in families. About 59 percent of the client exit survey respondents indicated that they increased use of the mosquito nets, and 51 percent indicated an increase of the mosquito net use by family members. Focus groups and key informant interviews provided anecdotal evidence that POLs positively influence health behavior. In particular, people became more aware of specific products and health behavior habits such as purifying water, either by boiling or with tablets, giving birth at health facilities, and receiving HIV counseling and testing. Yet only 8 percent of the client exit survey participants indicated that they participated in meetings with community leaders organized by AFFORD, which are most probably the meetings led by POLs. Key informants from the MARPs network reported that AFFORD has had a positive influence on its target populations. In particular, they indicated that CSWs and other at-risk groups have started using condoms more often, and even learned to demand condoms from partners or to market sex without condoms as a more expensive service. AFFORD also facilitated access to HIV testing that was previously unavailable because of fear of legal prosecution for engaging in prostitution. Since fear of prosecution is still a significant concern among CSWs and the doctors providing services, information about uptake of services was not recorded during this study. The surveys also provide some evidence of the impact of AFFORD activities on behavior. For example, 66 percent of respondents who heard or saw at least one of the AFFORD electronic media shows said they had increased use of mosquito nets, compared with only 33 percent among those who did not see or hear the shows. In general, the percent of positive changes in behavior is significantly higher than the percent of negative change. And among those respondents to the client exit survey who indicated either positive or negative change in their health behavior, 55 to 76 percent had heard or participated in at least one of the AFFORD health promotion activities. However, this evidence is only suggestive. Attribution of impact to AFFORD would require a more complex study that can statistically exclude the impact of other public health campaigns that are ongoing in Uganda. AFFORD performance measures related to Objective 2 illustrate the initiative‘s good progress. For example, AFFORD met the 19 percent target in 2007 for percentage of women in reproductive age using pills, injectables, condoms. Over 20 percent of persons used condoms for family planning: in AFFORD districts, the upper bound percentage for pills or injectable use was 34 percent compared to 26 percent for control districts. AFFORD reached its 2008 target, 100, of number of service outlets providing counseling and testing according to national or international standards (PEPFAR 9.1). By 2008, AFFORD had reached 50 percent of its target for number of individuals who received counseling and testing for HIV and received their test results, disaggregated by sex (PEPFAR 9.2): 26,706 out of 40,000 targeted by 2010. Finally, AFFORD already exceeded its life of project target of 44 million litres of water made safe to drink with AquaSafe with an impressive 155 million litres. 34 Objective 3: Strengthen and establish an indigenous organization and distribution systems for the sustainable and self-sufficient delivery of key health marketing functions, including management, distribution, and promotion. At the end of the first year of implementation, AFFORD registered and established UHMG as a Ugandan organization. UHMG continues to evolve and develop but already acts as a full￾scale partner of the AFFORD initiative. UHMG has gradually taken over social marketing activities and has introduced more products to the market. These products are introduced at prices that allow full recovery of the product, packaging, and distribution costs, meaning that the foundation for the sustainability of UHMG is already in place. However, this organization needs further development and support if it is to survive on its own. Establishment of UHMG AFFORD took early, critical steps to achieve its third objective. The UHMG was officially registered in October 2006 and has a statute, work plan, operations manuals, and other formal documents developed in consultation with international experts that meet national requirements. UHMG has an active board of directors currently comprising Ugandan nationals representing the directors and managers of the AFFORD partner organizations that established UHMG. UHMG also has a managing director, Dr. Magumba, who has worked with the organization for over two years, and four directors –director of finance and administration, director of programs and services, director of operations, and director of research and monitoring. The director of research and monitoring position was not filled at the time of the evaluation. UHMG also employs 37 staff members (see ANNEX B) that were transferred from AFFORD. Transition of social marketing activities The current AFFORD strategy is to gradually transfer all social marketing activities to UHMG. The project envisions that the UHMG will assume such social marketing functions on the AFFORD project as product development, distribution, and promotion, as well as research and evaluation. The current project plan is to transfer all social marketing activities to UHMG or other local partners, and to use some of the international experts as part-time advisors to these organizations. AFFORD is planning to implement this transition by fall 2009. However, key informants admit that the current capacity of the UHMG staff is insufficient to fully replace support from international experts and that further training and close collaboration with international experts is still necessary to maintain the quality of the work. Key informants suggested that for an indigenous organization to be viable, it needs individuals in top management positions who are not only qualified to lead the organization, but who are also determined to stay with the organization through the early, fragile stages of development—even if there is a risk of a decrease in their income. The interviews with AFFORD’s creation of UHMG was exceptional. Many projects have been initiated and ended like AIMS and UPHOLD but no project had such an initiative…. Projects like [AIM and UPHOLD] started, work some time, and then abruptly end. The only thing that is left is the Land Rover they gave to UNESCO. [AFFORD represents] good practice for building indigenous organizational capacity. —MOH Staff The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 35 UHMG‘s current top management suggest that these individuals have extensive experience working with international projects and had previously moved from project to project every two to three years. All of them admitted that what attracted them to work with UHMG was the possibility of a much more stable job, even if it offered a lower income. Steps toward sustainability UHMG has taken notable steps to secure additional sources of funding and to improve its financial management capabilities. The Director of UHMG is involved in the development of several project proposals to USAID, and UHMG is currently a local partner for two other USAID-funded projects. Still, most of the business development activities, including the participation in these two projects, were conducted under close supervision and in collaboration with the current international partners of the AFFORD project. Though most attention is devoted to the USAID projects and related financial management, UHMG management is also meeting with other donor organizations to explore funding opportunities. UHMG commissioned a strengths, weaknesses, opportunities, and threats (SWOT) broad analysis by an independent evaluator and currently is undergoing a process to establish a negotiated indirect cost recovery agreement (NICRA) with USAID. UHMG has also started developing a medium-term financial plan. AFFORD‘s work with the private sector to develop its distribution network is widely and favorably recognized by key informants as a unique and promising practice. This distribution network has been transferred under the UHMG management to provide a foundation for sustainability of the organization‘s distribution activities. UHMG provides products to 12 regional distributors, which are assigned districts by AFFORD, although their distribution is not limited to these districts. The distributors store products in the districts, and sell them to retailers at the prices recommended by AFFORD. In return, they receive a set margin to generate profits. Larger retailers also sell products to small retailers, such as drug shops and to individual customers. Finally, small marketing entrepreneurs—individuals who receive drugs from AFFORD distributors or large retailers—sell them in remote villages where the drugs are not otherwise distributed using AFFORD-branded bicycles. Although the distributors are assigned to particular regions, they are not limited to distribute products only in the districts of the assigned region. Distributors also work in districts not targeted by AFFORD as well as in the targeted districts. However, in the AFFORD-targeted districts the project refers retailers to the distributors as a part of other project activities. About 79 percent of respondents to the retail store survey in AFFORD-targeted districts and One of the major decisions/lesson learned was to begin to visualize the indigenous organization from the day one. When recruiting people, we were looking for people who will stay with the organization after the project ends. —AFFORD senior staff [Having the] Initiative go to private sector is important because in Uganda, people try to seek help in the private sector before going to government. —Partner staff AFFORD is unique because it is targeting private sector. Other organizations are targeting the public sector, and only on a limited level [are they targeting] the private sector. —MOH official 36 about 54 percent of respondents in comparison districts indicated that the drug shop/pharmacy received supplies from an AFFORD/UHMG distributor. Conversations with distributors and drug shop owners also revealed several weaknesses with distribution. First, there is poor communication between shop owners and distributors. The distributors do not survey the needs of the shops before bringing drugs to a specific area. As a result, there are situations when a distributor cannot satisfy the needs of store owners for a particular product, although this product is available in the warehouse. Another problem resulting from this lack of coordination is that store owners may spend financial resources on other products before a distributor comes to the shop, although store owners may have preferred to wait a day and use these resources to purchase the AFFORD products. There is also limited awareness among shop owners about the AFFORD distribution network. A lot of small drug shop owners purchase their supplies from local shops and pharmacies instead of contacting distributors directly because they do not have contact information for the distributors. Finally, the store owners complained about their limited basic knowledge and information about AFFORD products, especially side effects and advantages of contraceptives such as SoftSure, NewFem, and Pilplan. They noted that information provided with the products is too short and too general. They requested more specific information in order to provide educated recommendations of these products to their customers. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 37 Discussion This section discusses the findings of the AFFORD mid-term evaluation to: a) validate results; b) identify effectiveness and further potential of activities; and c) assess the future sustainability of UHMG. Framework for the Discussion There are two main approaches to social marketing recognized by researchers and practitioners: the NGO model and manufacturers‘ model (Armand 2003). Under the NGO model, an organization takes on the functions of branding and distributing products provided by manufactures or donors, often by creating its own distribution network and retail outlets. This limits the organizations‘ control over the cost of the products; and often makes the distribution of the products unsustainable without donor funding. The manufacturers‘ model implies that social marketing is conducted in partnership with manufacturers of the products, who are often responsible for branding and distribution of the product. The advantage of this approach is that the manufacturer has control of costs and usually sets the price of products above the cost to ensure sustainability. A disadvantage is that often, after completion of a project, manufacturers decide to increase the price of the product which negatively affects distribution and coverage. Development of social marketing concepts under financial limitations of government and donor funding resulted in the development of hybrid models, such as the total market approach to social marketing (Pollard 2007). Total marketing intends to reduce the financial burden of achieving a social goal by increasing targets for free or subsidized products to the poorest segments of the population while increasing the proportion of the population that buys the products on the commercial market. The approach is to introduce specialized, low￾cost, mass-produced products on the market specifically targeted to the low-income groups that cannot afford products currently available on the market. At the same time, these commercially distributed products should be more attractive than the free or subsidized versions. Social marketing is used to create an increased demand for these and other commercially available products on the market. The marketing campaigns are supplemented by efforts to create supply and distribution systems for the low-cost products. The success and sustainability of the total market approach depends on several factors, some of which are not necessarily under control of social marketing projects: 1. There should be sufficiently large, unsatisfied demand for low-priced, socially￾marketed products, so the high-volume of the market can compensate for the low profit margin on these products. 2. The existing market organizations (public and private) have logistic and management capacities, both to handle the volume and provide low-cost manufacturing and distribution of these products. 3. The commercial market should be willing to expand into the low-income population segment with low-priced products. One of the conditions to achieve this goal would 38 be that the low-priced products do not enter the high-profit segments and do not substitute the currently available high-profit products. Validating Project Results AFFORD is a social marketing initiative that includes all three major goals of social marketing in line with the total market approach discussed above. Objective 1 of the AFFORD initiative—increased availability and access to the products—corresponds to the goal of delivering affordable health products to low-income population. Objective 2 of AFFORD was designed to achieve the social goal of increased use of the health products and adoption of positive health behavior. Finally, Objective 3 of AFFORD aims for sustainability of the initiative. It was designed to sustain the social marketing achievements by coupling them with commercial groups in a total marketing approach. Below is a discussion of the current project results which demonstrate progress towards achieving its objectives. Maximizing product sales and delivery AFFORD is clearly on the way to successfully achieving Objective 1 of the project, as well as one of the main components of the social marketing framework, which is to increase availability of low-cost products to the population. The pure numbers indicate that sales of USAID socially-marketed brands (Pilplan, Injectaplan, and Protector) are increasing (Table 7). Protector sales started in 1993 with volume of about 1.8 million condoms sold. At year three of the AFFORD initiative sales reached 17.4 million. Similarly, Pilplan started in 1993 with sales of 66,000 packs. In 2002 (during the implementation of the predecessor of AFFORD) sales were about 970,000; and in year three of AFFORD project, the sales volume is now above 1.5 million. Finally, Injectaplan was introduced in 1996 with sales volumes of 4,000 packs. By 2002, sales volume had hit 540,000, and by the third year of AFFORD implementation over 1.7 million packs had been sold. These brands also are now widely available on the market—sold in 85 percent to 91 percent of private drug stores in the counties surveyed, depending on the product. This important result, however, is a success that can only be partly credited to AFFORD. The studies conducted by AFFORD‘s predecessor showed, for example, that in 2000 Protector condoms were available in 91 percent of pharmacies and 86 percent of drug shops (Commercial Market Strategies 2003). Only Injectaplan demonstrated an impressive 110 percent growth in sales between year one and year three of the project. Table 7 Comparison of USAID brands availability and sales Product Availability in Retail Stores Before AFFORD (2000) % At 2009 Mid-term Evaluation % Sales Volume Before AFFORD Sales Volume After AFFORD (2009 Evaluation) Pilplan n/a 91 970,000 (2002) 1.5m Injectaplan n/a 88 540,000 (2002) 1.7m Protector 86 85 1.8m (1993) 17.4m The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 39 One of the reasons for these results may be saturation of the market with condoms targeted at the lowest-income population group. The saturation of this market segment would imply that any further growth of the sales for Pilplan and Protector is only possible through substitution of other socially-marketed brands. The problem of competition between socially-marketed brands of health products distributed by different donor campaigns remains unresolved in Uganda. In their study in 2003, the Commercial Market Strategies project in Uganda found that the Protector and Lifeguard condoms (distributed by Marie Stopes) traded market shares. Since 2006, in addition to these two brands, there was a Trust brand introduced to the market. Provision of socially-marketed brands by a number of donors creates a large supply and great availability of the products. However, while one brand may command the highest market share, the total market (and, therefore, the total use of these products) is unlikely to change. Although a goal of social marketing is to increase demand for all socially important products, (and, therefore, for all brands of low-cost condoms) the basic marketing principles still apply. The necessity to achieve distribution goals for a specific brand requires more resources to protect the existing market share. As a result, fewer resources remain for market expansion. Therefore, the saturation of the market with multiple products leads to inefficient use of donor resources unless a concurrent effort is made to increase the size of the total market. In this sense, the AFFORD model is ideal because it combines behavior change communication alongside efforts to boost supply. It should be noted that widespread, sustained distribution of free condoms can destroy the commercial market for products in the same market segment. If that happens it may become impossible to achieve long-term sustainability of a product market without continuing external funding of subsidized brands. A significant achievement of the AFFORD initiative is the introduction of nine new brands of health products. These products are targeted at an income group slightly above the poorest population. This is an important step that allows a division of the low-income market into two separate parts: one that is capable of paying at least the at-cost price and another that is not. This differentiation is important for the introduction of the total market model because it can lead to commercial sustainability of at least part of the market. As mentioned earlier, the important element to ensure such sustainability is the size of the market segment that can pay at cost. Influencing a target population to achieve a social goal Objective 2 is the core of AFFORD‘s social marketing component. The initiative introduced and completed a variety of health promotion programs and activities in order to achieve increased use and adoption of healthy behavior. A number of these activities, such as the Good Life! show on television and radio, Under the Mango Tree radio show, and HIV/AIDS films, are well-recognized by the population and were listened to and watched by significantly-sized audiences. Other activities, such as the PULSE activation, and, especially, the POL program, are less well-known to the general population because of their local nature. However, these activities are also regarded as highly effective and innovative by district authorities and community members in the areas where they were implemented. At the time of evaluation, most of the health promotion activities connected to the initiative had been completed and there were plans to concentrate on product advertising for the remaining life of the project. Therefore, activities supporting Objective 2 can be considered almost completed at this point. At this early point in the initiative, the evaluation could not determine the impact of these activities on health behavior. The market response to the social marketing 40 campaigns of AFFORD is reported by some drug shop owners to have increased the number of customers for AFFORD products. However, the behavior change or stable market response could only be verified and become visible more than a year after campaigns end. There is some indication of a relationship between the AFFORD activities and positive changes in the health behavior. It is important that—as noted above—AFFORD experienced limitations inherent to all social marketing projects, especially ones that are based on the NGO model. Most of the successful health promotion campaigns implemented by AFFORD cannot be sustained without external funding. As described earlier, the most highly regarded health promotion campaigns are the Good Life! show and POL program. These programs also are very costly. The television time to run the Good Life! show and the production costs are the most expensive parts of the promotion campaign. If these costs are distributed onto the socially-marketed products in the total market model, it could put the products out of reach of the target low-income group. There is a similar dilemma regarding the POL program. By design, the POL program has very limited geographical impact since each POL covers only a small catchment area. Further, the POL program is dependent upon the motivation of each volunteer and material support (through free drugs, raincoats, bikes) by the initiative. Although the cost of a single POL is relatively low, the program could have significant influence on the health behavior of the population and reach its social goal only if current pilot activities are scaled-up though the country. There is an expectation from local health officials and NGO partners that UHMG will continue health promotion activities after the completion of AFFORD. Unfortunately, due to the nature of these activities and absence of government funding, it seems unlikely that they can be conducted by local organizations without external financing from donor organizations. Sustainability of social marketing results At the time of the evaluation, AFFORD also was on its way to achieving Objective 3: creating a sustainable, indigenous social marketing organization. UHMG was created, registered, and has been functioning for at least two years. AFFORD is gradually transferring staff and responsibilities for social marketing activities to UHMG. It has also adopted a model to transform the international experts (who started AFFORD) into short-term UHMG consultants in order to facilitate further capacity building. Key informants—including MOH officials, USAID, NGOs, international partners, AFFORD/UHMG partners—indicated that AFFORD has already achieved more in creating an indigenous organization than other, similar international projects have achieved at this point in their life cycle. In the opinion of these informants, the early establishment of UHMG and the steps taken to transfer responsibility for key organizational functions and activities is one of the promising practices introduced by AFFORD. The expressed willingness of senior managers to stay with UHMG despite lower salaries demonstrates their strong commitment to the organization. Such commitment, if fulfilled, is necessary but insufficient on its own to ensure sustainability. And while current UHMG staff was trained and obtained experience in social marketing, they lack skills in fundraising and organizational management and development—all essential skills for the sustainability of the organization. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 41 Objective 3 of the program is clearly aimed at creating a commercially-sustainable structure according to the total market model discussed above. As mentioned, AFFORD introduced nine new products that allow market segmentation into two groups, those able to pay and those who can‘t. Another important component for this model‘s success is the existence of a sales and distribution network that can handle the volume of the products necessary for sustainability. An operational distribution network that does not require significant financial support and therefore, should continue to function independently after AFFORD‘s work ends. Operations management for product development, marketing, and distribution has already been transferred to UHMG. For all brands, except USAID-supported brands, operations are conducted on a cost-recovery basis. Although the private distribution network has an incentive to support the distribution of a product if there is commercial interest, observations by the evaluation team showed that current distributors and store owners are not sophisticated in business methods and often fail to establish efficient communications with each other for the purpose of exchanging information about current needs for supply of particular product. As a result, the current profit margin of the socially-marketed products is sufficient for distributors as long as there is external support for cultivating new clients, promoting products, and ensuing consistent stock. In part, this is the expected result for a total market model at the early stages, when the at-cost market is just developing. As mentioned earlier, an important condition for the sustainability of the at-cost market segment is sufficient size of the market. The nine new products for this market segment were introduced by UHMG only one or two years ago and currently these products are present in about one-fourth of the drug stores surveyed. Still, this reach may not be sufficient for sustainability. At the moment, UHMG/AFFORD plays a significant role developing the market, in bringing new customers to product distributors and informing distributors about the need to bring products to stores. Because stores do not call distributors for new batches of products and distributors do not regularly survey stores for their needs, the sustainability of the distribution network in the absence of AFFORD is questionable. Key informants, and UHMG distributors in particular, indicated that there are also some operational problems with product distribution, such as low quality of packaging, incorrect invoices, and poor coordination between retailers and distributors. However, these are minor problems that UHMG is well-equipped to handle and the organization already is taking steps to resolve some of them, including changes in packaging technology and the templates for the receipts. Therefore, although the initiative achieved admirable progress towards the third objective, there is still significant work to be done in the next two years to ensure that UHMG has sufficient institutional capacity to be sustainable. Effectiveness of AFFORD Activities At this point of implementation, it is too early to expect measureable impact or observed results to specific AFFORD activities. Yet, there are some indications from key informants that AFFORD activities are effective. Here, we look at the effectiveness of AFFORD from two perspectives: first, the effectiveness as a USAID project and second, the effectiveness of specific project activities. 42 Key informants suggested that the AFFORD team is functioning effectively and efficiently. One of the indicators noted by these informants was the absence of any visible conflict between implementation partners. Uncoordinated activities and compartmentalized project work in which each partner implements assigned activities as separate project are common problems observed in unsuccessful projects. There is no evidence that this is happening with AFFORD. From direct observation and interviews with project employees and partners, it seems that different implementation partners share information and coordinate activities well within the organization and with external stakeholders. Conversations with stakeholders suggest that the initiative‘s success could be attributed to the professional skills and strong leadership of AFFORD. The choice of strong and well￾qualified leadership is one of the AFFORD best practices. The leadership style was essential to developing and maintaining good relations with the central and local government, as mentioned by the key informants. Good leadership was instrumental in uniting all AFFORD partners around the idea of branding all initiative activities as AFFORD rather than as activities of individual partners. AFFORD leadership identified and involved other partners, such as the NGOs of the MARP network, with relevant expertise. Finally, good leadership was the impetus behind the early development of UHMG. There are also indications that AFFORD is effective at social marketing. The data collected during the evaluation suggests that health promotion activities reached up to 30 percent or more of the intended audiences (see ANNEX N), and there is evidence of a relationship between these activities and positive changes in health behavior of the population. If we assume that drug shops are small commercial enterprises that can only carry the products that they can sell, then the availability of the products in the shops also indicates the demand for AFFORD products. At least one of the AFFORD products is available in 99 percent of the drug shops in target districts, and 95 percent in comparison districts (see Table 4). This evidence suggests that the social marketing communication campaign was successful in creating at least temporary demand for these products. One of the reasons for this success may be the fact that AFFORD went beyond the ―standard‖ practices for social marketing that relies on use of multimedia campaigns. AFFORD used complementary communication techniques to reach different audiences: The Good Life! show on television had a high success rating. It engaged people and stimulated their interest in health questions posed on the program. It is a multimedia technique that uses an unusual format of a game-show to draw viewers and stand out from other programs and advertisements on television. Yet it is only accessible to those who are English-speaking and live in areas that have access to Ugandan television programs. The POLs program uses a completely different format to extend health messages into rural areas. It is highly regarded by district officials, and there is evidence it has positively influenced health behaviors. Unfortunately, because each POL has a very limited catchment area and the program was implemented in only several districts, the overall marketing effect of the program was small. Only 8 percent of client exit participants indicated that they participated in meetings with POLs. However, the POL activities as a pilot for a new approach to social marketing demonstrated strong potential if implemented on a larger scale. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 43 Promising and best practices The results of the evaluation and opinions of the key informants who are experts in social marketing suggest that AFFORD was successful in developing and implementing promising and best practices for the development of a sustainable hybrid social marketing model based on total market approach. These practices include: Reliance on private distributors and distribution network for sustainability: One of the successful features of AFFORD is its reliance on the private sector for distribution. Key informants remarked that social marketing projects in Uganda and elsewhere usually develop a new NGO-owned distribution network and cooperate with government health facilities. However, officials of the Government of Uganda underscored that reliance on private distributors, and especially private clinics, allows AFFORD to tap into new areas previously not covered by assistance programs. This approach avoids overstretching resources of government clinics and builds a broader base for development of the health sector. AFFORD products are distributed through a self-sustaining network of private distributors that does not require financial support from the initiative. This is an efficient low-cost method for distributing USAID-supported socially-marketed products. If the UHMG products expand the market sufficiently enough to become self-sustaining, this distribution network also would be an efficient, low-cost method for getting these products to market, too. Proactive introduction of new health products onto the market: AFFORD is often complimented for its proactive role in introducing new products to the market. For example, it assisted the government in developing MOH policies that introduced a new type of oral rehydration salt in lieu of an older, less-effective product. AFFORD also introduced products for middle-income customers not historically targeted by social marketing projects, such as the ―O‖ condom. These products closed an existing gap by enabling customers who have the capacity to pay for products, but could not afford high-end products, to purchase these subsidized items. Sustainability of UHMG Sustainability is an important component of a hybrid social marketing concept. However, the AFFORD initiative also has a specific deliverable of developing an indigenous NGO oriented to social marketing and product delivery. By definition, this is an operational NGO, which has quite complex organizational structure compared to a simpler advocacy NGO. An operations NGO has to be able to mobilize financial and labor resources in order to implement program activities defined by its charter of incorporation. In the case of a social marketing NGO, such activities are the distribution of socially-valuable products and implementation of marketing campaigns both for these products and for behavioral change in general. Therefore, the social marketing NGO needs general capacities to prepare applications, to obtain grants or contracts from government or other donors, and to conduct budgeting, accounting, and reporting on these funds. In addition, it requires capacities specific to its area of expertise: market research, social marketing campaigns, and product acquisition and delivery. Therefore, the operational NGO has to possess a headquarters with qualified staff, and, in the case of a social marketing NGO, a network of local representatives or partners. 44 Establishment of an NGO runs through several stages. These include: creation and registration; development of key personnel, developing policies and bureaucracy/management capacities; establishing core activities and developing capacities and staff to deliver the core activities; establishing fundraising and financial management capacities; managing on-going fundraising and project/activities implementation; and developing an image and maintaining relations with donors and government. The initiative already established an indigenous NGO, UHMG, which has already passed the creation and registration stages, as evidenced by the policies and management capacity it developed. The NGO benefitted from early establishment at the outset of AFFORD. These actions are in line with the best practices of international projects in other countries, and experience suggests that early creation of a local entity is a strong step to ensure its sustainability at the end of the project. Because of the early establishment, the UHMG inherited staff and experience in delivering core social marketing activities as well as product delivery network from AFFORD, and was able to conduct capacity building on the delivery of these core activities over substantial time period during AFFORD implementation. The organization also started developing fundraising and financial management capacities, however the development of these components could be slowed because the fundraising activities do not coincide with direct activities of the AFFORD project. The mid-term evaluation corroborated the findings of a SWOT analysis conducted by AFFORD, particularly in terms of weaknesses that require redress. At the moment a major challenge to the sustainability of UHMG is the limited experience of the management in fundraising, project and institutional development. AFFORD and its international partners are aware of this weakness and are taking steps to address it. It is clear from the initiative‘s performance measures that AFFORD has made steady progress on indicators that support the intermediate result 3.1; establishment and operationalization of UHMG and the increased operational capacity of UHMG by strengthening its partners. For example, as mentioned above, AFFORD is legally established with a certificate of incorporation and business plan. In addition, almost all of the key staff positions are filled. They have met intermediate results 3.2 targets for number of staff in partner organizations trained in financial management (8 in 2007), number of partners that include M&E findings in their periodic reports (4 in 2007; UHMG records), and have almost met number of local organizations provided with TA for HIV-related institutional capacity building (meeting 6 of 8 target for 2008). Challenges There are several challenges specific to NGOs in developing countries. They are primarily a result of limited capability of a country‘s economy to support operational NGOs. Scarce financial resources mean that the salary levels for highly qualified staff at indigenous NGOs may be substantially lower than those offered in the commercial sector. The difference rises with the level of expertise, and will be especially high for the top management. Therefore, the motivation of top management is an important factor to compensate for lost income. Second, the motivation for the social marketing programs in the developing countries is often external, coming from people in more industrialized countries. Therefore, sustained support from the developed countries is important to keep an NGO‘s motivation strong. Finally, The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 45 domestic sources of funding, either from the government or private sector, are rarely available in developing countries. Operational NGOs in such countries must therefore rely on external donors for funding their activities. Because of the relatively low cost of operation for an indigenous NGO, a single donor or single project often can sustain such NGO activities. However, the funding streams coming from a single donor are highly volatile and if the NGO relies on a single source of funding, it risks losing most of its qualified staff and capacities in tough financial times when funding from this single source is put on hold. Therefore, diversification of the funding sources and fundraising efforts are highly desirable for an NGO in developing country. Based on the above-mentioned general hurdles for NGOs, the team identifies several outstanding challenges to UHMG‘s sustainability beyond the initiative: USAID as the single source of funding and expectations for continuation of funding: Because UHMG was established with the help of a USAID-funded initiative, there is a natural bias in the organization toward continued reliance on USAID. Most international experts providing capacity building to the organization, including fundraising advice, have vast experience with USAID-funded projects, but limited experience developing proposals for other donors. Key informant interviews also suggest that there is an expectation from the management and some advisors that USAID may provide continued support or preferential treatment to UHMG because it was created under its auspices. As a result, UHMG focuses most of its efforts on strengthening relations with USAID and competing for USAID-funded projects. This strategy could threaten the sustainability of UHMG if the organization fails to secure significant USAID funding after the end of AFFORD. Lack of formal linkages with the main partners that created UHMG: As mentioned previously, the chief of party and international partners played a significant role in the development of UHMG. They could be among the main motivators for the top management of UHMG to further proceed with the NGO when the AFFORD initiative ends. At the same time, current by-laws of and procedural manuals for the organization do not provide any official role for these partners in the future. After AFFORD ends, there is a chance that UHMG will lose its connection to these people as motivators as well as the technical support and other resources international partners can provide. Low involvement of UHMG staff in all stages of financial planning and fundraising activities: Although UHMG has won two projects as a part of international consortia and submitted several other proposals, these were developed largely by international experts. While these wins are important, the proposal development process has not provided significant capacity building for new business development of UHMG. 46 Conclusions and Recommendations Conclusions The conclusions are organized by the AFFORD initiative objectives. AFFORD has made significant progress towards achievement of its goals and objectives. AFFORD has nearly achieved Objectives 1 and 2. The evaluation validated several results reported by the initiative, such as availability of socially marketed products in drug stores, short-term effects of social marketing campaigns on demand for the AFFORD-branded products, and ability of the distribution network to deliver the products around the country. For example, socially marketed products, especially USAID brands Pilplan, Injectaplan, and Protector are sold in over 85 percent of the drug stores visited by the evaluation team. In addition, other brands were successfully introduced and are sold on the market. AFFORD established a sustainable private distribution network for these products. The AFFORD health promotion campaigns are recognized by the target population and highly regarded by key informants. There is also evidence suggesting that positive changes in health behavior are linked to them. Some of the products, such as newly￾introduced UHMG brands of contraceptives, condoms, and child health products, still have a small sales volume and limited presence on the market. Increase in availability of these products and in sales remains important tasks for the remaining years of project implementation. Further information about these products need to be provided to both retailers and the population to stimulate demand for the products. AFFORD has made significant progress on Objective 3, but must focus further efforts on this objective during the remaining years of the initiative. AFFORD established UHMG, an indigenous social marketing organization. It provided staff, structure, resources, and training to ensure that the organization will continue the distribution of AFFORD products on a cost-recovery basis. However, UHMG requires significant capacity building in financial management and new business development. Several of AFFORD’s activities have potential for further implementation. Working with private distributors who deliver socially marketed products without charging for their services is a highly effective and sustainable method. AFFORD created a network of private regional distributors, which in turn support regional networks of smaller wholesalers and retailers. The distributors and retailers participating in the network are commercial entities that exist without external funding by donors. Therefore, the network is sustainable. This system does not require the project to support any costs except distribution costs for the products that are provided though a normal price margin, similar to other (non-socially-marketed) products on the market. Currently, the network successfully distributes AFFORD branded products. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 47 There are outstanding gaps in the distribution network. Although functioning properly, the private distribution network presents some challenges that could be improved. These challenges include the limited knowledge of the distributor and retailers about the products, their uses and potential side effect. Health promotion activities, while popular, may not be sustainable. The most highly regarded efforts are the Good Life! show on television and radio, the Under the Mango Tree radio program, and POL program. It is clear that these program activities are expensive and likely cannot be sustained by a local organization without supplemental donor funding. The activities are implemented by local expertise via AFFORD/UHMG, but require supplemental funding. MARP activities, while valuable, may not be sustainable. AFFORD has made great progress in working with MARPs in Uganda—groups that are at high risk and often require innovative approaches to reach. However, despite the progress, these programs are not likely sustainable after AFFORD without additional donor funding. The huge demand for local level outreach efforts, such as the POL program, local radio shows, and capacity building for local government needs to be addressed. There is a huge demand from the district authorities for health promotion programs similar to those developed and implemented by AFFORD. Key informants spoke of a desire to extend these programs in their districts. In the case of local radio shows, some district authorities suggested that they could eventually find funds to support such programs if initial capacity building were funded by an international project. Therefore, an extension of the lives of these programs needs to be considered. UHMG requires additional capacity building and support. UHMG’s socially marketed brands are distributed on a cost-recovery basis. However, USAID brands (Pilplan, Injectaplan, and Protector) need ongoing support from USAID. Prices for the UHMG brand products are based on their cost, while prices for USAID products are set below their cost. As a result, continuing distribution of the latter is possible only if USAID covers at least the difference between the cost and price. At the same time, UHMG is a qualified and efficient distributor for USAID socially marketed products in Uganda, and there is a definite advantage in using it to distribute these brands in the future. UHMG lacks experience in financial management and new business development. UHMG‘s staff has been trained in distribution and marketing of the AFFORD products but has significantly less experience with new business/project development. Without this expertise, UHMG sustainability is threatened because the market capacity for UHMG products alone may be insufficient to provide revenue for sustaining the organization. 48 Recommendations Recommendations in the near term (AFFORD/UHMG) These near-term objectives are meant to be considered for the remaining time period of the AFFORD initiative (approximately two years). Since activities for objectives 1 and 2 are almost complete, the evaluation team provides some near-term recommendations that concentrate on strengthening UHMG to ensure its sustainability. However, some of these activities should also support the achievement of the other two project goals. Improve product distribution, which will indirectly support the UHMG distribution efforts. In particular, AFFORD should address several weaknesses, including: poor communication between shop owners and distributors, limited awareness of the shop owners about distribution networks, and limited basic knowledge and information about the products (especially SoftSure, NewFem, and Pilplan). Within the distribution network, instruct distributors on best business practices, such as training them to order products when current supplies are low, to survey the needs of shops by phone before making deliveries, and to keep an accurate list of contacts. Provide store owners with a simple re-order form at the bottom of each bulk package of drugs. The form should list contact telephone numbers for all 12 UHMG distributors in the area and a short request for re-supply of products, which can be sent by mail or facsimile (fax). Introduce a detailed description of medication into each bulk package of drugs so that trained pharmacy staff can learn about side-effects and the advantages of informing drug shop owners about the product. Transition selected promotional activities for implementation by district-level authorities. Even though these activities are costly to implement, local government and/or NGOs could sustain at least some of them, such as local education campaigns or local radio programs. Continuing them would improve visibility of the initiative in rural areas, build capacity of local authorities in health promotion using local resources such as radio stations and community meetings, and involve existing structures such as VHT and community health workers (especially in Lira District). In particular, district-level activities should seek to address the lack of understanding of the differences between AFFORD and UHMG. This misunderstanding hinders the establishment of UHMG as an independent indigenous institution. Encourage the involvement of UHMG senior management in meetings with local authorities to discuss local health issues and program solutions. Formulate and disseminate a clear statement of what should and should not be expected from UHMG compared to current AFFORD activities that the meetings identify. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 49 Extend AFFORD‘s practice of collaborating with local government entities, especially the district health office, through regular review meetings, sharing of information, and networking. Facilitate the integration of AFFORD activities into district-level plans and services to build capacity to carry out these activities in the districts. Create an understanding about sources of continued funding for these activities after AFFORD ends. In some cases, district health offices will have to find replacement funding or drop programs. Where possible, extend training and other activities conducted under the POL program to VHT and community health worker program participants. Collaborate with local government to provide information about UHMG products and Good Life! clinics to the participants in these programs and encourage them to in turn educate their communities about these products and services. Continue the practice of identifying and engaging local partners, including NGOs, CBOs, and district structures in rural areas to promote AFFORD/UHMG products and services with the goal of creating a wider network that increases impact beyond the initially-targeted population. AFFORD has made great strides in supporting Uganda MARPs by linking the networks, promoting healthy behaviors, and encouraging correct use of health products. AFFORD/UHMG should continue its valuable work with MARPs as much as possible. However, since it is not likely to be sustainable, UHMG will need to seek an outside funding source to continue this work after AFFORD ends. Strengthen UHMG to increase the probability of its survival beyond AFFORD’s end date. AFFORD should consider formalizing the continued participation of the international organizations that helped to create UHMG. For example, an informal network or an advisory board could be established that includes the current the chief of party of AFFORD, representatives of USAID, and experts from other international AFFORD partner organizations. The goal of the network or advisory board would be to advise the board and the managing director of UHMG about the current state of the organization, opportunities for development, progress, and new best practices in UHMG‘s main competencies, as requested by UHMG. AFFORD should develop medium-term financial plans that contain several funding scenarios, including one that assumes AFFORD funding ends according to its initial time-table. The plans should include measures to trim spending and identify priority activities that should be supported in the face of limited funds. Build UHMG capacities in new business development, for example, by arranging for the managing director or other senior staff to visit the U.S. offices of one of the AFFORD implementing partners to participate in the development of a proposal from start to finish. Develop UHMG contacts with other (non-U.S.) international donor organizations, such as the World Bank and other groups involved in government development 50 programs in Uganda. UHMG should be proactive and should not expect that USAID will make these introductions. Recommendations for long-term programs (USAID) The evaluation team offers the following long-term recommendations for USAID to consider after the conclusion of AFFORD. Support an extension of AFFORD activities, taking into consideration the following points: AFFORD as a brand has high visibility and positive recognition in Uganda. Therefore extension of the initiative or continued use of the name in a follow-on activity (for example, ―AFFORD-2‖ of ―AFFORD-plus‖) would capitalize on its current reputation. USAID/Washington and USAID missions have used this example specifically to extend the longevity and influence of a project name; such as the BASICS project and the Benin PISAF project. AFFORD activities supporting LLIN distribution are highly regarded in the districts, and authorities have indicated the need both for further net distributions and promotion of correct use. This activity could be implemented as a part of an AFFORD initiative extension or as a separate activity through another project. Schools are a major mechanism in Uganda for reaching communities with information that promotes demand and use of health products. Yet the current AFFORD design does not engage this resource on an adequate scale. An extension of or follow-up to the project could engage the Ministry of Education, especially at the district level, to implement health promotion through schools, such as the PIASCY program currently implemented in schools. Ensure the long term sustainability of UHMG with limited USAID support: AFFORD successfully transitioned the distribution of USAID branded products, such as Protector, PilPlan and InjectaPlan, to UHMG. USAID should continue distribution of these products with UHMG beyond the end-date of AFFORD, which will both support UHMG and prevent interruptions to product distribution. Provide financial support to UHMG beyond the remaining two years of AFFORD through a mix of competitive and noncompetitive grants and contracts to ensure a smooth transition to independence and help avoid the loss of qualified staff during the critical few years after AFFORD ends. Encourage UHMG‘s continued access to technical experts beyond the remaining two years of the AFFORD initiative. This periodic injection of technical expertise will help continue capacity building as the organization becomes independent and the staff faces new challenges. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 51 References AFFORD. 2005. Project Overview. Accessed at http://www.jhuccp.org/africa/uganda/afford.html. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2005. Original Project Proposal. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2006. AFFORD Health Marketing Initiative Annual Report, Year 1. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2007. AFFORD Health Marketing Initiative Annual Report, Year 2: October 1, 2006 to September 30, 2007. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2008a. AFFORD Health Marketing Initiative Annual Report, Year 3: October 1, 2007 to September 30, 2008. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2008b. Annual Work Plan Narrative: October 1, 2008 – September 30, 2009. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. 2008c. Baseline Survey for the AFFORD Project, Technical Proposal. Kampala, Uganda: Makerere University. ———. 2008d. Program Monitoring Plan. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. n.d.a. Baseline Methodology Description. Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. ———. n.d.b. Media Report (Regional). Baltimore, MD: Johns Hopkins Bloomberg School of Public Health. AFFORD and Uganda Health Marketing Group (UHMG) presentation to the Evaluation Team, November 19, 2008. Armand, Francoise. 2003. Social Marketing Models for Product-Based Reproductive Health Programs: A Comparative Analysis. Washington, DC: USAID/Commercial Market Strategies Project. Commercial Market Strategies (CMS). 2003. Uganda: Country Profile. Washington, DC: CMS. Evaluation of Popular Opinion Leaders‘ Performance, Wilsken Agencies, Ltd., 2007. Ministry of Health (MOH) Uganda and Micro International Inc. 2007. Uganda Atlas of HIV/AIDS Indicators. Calverton, MD: MOH and Micro International Inc. Pollard, Richard. 2007. ―Social marketing: An introduction to the total market approach to commodities and services supply in low-income countries,‖ presentation at George Washington University, Washington, DC, November 6. 52 Social Marketing Quarterly. 2006. Accessed at http://www.socialmarketingquarterly.com/. Washington, DC: Academy for Educational Development (AED). UHMG. 2008. Uganda Health Marketing Group Annual Report 2007. Kampala, Uganda: UHMG. UHMG. Uganda Health Marketing Group Annual Work Plan Narrative, October 1, 2008 – September 30, 2009. Kampala, Uganda: UHMG. UHMG. Group Organization Chart. Kampala, Uganda: UMHG UHMG. Management Reports. Kampala, Uganda: UHMG. Uganda Bureau of Statistics and Macro International Inc. 2007. Uganda Demographic and Health Survey 2006: Key Findings. Kampala, Uganda and Calverton, MD: Uganda Bureau of Statistics and Macro International Inc. ———. 2007. Uganda Demographic and Health Survey 2006. Calverton, MD: Uganda Bureau of Statistics and Macro International Inc. Uganda Ministry of Health and ORC Macro. 2006. Uganda HIV/AIDS Sero-behavioural Survey 2004-2005. Calverton, MD: Ministry of Health and ORC Macro. Uganda HIV Factsheet 2004-2005. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 53 Annexes ANNEX A : AFFORD ACTIVITIES TIMETABLE The table bellow indicates when products or services produced or supported by AFFORD became available to the intended target group or became visible outside the project. This timeline does not include the time spent by AFFORD internally to prepare 'launch" of product, service or activity. ACTIVITY October 05 - September 06 October 06 - September 07 October 07 - September 08 October 08 - September 09 O N D J F M A M J JL A S O N D J F M A M J JL A S O N D J F M A M J JL A S O N D J F M A M J JL A S HIV AIDS Products Protector "O" Cotramox (Cotrimoxazole 960mg) Clovirex (Aciclovir Tablets and Cream) Vitaboost (Multivitamin Tablets) Family Planning Products Pilplan (COC) Injectaplan (MPA) MoonBeads (Natural FP Method) NewFem (COC) SoftSure (POP) Child Care Restors (ORS) Zinkid (Dispersible Zinc Tablets) Aquasafe (NaDCC 70mg) Malaria Protection ACTs Training in the private sector LLINs- NGO distribution LLIN-ANC Distribution LLIN-Global Fund LLIN-Malaria No More Mass Campaign PMI Disitribution Distributors Network Distributors Contracted Distributors Training SSE Contracted SSE Training MARPS– Interventions Training of Peer Educators Grants to WAWI, WAYS, Uganda Police, UPDF, SSECODA Referral for HCT, STI and other needed services Institutional Capacity Building Good Life Clinics Supply of test kits and other reagents HCT data collection Referral for HCT, STI and other needed services Training in Health care waste management (with MMIS) Linkage through referral network between private & public facilities Training of consulars and staff HIV/AIDS Program Palliative Care Training of staff referral mechanism from VCT to palliative care support materials and job aids to health workers Support PLHAs to access palliative care products Establish data collection ACCESS TO SERVICES Objective I: Increase accessibility to products and services PRODUCT DISTRIBUTION TO POPULATION 54 ACTIVITY October 05 - September 06 October 06 - September 07 October 07 - September 08 October 08 - September 09 O N D J F M A M J JL A S O N D J F M A M J JL A S O N D J F M A M J JL A S O N D J F M A M J JL A S Community-Based Vehicles Popular Opinion Leaders Pulse Activations HIV Prevention Film shows Good Life Show TV episodes (English) radio episodes press columns radio spots Prize giving PR events Other Good Life Campaign Vehicles Under the Mango Tree Everyday Health Matters Other radio spots (Malaria & FP) Establishment of UHGM Legal registration Established a Governance system Organizational policies & systems Developed strategic and business plans Recruit staff UHGM Activities Product Facility Pharmacy Consultancy Market Research COMMUNICATION UGANDA HEALTH MARKETING GROUP Objective II: Enhance healthy behaviors and lifestyles within communities and families Objective III: Establish and strengthen capacity of a sustainable indigenous health marketing organization The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 55 ANNEX B : UHMG ORGANIZATIONAL STRUCTURE 56 ANNEX C : KEY INFORMANT INTERVIEW METHODOLOGY Key informants interviews were the most broadly used method of data collection for the AFFORD evaluation. Interviews were used to: deepen the team‘s understanding AFFORD and its expected influence; validate or refine methodologies for the retail store survey and the client exit survey; obtain expert opinions on the success of current AFFORD activities and progress towards objectives; and inform recommendations for future programming. AFFORD provided the initial list of key informants. The evaluation team expanded and refined the list based on suggestions of key informants and recommendations from USAID. Key informant interviews started by phone before the field work began. However, the majority of the interviews were conducted in Kampala during the first two weeks of field work. The key informant interviews in Kampala were conducted by the International Evaluator and the Senior Evaluator. Key informant interviews in districts outside Kampala were conducted during the planned visits by two data collection/survey teams: 1) the International Evaluator and Research Assistant as one team and 2) the International Evaluator and Lead In-country Evaluator as another team. Key informants were contacted to schedule interviews in advance. If they were not able to meet during the proposed time, the scheduler asked for a suggested colleague who was equally familiar with the topic. All key informants interviews were conducted in English. The evaluation team developed a semi-structured instrument with open-ended questions, which was refined after the first week of interviewing. The instrument was also used as the basis for the summary prepared in electronic format after each interview. Key informants were provided information about the study and asked for informed consent before each interview. The key informant interviews in electronic and paper form were reviewed, summarized, and encoded into a Microsoft Excel table by the Data Analyst. Responses to each question were organized by district. The data was further reviewed and aggregated in a table to link individual responses to the main questions for the evaluation. The team used a summary of the key informant interview findings to write the report, also presented as ANNEX K. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 57 ANNEX D : RETAIL STORE SURVEY METHODOLOGY The retail store survey‘s purpose is to evaluate the AFFORD initiative specifically and not to collect information about the impact of specific activities, policies or measures of the whole population of Uganda. The retail store survey was designed to assess the effect of AFFORD/UHMG‘s direct work with distributors and retail drug stores. The QED team collected data to assess the difference in availability of AFFORD/UHMG supported brands in drug stores that work with one of twelve AFFORD/UHMG distributors and, by comparison, in the stores that are not direct partners of such distributors. In addition, the survey also collected data on the impact of AFFORD/UHMG promotional campaigns on the demand of the AFFORD/UHMG-supported products in these stores. Both the client exit surveys and retail store survey have mixed sampling strategies, unified geographical coverage based on convenience sampling. District sites criteria are: 1. The treatment group should cover at least one district within the designated areas of each of the twelve AFFORD/UHMG distributors. Comparison group districts should be adjacent to the treatment group districts and should not be included in primary activity areas for AFFORD/UHMG. 2. Districts should be a part of the areas where AFFORD/UHMG conducted different types of malaria prevention activities: campaign-style LLIN distribution, ACT training in the private sector, LLIN ANC distribution, and malaria prevention activities. 3. The number of main languages spoken in these districts should be minimal in order to recruit an adequate number of fluent interviewers Based on these criteria, the following districts were selected: Treatment Group Districts Comparison Group Districts Arua Nebbi Kabarole Kamwewge Mbarara Ntunaamo Wakiso Mpigi Masaka Rakai Jinja Iganga Mbale Budaka Lira Pader Gulu Oyam Kampala NA Kiboga NA Two districts, Kampala and Kiboga, did not have formal comparison group districts identified due to their uniqueness. Kampala was the capital of the country and the primary location for AFFORD/UHMG activities not targeted to the rest of the country, such as the English language ―Good Life! show‖ on television. Kiboga District was in the distribution area for AFFORD/UHMG. However, there were no AFFORD activities conducted in this region except for the distribution of AFFORD/UHMG products. The retail stores in each district were randomly selected from two sources: 1. List of all licensed drug retail stores provided by a District Assistant Drug Inspector (DADI). 58 2. List of retail drug stores working with AFFORD/UHMG distributor that was responsible for the distribution in this district. Selection criteria: Based on the two sources above, in each district six main and six alternative locations (retail stores) were randomly selected from the list of retail drug stores working with AFFORD/UHMG distributor. These stores were in the treatment group sample. From the lists of all licensed drug stores in a district provided by a DADI, all stores that are not on the list provided by the AFFORD/UHMG distributor were selected for the comparison group list. From this list, the store geographically closest to each in the treatment group sample was selected. These stores formed the comparison group sample. Recruitment procedures and selection requirements: Interviewers received a list of two retail drug stores from the treatment group and the corresponding names of two stores for the comparison group. Each interviewer was instructed to proceed as follows: In a main store from the treatment group, identify a person who was either a store owner or main store manager on duty. Directly ask for such a person and probe with such criteria: a person who was responsible for purchasing products/supplies for the store, control of the stock, and dealing with distributors. If such a person was identified, the interviewer conducted an interview. However, if such a person was not present in the store, the interviewer could either wait (if arrival of such person was expected) or move to an alternative location/store from the treatment group. After completing a survey with the retail store owner or main manager, the interviewer remained near the store for an hour and conducted the exit survey interviews. The interviewer then moved to the location identified as a corresponding location in the comparison group and repeated the procedure. Sample size: The retail store survey sample size was limited by geographic dispersion, time constraints for the survey, and the number of Interviewers in each district. For each district, an interviewer was expected to conduct two Retail Stores Checks: one treatment group store, and a control group store. Therefore, the expected number of retail store survey interviews for the treatment sites was 132 (6 interviewers x 2 stores x 11 districts). For the comparison sites, the team estimated 108 interviews (6 interviewers x 2 stores x 9 districts). Inducements for participation: Interviewers were instructed to provide a clear explanation of the survey‘s purpose and provide convincing evidence for potential respondents to participate in the retail store survey. However, no special inducements for participation, such as monetary or non-monetary rewards or compensation, were given to the survey participants. Interviewers were instructed to receive informed consent from the respondents and provide reassurance that their relations with AFFORD/UHMG would not be affected by the respondent‘s decision to participate. The store management was also provided with a copy of an introduction letter written by UHMG/AFFORD to support the study. The letter clearly stated that there is no direct benefit for participation in the study or punishment for nonparticipation. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 59 ANNEX E : EXIT SURVEY METHODOLOGY The purpose of the client exit survey is to evaluate the AFFORD initiative‘s progress specifically; but not to collect information about the impact of activities, policies or measures on the whole population of Uganda. The client exit survey sought to assess specific health behaviors targeted by AFFORD; such as malaria prevention by using nets properly, HIV prevention by wearing condoms during sex, family planning through use of contraception, and child health through use of clean water, vitamins, and oral rehydration solution treatment during diarrhea. The survey also focused on a population assumed to be the most health conscious as suggested by their patronage of Good Life! clinics and retail stores (such as pharmacies and dukas). Specifically, the survey recruited respondents exiting AFFORD￾supported clinics and retailers/clients. This sub-population ostensibly is expected to show a higher than average positive response to the AFFORD products and activities. The results provide an upper bound estimate of AFFORD‘s ability to enhance healthy behaviors and lifestyles within communities and families. However, if the estimate of the upper bound is lower than AFFORD‘s self-assessment results from their internal mid-term evaluation, it indicates potential problems with the initiative‘s monitoring systems and/or with implementation. Both the client exit survey and retail store survey have mixed sampling strategies, unified geographical coverage based on convenience sampling. District sites criteria are: 1. The treatment group should cover at least one district within the designated areas of each of the twelve AFFORD/UHMG distributors. Comparison group districts should be geographically adjacent to the treatment group districts and should not be included in primary activity areas for AFFORD/UHMG. 2. Districts should be a part of the areas where AFFORD/UHMG conducted different types of malaria prevention activities: campaign-style LLIN distribution, ACT training in the private sector, LLIN ANC distribution, and malaria prevention activities. 3. The number of main languages spoken in these districts should be minimal in order to recruit an adequate number of fluent interviewers. Based on these criteria, the following districts were selected: Treatment Group Districts Comparison Group Districts Arua Nebbi Kabarole Kamwewge Mbarara Ntunaamo Wakiso Mpigi Masaka Rakai Jinja Iganga Mbale Budaka Lira Pader Gulu Oyam Kampala NA Kiboga NA Two districts, Kampala and Kiboga, did not have formal comparison group districts identified due to their uniqueness. Kampala, the capital of the country, was the primary location for 60 several AFFORD/UHMG activities which are not targeted to the rest of the country, such as the English language Good Life! show on television. Kiboga District was in the distribution area for AFFORD/UHMG. However, there were no AFFORD activities conducted in this region except for the distribution of AFFORD/UHMG products. Selection criteria: Clinics in treatment districts are selected from the list of Good Life! clinics supported by AFFORD/UHMG. If a district has more than six Good Life! clinics, the team selected those clinics with the largest expected number of patient visits per day based on UHMG information. In comparison districts, six clinics were selected in the sample based on recommendations provided by the DHO. (Note: retail stores for the client exit survey are the same as retail stores for the retail store survey). Recruitment procedures and selection requirements: Interviewers were instructed to approach each person exiting a clinic, with three exceptions: 1. Do not approach a person that appears to be a family member of a person that was already interviewed. 2. Do not approach a person that appears to be a medical or health worker of the clinic (such as persons wearing white coat or seen working at the reception in the clinic). 3. Do not interview persons under 18 years of age. Sample size: The client exit survey sample size was not identified a priori because it was expected to be limited by the following factors: 1. The number of people visiting a clinic or retail store on a given day was a factor. Since most Good Life! clinics are small in size; the team expected an average of about five people per day. 2. The number of interviewers and number of people interviewers could process in an hour was a factor. Only six interviewers were working in each district, and only one Interviewer deployed at each clinic. The expected completion time of a single client exit survey interview in a local language was 20–25 minutes, therefore an interviewer was able to interview only up to two people an hour. The total number of hours that an interviewer spent in a particular location was limited to eight hours. 3. The availability of trained interviewers with particular language skills was a factor. Consequently, the total expected maximum number of interviews conducted at clinics in AFFORD districts exits was 1,056 (6 clinics x 11 districts x 8 hours x 2 respondents). Among the comparison clinics, the team expected 864 interviews (6 clinics x 9 districts x 8 hours x 2 respondents). In addition, 264 Retail Store Exit Interviews conducted in AFFORD districts and 216 in comparison districts. The actual number of Client Exits Surveys (combined clinics and retail stores) completed in AFFORD districts is 706 and in comparison districts is 509. Inducements for participation: Interviewers were instructed to provide a clear explanation of the survey‘s purpose and provide convincing evidence for potential respondents to participate in the client exit survey. However, no special inducements for participation, such as monetary or non-monetary rewards or compensation were provided to the survey participants. Interviewers were directed to receive informed consent from the respondents and provide reassurance that future services received at the clinic would not be affected by the respondent‘s decision to participate. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 61 Clinic and retail store management were contacted in advance to receive permission for conducting the survey near their premises. They were provided full information about the survey, but did not receive any form of compensation. The establishment‘s management was also provided with a copy of an introduction letter written by UHMG/AFFORD in support of the study. The letter clearly stated that there was no direct benefit for participation in the study or punishment for nonparticipation. Ethical Considerations Both the client exit survey and retail store survey procedures did not request any information that could lead to the identification of a respondent. Neither type of surveys asked for a respondent‘s name or information that can be used to identify the person or his/her family; such as, address, phone number, etc. The surveys did not collect sensitive personal or business information, such as health status, income level, business profits or revenues. Additionally, there were several extra precautions taken to ensure the protection of respondents‘ rights and to take into consideration ethical issues: During training, each interviewer was instructed to begin a survey after the receipt of informed consent from the respondent. Interviewers were instructed on how to request informed consent, and were provided with informed consent forms. There was one form per every questionnaire which has a code number that links it to a particular questionnaire. The form requires signature of interviewer (but not the respondent) before the start of the interview. Also during the training, each interviewer was instructed how to ensure that the interview was conducted in a confidential setting, one-on-one with a respondent so no one else could hear the respondent‘s answers. Interviewers‘ contractual letters of agreement explicitly stated that he/she should safeguard confidentiality of completed questionnaires, provide all completed questionnaires to the Evaluation Supervisor at the end of each day, and should not discuss completed questionnaires with anybody except a designated data entry person, the Lead In-country Evaluator or the International Evaluator. The following is an excerpt from the Interviewer agreement: ...Interviewer is responsible for safeguarding completed questionnaires and other personal and confidential data in strict accordance to the study protocol regarding sensitive/confidential data. He/she should not provide completed questionnaires to anybody except Lead In-Country Evaluator unless otherwise specified in the study protocol. Interviewer should not discuss answers to questionnaires or other confidential data with anybody including other interviewers and survey participants, except with the Lead In￾Country Evaluator, Senior Evaluator or Team Leader, unless otherwise required by survey protocols and methodology provided to the interviewer in advance…. Since no personal information, such as a name, address or telephone number were collected, such information is not present in the survey database. Each record in the survey database is identified only by a questionnaire code number in the following format: ―2-digit district number—2-digit interviewer number—3-digit interviewee number within the district.‖ 62 All evaluation team members signed QED‘s Confidentiality Statement which confirms their commitment to keep information collected from respondents safe and confidential. This applies to both verbal conversations and electronic information. The data analysts had secure computers that were password protected. Discussions on the data collected remained internal to the evaluation. Reporting of data was on an aggregate level, protecting the identification of the respondents. The Lead In-Country Evaluator, Evaluation Assistant and Research Assistants were in control of their written notes at all times. When the notes were not in their personal possession, they remained in a locked cabinet in the Lead-In Country Evaluator‘s office. After the interviews and discussions were completed, the team merged their notes into one Microsoft Word document per interview which was transmitted by email to the Team Leader and protected by QED‘s network firewall. In the rare event the data are transmitted to a wrong address, all QED emails have a confidentiality statement included as an email footer. Since no personal information, such as a name, address or telephone number were collected, such information could not be present in the written questionnaires or electronic database. Each record in the database is identified only by a unique identifier code as noted earlier. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 63 ANNEX F : FOCUS GROUP METHODOLOGY The QED evaluation team conducted focus groups with local communities specifically to assess the effect of the POL program, LLIN distribution, and other anti-malaria activities implemented by AFFORD and UHMG. FGDs were conducted in the same districts included in the client exit surveys and the retail store survey. Four districts with a variety of anti-malaria activities were selected for the focus groups: Lira in Northern Uganda where UHMG conducted a distribution of the LLIN through ANC services; Fort Portal (Kabalore) in Western Uganda where no mosquito net distributions were conducted; and Mbale in Eastern Uganda, where mosquito nets were conducted through campaign and civil society organization distributions; and Wakiso, as a unique region close to Kampala. Selection criteria: The evaluation team intended to conduct two FGDs per district; one with POLs, and another with participants living in areas covered by the AFFORD/UHMG‘s POL and anti-malaria activities. POLs were contacted based on information provided AFFORD/UHMG. In two districts (Kabalore and Lira) the schedule allowed the team to conduct two groups with POLs and one with community members. Community FGD participants were selected from one of the villages in the district that has a POL, and helped recruit group participants. Participants averaged approximately 5-10 persons per focus group. The evaluation team chose groups representing persons of mixed gender and age between the ages of 18 and 60 years of age. Recruitment procedures and selection requirements: AFFORD/UHMG and its partners in districts assisted the evaluation team in identifying FGD participants from areas included in Lira, Fort Portal, Mbale and Wakiso Districts programs and anti-malaria activities. Inducements for participation: None of the focus group participants received any form of inducement or incentive to participate in the study. Persons who had questions or showed interest about AFFORD/UHMG-supported products or services received referral information. The evaluation team members conducting the FGDs reiterated their external role. They explained that participation in the focus group was strictly voluntary; and questioning begun only after informed consent has been secured. FGD participants received light snacks and beverages after the end of each FGD as is customary for such a gathering. The FGDs generated qualitative data about health behaviors, community education and effectiveness of the POL approach implemented by AFFORD/ UHMG. They shed light on AFFORD/UHMG‘s ability to influence knowledge and health behaviors at the parish level. 64 Uganda AFFORD Mid-term Evaluation Popular Opinion Leader Focus Group Guiding Questions 1. WHAT IS YOUR ROLE AS A POPULAR OPINION LEADER? WHAT DO YOU DO ON A DAILY BASIS? IS THERE ANYTHING ELSE YOU DO ON A MONTHLY OR QUARTERLY BASIS? (PROBE FOR MEETINGS WITH UHMG/CDFU REPRESENTATIVES, FOLLOW UP TRAINING, ETC)? 2. WHAT ARE THE BIGGEST HEALTHCARE PROBLEMS IN YOUR VILLAGE/COMMUNITY? 3. WHAT DO YOU KNOW ABOUT THE AFFORD PROJECT? HOW ABOUT UGANDA HEALTH MARKETING GROUP (UHMG)? 4. DO YOU KNOW THE DIFFERENCE BETWEEN AFFORD AND UHMG? PLEASE EXPLAIN. 5. HAVE YOU EVER ATTENDED ANY AFFORD/ UHMG TRAINING SESSIONS OR MEETINGS? WHAT TRAINING DID YOU RECEIVE? WHAT ARE YOUR IMPRESSIONS ABOUT THE AFFORD/UHMG TRAINING THAT YOU RECEIVED? 6. DO YOU THINK ANYTHING WOULD BE DIFFERENT IN YOUR COMMUNITY IF YOU HAD NOT RECEIVED TRAINING AS A POL? PLEASE EXPLAIN.WHAT ARE PEOPLE‘S REACTIONS WHEN YOU TELL THEM ABOUT THE FOLLOWING: A. HIV TESTING B. USE OF FAMILY PLANNING METHODS c. CONDOMS D. MALARIA PREVENTION PRODUCTS SUCH AS MALARIA NETS e. CHILD HEALTH PRODUCTS SUCH AS RESTORS AND ZINKIDS 7. DESCRIBE IF THERE HAVE BEEN ANY SIGNIFICANT CHANGES IN HEALTH BEHAVIORS, IN YOUR PARISH/VILLAGE OVER THE LAST 3 YEARS? (PROBE FOR CHANGES IN BEHAVIOR REGARDING USE OF MALARIA NETS, HIV/AIDS, CHILD HEALTH AND USE OF SAFE WATER) 8. IN THE LAST 3 YEARS, HAVE THERE BEEN ANY OBSERVABLE CHANGES IN RESPECT TO THE AVAILABILITY OF HEALTHCARE PRODUCTS (DRUGS, MALARIA PREVENTION PRODUCTS E.G. MALARIA NETS, VITAMINS, ETC.?) EXPLAIN. (PROBE: WHAT PRODUCTS/CATEGORIES OF PRODUCTS? WHAT DO YOU THINK BROUGHT ABOUT THIS CHANGE?) 9. WHERE THERE ANY CHALLENGES OR BARRIERS TO PARTICIPATING IN THE AFFORD EVENTS? IF SO, WHAT WERE THEY? The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 65 ANNEX G : INTERVIEWER TRAINING MANUAL AFFORD Mid-term Evaluation USAID/Uganda HIV/AIDS Evaluation, Assessment, and Formative Research INTERVIEWER TRAINING MANUAL The QED Group, LLC Project SEARCH February 5, 2009 Contract No. GHH-I-02-07-00034-00 66 TABLE OF CONTENTS I. Background and the Purpose of the Study 67 II. Administrative Matters 68 II.A. Schedule of the Survey Work, Composition of the Team and How to Address Your Questions 75 II.B. How to Prepare for Interviews 69 II.C. Materials You Will Need on this Study 70 II.D. What Is Expected of You 70 II.E. Asking for Help 70 II.F. Your Most Important Jobs 70 III. General Point of Interviewing 71 III.A. Introducing Yourself 71 III.B. How to Answer General Questions and Complaints 72 III.C. How to Maintain Confidentiality 72 IV. Basic Techniques for Interviewing 73 IV.A. How to Read Questions 73 IV.B. How to Get Adequate Answers 73 IV.C. How to Record Answers 75 V. How To Administer the Questionnaires 76 V.A. Exit Survey Questionnaire 76 V.B. Retail Store Check Survey Questionnaire 78 The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 67 I. BACKGROUND AND THE PURPOSE OF THE STUDY The purpose of this research is to gather information for the mid-term evaluation of the AFFORD project, funded by the United States Agency for International Development (USAID). The rationale of the mid-term evaluation is to determine whether the implementation of AFFORD activities during its three-year base period have achieved the desired results. The evaluation will determine the effectiveness of the marketing and awareness creation components. It will also assess the extent to which the products and services promoted by AFFORD are made available by local institutions in a sustainable manner. AFFORD is a five-year project funded by USAID that was designed to increase the sustainable marketing of products and services for HIV prevention and treatment, reproductive health, child health, and malaria. AFFORD also intends to enhance the knowledge and correct use of these products and services. The core of the AFFORD interventions has been to facilitate the development and implementation of innovative social marketing and communication strategies to ensure that Ugandans obtain access to affordable and high quality health products and services. One of the key goals (and an expected major result) of AFFORD is the creation of an indigenous organization that will continue in the footsteps of AFFORD after the end of the project. The Uganda Health Marketing Group (UHMG) is a new entity that was created by AFFORD and is being developed and nurtured to assume AFFORD‘s role. The two surveys that will be conducted during this study will collect information to help evaluate two main objectives of the AFFORD project and answer the following questions: 1. What is the effect of AFFORD activities on the volume of sales/distribution of the AFFORD-supported healthcare products and substitutes? 2. What is the effect of AFFORD-supported activities on the health behavior of the general population? In order to collect this information, we will conduct two surveys. First, we will carry out a population survey conducted at the exits of Good Life! clinics and Retail Stores called the ‗Exit Survey‘. Second, we will perform a survey of owners or managers of stores that sell drugs and medical products, conducted at their premises called the ―Retail Store Check‖. 68 II. ADMINISTRATIVE MATTERS II.A. Schedule of the Survey Work, Composition of the Team and How to Address Your Questions 1. The surveys will be conducted throughout the districts of Uganda, during a period of 4 days in each district or surrounding area(s). The work schedule in each district will be as follows: a. Day 1. Exit Survey at the exit/entrance to a Good Life! clinic or other clinics in a district. Each interviewer works in separate clinic. b. Day 2. Exit Survey at the exit/entrance to a Good Life! clinic or other clinics in neighboring district. Each interviewer works in separate clinic. c. Day 3. Retail Store Check Survey and Exit Survey at the exit/entrance to a retail store. Each interviewer will visit 2 retail stores in the districts, and conduct the Retail Store Check survey at a store and then interview visitors at the same store. d. Day 4. Retail Store Check Survey and Exit Survey at the exit/entrance to a retail store. Each interviewer will visit 2 retail stores in this or neighboring district, and conduct the Retail Store Check survey at a store and then interview visitors to the same store. 2. Tentatively, the survey work will start on February 15, 2009, and will end at the end of March 2009. 3. The interviewers will be assigned to cohorts/teams according to the language they speak. Each cohort will work in districts that speak their language. Each cohort will have an individual schedule (start and end dates for their work). Individual schedules for each cohort will be communicated separately by you project contract. 4. Each team will be accompanied by a project staff member, who will conduct supervision, quality checks, and will also conduct key informant interviews in the districts. 5. Each team will also have a supervisor selected from among the interviewers based on the demonstrated results of the training. 6. The work of each individual interviewer will be conducted in three stages: a. First, all interviewer candidates will receive a two-day theoretical training on the survey instruments. b. Second, all interviewer candidates will participate in the pre-test of the survey instruments, a third day of training, and a test of their survey skills. Four interviewer candidates selected to be supervisors will receive another day of training after the pre-test. c. Third, interviewer candidates who successfully pass the training will be invited to participate on a team and conduct interviews. Each interviewer is The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 69 expected to spend at least 6 days and no more than 42 days in the districts including travel days. Sunday are non-work days, and will be used either for rest or travel between districts. 7. Each individual interviewer will be paid in at least three installments: a. All interview candidates who participated in the training will receive payment for the days of training b. For the individuals selected as interviewers: interviewers will receive Uganda government level per-diems which include lodging and meals allowances prior to travel. If transportation is not provided by the project, a transportation allowance for travel to or from a district to or from Kampala will be provided. c. At the end of the work conducted by a language cohort, each interviewer will be evaluated individually. Interviewers who completed their work according to the survey design discussed in this manual and who submitted completed questionnaires to the project staff will receive payment for the amount of days worked. Please note that Sundays are not counted as work days, however per￾diems will be provided for interviewers who are staying in the districts or traveling on Sundays. 8. Documentation of interviewer contractual obligations will be done in the following way: 9. Each interviewer candidates will sign a Letter of Agreement with The QED Group, LLC, which will serve as a contract between you and the company. 10. At the time of payment, each interviewer will sign a receipt for payment which will also be signed by a witness. 11. Per-diems are not required to be supported by receipts. 12. Travel allowances will be provided based on the receipt for a bus and/or bus ticket. II.B. How to Prepare for Interviews 1. Read all the Training Manual material, familiarize yourself with all the forms and their use, and prepare the introduction you plan to use in beginning the interview. 2. Interview yourself, answering as if you were the respondent, and record your answers just as if someone else were conducting the interview. Take your time and make sure you get a feeling for the skip directions 3. Conduct a practice interview with a friend. The purpose of the practice interview is so you can become accustomed to asking the interview questions, to help you recognize inadequate responses, to indicate which parts of the instructions need your review, to identify problems which these instructions do not cover adequately, and to reveal problems we need to cover in future instructions to interviewers. 4. Jot down notes of anything that is unclear or any questions that you wish to ask, or sections that benefit from further instruction. Since the questionnaires are provided in 70 English, select a master copy of each questionnaire and mark difficult places with translation in the local language for your quick reference. II.C. Materials You Will Need on this Study The following is a list of materials designed for this study and a brief description of how to use them. Detailed instructions on how to use all forms appear later in this manual. 1. Interviewer Training Manual. Read this manual carefully and consult it whenever you have a question about administrative matters, filling out forms, or the interview itself. 2. Exit Survey Questionnaire. You will use this form to interview respondents for the Exit Survey at exit/entrances to clinics and retail stores. 3. Retail Store Check Survey Questionnaire. You will use this form to interview owners or managers of retail stores/drug stores. 4. List of assigned clinics and retail stores assigned to you individually. II.D. What Is Expected of You 1. Interview locations will be assigned to you individually on a daily basis. You will get to the interview location and conduct interviews one at a time. It is your responsibility to make contact with the management of clinics or retail stores to explain the purpose of the survey and receive their authorization to work near their establishment. If problems occur, it is your responsibility to notify your supervisor immediately to get reassigned to a different location. 2. Conduct interviews according to the rules and general concepts outlined in this manual. Supervisors will conduct random checks of each individual interviewer at their location during the day. 3. You expected to be at your first location for the day at 9AM and continue the interview process until 6PM unless you are given your supervisor‘s permission to shorten the interview process. 4. Organize your materials at the end of the day. Bring in all questionnaires and have them arranged in ID # order. Your supervisor will review your completed work and go over those cases you have not yet completed. II.E. Asking for Help 1. Ask for help before major problems develop. Almost every interviewer will need individual help with some aspects of the study. If there is anything you don‘t understand about the study or the interview procedure, please don‘t hesitate to ask questions. We want to help you do the very best job you can when interviewing the respondents. II.F. Your Most Important Jobs 1. Persuade people to cooperate in the study by giving them the necessary time to answer our questions honestly. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 71 2. Get a clear, complete, and unambiguous statement of a person‘s responses. To do this you should (a) help the respondent feel relaxed enough to answer all the questions, regardless of the popularity of the answer, and (b) allow the respondent to answer the question without suggesting answers, even subtly. Therefore, do not appear shocked, pleased, or upset by anything respondents say. Do not contaminate the interview with your own opinions. Instructions will be given later in this manual about how to use neutral probes in interviewing. 3. Record the respondent‘s ideas accurately on the form which will give someone else (such as the Evaluator and Data Manager) a clear picture of what happened in the interview. The best interview in the world is worthless if what happened is a secret between the interviewer and the respondent. III. GENERAL POINT OF INTERVIEWING III.A. Introducing Yourself 1. Some General Rules a. Know roughly what you will say before the interview. Try not to sound anxious or hesitant when you talk to the respondent. Know in advance what you want to say, but don‘t use a ―canned‖ speech that will sound phony. Say it in an easy, relaxed manner. b. Don‘t say more than necessary. Your introduction should be done in the shortest way possible. You should not spend much time giving explanations that the respondent did not ask for because this may cause the respondent to not want to be interviewed. c. Be business-like, yet friendly. 2. Suggested Introduction: a. Each questionnaire has an individual consent form that also provides an introduction for the interview. Please learn the introduction before the interview and follow it closely when introducing yourself to an interviewee. The survey methodology requires you to receive consent for the interview before proceeding. b. When conducting the Exit Survey at an entrance to a clinic, introduce yourself to the management or clinic staff before conducting the interview and obtain their permission to interview people near their establishment. You will be provided with a short introduction statement, and you can also show the questionnaire to the personnel, if requested. 72 III.B. How to Answer General Questions and Complaints Try to answer all questions simply and don‘t go out of your way to offer long detailed explanations that are not needed. Any complaints by respondents should be noted in writing, given to your team supervisor when you turn in your questionnaires, and the respondent should be told that s/he can call in any complaint to the supervisor if s/he would like. Below are some questions you may be asked and our suggested answers; but you should try to put the answers in your own words. Be familiar with the answers so that you can respond in a smooth and comfortable manner if you are asked any of these questions. However, don‘t try to say too much because you may lose your respondent or find yourself answering many more questions and wasting valuable time. 1. “What is this all about?” ―We are conducting an evaluation of a USAID-funded project. Your answers will help to determine if the project achieved its desired effect.‖ 2. “Do you know my name?” ―No, I don‘t know your name or any other information such as your address or telephone number, and no interviewers will ever know this information. Each questionnaire is coded with an individual ID, which is not linked to your name. However, we do ask for the name of your town/village in order to record the town/village each of the respondents who participate in this Survey.‖ 3. “Can I refuse to answer any question in this interview?” ―Yes, the interview is totally voluntary. But I would appreciate any help you can give us. I might add that if you would rather not answer certain questions, all you have to do is say so, and I will skip them.‖ 4. “I don’t want to be a part of your data bank.” ―Since we do not ask for your name or contact information, you will not be a part of any data bank.‖ 5. “How long will the interview take?” For the Exit Survey: ―That‘s hard to say because it varies depending on how many answers you tell us, but it usually takes about 20 minutes.‖ For the Retail Store Check Survey: ―That‘s hard to say because it varies depending on how many answers you have, but it usually takes about 1 hour. If during this time you need to attend to your business, for example, talk to a client, we can interrupt the survey and continue after you finish with the client‖ 6. “Who’s funding and conducting this study?” ―The study is funded by USAID and conducted by The QED Group, LLC, a consulting company based in Washington, DC.‖ III.C. How to Maintain Confidentiality You are bound by strict ethical procedures which prevent you from ever revealing what any respondent has said. While the respondents will be answering many questions about their The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 73 private lives, they will confide in you because you can promise them strict confidentiality— by this we mean the luxury of being totally open and honest without fear of being censured or quoted personally. This is very important if respondents have questions about confidentiality. Explain to them that their names will never be connected with what they tell you. When you get an interview assignment, the respondents‘ name will not be divulged to you so that even you are blinded to who the respondent is and to his/her responses on the questionnaire after the interview. IV. BASIC TECHNIQUES FOR INTERVIEWING IV.A. How to Read Questions 1. All questions should be read in a natural conversational rhythm in a normal tone of voice. We tried to write the questions in the everyday language that people use in conversation. You must learn to read them exactly as written but should as though you are talking rather than reading. Do not read a question too fast. A respondent may not feel free to ask you to repeat it and may guess at what the question was intended to be. If the respondent does not understand a question, repeat it exactly as written. No unusual emphasis should be used unless (a) it is clear that the respondent did not understand certain words, in which case you may want to emphasize the words that were missed, or (b) the words are underlined in the question, indicating that they should be emphasized. 2. Be aware of your tone of voice and facial expressions. How you ask a question or probe can be just as important as the wording of the question. Be careful that a tone of censure of criticism does not creep into your voice. Also your face may give you away. The best way to avoid such problems is to remember that your job is not to get people to agree with you, but to try to understand how people feel and think and what they do. 3. Reading Verbatim. Ask the questions exactly as they are worded and in the same order as in questionnaire. Even minor changes in the questionnaire wording can completely change the meaning of a question. So every interviewer must ask the questions just as worded or the responses are meaningless. Also you must follow the order of the questions and never ask questions out of order UNLESS there are special instructions to do so, such as in Question X of the Retail Store Check Questionnaire. In this case, follow the skip instructions just as printed. IV.B. How to Get Adequate Answers 1. General Guidelines a. Learn the purpose of each question. In order to do a good job of interviewing, you must understand the kind of information we are trying to get by asking a question. Unless you understand its purpose, you will not be able to judge when a response is adequate and you must probe for clarity or for more information. b. Don‘t try to explain the question. Be neutral. As mentioned before, if a respondent does not understand a question, repeat the question slowly and clearly. Give the respondent time to think about the question. Unless you have 74 other information about handling specific questions, the only acceptable answer for a respondent who wants to know what a question means is ―Whatever it means to you.‖ Never explain the meaning or purpose of a question unless the interviewer instructions tell you to do so. c. Don‘t define terms used in questions. Some respondents may ask you what we mean by a word used in a question. Leave the definition to the respondent, except where the written instructions give you a definition or alternate wording. Instead of giving your own definition, just say “Whatever________ means to you” or “However you use the term ___________.” The only exception to this rule is that for certain factual questions you will be given specific instructions on how to probe for accurate information, such as Question X. d. Don‘t leave a question until you have an adequate answer or find that a respondent can‘t give a clearer answer. e. Don‘t accept a ―don‘t know‖ without probing at least once. When you ask a question, people often say ―I don‘t know‖ just to give him/her time to collect their thoughts. If anyone says ―don‘t know,‖ then probe by saying “Well, what do you think?” or “I just want your opinion” (if the question asks for opinions rather than facts). Or, if the question asks for facts, we prefer if you probe further by saying “What’s your best guess?” or “Approximately?” to indicate that 100% accuracy is not required. f. Be sure that the respondent understands the time frame of the question being asked. Both surveys have only two time frames for the recall periods: one month and one year. Please be careful and remind individuals of specific time frame with each question. You can facilitate the respondent‘s recall by asking them to think back 1 month or a year, to a specific date, season or event to ‘anchor‘ their recall. Then ask, for example, "Since (date, season or event), how often have you ________?". g. Respondents may have difficulty with questions having multiple response options. Please note that there are four types of multiple response option questions and that all of these questions have four response categories. For these questions, you will be provided with set of responses. You can ask individuals to provide answers from each category in their own words, and then code the answers according to the set of responses provided in questionnaires. 2. Probing In most interviews, you will have to ask additional questions in order to clarify the respondent‘s response. This is called probing. There are several issues to keep in mind when probing for clarification: a. Use neutral probes that do not suggest answers. Probes are needed to get more complete and detailed answers. All probes must not be ―leading‖—that is, your probe must not suggest any particular answer to the respondent. Probes The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 75 should be used whenever the respondent is hesitant to answer a question; when s/he seems to have trouble expressing her/himself; when s/he seems to shy to speak; whenever there is any reason for the interviewer to think that the respondent has not given a complete report of her/his thinking; and, finally, when reassuring is needed because a respondent seems to lack confidence. b. Many interviewers forget to use two of the most effective neutral probes: (1) silence and (2) repeating the original question. The interviewer who can wait patiently and quietly will soon find that 15 seconds of silence is more than most respondents can take, and the respondent will often expand or clarify a previously inadequate answer if you simply wait. And repeating the question is another neutral way of probing. Be sure to repeat only the question as it is written in the questionnaire. c. Avoid ―depends‖ or ―qualified‖ answers. Never accept a ―depends‖ or ―qualified‖ answer the first time it is given as a response to a question. Respondents often use phrases such as ―Well, that depends.‖ When the respondent gives a qualified answer, we advise one of the following probes: Repeat the question (unless this response will sound like you are not listening). Preface the question with a phrase like “Well, in general” or “On the whole.” Remind the respondent that we want to know which code comes closest to her/his views; use an introductory phrase such as “Well, if you had to choose” or “Even though you are somewhere in the middle, which way do you lean?” and repeat the question. If the respondent insists upon answering in qualified terms after probed enough, simply record verbatim her/his entire response, alone with your probes. IV.C. How to Record Answers 1. General Guidelines a. Be ready to write. Have your pen or pencil ready as you ask your question. b. Learn to look up at your respondent and to look interested while you are interviewing. c. Never erase. If you make a mistake, cross out the wrong part instead of erasing. It‘s faster and often easier to read. 2. How to Handle Precoded Questions Nearly all the questions in the questionnaire are precoded. For most questions, you will simply circle a number or fill in a box with a number the respondent gives you. Keep the following in mind when asking precoded questions. a. Show corrections if you need to change a code. If you give the wrong precoded response by mistake, or the respondent changes her/his mind about an answer, cross it out and make the correction. b. Be sure to record any comments the respondent volunteers in answering a precoded question whenever you have any doubt about whether you entered the right code. 76 3. How to Record Verbatim Responses When asking the few open-ended questions, you must record exactly what the respondent tells you. Use the following guidelines when recording responses word for word. a. Don‘t hesitate to ask the respondent to ―Please slow down.‖ Many times respondents will talk much more quickly than you can write. So asking them to talk slower is understandable in order for you to take their comments down correctly. b. Always use a person‘s own language. Quote your respondent directly. Don‘t polish what the respondent says even if s/he uses slang language or profane language. c. If there‘s not enough space for the respondent‘s answer, use the margins (top, side, or bottom) but be sure to label the continuation of your notes clearly. V. HOW TO ADMINISTER THE QUESTIONNAIRES V.A. Exit Survey Questionnaire 1. Survey Code Number Q.N is an important part of questionnaire. When you preparing for the next interview, you have to enter the Survey Code Number in the following format: Two digit district code-two digit interviewer number –three digit interview number For example: 01-01-001 A unique district code is assigned to each district. Please locate your current district code in the following table: District Code District Name District Code District Name 01 Arua 21 Nebbi 02 Kabarole 22 Kamwewge 03 Mbarara 23 Ntunaamo 04 Wakiso 24 Mpigi 05 Masaka 25 Rakai 06 Jinja 26 Iganga 07 Mbale 27 Budaka 08 Lira 28 Pader 09 Gulu 29 Oyam 10 Kampala 11 Kiboga A unique two-digit interviewer number will be assigned to each interviewer before going to the field. This number will be constant throughout the field work. You are responsible of keeping track of the number of Exit Surveys you conducted in particular district. First questionnaire that you complete in a new district has three￾digit interview number 001. When moving from location to location within the same The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 77 district, you have to continue increasing this number each time you complete a questionnaire. 2. Most other questions of the questionnaire are self-explanatory. You have to follow the general guidelines to conducting survey described above. If you have any questions, please do not hesitate to ask your supervisor. Exceptions from this rule are discussed bellow. 3. Questions Q.1, Q.2.1 – Q.2.3: According to your assignment, your location is either Clinic or Retail Store. Record this location type in Q.1. On each questionnaire that you complete at particular location (either clinic or retail store exit), complete questions Q.2.1 – Q.2.3 with the address of the establishment as it is written on your list of designated locations. If you know a name of the owner/manager of this location and his/her contact number, please also record it in these questions. 4. Q.4.1 Please ask interview what village or town he/she permanently leaves for the last year. Record this number. For this address please ask respondent if it is a town or a village. Answer code 1 for this question is only applicable to Kampala 5. Q.4.4 and Q.4.5: If respondent is confused about definition of the family, please provide an explanation that by family we mean all relatives leaving in the same house as the respondent. Question Q.4.5 records how many children are among these individuals. If an individual leaves alone, both Q.4.4 and Q.4.5 should be coded as 0. 6. Q.4.6 If occupation of a respondent cannot be coded as one of the choices provided for the question, please mark choice Other-7, and record respondents‘ occupation in the space for variable Q.4.6_O 7. Ask question Q.5 only if answer to question Q.1 is 1 (Clinic Exit Survey) Otherwise, leave the question blank 8. Ask question Q.6 only if answer to question Q.1 is 2 (Retail Store Exit Survey) Otherwise, leave the question blank 9. Questions Q.7 and Q.8. Read each choice for the question as individual question, repeating the top (common) portion of the question. Please make sure that respondents understand that the duration of recall period is one month. 10. Q.8: Please explain to respondent that he/she should use the same definition of family as used for answering question Q.4.4 11. Q.9 and Q.10: Please read the top part of the question to the respondent, and then ask he/she to identify if there was behavioral change with each specific area. Present each choice separately. If respondent indicates that there is a behavioral change (increase or decrease in use of some products, coded 1 or 3 for the question), please follow this question with question Q.9_R or Q.10_R, asking individual to identify possible sources of this change. Based on the respondents‘ response, please code it with choices presented at the bottom of the page, but do not read these choices to the responded prior to the answer. If respondent gives more than three reasons, please ask him/her to identify three most important. Number of completed choices does not necessarily have to be 3, it can be smaller. Three main reasons are not required to be coded in order of priority. 78 V.B. Retail Store Check Survey Questionnaire 1. Retail Store Check Survey Questionnaire is designed to survey owners or primary managers of retail stores that sell drugs (such as drugs stores and pharmacies) that work with AFFORD/UHMG distributors or are similar to these stores. Therefore, it is important when coming to the store to identify such person. Start with talking to a person behind the counter. Since most stores are small, this will probably be the person that you are looking for. If it is not the owner or primary manager, please enquire if such person is around for an interview, or will return to the store soon. If the person is unavailable at the moment, but can be available later, please consider going to the next store on your list and then returning to this store for the interview later. Finally, if a store owner or primary manager is not available, go to the next location on your list and include in your daily schedule nearest store from your reserve locations list. If you have any questions or foresee problems, please do not hesitate calling your supervisor. 2. Most questions of the questionnaire are self-explanatory. You have to follow the general guidelines to conducting survey described above. If you have any questions, please do not hesitate to ask your supervisor. Exceptions from this rule are discussed bellow. 3. You can pre-complete first page of the questionnaire with the information about the location available from your list of locations (such as store address) before coming to the field. However, you expected to record exact time (not a guess) of the interview start and end, and sign the first page of the questionnaire only after asked respondent for this informed consent using provided form. 4. Survey Code Number Q.N is an important part of questionnaire. When you preparing for the next interview, you have to enter the Survey Code Number in the following format: Two digit district code-two digit interviewer number –three digit interview number For example: 01-01-001 A unique district code is assigned to each district. Please locate your current district code in the following table: District Code District Name District Code District Name 01 Arua 21 Nebbi 02 Kabarole 22 Kamwewge 03 Mbarara 23 Ntunaamo 04 Wakiso 24 Mpigi 05 Masaka 25 Rakai 06 Jinja 26 Iganga 07 Mbale 27 Budaka 08 Lira 28 Pader 09 Gulu 29 Oyam 10 Kampala 11 Kiboga The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 79 A unique two-digit interviewer number will be assigned to each interviewer before going to the field. This number will be constant throughout the field work. You are responsible of keeping track of the number of Retail Store Check Surveys you conducted in particular district. First questionnaire that you complete in a new district has three-digit interview number 001. When moving from location to location within the same district, you have to continue increasing this number each time you complete a questionnaire. 5. SECTION 1 is related to information about the store. Please make sure that respondent understands that recall period for this ONE YEAR. Read each question individually, listen to the respondents answer, and code it based on provided set of responses. Some questions lead to a skip pattern. This means that certain answer to a question (either yeas or no) will prompt you to skip several following questions and start your questioning from the question to which point instructions to this question. 6. SECTION 2 is related to information about products sold in this store. It contains general section and subsections specific to each type of products marketed by UHMG/AFFORD. First, ask question Q.2 to determine which types of products are not sold in the store. If particular type of products is not sold in the store, you should skip corresponding sub-section of questionnaire. 7. SUBSECTION 2.A has nonstandard for this questionnaire pattern. First two columns of the section ask about the bands marketed by UHMG/AFFORD. Third column asks about all other bands of condoms. If store sells other brands of condoms, please find out how many other bands NOT including UHMG/AFFORD are sold in the store currently. Ask respondent which of this brands is sold the best. When asked about volume of sale, please ask total amount of packs sold by ALL other brands. All other questions in column three should be asked about the brand that respondent identified as BEST BRAND among bands that are not marketed by UHMG. 8. SUBSECTIONS 2.B – 2.D in the last column asks respondent identify up to five other best selling products in this category. Other questions in this subsection should NOT BE APPLIED to these brands. 9. If a store does not sell a product identified at the head of column in SUBSECTIONS 2.A - 2.D, please skip all questions and go to the last question in this column. If a store sells the product, please skip the last question for this product. 10. SECTION 3 asks about typical consumer coming in the store for particular type of products. If particular type of products is not sold in the store, please code all questions in a raw with code for N/A. Explain to the store owner that we are looking for his personal opinion about the typical costumers, not a scientifically confirmed answer. If respondent asks, offer him/her to recall typical costumers for a period of LAST MONTH. Please listen to respondent‘s answer, and then code it with a choice provided in the list of choices. Do not read choices to the respondent with the question. 11. SECTION 4 asks about store sources of drugs and medical products and their experience with suppliers. Read each question and choice of answers to the respondent. For the questions Q.4.2 – Q.4.4, ask respondent to select only one 80 response that is the most applicable to the situation in the store. If respondent asks, provide him a recall period of ONE YEAR. 12. SECTION 5 asks about experience with AFFORD. Question Q.5.1 relates to the store experience with UHMG/AFFORD distributor if they can clearly identify such person. All other questions in this section refer to the respondents personal experience with AFFORD or UHMG activities. 13. After SECTION 5 your interaction with a respondent should be completed. Please thank the respondent for taking part in our research. 14. SECTION 6 of the questionnaire is based on interviewer‘s observations. Take a quick look around the store. For each item on the list in the section, please identify if this item is present and visible in the store. However, please DO NOT ASK store owner, manager or sales person to point to a product or other material mentioned in this section. The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 81 ANNEX H : KEY INFORMANT INTERVIEW GUIDE KEY INFORMANT INTERVIEW Date/Location Jan. 27, 2009 9:00am AFFORD OFFICE Introduction: Hello. My name is _________________ and I‘m part of The QED Group, LLC (QED) team which is leading the mid-term evaluation of the USAID-financed AFFORD project. QED is a full-service international development company located in Washington, DC which specializes in project monitoring and evaluation. The purpose of this evaluation is to determine if the AFFORD project activities have achieved the desired results during the first 3 years of operations. The objectives of the evaluation are to: a) validate the project results; b) identify the effectiveness and further potential of project activities; and c) assess the future sustainability of the Uganda Health Marketing Group (UHMG). Your views will help to provide a better understanding of the changes in the health sector in Uganda and the effect of current AFFORD programs. Information collected from key informant interviews will be used to highlight best practices and make recommendations for improvements to AFFORD‘s current and future mixture of activities. Your responses will also be used to test the initial assumptions that led to the development of AFFORD, to inform USAID/Uganda‘s decisions on the continuation of the program, and to United States Government‘s investments in marketing strategies for the health sector in Uganda. Participation in this interview is voluntary. We ask to interview you alone in order to facilitate your honest responses. If you choose, the information you provide can be kept strictly confidential and not be shown to AFFORD or UHMG representatives. However, summary results based on key informant interviews will be shared with AFFORD and USAID/Uganda officials During the interview, if we come to any question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this interview since your views are important. At this time, do you want to ask me anything about the survey? May I begin the interview now? Signature of Interviewer: ______________________ Date: ________________________ 82 A-1. PLEASE TELL AS BRIEFLY WHAT DOES YOUR ORGANIZATION OR DEPARTMENT DO, AND WHAT ARE YOU ROLE AND RESPONSIBILITIES? A-2. DO YOU KNOW ABOUT THE AFFORD PROJECT? DO YOU HAVE ANY AFFILIATION WITH AFFORD? WHAT ABOUT UHMG? DO YOU KNOW ABOUT OR HAVE ANY INVOLVEMENT WITH UHMG? A-3. DO YOU KNOW ABOUT SIGNIFICANT CHANGES IN HEALTH IN UGANDA OVER THE LAST 3 YEARS? IN YOUR OPINION, WHAT WAS THE SOURCE OR CAUSE OF THESE CHANGES? A-4. IN YOUR OPINION, WERE THERE CHANGES IN THE HEALTH BEHAVIOR OF THE GENERAL POPULATION AND SPECIFIC AT-RISK GROUPS IN THE COUNTRY OVER THE LAST 3 YEARS? WHAT WAS THE SOURCE OR CAUSE OF THESE CHANGES? A-5. IN YOUR OPINION WAS THERE A CHANGE IN AVAILABILITY OF HEALTHCARE PRODUCTS (DRUGS, ETC.) TO THE POPULATION IN THE LAST 3 YEARS? WHAT PRODUCTS/CATEGORIES OF PRODUCTS, AND WHAT IS THE SOURCE OR CAUSE OF THIS CHANGE? A-6. HAVE YOU EVER PARTICIPATED IN ANY AFFORD ACTIVITIE(S)? PLEASE DESCRIBE THE ACTIVITIE(S), AND YOUR PARTICULAR ROLE. A-7. WHAT IS YOUR EXPERINECE WORKING/COOPERATING/COMMUNICATING WITH AFFORD? WHAT WERE THE ADVANTAGES? WHAT WERE THE LESSONS LEARNED FROM THIS EXPERIENCE? WHERE THERE ANY CHALLENGES OR BARRIERS? IF SO, WERE THERE ANY STEPS TAKEN TO RESOLVE THEM AND MOVE FORWARD SUCCESSFULLY? A-8. IN YOUR EXPERT OPINION, WHAT IS THE EFFECT OF AFFORD IN THE FOLLOWING AREAS OF WORK: A) PRODUCT AVAILABILITY, B) POPULATION HEALTH BEHAVIOR, C) NON-PROFIT AND FOR-PROFIT HEALTHCARE SECTORS, D) OR OTHER AREAS OF LIFE IN UGANDA? A-9. IN YOUR EXPERT OPINION, WHAT ARE THE MAIN AFFORD SUCCESSES OR BEST PRACTICES? A-10. IN YOUR EXPERT OPINION, WHAT ARE AFFORD‘S MAIN CHALLENGES OR BARRIERS? The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 83 A-11. HAVE YOU HEARD ABOUT UHMG? A. DO YOU KNOW WHAT UHMG DOES? B. IF YES, CAN YOU COMMENT ON THE PERFORMANCE OF UHMG C. WHAT ARE THE SUCCESSES OF UHMG? D. WHAT ARE THE CHALLENGES OF UHMG? E. WHAT SHOULD BE DONE TO MAKE UHMG A SUCCESSFUL NONGOVERNMENTAL ORGANIZATION (NGO) IN UGANDA? A-12. IN YOUR EXPERT OPINION, WHAT PROBLEMS IN THE HEALTHCARE SECTOR MOST NEED UHMG ATTENTION AND SUPPORT IN THE NEXT 5 YEARS? A-13. INTERVIEWER, PLEASE SELECT APPROPRIATE QUESTION RELATED TO THE AREA OF EXPERTISE OF RESPONDENT AND RECORD ANSWERS? 1. In your opinion, has there been a change in marketing or volume of sales/distribution of the healthcare products in Uganda over the past two years? If so, what products, and what is the source or cause of this change? Has AFFORD or UHMG played any role in this change? 2. In your opinion, has there change in availability and/or use of malaria prevention products over the past two years? If so what products in particular? What is the source or cause of this change? Were there also changes in malaria prevention behaviors? What is the source or cause of these changes? Has AFFORD or UHMG played any role in this change? 3. In your opinion, has there been change in HIV/AIDS understanding and related behaviors? What is the source or cause of this change? Was there change in availability and/or use of HIV/AIDS protection/prevention products and/or services? What is the source or cause of this change? Has AFFORD or UHMG played any role in this change? 4. In your opinion, has been there any change in the NGO/civil society organization environment in healthcare sector in Uganda? What is the source or cause of this change? Has AFFORD or UHMG played any role in this change? 5. In your opinion, what progress has UHMG achieved at the moment? What should be done to ensure long-term sustainability of UHMG? A-14. DO YOU HAVE ANY OTHER SUGGESTIONS OR RECOMMENDATIONS RELATED TO THE TOPIC OF TODAYS INTERVIEW? A-15. CAN YOU RECOMMEND ANY OTHER EXPERTS WHO‘S OPINION MAY BE USEFUL FOR THIS EVALUATION? 84 ANNEX I : RETAIL STORE SURVEY QUESTIONNAIRE Retail Store Study Survey Questionnaire Date………………………. District : Town/Village: Starting time of interview Ending time of interview Type of Outlet RETAILER/PHARMACY Name of Outlet Address Telephone # Supervisor’s name and signature Interviewer’s name Interviewer’s signature Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 85 SECTION A: Interviewer/Pharmacy Background Q.A.1 Are you the proprietor or main manager or main sales person for this establishment (drug store, pharmacy, retail store)? Yes .. 1 No.. 2 (If no, stop interview) Q.A.2 Are you the proprietor of the establishment? Yes .. 1 go to Q.A.4 No.. 2 Q.A.3 Who is proprietor of this establishment (drug store, pharmacy, retail store)? 1. Respondent’s family 2. Respondent’s friend or neighbor 3. Unrelated businessperson 4. Government of Uganda 5. Uganda nongovernmental organization 6. National company/business 7. International company or individual 8. International project/donor organization 9. DNK (Do not know) _______ Q.A.4 Do you usually work in the establishment (drug store, pharmacy, retail store) most of the time? Yes .. 1 go to Q A. 6 No.. 2 Q.A.5 Who is the main person who usually works in this establishment (drug store, pharmacy, retail store)? 1. Owner 2. Member of respondent’s family who is not the owner 3. Friend/neighbor of the respondent who is not the owner 4. Other unrelated employee who is not the owner 9. DNK (Do not know) _______ Q.A.6 How long has this establishment existed here (Number of years)? _______ Q.A.7 How long have you worked in this establishment (Number of years)? _______ Q.A.8 What is your (respondent) level of education?: 1. No Education 2. Incomplete Primary School (less than 7 years) 3. Complete Primary School ( 7 years) 4. Incomplete O-level (less than 11 years.) 5. Complete O-level (11 years 6. Incomplete A-level (less than 13 years) 7. Complete A-Level (13 years) 8. University or training institution 99. DNA (Did not answer) _______ Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 86 SECTION 1: Interviewer and pharmacy store information Q.1 FOR THE LAST ONE YEAR, UNLESS OTHER TIME FRAME SPECIFIED, PLEASE ANSWER THE FOLLOWING?: Q.1.1 Did you have any health or healthcare education? Yes .. 1 No.. 2 Q.1.2 Do you know if the establishment/store received supplies from AFFORD/United Health Marketing Group (UHMG) distributors? Yes .. 1 No.. 2 Q.1.3 Do you know if the establishment sold/still sells products marketed by AFFORD or UHMG? Yes .. 1 No.. 2 Q.1.4 Do you know if the establishment sold/still sells products marketed by international projects or donor organizations such as the USAID or the World Bank? Yes .. 1 No.. 2 SECTION 2: Product information Q.2 DO YOU SELL THE FOLLOWING PRODUCTS?: Q.2.1 Male condoms (if no, skip subsection 2.A) Yes .. 1 No.. 2 Q.2.2 Other contraceptives/family planning products (if no, skip subsection 2.B) Yes .. 1 No.. 2 Q.2.3 Vitamins Yes .. 1 No.. 2 Q.2.4 Child health related drugs/supplements (if not, skip subsection 2.C) Yes .. 1 No.. 2 Q.2.5 HIV/AIDS and other sexually-transmitted infections (STI) treatments (if no, skip subsection 2.D) Yes .. 1 No.. 2 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 87 SUBSECTION 2.A: Condoms (Note: The brands indicated here are AFFORD/UHMG Products. Show the pictures of these brands to the respondent) FOR THE FOLLOWING BRANDS: Protector “O” Other Brands 2.A.1 Do you sell this brand of male condoms? Q.2.A.1_1 1. Yes 2. No (go to Q.2.A.10_1) Q.2.A.1_2 1. Yes 2. No (go to Q.2.A.10_2) Q.2.A.1_3 1.Yes (#of other brands) ____ 2. No (go to Q.2.A.10_3) 2.A.2 How many packs of this brand do you usually sell a month? Q.2.A.2_1 Q.2.A.2_2 Q.2.A.2_3 (Total for other brands) 2.A.3 How many condoms are in a pack (if several sizes are available, the best selling pack size)? Q.2.A.3_1 Q.2.A.3_2 Q.2.A.3_3 (Best selling brand) 2.A.4 What is the current price per pack (if several sizes are available, the best selling pack size)? Q.2.A.4_1 Q.2.A.4_2 Q.2.A.4_3 (Best selling brand) 2.A.5 How often do you run out of supply for this brand? 1. More than once a month 2. Once a month 3. Once a quarter 4. twice a year 5. Once a year 6. Never 99. DNA Q.2.A.5_1 Q.2.A.5_2 Q.2.A.5_3 (Best selling brand) 2.A.6 What do you do when you run out of supply for this brand? 1. Call/place an order with the supplier 2. Wait for the supplier to visit 3. Buy from another source 4. Never run out of supply 5. Other 99. DNA Q.2.A.6_1 Q.2.A.6_2 Q.2.A.6_3 (Best selling brand) 2.A.7 How would you rate sales of this brand among all other brands available at your store? 1. Best 2. Average 3. Worst 99. DNA Q.2.A.7_1 Q.2.A.7_2 Q.2.A.7_3 (Best selling brand) 2.A.8 How often do customers ask for this specific brand? 1. More than once a day 2. Once a day 3. At least once a week 4. Less than once a week 99. DNA Q.2.A.8_1 Q.2.A.8_2 Q.2.A.8_3 (Best selling brand) 2.A.9 How do you usually get this brand? 1. Sales person visits me 2. Buy it from wholesaler 3. Buy from another retailer 4. Other 99. DNA Q.2.A.9_1 Q.2.A.9_2 Q.2.A.9_3 (Best selling brand) 2.A.10 What is the reason you do not sell this brand? 1. No distributor 2. Not available 3. No credit/cash to buy 4. Not interested 5. Other (specify) ……………………. 6. DNK 99.DNA Q.2.A.10_1 Q.2.A.10_2 Q.2.A.10_3 (Best selling brand) Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 88 SUBSECTION 2.B: Other contraceptives/family planning products (Interviewer show these brands to respondent) FOR THE FOLLOWING BRANDS: Pilplan Injectaplan® MoonBeads NewFem SoftSure Other Products 2.B.1 Do you sell following brands of contraceptives or family planning products? Q.2.B.1_1 1. Yes 2. No (go to Q.2.B.9_1) Q.2.B.1_2 1. Yes 2. No (go to Q.2.B.9_2) Q.2.B.1_3 1. Yes 2. No (go to Q.2.B.9_3) Q.2.B.1_4 1. Yes 2. No (go to Q.2.B.9_4) Q.2.B.1_5 1. Yes 2. No (go to Q.2.B.9_5) Please list up to 5 other best￾selling products 2.B.2 How many packs of this brand do you usually sell a month? Q.2.B.2_1 Q.2.B.2_2 Q.2.B.2_3 Q.2.B.2_4 Q.2.B.2_5 2.B.3 What is the current price per pack? Q.2.B.3_1 Q.2.B.3_2 Q.2.B.3_3 Q.2.B.3_4 Q.2.B.3_5 2.B.4 How often do you run out of supply for this brand? 1. More than once a month 2. Once a month 3. Once a quarter 4. twice a year 5. Once a year 6. Never 99. DNA Q.2.B.4_1 Q.2.B.4_2 Q.2.B.4_3 Q.2.B.4_4 Q.2.B.4_5 2.B.5 What do you do when you run out of supply for this brand? 1. Call/place an order with the supplier 2. Wait for the supplier to visit 3. Buy from another source 4. Never run out of supply 5. Other 99. DNA Q.2.B.5_1 Q.2.B.5_2 Q.2.B.5_3 Q.2.B.5_4 Q.2.B.5_5 2.B.6 How would you rate sales of this brand among all other brands available at your store? 1. Best 2. Average 3. Worst 99. DNA Q.2.B.6_1 Q.2.B.6_2 Q.2.B.6_3 Q.2.B.6_4 Q.2.B.6_5 2.B.7 How often do customers ask for this specific brand? 1. More than once a day 2. Once a day 3. At least once a week 4. Less than once a week 99. DNA Q.2.B.7_1 Q.2.B.7_2 Q.2.B.7_3 Q.2.B.7_4 Q.2.B.7_5 2.B.8 How do you usually get this brand? 1. Sales person visits me 2. Buy it from wholesaler 3. Buy from another retailer 4. Other 99. DNA Q.2.B.8_1 Q.2.B.8_2 Q.2.B.8_3 Q.2.B.8_4 Q.2.B.8_5 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 89 FOR THE FOLLOWING BRANDS: Pilplan Injectaplan® MoonBeads NewFem SoftSure Other Products 2.B.9 What is the reason you do not sell this brand? 1. No distributor 2. Not available 3. No credit/cash to buy 4. Not interested 5. Other(specify)…………………… 6. DNK 99.DNA Q.2.B.9_1 Q.2.B.9_2 Q.2.B.9_3 Q.2.B.9_4 Q.2.B.9_5 SUBSECTION 2.C: Child health related drugs/supplements (Interviewer show the pictures of these brands to respondent) FOR THE FOLLOWING BRANDS: Restors (ORS) Zinkid (Zinc Tablets) Aquasafe (NaDCC 70mg) Other Products 2.C.1 Do you sell following brands of child health related drugs/supplements? Q.2.C.1_1 1. Yes 2. No (go to Q.2.C.9_1) Q.2.C.1_2 1. Yes 2. No (go to Q.2.C.9_2) Q.2.C.1_3 1. Yes 2. No (go to Q.2.C.9_3) Please list up to 5 other best-selling 2.C.2 How many packs of this brand do you usually sell products a month? Q.2.C.2_1 Q.2.C.2_2 Q.2.C.2_3 2.C.3 What is the current price per pack? Q.2.C.3_1 Q.2.C.3_2 Q.2.C.3_3 2.C.4 How often do you run out of supply for this brand? 1. More than once a month 2. Once a month 3. Once a quarter 4. twice a year 5. Once a year 6. Never 99. DNA Q.2.C.4_1 Q.2.C.4_2 Q.2.C.4_3 2.C.5 What do you do when you run out of supply for this brand? 1. Call/place an order with the supplier 2. Wait for the supplier to visit 3. Buy from another source 4. Never run out of supply 5. Other 99. DNA Q.2.C.5_1 Q.2.C.5_2 Q.2.C.5_3 2.C.6 How would you rate sales of this brand among all other brands available at your store? 1. Best 2. Average 3. Worst 99. DNA Q.2.C.6_1 Q.2.C.6_2 Q.2.C.6_3 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 90 FOR THE FOLLOWING BRANDS: Restors (ORS) Zinkid (Zinc Tablets) Aquasafe (NaDCC 70mg) Other Products 2.C.7 How often do customers ask for this specific brand? 1. More than once a day 2. Once a day 3. At least once a week 4. Less than once a week 99. DNA Q.2.C.7_1 Q.2.C.7_2 Q.2.C.7_3 2.C.8 How do you usually get this brand? 1. Sales person visits me 2. Buy it from wholesaler 3. Buy from another retailer 4. Other 99. DNA Q.2.C.8_1 Q.2.C.8_2 Q.2.C.8_3 2.C.9 What is the reason you do not sell this brand? 1. No distributor 2. Not available 3. No credit/cash to buy 4. Not interested 5. Other (specify) …………….. 6. DNK 99.DNA Q.2.C.9_1 Q.2.C.9_2 Q.2.C.9_3 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 91 SUBSECTION 2.D: HIV/AIDS and other sexually-transmitted infections (STI) treatments (Interviewer, show the picture of this product to respondent) FOR THE FOLLOWING BRANDS: Cotramox (Cotrimoxazole 960mg) Other Products 2.D.1 Do you sell following brands of HIV/AIDS and other STI treatments? Q.2.D.1_1 1. Yes 2. No (go to Q.2.D.9_1) Please list up to 5 other best￾selling 2.D.2 How many packs of this brand do you usually sell a month? Q.2.D.2_1 products: 2.D.3 What is the current price per pack? Q.2.D.3_1 2.D.4 How often do you run out of supply for this brand? 1. More than once a month 2. Once a month 3. Once a quarter 4. twice a year 5. Once a year 6. Never 99. DNA Q.2.D.4_1 2.D.5 What do you do when you run out of supply for this brand? 1. Call/place an order with the supplier 2. Wait for the supplier to visit 3. Buy from another source 4. Never run out of supply 5. Other 99. DNA Q.2.D.5_1 2.D.6 How would you rate sales of this brand among all other brands available at your store? 1. Best 2. Average 3. Worst 99. DNA Q.2.D.6_1 2.D.7 How often do customers ask for this specific brand? 1. More than once a day 2. Once a day 3. At least once a week 4. Less than once a week 99. DNA Q.2.D.7_1 2.D.8 How do you usually get this brand? 1. Sales person visits me 2. Buy it from wholesaler 3. Buy from another retailer 4. Other 99. DNA Q.2.D.8_1 2.D.9 What is the reason you do not sell this brand? 1. No distributor 2. Not available 3. No credit/cash to buy 4. Not interested 5. Other (specify)………………………………………. 6. DNK 99.DNA Q.2.D.9_1 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 92 SECTION 3: Customer information FOR EACH TYPE OF THE PRODUCTS, PLEASE INDICATE THE FOLLOWING: Q.3.1 Who is the main customer for this category of products? Q.3.2 How do the customers select a specific brand? Q.3.3 Do your customers often? Q.3.4 What do you think influences most customers to choose a specific brand?: Select from following choices: 1. Male 2. Female 3. Both 4. DNK (Do not know) 98.N/A (if do not sell these products) 99. DNA (Did not answer) 1. Ask for specific brand 2. Ask for your advice 3. Buy the least expensive 4. Buy the most expensive 5. Other (specify) 98. N/A 99. DNA 1. Ask for brand/product advertised on TV 2. Ask for brand/product advertised on radio 3. Ask for brand/product advertised in newspaper 4. Ask for brand/product advertised in your store 5. Ask for brand product advertised 6. Ask for brand/product suggested by doctor 7. others (Specify)……………………. 98. N/A 99. DNA 1. Price 2. Quality 3. Brand recognition 4. Recommendation 5. Simplicity of use 6. others(Specify)……………… 7. DNK 98. N/A 99. DNA Male condoms Q.3.1_1 ________ Q.3.2_1 ________ Q.3.3_1 ________ Q.3.4_1 ________ Other contraceptives/family planning products Q.3.1_2 ________ Q.3.2_2 ________ Q.3.3_2 ________ Q.3.4_2 ________ Vitamins Q.3.1_3 ________ Q.3.2_3 ________ Q.3.3_3 ________ Q.3.4_3 ________ Child health related drugs/supplements Q.3.1_4 ________ Q.3.2_4 ________ Q.3.3_4 ________ Q.3.4_4 ________ HIV/AIDS and other sexually-transmitted infections (STI) treatments Q.3.1_5 ________ Q.3.2_5 ________ Q.3.3_5 ________ Q.3.4_5 ________ Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 93 SECTION 4: Product distribution network Q.4.1 FOR THE LAST YEAR, WHICH OF THE FOLLOWING DID YOU USE TO GET MEDICAL PRODUCTS/DRUGS THAT YOU SELL?: Q.4.1_1 Other stores like yours Yes .. 1 No.. 2 Q.4.1_2 Larger store or pharmacy Yes .. 1 No.. 2 Q.4.1_3 Wholesale distributor Yes .. 1 No.. 2 Q.4.1_4 Company that produces this drug Yes .. 1 No.. 2 Q.4.1_5 Government (for free distribution) Yes .. 1 No.. 2 Q.4.1_6 Government (for sale) Yes .. 1 No.. 2 Q.4.1_7 NGOs or International Projects Yes .. 1 No.. 2 Q.4.2 DURING THE LAST YEAR, WHAT WAS THE MAIN SOURCE THAT YOU USED TO GET MEDICAL PRODUCTS/DRUGS THAT YOU SELL?: Please select only one source, mark box with X 1 Other stores like yours □ 2 Larger store or pharmacy □ 3 Wholesale distributor □ 4 Company that produces this drug □ 5 Government (for free distribution) □ 6 Government (for sale) □ 7 NGOs or International Projects □ Q.4.3 DURING THE LAST YEAR, WHAT IS THE FURTHEST PLACE THAT YOU TRAVELLED TO GET THE MEDICAL PRODUCTS/DRUGS THAT YOU SELL?: Please select only one choice, mark box with X 1 Did not travel/All products were delivered to me by suppliers □ 2 Nearest trading center (other than current location, if located in trading center) □ 3 Capital of my district (other than current location, if located in capital of the district) □ 4 Neighboring district □ 5 Kampala (only if it does not qualify as choice 1,2 or 3) □ 6 Districts far from my district □ 7 Another country □ Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 94 Q.4.4 DURING THE LAST YEAR, HOW DID YOU USUALLY GET NEW BRANDS/MEDICAL PRODUCTS TO SELL AT YOUR STORE?: Please select only one source, mark box with X 1 Look for distributors of new products that I want to sell by myself □ 2 Buy new products that available in neighboring stores □ 3 I am contacted by distributor who offers new products □ 4 Other □ Q.4.4_O Specify _______________________________ 99 DNA □ SECTION 5: Experience with AFFORD Q.5.1 FOR THE LAST ONE YEAR, PLEASE RECALL THE FOLLOWING?: Q.5.1_1 Did you work with a distributor who clearly identified that he is working with UHMG or AFFORD project? Yes .. 1 No.. 2 (Go to Q5.2) Q.5.1_2 Is this distributor the main source of medical products/drugs that you sell in the store Yes .. 1 No.. 2 Q.5.1_3 Is this the only source of medical products/drugs that you sell in the store Yes .. 1 (Got to Q5.2) No.. 2 Q.5.1_4 What is your experience working with this distributor compared to other distributors? Better than the other…….1 Same as the other……….2 Worse than the other…....3 N/A………………………98 _______ Q.5.1_5 What is most attractive to you in this distributor? Brands of the products offered…………..1 Prices……………………………………….2 Distributor delivers to my store…………..3 Distributor responds to requests quickly..4 Distributor always has supplies………….5 Other (specify)……………………………..6 N/A…………………………………………98 _______ Q.5.2 DID YOU EVER HEAR/SEE THE FOLLOWING?: Q.5.2_1 Good Life Show on TV Yes .. 1 No.. 2 Q.5.2_2 Good Life Show on radio Yes .. 1 No.. 2 Q.5.2_3 Good Life Show column in newspapers Yes .. 1 No.. 2 Q.5.2_4 Under the Mango Tree on radio Yes .. 1 No.. 2 Q.5.2_5 Everyday Health Matters newspaper Yes .. 1 No.. 2 Q.5.2_6 HIV prevention films Yes .. 1 No.. 2 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 95 Q.5.3 DID YOU EVER PARTICIPATE IN THE FOLLOWING?: Q.5.3_1 Good Life Show prize drawings Yes .. 1 No.. 2 Q.5.3_2 Pulse Activation theatre/show Yes .. 1 No.. 2 Q.5.3_3 Meeting with community leaders to discuss health related issues organized by the AFFORD Project (AFFORD) or the Uganda Health Marketing Group (UHMG) Yes .. 1 No.. 2 Q.5.3_4 Training on healthcare issues or other events organized by AFFORD or UHMG Yes .. 1 No.. 2 Q.5.3_5 Malaria net distribution from AFFORD, USAID, or UHMG Yes .. 1 No.. 2 Q.5.4 DID YOU NOTICE MORE CUSTOMERS COMING TO YOUR STORE AFTER THE FOLLOWING?: (You can only record 98 for these questions if the answers to the corresponding questions in Q. 5.2 are no (2)) Q.5.2_1 Good Life Show on TV N/A .98 Yes .. 1 No.. 2 Q.5.2_2 Good Life Show on radio N/A . 98 Yes .. 1 No.. 2 Q.5.2_3 Good Life Show column in newspapers N/A . 98 Yes .. 1 No.. 2 Q.5.2_4 Under the Mango Tree on radio N/A . 98 Yes .. 1 No.. 2 Q.5.2_6 HIV prevention films N/A . 98 Yes .. 1 No.. 2 Q.5.2_6 Pulse Activation theatre/show in your town/village N/A . 98 Yes .. 1 No.. 2 Q.5.2_6 Meeting with community leaders to discuss health related issues organized by the AFFORD Project (AFFORD) or the Uganda Health Marketing Group (UHMG) in your town/village N/A . 98 Yes .. 1 No.. 2 THIS IS THE END OF THE INTERVIEW. THANK YOU FOR PROVIDING VALUABLE INFORMATION FOR OUR EVALUATION! Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 96 SECTION 6: Concluding observations (to be completed based on interviewer observations) Q.6 INTERVIEWER, PLEASE LOOK AROUND THE STORE AND MARK IF YOU SEE THE FOLLOWING UHMG/AFFORD SUPPORTED PRODUCTS AND ADVERTISEMENT. DO NOT ASK STORE PERSONAL TO HELP YOU FIND THESE PRODUCTS. ALL UHMG/AFFORD PRODUCTS/MATERIALS SHOULD BE DISTINCTLY MARKED WITH UHMG/AFFORD LOGOs: Q.6.1 Cotramox (Cotrimoxazole 960mg) Yes .. 1 No.. 2 Q.6.4 Restors (ORS) Yes .. 1 No.. 2 Q.6.5 Zinkid (Dispersible Zinc Tablets) Yes .. 1 No.. 2 Q.6.6 Aquasafe (NaDCC 70mg) Yes .. 1 No.. 2 Q.6.8 Pilplan Yes .. 1 No.. 2 Q.6.9 Injectaplan® Yes .. 1 No.. 2 Q.6.10 MoonBeads Yes .. 1 No.. 2 Q.6.11 NewFem Yes .. 1 No.. 2 Q.6.12 SoftSure Yes .. 1 No.. 2 Q.6.13 Protector Yes .. 1 No.. 2 Q.6.14 “O” Yes .. 1 No.. 2 Q.6.15 Signs or advertisements for at least one of the above products Yes .. 1 No.. 2 Q.6.16 Advertisements for Good Life Clinic Yes .. 1 No.. 2 Q.6.17 Advertisements for UHMG/AFFORD events/shows Yes .. 1 No.. 2 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 97 ANNEX J : EXIT SURVEY QUESTIONNAIRE Exit Survey Questionnaire Q.0.A INTERVIEWER’S Number ……………………... Q.0.B INTERVIEWER’S NAME…………………………... Q.3 Date: …………………………………..(MM/DD/YY) Q.4 Respondent’s Background Q.4.1 Respondent Address (Zone, Village, Division, District) ………………………………………………………………… Q.4.2 Type of area Capital…………………………..……1 District Capital/Town………………..2 Village/Rural…………………..……..3 Q.4.3 Gender : Male ……………..…………….1 Female………………………...2 Q.4.4 Marital status : Single ……………..………..…..1 Married/living together ………..2 divorced/separated…………….3 Widowed ………………….……4 DNA……………………. ……….99 Q.4.5 Age (approximately if not known): ………. Q.4.6 Number of people in your household: …. …… Q.4.7 Number of children in your household: … ..…. Q.4.8 Occupation of respondent Health worker .……..………….………..1 Industrial Worker.……………………...…2 Service Industry....……………………......3 Science/Professional/Management ...…4 Farmer……………..………………………5 Household & Domestic...…………….…..6 Other(specify)….……….…………………7 DNA…………………….………………….99 Q.4.9 Education level of respondent: No Education ………………………………………….……...1 Incomplete Primary School (less than 7 years) …………..2 Complete Primary School ( 7 years) ……………..………..3 Incomplete O-level (less than 11 years.) ……….…..……..4 Complete O-level (11 years)………………...…………..…5 Incomplete A-level (less than 13 years) ……………...….. 6 Complete A-Level (13 years) …………..….....…………… 7 University or training institution……………………….…… 8 DNA……………………………………………………….….99 Q.1 Exit Survey Type: Clinic Exit Survey ……..……1 Retail Store Exit Survey……2 Q.2.1 LOCATION ADDRESS: (address of clinic or retail store) Q.2.2 Contact authority/owner Name: (at clinic or retail store) Q.2.3 Tel. Number : (of contact authority) If Exit Survey Type 1 (Clinic) Q.5. Purpose of visit to clinic: Get services ……………………………1 Accompany someone..…………..……2 Did not come to the clinic ……………3 Other (specify) ………….………..……4 DNA …………………………………...99 If Exit Survey Type 2 (Retail Store) Q6. Purpose of visit to the store: Buy drugs/health related items………………..1 Buy items not health related………………….2 Accompany someone..……………………..…..3 Did not come to the store.……..….…………....4 Other (specify) ………….……………………...5 DNA ……………………………………………..99 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 98 Q.7 DURING THE LAST SIX MONTH, DID YOU? Q.7.1 Use Mosquito nets Yes .. 1 No.. 2 Q.7.2 Treat water by boiling, using filters or purification tablets before drinking Yes .. 1 No.. 2 Q.7.3 Take vitamin tablets Yes .. 1 No.. 2 Q.7.4 Use condoms for family planning Yes .. 1 No.. 2 Q.7.5 Use pills, injections, bids or other birth spacing/family planning products (for females only) Yes .. 1 No.. 2 98.. N/A Q.7.6 Consider the possibility of being exposed to a sexually-transmitted infection (STI) like HIV/AIDS before engaging in sexual activities Yes .. 1 No.. 2 Q.7.7 Use condoms for protection against HIV/AIDS and other STI’s Yes .. 1 No.. 2 Q.7.8 Consider taking HIV test Yes .. 1 No.. 2 Q.7.9 Consider the possibility of being exposed to an infectious disease like HIV/AIDS when health provider, other person or yourself is using needles to inject you with drugs or medication Yes .. 1 No.. 2 Q.8 HAS AT LEAST ONE PERSON IN YOUR HOUSEHOLD (excluding you) DURING THE LAST SIX MONTHS (Skip this question or recode 98 if the respondent stays alone ) Q.8.1 Used mosquito nets Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.2 All children under 5 years of age in your family slept under mosquito nets Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.3 Treated water by boiling, using filters or purification tablets before drinking Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.4 Taken vitamin tablets Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.5 Provided vitamins and mineral supplements for all children under 5 years of age in your family Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.6 Used condoms for family planning Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.7 Used pills, injections, bids or other birth spacing/family planning products Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.8 Discussed the possibility of exposure to an infectious disease like HIV/AIDS before engaging in sexual activates or using needles for injections Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.9 Used condoms for protection against HIV/AIDS and other STI’s Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.8.10 Considered taking or took HIV test Yes .. 1 No.. 2 DNK.. 3 98..N/A Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 99 Reasons for behavioural change 1 Information on TV 7 Experience/information from family members 13 No drugs/medicine available 2 Information on radio 8 Opinion of colleagues 14 No doctor/clinic available close by 3 Information in newspapers 9 Opinion of respected community leaders 15 Do not think it is effective 4 Information distributed by community leaders or at meetings 10 Recommendation by a doctor 16 Religious reasons 5 Greater availability of health products 11 Personal experience with being sick 17 Others (specify)………………………. 6 Greater availability of health services 12 Not getting sick any more 99 Did not answer (DNA) Q.9 OVER THE LAST 3 YEARS, DID YOUR PERSONAL BEHAVIOUR CHANGE REGARDING THE FOLLOWING?: ( If there was a change, probe to find out if the change was an increase or decrease. If question Q.9.1 – Q.9.8 is answered 1 or 3, please ask for the reasons for behavioural change (Q.9_r), and code the response with the choices given at the bottom of this page) Q.9.1 Use of mosquito nets Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.2 Use of water treatment for drinking water (through boiling or tablets) Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.3 Use of vitamins and mineral supplements Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.4 Use of condoms for family planning Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.5 Use pills, injections, bids or other birth spacing/family planning products (for females only) Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.6 Use of condoms or other products to prevent HIV/AIDS Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.7 Visits to health clinics Increased 1 No changes 2 Decreased 3 DN/A 99 Q.9.8 Visits to pharmacies/Drug shops Increased 1 No changes 2 Decreased 3 DN/A 99 Reason 1 Reason 2 Reason 3 Q.9.1_r1 ______ Q.9.1_r2 ______ Q.9.1_r3 ______ Q.9.2_r1 ______ Q.9.2_r2 ______ Q.9.2_r3 ______ Q.9.3_r1 ______ Q.9.3_r2 ______ Q.9.3_r3 ______ Q.9.4_r1 ______ Q.9.4_r2 ______ Q.9.4_r3 ______ Q.9.5_r1 ______ Q.9.5_r2 ______ Q.9.5_r3 ______ Q.9.6_r1 ______ Q.9.6_r2 ______ Q.9.6_r3 ______ Q.9.7_r1 ______ Q.9.7_r2 ______ Q.9.7_r3 ______ Q.9.8_r1 ______ Q.9.8_r2 ______ Q.9.8_r3 ______ Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 100 THIS PAGE IS FOR RECORDING REASONS FOR BEHAVIOUR CHANGE IN QUESTIONS Q.9 AND Q.10, IF ANSWER CAN NOT BE CODED USING CODES FOR REASONS PROVIDED AT QUESTION PAGES Q.9: Question starts at the PREVIOUS PAGE. If reason can not be coded with codes provided on previous page, please use code 17 and record the reason here. Q.9.1_rO Use of mosquito nets Q.9.2_rO Use of water treatment for drinking water (through boiling or tablets) Q.9.3_rO Use of vitamins and mineral supplements Q.9.4_rO Use of condoms for family planning Q.9.5_rO Use pills, injections, bids or other birth spacing/family planning products (for females only) Q.9.6_rO Use of condoms or other products to prevent HIV/AIDS Q.9.7_rO Visits to health clinics Q.9.8_rO Visits to pharmacies/Drug shops Q.10: Question starts at the NEXT PAGE. If reason can not be coded with codes provided on the next page, please use code 17 and record the reason here. Q.10.1_rO Use of mosquito nets Q.10.2_rO Use of water treatment for drinking water (through boiling or tables) Q.10.3_rO Use of vitamins and mineral supplements Q.10.4_rO Attention to the health care of children under 5 in the family Q.10.5_rO Use of condoms for family planning Q.10.6_rO Use pills, injections, bids or other birth spacing/family planning products (for females only) Q.10.7_rO Potential risks to infectious diseases Q.10.8_rO Use of condoms or other products to prevent HIV/AIDS Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 101 Reason 1 Reason 2 Reason 3 Q.10.1_r1 ______ Q.10.1_r2 ______ Q.10.1_r3 ______ Q.10.2_r1 ______ Q.10.2_r2 ______ Q.9.2_r3 ______ Q.10.3_r1 ______ Q.10.3_r2 ______ Q.9.3_r3 ______ Q.10.4_r1 ______ Q.10.4_r2 ______ Q.9.4_r3 ______ Q.10.5_r1 ______ Q.10.5_r2 ______ Q.9.5_r3 ______ Q.10.6_r1 ______ Q.10.6_r2 ______ Q.9.6_r3 ______ Q.10.7_r1 ______ Q.10.7_r2 ______ Q.9.7_r3 ______ Q.10.8_r1 ______ Q.10.8_r2 ______ Q.9.8_r3 ______ Reasons for behavioural change 1 Information on TV 7 Experience/information from family members 13 No drugs/medicine available 2 Information on radio 8 Opinion of colleagues 14 No doctor/clinic available close by 3 Information in newspapers 9 Opinion of respected community leaders 15 Do not think it is effective 4 Information distributed by community leaders or at meetings 10 Recommendation by a doctor 16 Religious reasons 5 Greater availability of health products 11 Personal experience with being sick 17 Others (specify)………………………. 6 Greater availability of health services 12 Not getting sick any more 99 Did not answer (DNA) Q.10 OVER THE LAST 3 YEARS DID YOUR FAMILY MEMBERS’ BEHAVIOUR CHANGE RELATED TO THE FOLLOWING? (Skip this question or recode 98 if respondent lives alone) (If there was a change, probe to find out whether the change was an increase or decrease. If questions Q.10.1 – Q.10.8 is answered 1 or 3 please ask for the reasons for behavioural change (Q.10_r) and code the responses with the choices given at the bottom of the page) Q.10.1 Use of mosquito nets Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.2 Use of water treatment for drinking water (through boiling or tables) Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.3 Use of vitamins and mineral supplements Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.4 Attention to the health care of children under 5 in the family Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.5 Use of condoms for family planning Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.6 Use pills, injections, bids or other birth spacing/family planning products (for females only) Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.7 Potential risks to infectious diseases Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.10.8 Use of condoms or other products to prevent HIV/AIDS Increased 1 No changes 2 Decreased 3 DNK 4 N/A 98 Q.N Questionnaire Code Number ( ) /format: dist_code - interviewer_n - quest_n/ 102 Q.11 DID YOU EVER HEAR/SEE THE FOLLOWING?: Q.11.1 Good Life Show on TV Yes .. 1 No.. 2 Q.11.2 Good Life Show on radio Yes .. 1 No.. 2 Q.11.3 Good Life Show column in newspapers Yes .. 1 No.. 2 Q.11.4 Under the Mango Tree on radio Yes .. 1 No.. 2 Q.11.5 Everyday Health Matters newspaper Yes .. 1 No.. 2 Q.11.6 HIV prevention films Yes .. 1 No.. 2 Q.12 DID YOU EVER PARTICIPATE IN THE FOLLOWING?: Q.12.1 Good Life Show prize drawings Yes .. 1 No.. 2 Q.12.2 Pulse Activation theatre/show Yes .. 1 No.. 2 Q.12.3 Meeting with community leaders to discuss health related issues organized by the AFFORD Project (AFFORD) or the Uganda Health Marketing Group (UHMG) Yes .. 1 No.. 2 Q.12.4 Training on healthcare issues or other events organized by AFFORD or UHMG Yes .. 1 No.. 2 Q.12.5 Mosquito net distribution from AFFORD, USAID, or UHMG Yes .. 1 No.. 2 The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 103 ANNEX K : RETAIL STORE SURVEY INFORMED CONSENT AFFORD Mid-term Evaluation Retail Store Survey Informed Consent (English) February 9, 2009 Using structured, signed informed consent forms, all survey respondents will be informed about the study objectives, procedures, benefits and risks. They will be assured that should they agree to participate in the study, they have the right to withdraw at any point in time and that they can refuse to answer any or all of the questions, even after consenting to participate in the study. After being fully informed about the study objectives, procedures, benefits and risks, the respondents will be asked to give oral consent if they wish to participate in the study. The interviewers will then sign and date the informed consent form, attesting to the fact that they received the respondent‘s oral consent. The informed consent to be used for each respondent follows: INFORMED CONSENT Hello. My name is ____________ and I am working with the QED Group evaluation team to evaluate AFFORD project. We are conducting a research study that asks retail store owners about their experience with drugs supported by AFFORD project and UHMG, as well as similar products. We would very much appreciate your participation in this survey. This information will help in the planning of health services and the AFFORD project activities. The survey usually takes about 1 hour to complete. However, we understand that you need to attend your business, and we can interrupt the survey at any minute so you attend your costumers. We would appreciate if you continue the survey after that. Your views will help in providing better health services to the greater community. The information collected in this survey will be utilized to improve programming related to increase access to and use of selected quality basic health products and services in the private sector, by women and men of reproductive age, as well as health products and services for children under 5 years of age. Participation in this survey is voluntary. We ask to interview you alone, to facilitate your honest responses. Whatever information you provide will be kept strictly confidential and will not be shown to other persons outside the research team. Please also note that participation or not participation in this survey would not in any way effect any current or future relations with AFFORD Project or UHMG. If we should come to any question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey since your views are important. At this time, do you want to ask me anything about the survey? Do you agree to participate in the survey and may I begin the interview now? YES RESPONDENT AGREES TO BE INTERVIEWED: [START INTERVIEW] NO RESPONDENT DOES NOT AGREE TO BE INTERVIEWED: [END] Signature of Interviewer: ___________________________________ Date: ________________________ 104 ANNEX L : EXIT SURVEY INFORMED CONSENT AFFORD Mid-term Evaluation Exit Survey Informed Consent (English) February 5, 2009 Using structured, signed informed consent forms, all survey respondents will be informed about the study objectives, procedures, benefits and risks. They will be assured that should they agree to participate in the study, they have the right to withdraw at any point in time and that they can refuse to answer any or all of the questions, even after consenting to participate in the study. After being fully informed about the study objectives, procedures, benefits and risks, the respondents will be asked to give oral consent if they wish to participate in the study. The interviewers will then sign and date the informed consent form, attesting to the fact that they received the respondent‘s oral consent. The informed consent forms have been translated into Luganda language. The informed consent to be used for each respondent follows: INFORMED CONSENT Hello. My name is ____________ and I am working with The QED Group, LLC evaluation team to evaluate AFFORD project. We are conducting a research study that asks men and women about various health issues. We would very much appreciate your participation in this survey. This information will help in the planning of health services and the AFFORD project activities. The survey usually takes between 10 and 15 minutes to complete. Your views will help in providing better health services to the greater community. The information collected in this survey will be utilized to improve programming related to increase access to and use of selected quality basic health products and services in the private sector, by women and men of reproductive age, as well as health products and services for children under 5 years of age. Participation in this survey is voluntary. We ask to interview you alone, to facilitate your honest responses. Whatever information you provide will be kept strictly confidential and will not be shown to other persons. If we should come to any question you don't want to answer, just let me know and I will go on to the next question; or you can stop the interview at any time. However, we hope that you will participate in this survey since your views are important. At this time, do you want to ask me anything about the survey? Do you agree to participate in the survey and may I begin the interview now? YES RESPONDENT AGREES TO BE INTERVIEWED: [START INTERVIEW] NO RESPONDENT DOES NOT AGREE TO BE INTERVIEWED: [END] Signature of Interviewer: ___________________________________ Signature of Interviewer: ___________________________________ Date: ____________ The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 105 ANNEX M : SHORT SUMMARY OF KEY INFORMANT INTERVIEWS The AFFORD initiative was widely recognized by almost all key informants at the national and district levels. However, the relationship and difference between UHMG and AFFORD was less clear. The majority of the key informants underscored the role of AFFORD distributor and promoter of certain health products and programs—such as LLINs and training of private health practitioners (basic palliative care; HIV counseling and testing)— who incidentally are often not targeted by government training and capacity building programs. ―AFFORD has been working with us since 2007; and it started with identifying and training of health [private] providers—clinics and drug shop. I think these were trained in basic palliative care. AFFORD later identified clinics with potential [for instance established infrastructure, qualified personnel] to provide HIV counseling and testing. They trained two staff from each of the identified health facility in HIV counseling and testing. AFFORD also distributed some mosquito nets in selected sub-counties [not sure how many], particularly to children under-five and pregnant mothers. They [AFFORD] also carried out promotional activities aimed at promoting use of mosquito nets and other AFFORD products such as Aqua safe.‖ (KEY Informant, Iganga District) AFFORD was lauded for its unique approach of working with the private drug distribution network—to increase the availability of health products. The volume of health products available in the different districts, covered during the study, was reported to have substantially increased over the last three years, due in part, to an increase in the number of private drug outlets (including clinics). These entities were reported to be playing a vital role in filling gaps left by the public health care system. However, monitoring use and dispensing of health products by private drug outlets remains a daunting challenge. Yet, as one moves away from urban centers to remote villages, most drug outlets are manned by ill-trained personnel which ultimately affects the quality of care. Key informants, at the district and national levels, observed several changes in health in Uganda, which were attributed to a confluence of factors, including specific interventions by civil society organizations, and targeted government health programs such as mass immunization, HIV campaigns, and others. The changes range from decreased incidence of malaria and diarrheal diseases to significant improvements in health care seeking behavior and the number of people seeking HIV counseling and testing services—albeit with variations between districts. While condom use was reported to have significantly improved; encouraging condom use among couples remains an up-hill task. Key informants were nonetheless concerned with increasing discordance rates, HIV prevalence, and the rising infections among married couples. According to The Modes of Transmission Study, 43 percent of new infections in Uganda occur among married couples (UNAIDS/UAC 2008). This increase was reported to be due to non-disclosure of HIV status between couples and an unwillingness of couples to be counseled and tested as couples. Therefore, key informants urged: renewing focus on identifying and promoting prevention messages; offering a more comprehensive and robust HIV counseling and testing service that enables ―couples to test as couples‖; and providing strategies that transcend the current client￾initiated HIV counseling and testing for couples. 106 Discussions also revealed that the contraceptive prevalence rates are generally low in most districts, despite slight improvements in the last three years. The fear of side effects coupled with refusal by male partners bar a significant number of women from using contraceptives. Key informants generally noted that men are very reticent and reluctant when it comes to discussing reproductive health issues, including family planning and HIV testing with their spouses. The initiative of training and using POLs to promote the utilization of health products socially-marketed by AFFORD and to encourage certain desirable health behaviors was praised as an innovative way to engage communities in the promotion of better health outcomes. POLs were reported to be playing a vital health promotion role including: counseling and referring community members with different health problems; dialoguing with community members on different health issues; and promoting use health products such as aqua-safe, LLINs, and contraceptives. However, key informants noted that POL activities are limited to a few sub-counties within different districts. Thus, their reach and effectiveness is limited. Nonetheless, a significant number of key informants in the districts expressed willingness to constitute and, where necessary, incorporate the trained POLs into VHTs, a much broader government community health initiative. Other key informant comments: Some AFFORD products such as moonbeads and mosquito nets, despite being subsidized, are still less affordable and limit demand. There is need to monitor the use of health products such as mosquito nets which will help inform the district strategies to improve utilization of health products. Staff turnover in private health clinics is significantly high. Some clinics lose staff trained which collaborate directly with AFFORD which affects continuation of programs. Consequently, there is a need for continuous and/or refresher training for staff in private health clinics. 107 Kampala WEST NORTH WEST NILE CENTER AND EAST GENERAL CONTEXT AND COMMENTS ACTIVITIES PERFORMED BY RESPONDENTS ON AFFORD PROJECT Trainer and supervisor in AFFORD projects; investigation of outbreaks; training and HIV testing; management of HIV test results; training and monitoring/follow up to the trainees; training. Distributor of AFFORD/UHMS products; focal person for AFFORD in district; trained by AFFORD and then offered training; supply of nets, and promotion of other health products; AFFORD focal person in district; training; training; distribution of AFFORD products; recruiting clients in districts for AFFORD products. Health educator/trainer; training of drug store owners; monitoring and evaluation of health activities; health promotion and education; mobilization and sensitization of communities on VHT; supply of long-lasting mosquito nets, and training in the use of these nets. Coordination and supervision of health-related activities; planning for health sector; training of health workers on new policy on malaria; supply of treated nets; training of drug store owners in their products; training on sanitation and HIV/AIDS and palliative care; distribution of mosquito nets. Training health workers; distribution of mosquito nets; worked with private sector and NGOs; training of private health practitioners; training of POLs; training of private healthcare services outlets and event promotional activities. 108 Kampala WEST NORTH WEST NILE CENTER AND EAST CHANGES IN HEALTH SECTOR IN UGANDA More private sector involvement in the health sector even in rural areas; people’s health seeking behavior improved; more demand for health care services; HIV care has greatly improved with more availability and affordable drugs. Reduction in malaria; increased use of contraceptives including condoms; prevalence of child diseases and epidemic diseases are under control; no major outbreaks; increase in deliveries in clinics; decrease in oral diseases among children; safe water coverage was improved leading to reduction of water related diseases; increased awareness leading to increase in VHCT; malaria cases have significantly decreased. Health seeking behavior has increased; attitude of health workers has been improving; male participation in family planning is still very low; malaria is on the increase; many people now embrace family planning; health seeking behavior is still negative; use of condoms has steadily increased; increase in use of treated mosquito nets; a change from preventive to curative measures; funding of the health sector has remained very low. There has been some improvement in health services; welfare of health workers has also improved; mosquito nets have greatly improved mothers’ health and reduced infant mortality; resources in health sector have been decreasing as compared to the health sector demands; motivation of health workers is quite low. There has been an overall reduction in the incidence and occurrence of diarrheal diseases; cases of malaria are still high although on a downward trend; HIV infection rates are rising among couples as is the discordance rate; HIV/AIDS prevalence has stagnated; non-disclosure between couple and unwillingness of couples to be counseled and tested together; general reduction in the incidences and severity of epidemics. CAUSES FOR CHANGES IN HEALTH SECTOR Increased community awareness by POLs at grassroots level; private sector involvement; distribution of the nets by AFFORD; reduction in stigmatization for using products; introduction of Aquasafe. Changed courses in counseling provided by: AIMS, UNICEF, NUMAT and AIDS Information Centre; flooding creates breeding ground for mosquitoes. Private and NGO involvement in the health sector e.g. malaria consortium. 109 Kampala WEST NORTH WEST NILE CENTER AND EAST EXPERIENCE WORKING WITH AFFORD PROJECT AS A PARTNER Training is well organized under AFFORD; good materials; support from office staff is good; good working relationships. AFFORD has very broad range of areas but networks with other organizations; no big difference from other partners the distributor works with; it’s fulfilling its objectives; work well with the district. AFFORD staff is good to work with; interaction and communication is good; AFFORD does not plan with the district; AFFORD is not clear about its activities; people to use VHT and Zinkid were not trained; AFFORD products are expensive; some products are not well promoted. AFFORD staff are cooperative; they follow the procedures and provide supervision and support for their activities. District is only involved at implementation stage, not at planning stage. CHALLENGES WORKING WITH AFFORD AS PARTNER Selection/identification of people to train was difficult initially. Out of stock of products; AFFORD funding is sent to district but indicative figures are not shown in the budget. 110 Kampala WEST NORTH WEST NILE CENTER AND EAST AFFORD BEST PRACTICIES Using the media to educate the public about their products; working closely with private sector (including testing); giving support to trainees after training; beating cultural misconception about HIV/AIDS through communication programs (e.g., Goodlife show); MARPs network; bringing organizations to work together; capacity building for organizations. Use of POLS; good working relationship with MOH and DHO; working within district policies and MOH policies; work closely with National Drug Authority; motivation of POLS; production of UHMG; translation of health booklets into local languages; working closely with private sector and government training to CBOs and POLs in health issues. Funding of trainings; use of the media radio talk shows; working with MOH and private sector; timely supply of AFFORD products; distribution of mosquito nets; introduction of purifiers; training health practitioners on palliative care; contribution to prevention of malaria; good advertisements; good communication with distributors; always enough stock. Follow set procedures; they do supervision and support for their activities; communication; distribution of nets attracted pregnant women for antenatal services; involvement of the district in its activities. Training of private health providers; working with private sector and NGOs; regular supervision and follow-up visits; capacity building of private health providers/POLS; working closely with private sector; provision of regular supervision and observance of strict timelines. CHALENGES IN HEALTH SECTOR THAT WILL NEED ATTENTION DURING THE NEXT 5 YEARS Coverage is still low, and covering everywhere is still a challenge. Male involvement in child health and prenatal care; UHMG should penetrate into villages; micro - element deficiency and nutrition; immunization against the 8 diseases; encourage/sensitize mothers to deliver at clinics. Sensitization of community on health issues; complement government efforts to have clinics at parish level. HIV counseling and testing; improving access to HIV prevention; treatment and support services; capacity building for health workers; nutrition for people living with HIV/AIDS; improving maternal and child health; increased access to vital drugs. 111 Kampala WEST NORTH WEST NILE CENTER AND EAST AFFORD OBJECTIVE I: Increasing access and affordability of health products and services through innovative marketing and distribution approaches CHANGE IN AVAILABILITY OF HEALTHCARE PRODUCTS (DRUGS, ETC.) TO THE POPULATION IN THE LAST 3 YEARS Basic drugs became more available even in rural areas; people are now more interested in socially marketed products because of quality; establishment of GoodLife clinics. Condoms, HIV products and FP products have increased; Restors and Zinkind; services brought closer by opening new drug shops through the work of AFFORD/UHMG; increase in number of pharmacies. Increase in availability health products e.g., condoms and Aquasafe, mosquito nets, ZinKid and other family planning products; mosquito nets, Aquasafe have increased; people have been sensitized on use of Aquasafe to prevent cholera. Increase in availability of health products e.g., condoms and Aquasafe, mosquito nets, ZinKid and other family planning products; moonbeads are less demanded; private involvement in health sector e.g., UHMG. Health products are increasingly available but expensive; volume of health products available in both private and public health facilities and on the market has increased; family planning products and condoms and water purifiers usage is limited by price; increase in the number of outlets; promotional activities and advertising campaigns have generated demand for certain health products. CHANGES THAT CAN BE ATRIBUTED TO AFFORD ACTIVITIES Increased sales and availability of AFFORD supported products; improved product availably since it is privately driven (due to work with AFFORD). Improvement in health behavior where POLS operate; increased availability of mosquito nets & moonbeads. Increase in supply of Protector condoms, Pilplan and Injectaplan, Aquasafe. Increased availably of health products in communities (due to POLs). 112 Kampala WEST NORTH WEST NILE CENTER AND EAST CHALLENGES There is a gap in linking clinics to distributors; logistical support is weak; no job security in private sector; turnover is high; competition between condoms for sale and condoms that are free; AFFORD policy does not allow advertising of condoms; free distribution of condoms by MOH; competition from other marketing groups e.g., MSI and PSI; no motivation to make trained staff stay in the clinic for long. Not involving males in PMTCT; AFFORD never leaves reports at the district; pilplan has no margin: cost price and selling price is the same. Men take the nets supplied to pregnant women; promotion of moonbeads and O condoms is low; they do not involve district authorities and VHTs; running out of stock of supplies especially nets; competition from LifeGuard; competition from other suppliers e.g., PSI; usage of nets being ignored; private clinics may not take training seriously; free condoms find their way into the market and are sold cheaply; mosquito nets were not enough; some NGOs buy in bulk and supply products for free; some pharmacies buy direct from Kampala and sell cheaply. There was demand for the products but products could run out of stock; procurement of stock is a length process; people’s past experience with other USAID projects e.g., AIMS and UPHOLD which wind up before impact is felt; irregularity of review meetings with the district officials; small margin on all products; packaging of protector is not attractive to customers. Competition from other condom producers; high prices for the products; people want free condoms; AFFORD activities are not incorporated in district plans; AFFORD’s pricing structure leaves room for customers to be overcharged; drug shops and clinics are not authorized to administer injectable contraceptives; new AFFORD products have not been promoted; staff turnover in private health clinics is significantly high; confinement to few subcounties; it doesn’t work closely with the district; AFFORD products are expensive. 113 Kampala WEST NORTH WEST NILE CENTER AND EAST AFFORD OBJECTIVE II: Enhancing knowledge, self efficacy and correct use of products and services to encourage health lifestyles. CHANGES IN HEALTH BEHAVIOUR OF POPULATION Health seeking behavior has improved. People's health service seeking behavior has improved including HIV testing; people look for health information (radios and talk shows); VCT has increased; seeking treatment from witch doctors has decreased; mobilization and sensitization undertaken by the district and the efforts of NGOs. People’s awareness on health issues has increased; valuing highly the socially marketed health products; condom use has increase; increased awareness on basic health issues. Contraceptive prevalence rate has been stagnant; improvement in mosquito net utilization in communities; condom use is slightly higher among the youth compared to adults; people feel comfortable paying for condoms; condom use among PLHIV and discordant couples is still poor; use of mosquito nets has slightly increased; increase in awareness; contraceptive use has also improved; improvements in latrine use; increasing normalization of HIV/AIDS and the risky behaviors spurred by availability of ARVs. 114 Kampala WEST NORTH WEST NILE CENTER AND EAST CHANGES THAT CAN BE ATRIBUTED TO AFFORD ACTIVITIES Significant improvement in health behavior where POLS operate. CHALLENGES There is a gap in linking clinics to distributors; logistical support is weak; no job security in private sector; turnover is high; competition between condoms for sale and condoms that are free; AFFORD policy does not allow advertising of condoms; free distribution of condoms by MOH; competition from other marketing groups e.g., MSI and PSI; no motivation to make trained staff stay in the clinic for long. Not involving males in the PMTCT; AFFORD never leaves reports at the district. Men take the nets supplied to pregnant women; promotion of moonbeads and O condoms; they do not involve district authorities and VHT; AFFORD does not work with district health team. There was demand for the products but products could run out of stock; procurement of stock is a lengthy process; people’s past experience with other USAID projects e.g., AIMS and UPHOLD which wind up before impact is felt; irregularity review meetings with the district officials; small margin on all products; packaging of protector is not attractive to customers. People want free condoms; monitoring use and dispensing of health products by private drug outlets; new AFFORD products have not been promoted and so demand is low; confinement to few subcounties. 115 Kampala WEST NORTH WEST NILE CENTER AND EAST AFFORD OBJECTIVE III: Strengthening/establishing indigenous organization and distribution systems for sustainable delivery of health marketing functions KNOWLEDGE ABOUT UHMG AND AWARENESS ABOUT ITS ACTIVITIES No visible difference between AFFORD and UHMG; AFFORD activities are actually implemented by UHMG; still a confusion between UHMG and AFFORD. There is no difference between UHMG and AFFORD. Find it a problem differentiating AFFORD from UHMG; UHMG has wide and varied services; UHMG is part of AFFORD. Find it a problem differentiating AFFORD from UHMG; knows AFFORD but not UHMG. UHMG is to take over from AFFORD. 116 Kampala WEST NORTH WEST NILE CENTER AND EAST RECOMMENDA￾TIONS FOR AFFORD AND USAID AFFORD should collaborate with other service providers/ projects; AFFORD should support whole range of services related to HIV e.g., testing and counseling; repackage protector condoms. AFFORD should work with UHMG for 5 years before its left on its own; AFFORD to involve districts in their plans; POLS should be spread to grass roots; Packaging of rectors is not good; Drug shops shouldn’t sell Injectaplan® since it has to be administered by a doctor; price for O condoms is too high, needs to be reduced. AFFORD should decentralize its services; motivate VHTs in form of T-shirts and caps; work directly with districts; regularly hold review meetings with the stakeholders; communicate before visiting distributors; revive radio talk shows; emphasis on utilization of mosquito nets instead of distribution of nets; increase marketing and making products known; have a desk office in each district e.g., through the district health officer; impact assessment of the AFFORD products on community; more training is needed; continue funding field activities; involve districts in planning, not just implementation; design a clear reporting and data storage system that works from the health facility level upwards. AFFORD project should be extended for additional years so as to have significant impact on the community or to be replicated in other areas; AFFORD should be flexible and open to collaboration in handling emergencies that are outside their program activities e.g., meningitis; AFFORD should facilitate districts/person implementing the project at the district with things like computers; AFFORD should not restrict distributors to regions; AFFORD officials should communicate before visiting distributors. Promotional materials e.g., t-shirts are needed; AFFORD needs to monitor proper utilization of the health care products e.g., mosquito nets; packaging of protector needs to be revisited; AFFORD needs to work closely with the districts. 117 Kampala WEST NORTH WEST NILE CENTER AND EAST SUSTAINABILITY OF UHMG AFFORD should not pull out resources after it closes, should leave everything for UHMG. UHMG should look for funding for its activities; UHMG should establish branches at districts. UHMG should increase publicity of the products; reduce the gap between providers and beneficiaries; UHMG should ensure that distributors especially in Kampala adhere to recommended prices; UHMG should reach out to rural areas. UHMG should source for funding from the government and other donor organizations. 118 ANNEX N : STATISTICS FOR RETAIL STORE SURVEY Main characteristics of the drug shops/pharmacies and respondents in the sample AFFORD or comparison district Primary AFFORD target Comparison Are you the proprietor or main manager or main sales person of the establishment 100.0% 100.0% Are you the proprietor of the establishment? 42.0% 45.4% Do you usually work in the establishment most of the time 96.2% 96.3% Did you ever have any health or health care education? 73.3% 75.9% Do you know if the establishment received supplies from AFFORD/UHMG distributors 79.4% 57.4% Do you know if the establishment sold/still sells products marketed by AFFORD/UHMG? 89.3% 76.9% Do you know if the establishment sold/still sells products marketed by International projects or donor organizations e.f USAID? 39.7% 35.2% How long has the establishment existed (in years) 6.62 5.19 How long have you worked in this establishment (in years) 4.43 3.76 Did respondent ever heard, saw or participated in any of the AFFORD/UHMG events? AFFORD or Comparison district Total Comparison % % Primary AFFORD target % Yes 79 80 79 No 21 20 21 Total 100 100 100 Is there a visible advertisement for AFFORD/UHMG events in a store? AFFORD or Comparison district Total Comparison % % Primary AFFORD target % Yes 59 76 68 No 41 24 32 Total 100 100 100 Does a store sell at least one of the AFFORD/UHMG marketed brands? AFFORD or Comparison district Total Comparison % % Primary AFFORD target % Yes 96 99 97 No 4 1 3 Total 100 100 100 The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 119 Sales of the AFFORD/UHMG marketed brands by retailers in the sample: Yes % No % Not applicable % Not answered % Do you sell Injectaplan? 87.9 6.7 5.4 Do you sell Pilplan? 90.8 3.8 5.4 Do you sell Protector condoms? 85.4 10.0 4.6 Do you sell Cotramox? 21.3 47.3 31.4 Do you sell Aquasafe? 39.7 51.0 9.2 Do you sell Zinkid? 45.2 45.6 9.2 Do you sell Restors? 64.4 26.4 9.2 Do you sell SoftSure? 4.6 89.5 5.4 0.4 Do you sell "O" condoms? 15.9 79.5 4.6 Do you sell NewFem? 3.8 90.8 5.4 Do you sell moonbeads? 12.6 82.0 5.4 Noticed that more customers coming to a store after AFFORD events/shows (overall sample)? Yes % No % Not applicable % Not answered % after Good Life shows on TV 10.5 18.8 70.7 after Good Life shows on Radio 33.9 20.5 45.2 0.4 after Good Life show column in newspapers 18.0 18.0 64.0 after under the mango tree on radio 16.3 14.2 69.0 0.4 after HIV prevention films 22.6 48.1 29.3 after Pulse activation theatre/show in your town/village 6.3 24.7 69.0 after meeting with community leaders to discuss health related issues with AFFORD/UHMG 14.6 29.7 55.2 0.4 120 Did respondent noticed more costumers coming to the store given that there was this type of show in the store location (respondent‘s participation/knowledge of the show as proxy for the availability of the show/even in the location). Did you ever hear/see/ participate in the show/event % Total % Type of show/event: Noticed increase in customers Yes No Not answered Good Life shows on TV Yes 35 0 0 10 No 61 0 100 19 Not applicable 4 100 0 71 Good Life shows on Radio Yes 61 0 100 34 No 37 0 0 21 Not applicable 1 100 0 45 Not answered 1 0 0 0 Good Life show column in newspapers Yes 48 0 18 0 No 48 0 18 0 Not applicable 4 100 64 0 Under the Mango Tree on radio Yes 48 0 16 0 No 42 0 14 0 Not applicable 9 100 69 0 Not answered 1 0 0 0 HIV prevention films Yes 33 2 23 0 No 61 24 48 0 Not applicable 6 73 29 0 PULSE activation theatre/show in your town/village Yes 44 4 0 6 No 25 25 0 25 Not applicable 31 72 100 69 Meeting with community leaders to discuss health related issues with AFFORD/UHMG Yes 73 3 15 0 No 23 31 30 0 Not applicable 5 65 55 0 Not answered 0 1 0 0 Over the last year, how did the total sales of this product category changed: Increased % Not changed % Decreased % Not applicable % Not answered % Male condoms 61.1 18.0 16.3 4.6 Other contraceptives 77.0 11.7 6.3 4.6 0.4 Vitamins 57.7 29.3 3.8 8.8 0.4 Child health-related drugs 63.2 20.5 6.7 9.2 0.4 HIV/AIDS and other STI treatments 51.9 10.9 5.0 31.4 0.8 The AFFORD Health Marketing Initiative in Uganda: Mid-term Evaluation 121 ANNEX O : STATISTICS FOR EXIT SURVEY AFFORD Comparison Total Number of observations 706 509 1,215 including Clinic exit survey 553 380 933 Retail exit survey 153 129 282 Type of area where respondents live Capital 48 48 District capital/town 326 146 472 village/rural 328 363 691 did not answer 4 4 Gender Male 41% 39% 40% Female 59% 61% 60% Marital status Single 25% 20% 23% Married/living together 65% 69% 67% Divorced/Separated 6% 6% 6% Widowed 4% 4% 4% Did not answer 1% 1% 1% Age of respondent (average) 31 32 31 Household size (average) 6 6 6 Number of children in your household (average) 3 3 3 Did respondents even heard, saw, or participated in AFFORD/UHMG show or event Primary AFFORD district % Comparison district % Total % Yes 66 58 63 No 34 42 37 Respondents knowledge about specific AFFORD promotion activities Yes % No % Did not answer % Did you ever hear/see Good Life show on TV 21 78 1 Did you ever hear/see Good Life show on radio 48 51 0 Did you ever see good life show column in newspapers 16 84 0 Did you ever hear under the mango tree on radio 28 72 0 Did you ever see everyday health matters newspaper 11 89 0 Did you ever see HIV Prevention films 53 47 0 Did you ever participate in Good Life show prize drawings 3 97 0 Did you ever participate in pulse activation theatres/show 5 95 0 Did you ever participate in meeting with community leaders to discuss health related issues organized by AFFORD or UHMG 8 92 0 Did you ever participate in training on healthcare issues or other events organized by AFFORD or UHMG 4 95 0 Did you ever participate in mosquito net distribution from AFFORD, USAID or UHMG 8 92 0 122 Over the last 3 year, how did respondents personal health behavior changed Increased % No changes % Decreased % Non applicable % Did not answer % use mosquito nets? 59 34 7 0 use of water treatment for drinking water (though boiling or tablets)? 48 49 3 0 use of vitamins and mineral supplements? 19 79 2 1 use of condoms for family planning? 16 80 3 1 use of pills, injections or other birth FP products? 14 41 5 40 0 use of condoms or other products to prevent HIV/AIDS? 26 70 3 1 visits to health clinics? 54 31 15 0 visits to pharmacies and drugshops? 45 40 14 1 Over the last 3 year, how did respondents family health behavior changed Increased % No changes % Decreased % Did not know % Not applicable % Did not answer % use mosquito nets? 51 30 5 1 12 1 use of water treatment for drinking water (though boiling or tablets)? 40 42 2 2 12 2 use of vitamins and mineral supplements? 28 47 2 8 12 3 attention to the health care of children under 5 in the family? 46 20 3 3 27 1 use of condoms for family planning? 9 51 1 24 12 4 use of pills, injections or other birth FP products? 6 41 0 10 40 3 considering potential risks to infectious diseases? 22 35 7 23 12 2 use of condoms or other products to prevent HIV/AIDS? 9 40 1 36 12 4 Knowledge and participation in AFFORD shows among those who had positive and negative change in specific health behavior: Increased (positive behavior change) Decreased (negative behavior change) Health behavior Heard/saw/participated in AFFORD shows Heard/saw/participated in AFFORD shows No % Yes % No % Yes % use mosquito nets? 33.70 66.30 43.00 57.00 use of water treatment for drinking water (though boiling or tablets)? 36.20 63.80 36.40 63.60 use of vitamins and mineral supplements? 24.00 76.00 34.80 65.20 use of condoms for family planning? 28.10 71.90 32.40 67.60 use of pills, injections or other birth FP products? 45.00 55.00 32.70 67.30 use of condoms or other products to prevent HIV/AIDS? 24.30 75.70 24.30 75.70 visits to health clinics? 29.40 70.60 29.20 70.80 visits to pharmacies and drug shops? 30.80 69.20 26.70 73.30