USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION JUNE 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Tiffany Lillie, Serigne Seye, and Bernard Sene through the Global Health Technical Assistance Bridge Project. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-01-038) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00004. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION i ACKNOWLEDGMENTS For their assistance in the evaluation, the GH Tech Bridge evaluation team thanks the following individuals, in particular Dr. Seynabou Mbengue, ADEMAS executive director, and Dr. Cheikh Sarr, deputy director, for offering their office space and making themselves available to the team throughout the mission. The team also would like to recognize the ADEMAS staff for their assistance in managing the logistical aspects of the evaluation and for providing us with the necessary support in conducting interviews and focus group discussions in the field. The team also thanks ADEMAS’s partners, the Ministry of Health, and the Senegalese people who participated in the evaluation. The team is particularly appreciative of the guidance and support received from USAID/Senegal in completing this evaluation. Lastly, the team recognizes the wonderful opportunity this evaluation gave in working with ADEMAS and the stakeholders in the USAID/Senegal Social Marketing Program Component of the 2006–2011 portfolio. ii USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION iii CONTENTS ACRONYMS ........................................................................................................................................ v EXECUTIVE SUMMARY ................................................................................................................. vii Findings ......................................................................................................................................... vii Challenges and Limitations ...................................................................................................... viii Recommendations ......................................................................................................................ix I. INTRODUCTION ....................................................................................................................... 1 II. METHODOLOGY ....................................................................................................................... 3 Purpose........................................................................................................................................... 3 Technical Approach ..................................................................................................................... 3 Objectives ...................................................................................................................................... 3 Data Collection ............................................................................................................................ 3 Interviews and Focus Groups .................................................................................................... 4 Literature Review ......................................................................................................................... 4 Evaluation Team ........................................................................................................................... 4 III. FINDINGS ..................................................................................................................................... 7 Objective 1: To what extent were the program’s stated objectives met? How effective was the program approach in meeting the stated objectives? .......................... 7 Objective 2: Were program activities and implementation sound? ...............................19 Objective 3: Did the program achieve the coverage expected? ..........................................29 Objective 4: To what extent were the program approaches sustainable? ...................35 IV. CONCLUSIONS ........................................................................................................................39 V. FUTURE PERSPECTIVES AND RECOMMENDATIONS .................................................41 Sustainability ................................................................................................................................41 iv USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION ANNEXES ANNEX 1. STATEMENT OF WORK .........................................................................................47 ANNEX 2. LIST DES PERSONNES RENCONTRÉES .............................................................59 ANNEX 3. BIBLIOGRAPHY ..........................................................................................................61 ANNEX 4. CVS .................................................................................................................................63 ANNEX 5. QUESTIONNAIRE ADEMAS ...................................................................................81 ANNEX 6. QUESTIONNAIRE IDI USAID.................................................................................85 ANNEX 7. QUESTIONNAIRE IDI MSP ET PARTENAIRES ADEMAS ...............................89 ANNEX 8.QUESTIONNAIRE IDI PARTENAIRES PRIVES ADEMAS..................................91 ANNEX 9. POPULATION QUESTIONNAIRE FDG ..............................................................95 FIGURES Figure 1. SMP Commercial Strategy for Condoms and Aquatabs .........................................12 Figure 2: Cost Effectiveness Key Parameters .............................................................................23 TABLES Table 1. In-depth Interviews ............................................................................................................. 4 Table 2. Focus Group Discussions .................................................................................................. 4 Table 3. Total Program Expenditures during Program Budget Implementation ................24 Table 4. Funds Returned to the Project, 2007–2011 ...............................................................24 Table 5. Cost Allocations ................................................................................................................25 Table 6. Distribution Channels for PROTEC Condoms ..........................................................26 Table 7. Aquatab Sales by Distribution Channel ........................................................................27 Table 8. POS Numbers, 2007–2011.............................................................................................27 Table 9. Achievement of Program Indicators .............................................................................29 Table 10. Recommendations Related to Organizational Structure .......................................41 Table 11. Recommendations Related to Donor Dependency ................................................42 Table 12. Recommendations for Overcoming Barriers ...........................................................43 Table 13. Recommendations Related to Communications .....................................................44 Table 14. Recommendation Related to Product Placement ...................................................45 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION v ACRONYMS ADEMAS Agence de Développement du Marketing Social BCC Behavior change communication CA Contracted agency CBO Community-based organization CHW Community health workers CME Continuing medical education CNLS National Council for the Fight against AIDS in Senegal CPR Contraceptive prevalence rate CYP Couple years of protection DHS Demographic Health Survey DPL Direction de la Pharmacie et des Laboratoires DPIC Direction de la Protection Individuelle et Collective DSR Division de la Santé Reproductive FGD Focus group discussions FP Family planning GOS Government of Senegal HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome IA Implementing agency ICS Industrie Chimique du Sénégal IDI In-depth interviews IEC Information, education, and communication IR Intermediate results IRIS Institute of Research and Investigations by Survey KfW KfW Entwicklungsbank, German development bank M&E Monitoring and evaluation MCH Maternal and child health MHO Mutual health organization MOH Ministry of Health NGO Non-governmental organization ONPS Ordre National des Pharmaciens du Sénégal OC Oral contraceptive ONMS Ordre National des Médecins du Sénégal PEPAM Programme d’eau potable et d’assainissement du Millénaire vi USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION PDIS Programme de Développement Intègre de la Santé et l’Action sociale PMC Population Media Center PNA Pharmacie Nationale d’Approvisionnement PNDSS Plan National de Développement Sanitaire et Social POS Point of sale PPP Public-private partnership PSI Population Services International PSP-One Private Sector Partnerships-One R&D Research and development RH Reproductive health RPF Return to project funds RTI Research Triangle Institute RTS Radio Television Senegal SIR Sub-Intermediate Results SMO Social marketing organization SMP Social Marketing Program SNEIPS Service National de l’Education, Information et Promotion de la Santé SODEFITEX Société de Développement des Fibres et Textiles STI Sexually transmitted infection UNACOIS Union nationale des commerçants et industriels du Sénégal UNFPA United Nations Population Fund USAID United States Agency for International Development WHO World Health Organization USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION vii EXECUTIVE SUMMARY At the request of the United States Agency for International Development (USAID)/Senegal, the GH Tech Bridge project conducted a performance evaluation of its Social Marketing Program (SMP). The goal of the program, implemented by ADEMAS, the Agence de Développement du Marketing Social, was to seek innovative ideas about health commodities and services that could be marketed to Senegalese providers and the public. The challenge is to expand the range of products and services that are socially marketed. The evaluation team reviewed activities to date (2006–2011) and identified successes, gaps, and constraints in program implementation. The team conducted in-depth interviews (IDI), focus group discussions (FGD), and a literature review. Twenty-seven IDIs were conducted with central-level stakeholders (e.g., USAID, ADEMAS, the Ministry of Health [MOH], ADEMAS partners); private sector participants (e.g., pharmacists, private physicians, boutiques); and public sector participants (e.g., midwives, nurses), and there were six FGDs with ADEMAS target populations. FINDINGS The evaluation team found that the SMP served its social mission by increasing access to family planning (FP)/reproductive health (RH), sexually transmitted infections (STI)/Human Immunodeficiency Virus (HIV)/Acquired Immunodeficiency Syndrome (AIDS), and diarrheal disease products and services in Senegal. Because no other organization was active in this area, the SMP filled a serious gap in the Senegalese health care system. In general the SMP not only met but surpassed its original targets. For instance, FP educational sessions reached 31,307 people (139% of the original target), FP refresher training sessions reached 1,228 people (113%), 1,246,396 cycles of Securil were distributed (117%), and 49,919 Depo-Provera injections were dispensed (272%). SMP products were leaders in their markets; each was among the lowest-priced products available and also among the most accessible. The SMP was also able to add to its product portfolio an injectable contraceptive, Depo-Provera, and diarrheal disease prevention tablets, (Aquatabs). The SMP developed successful partnerships and good management systems in order to achieve these results. In elaborating the program the SMP brought in private, public, and other partners. The number of points of sales (POS) increased by about 27% over the five years, from 5,367 to 6,788. Collaboration with the public sector focused on conducting trainings and distributing certain products, such as Aquatabs. Collaboration with the private sector focused on attending trainings and the promotion and sales of the SMP’s products. The SMP also served as an important link between the private and public sectors. However, based on the IDIs and FDGs, the private and public sectors would like to be more involved in the program. Examples included wanting to become more formal partners and acting as both implementers of activities and distribution points (e.g., through the Midwives Association); working together to draft work plans and set objectives; profiting from ad disseminating SMP lessons learned; and increasing knowledge of the products. The SMP focused more on the distribution and supply of products and less on the social or behavior change aspects of social marketing. It met its targets for “people reached” and viii USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION “trained,” but while the quantitative targets were met, the quality and effectiveness of these types of programs fell short, perhaps due to the lack of financial resources. Most program activities were one-time rather than repeat interactions with the same group of people. A notable exception was the radio serial drama, Ngelawu Nawet, with accompanying listening groups; this should be continued. The public sector stated that the SMP should have more behavior change communication (BCC) activities in the field. Representatives of both the private and the public sectors felt that the SMP staff was too concentrated on Dakar and should have an expanded regional presence, which would both increase the number of BCC activities and give the private sector more face-time with staff, as its representatives desire. The sustainability of the SMP is a major challenge. The program was slow to act on the recommendations of several assessments conducted during the five-year project. Sustainability issues identified related, for instance, to product lines needing to be added and to be more profitable; the need to align organizational and management systems with a new for-profit structure; and the need for more staff. CHALLENGES AND LIMITATIONS ADEMAS, the implementing agency (IA) of the SMP, reported several challenges—too little time to design the project, financial resources, and cultural and social norms—that affected how the program operated. Originally, ADEMAS said, the SMP was not intended to be a stand-alone project but was integrated into the larger USAID maternal and child health program being managed by IntraHealth. When it was apparent that the SMP needed its own IA, USAID asked ADEMAS for a quick proposal for a five-year $3.5 million project. ADEMAS drafted the proposal and was granted the project, but the lack of planning time and the change of status meant that the organization did not have enough staff to execute the larger-scale program; nor did it have enough time to strategize on a system that would be more sustainable. A second problem was that the SMP had only $3.5 million for a five-year project that had to have national coverage. Staff had to be kept to a minimum, which reduced interactions with partners and reduced the number of SMP activities. The high cost of advertising in Senegal meant that the promotional campaigns and TV/radio spots had to be pared. Community-based BCC activities also had to be kept to a minimum compared to the planned scope of activities that would have made the SMP fully operational. The ADEMAS budget for research was quite low, which affected its ability to gauge the population’s needs and track sales and use of the products. The social context was another challenge. Though use of modern FP/RH products in Senegal is quite low, leaving much room for growth, it was difficult to transcend gender and religious norms. The SMP is working to change these norms, but it will take time and considerable effort. A limitation of this performance evaluation was that it was conducted more than eight months after the SMP ended and a year after the last field activities. Therefore, participants may have forgotten or been less accurate then they could have been if the evaluation had been closer to the end date. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION ix RECOMMENDATIONS Based on the findings, the following are recommended: Conduct studies on potential social marketing activities and new areas of business to increase the SMP’s product lines and revenue sources. Identify the ideal legal status for IA sustainability based on available data by choosing the most salient option, specifying the person responsible, and executing a time-bound plan. Formulate a financial sustainability strategy that takes into account all possible aspects of profit—commercial, logistics, and production; draft an action plan; and appoint a professional to be responsible for its realization. Establish a new organizational and management system for the IA, with sufficient staff to support the sustainability plan. Create formal and sustained partnerships with both private and public sectors and associations and other groups, to strengthen ownership, increase distribution points, and expand BCC activities. Create a social marketing training center as a revenue center to build the capacity of health providers and social marketing practitioners throughout West and Central Africa. Expand SMP monitoring and evaluation to incorporate formative assessments; baseline, mid￾term, and final evaluations; and specification of both quantitative and qualitative indicators (e.g., behavior change process of group, pre/post tests for trainings). Conduct a formative assessment of target populations to explore attitudes, beliefs, values, barriers, life goals, media outlets, sources of information, health care practices, and other behaviors related to FP/RH, STI/HIV, and diarrheal disease. Draw up a strategic communications report, based on data and behavioral theory, to guide the SMP in implementing a sound, evidence-based social marketing plan that has a communications and behavior change focus, and identify or create tools to support it. Define a system to document lessons learned and share findings with partners to create dialogue to continually improve the SMP. In conclusion, there is still much that the USAID SMP needs to accomplish if it is to increase demand for and use of FP/RH products. Knowledge of contraceptive methods in Senegal is almost universal (over 90% knew of at least one modern method), but only 10% of married women used a modern method. However, the SMP has made great strides in overcoming logistical, management, social, and cultural barriers, achieving success on a number of indicators. Continued support will help the SMP to establish a more sustainable system that can ensure that FP/RH products and services will be available to all those who desire them. x USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 1 I. INTRODUCTION USAID/Senegal’s Social Marketing Program (SMP) was established to ensure the continued existence of socially marketed products in Senegal and to foster growth of the private sector’s contribution to family planning (FP). Its main goal was to “contribute to improvements in the well-being and quality of life of the population in general and, specifically, of families.” To accomplish this, ADEMAS promoted good health, especially reproductive health (RH), behaviors through information, education, and communication (IEC) activities, research, and training and by making health products and services more accessible. ADEMAS, a Senegalese nongovernmental organization (NGO), was created in 1998 and operated as a subgrantee to the USAID Commercial Marketing Strategies project. As part of this agreement, ADEMAS as a USAID implementing agency (IA) received technical assistance and funding for social marketing the condom Protec®; the oral contraceptive Securil® was added to the ADEMAS portfolio in 2002. Since September 2003 ADEMAS has been implementing USAID/Senegal’s health social marketing activities under the initial cooperative agreement. In 2006 ADEMAS became a sole recipient of USAID funds for a five-year project that was part of the USAID/Senegal Health Sector Strategy. A new five-year contract was signed in March 2012 for completion in 2016. The results of this performance evaluation will feed into the new project and work plan. Products socially marketed as part of the USAID/Senegal Health Program were male condoms (Protec®); an injectable contraceptive (Depo-Provera®); an oral contraceptive (Securil®); and a point-of-use water purification tablet (Aquatabs®). Also implemented were behavior change/social marketing activities for improved hygiene and sanitation within the USAID/RTI/PEPAM program, funded through the USAID/Senegal Economic Growth Office, although the primary support came from the USAID/Senegal Health Office. In addition, the Social Marketing Program (SMP) was designed to contribute to the following Intermediate Results (IR) and Sub-Intermediate Results (SIR): IR 1: Improved quality of services, health products and information SIR 1: Sufficiently trained, equipped and supervised service providers to apply the norms and protocols of the Ministry of Health (MOH) IR 2: Increased use of health services, products and information SIR 1: Increased demand for health services and products and for healthy behaviors SIR 2: Improved access to services, products and information ADEMAS targeted the following specific results for the SMP: Increased sales of Protec male condoms and Securil oral contraceptives Launch of the injectable contraceptive, Depo-Provera Increased use of socially marketed products to help increase the total couple-years of protection (CYP) 2 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Behavior change promoted in an innovative entertainment-education approach through a radio serial drama Increased social acceptance of targeted healthy behaviors to support the USAID health strategy A development plan drafted and implemented to work toward ADEMAS sustainability. Most of the SMP products were supplied to ADEMAS free by USAID or KfW (KfW Entwickungsbank, the German development bank), except for Depo-Provera, which Pfizer has supplied since 2009. The products were packaged, stored, and distributed by Valdafrique, ADEMAS’s main commercial partner. The other formal implementing partners were the Population Media Center (PMC), ChildFund, and IntraHealth. PMC created and broadcast the radio serial drama, Nguelawu Nawet. ChildFund and IntraHealth implemented other BCC activities, such as listening groups and community outreach. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 3 II. METHODOLOGY PURPOSE USAID/Senegal believes that social marketing is a useful resource for promoting public health products and services in Senegal. The purpose of this performance evaluation was to (1) determine the extent to which the expected social marketing results were achieved; and (2) provide pertinent information on how the program was implemented that will inform USAID/Senegal’s decision-making about its new social marketing and health communication and promotion program. This evaluation will provide a comprehensive look at the SPM component of the USAID/Senegal Health portfolio as implemented by ADEMAS from 2006 through 2011. TECHNICAL APPROACH USAID sought innovative ideas about health commodities and services that could be marketed to providers and the public. The challenge was to expand the range of health products and services that were socially marketed in Senegal. The evaluation team therefore reviewed activities implemented to date and identified successes, gaps, and constraints. OBJECTIVES According to the Scope of Work (Annex 1), the objectives of the study were to: Determine if program objectives were met. Determine the effectiveness of the program’s approach in meeting the objectives. Determine if program activities were sound. Determine if the program achieved the coverage expected. Determine if program approaches are sustainable. Provide perspectives and recommendations for the future. DATA COLLECTION Data collection was based on document review and qualitative methods. Data from the documents reviewed was triangulated. In-depth interviews (IDI) and focus group discussions (FGD) were used for the qualitative portion of the study. The SMP worked in all 14 regions of Senegal, but each health domain (e.g. family planning [FP], HIV, and water and sanitation) was not necessarily represented in each region. The evaluation focused on Dakar and two regions, Thies and Kaolack, to ensure geographic and population representation. The three regions were chosen based on both the representation of the SMP’s social marketed products and on timing constraints for the evaluation. All products and services were available in Kaolack. The only product not available in Thies was Aquatabs, because diarrheal disease is minimal there. The radio program, Ngelawu Nawet, targeted rural areas; it was not broadcast in Dakar, where residents are more interested in television than radio. Participants who were selected for the evaluation had collaborated with the SMP at some point during the program and 4 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION were aware of the project and its products, which aided the evaluation. Participants for IDIs and FGDs were sampled from all three regions. Criteria for participant selection were based on participation in SMP activities between 2006 and 2011; therefore, the sampling was purposeful rather than random. INTERVIEWS AND FOCUS GROUPS The objective of the IDIs and FGDs was to elicit stakeholder feedback on SMP activities to date and identify accomplishments, gaps, and constraints in those activities. Eleven central-level stakeholders in SMT activities were interviewed in Dakar. Five to six public and private sector representatives per region were interviewed. Two FGDs per region were conducted with the target population. In total, 27 IDIs and 6 FGDs were conducted (see Tables 1 and 2 for illustrative sampling and Annex 2 for the list of interviewees). Table 1. In-depth Interviews Stakeholders Number Per Region (3 Total) Total Stakeholders in Dakar (e.g. MOH, USAID, KfW, Child Fund, IntraHealth, Valdafrique, ADEMAS) n/a 10 Public sector representatives (providers, midwives, nurses) 2–3 6–9 Private sector representatives (providers, pharmacists, points of sale) 2–3 6–9 Total 22–28 Table 2. Focus Group Discussions Stakeholders (10 participants per group) Number Per Region (3 Total) Total Target population (listening groups, women’s associations, FP/RH users, youth groups) 2 6 Total 6 LITERATURE REVIEW The evaluation team reviewed all documents pertaining to the USAID/Senegal Health Program, the USAID/Senegal SMP Component, and Government of Senegal (GOS) health activities. These documents served first as background reading and second to provide responses to questions in the Scope of Work (see Annex 3 for a list of documents). EVALUATION TEAM The evaluation team consisted of three public health experts plus one team member who was added for the three days of data collection in Thies. Serigne Mbaye Seye, the team leader, has over 20 years of experience in BCC, education, strategic planning, monitoring and evaluation (M&E), and project management, especially in sub-Saharan Africa. Bernard Noel Sene has over USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 5 15 years of experience in commercial and social marketing, especially in West Africa. Tiffany Lillie has over 15 years of experience in behavioral sciences, M&E, and BCC (see Annex 4 for CVs). The first three days of the evaluation were devoted to meetings with USAID and ADEMAS, reading background documents, drafting the methodology and questionnaires, and assembling a list of central and regional contacts. During this time, the team also began scheduling central and regional meetings and preparing for the field visits. ADEMAS helped the evaluation team to schedule the IDIs and FGDs. Finding another team member for data collection in Thies and organizing transportation were also on the first week’s agenda. The team conducted as many of the central-level IDIs as possible during the first week in Senegal. During the second week, the original team members and the Thies supplementary member were divided into teams of two to collect data in Kaolack and Thies. During the third and fourth weeks, two of the original team members continued to collect data in the Dakar region. After data collection, the team finalized the transcripts. Analysis was based on themes that emerged from all the transcripts, which were analyzed by target group—central level, public sector, private sector, and target population participants. 6 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 7 III. FINDINGS OBJECTIVE 1: TO WHAT EXTENT WERE THE PROGRAM’S STATED OBJECTIVES MET? HOW EFFECTIVE WAS THE PROGRAM APPROACH IN MEETING THE STATED OBJECTIVES? SMP’s Role in Increasing Demand for and Use of FP Products ADEMAS and other health partners have a great opportunity to increase demand and use of certain FP products. Based on the 2005 Demographic Health Survey (DHS), knowledge of contraceptive methods in Senegal was almost universal (over 90% knew of at least one modern method), but only 10% of married women used a modern method. The goal of the SMP was to improve maternal and child health and reduce HIV/AIDS prevalence. SMP provided an important, perhaps unique, social service to the Senegalese public health system. One government official stated: Les activités de marketing social pour le changement de comportement, viennent combler le vide laissé par les politiques et le gouvernement. English: The social marketing activities to change behaviors fill a gap in the Senegalese government. SMP had the funding, personnel, knowledge base, and experience in implementing social marketing programs that no other organization had. Therefore, they were filling an important gap in the current health system. Social marketing programs seek to bring together the social, commercial, and marketing fields to influence and support positive consumer behavior. The SMP was strong on the commercial side of the program, focusing as it did on the distribution and supply of products. With an innovative approach, SMP was highly successful in distributing products, increasing point-of-sale (POS) networks, and partnering with new businesses. At the end of the project, there were nearly 6,800 POS, 127% of the original target. However, as discussed below under communications strategy, the social and behavior change communication (BCC) aspects of the SMP were not as effective. Communication Strategies to Increase Demand The SMP’s communications approach incorporated trainings, awareness and sponsoring sessions, talk sessions, TV/radio spots, a serial radio drama with accompanying listening groups, and large￾scale events. ADEMAS itself led some activities, such as trainings, talk and awareness sessions, and large-scale events, but worked with other partners (PMC, Child Fund, and Intra Health) on the radio serial drama, Ngelawu Nawet, and BCC activities in the field. The SMP organized a large number of communications activities with supportive materials such as posters, brochures, calendars, pens, buckets, leaflets, t-shirts, and hats. Among the activities were these: Interpersonal Trainings (with capacity-building tools) Home visits 8 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Group Talk and awareness sessions Listening groups Community Ngelawu Nawet radio program Radio and TV spots Promotional items (posters, brochures, prescription pads, calendars, pens, buckets, leaflets, t-shirts, hats) Large-scale events (e.g., religious events, caravans) One limitation of the program was that most of its communications activities were one-time activities to raise awareness and increase knowledge. The notable exception was Ngelawu Nawet, which is discussed below. The majority of communication activities did not fully address such pertinent topics as the cultural barriers that women may experience or how to build communication, negotiation, and practical use skills to increase demand for and use of specific FP/RH products. While one-time events and activities are important, they should not predominate. Raising awareness and increasing knowledge are important communications objectives but again should not be the only messages emphasized. Because most Senegalese are already aware of modern FP methods, the emphasis of the communications approach should be on dismantling barriers to change and factors that motivate use of FP/RH methods. Data and behavior theory should be utilized in the communications strategy to enhance the impact of activities. Peer-reviewed studies of the evidence base for behavior change activities has found that common elements in the most effective programs are focusing on specific behavioral goals and specific cognitive factors that affect those behaviors (e.g., knowledge, perception of risk, values, attitudes, peer norms, skills, intentions); employing multiple participatory activities that address those cognitive factors; giving clear messages about behavior; and implementing a large number of activities with the same group. Since most of its communications activities were one-time events, the social marketing program did not go into detail on the curriculum or agenda used for them. There was little or no description of how or if behavior change theory was used in designing activities or what behavioral goals and specific cognitive factors were to be addressed; nor did it appear that participatory learning methods were used or that there were ever multiple sessions with the same group of people, which could have promoted long-term behavioral changes. It is recommended that the SMP formulate a communications strategy based on current epidemiological, social, and cultural data. The strategy should list target segmented populations, the behavioral and cognitive factors the activities are targeting, behavioral theories, participatory activities, curricula and materials for the activities, and the number of sessions for each activity and population. The strategy should be specific for each unique product; knowledge, awareness, barriers, motivational issues, populations, demand, and use will vary with each. Each activity, whether mass media, community, group, or individual, should have defined goals, objectives, messages, and target populations. Stakeholders, partners, and representatives of the target USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 9 population should be consulted on the design of the strategy and conceptualization and creation of materials to accompany activities. There should also be an implementation plan for each that lists the person or organization responsible, geographical area covered, schedule, and budget. Ngelawu Nawet Radio Program One of the strengths of USAID’s SMP was the Ngelawu Nawet radio serial and the listening groups. Mass media have been found to be most effective when reinforced with community activities; with these programs, the SMP appears to have been responsive to the evidence base. PMC created and produced the drama, and Child Fund organized and ran the rural listening groups. In the ADEMAS final report, PMC was shown to have followed best practices in broadcasting Ngelawu Nawet by involving stakeholders in design of the program, basing the script content on research results, training and selecting script writers based on evaluations, using culturally appropriate markers (e.g., music), pre-testing and validating pilots, and revising forthcoming episodes based on listener feedback. PMC used the SABIDO methodology in creating the entertainment-educational drama and drew on five theories of communication and behavior change. The main characters acted as “good,” “bad,” and “transitional” characters to demonstrate how choices affected health outcomes that were to be copied or avoided by the target population. PMC broadcast 168 episodes between February 2009 and late 2011. This performance evaluation and one conducted by the SMP reached conflicting conclusions about the level of awareness of the program. This evaluation found knowledge of the program to have been low. Of the seven public sector participants asked about Ngelawu Nawet, four knew of the program but one stated that it was not well known among the population. Only two of thirteen private sector participants reported knowing about the drama. Of the six FGDs conducted, only two actively listened to the program, but those two had very favorable reactions. One group in Thies reported: Ngelawu Nawet est une émission très appréciée. C'est un phénomène sociétal qui bénéficie d'une grande écoute. L'émission constitue une source d'apprentissage et d'éveil ; elle a un réel impact sur les comportements. English translation: The Ngelawu Nawet program is very much appreciated. It is a social program that has a large following. The program is a source of information and is increasing awareness; it has a real impact on behavior. L’émission est suivie par toutes les femmes du village. Le message de démarrage de l’émission se fait instantanément. A la fin de celle-ci les femmes discutent sur les épisodes traitant par exemple du traitement du rhume chez les enfants, de la conservation des aliments, des causes et traitements des maux de ventre, de la tuberculose, du SIDA, etc. English translation: The program is followed by all the women in the village. The messages are understood instantly. At the end of the program, women discuss the episodes, such as how to treat childhood colds, properly storing food, and causes and treatment for stomachaches, tuberculosis, AIDS, etc. Reasons FGD participants gave for not listening was that they did not listen to the radio (they watched TV instead) or did not listen to the channel on which Ngelawu Nawet was broadcast. Despite the evaluation team’s findings, the ADEMAS Final Report stated that about half (52%) of women had listened to Ngelawu Nawet program based on a monitoring evaluation that was 10 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION conducted. The themes that were most remembered were FP (44%), maternal health (16%), and malaria treatment (9%). The discrepancy between the results of the current evaluation and the SMP monitoring may have to do with differences in sample participants: FGDs for this evaluation were mostly conducted in urban and peri-urban areas, whereas the SMP monitoring exercise may have had a wider sample from rural populations. It is recommended that USAID retain Ngelawu Nawet in the current SMP but do a more focused study of such factors as who listened to the program; how many people listened and how often (number of episodes); where they were located (rural, peri-urban, urban); what program messages they retained and whether they discussed them with friends and family; and whether the program influenced their decisions and behaviors. A more focused study exploring these issues would help USAID modify the program to increase the number of listeners and tailor messages and programming. Activities and Materials in the Field Many interviewees and FGD participants recommended that SMP do more in the field, which supports the finding that the SMP emphasized one-time activities. They also commented that SMP did not have a strong presence in the regions but was too focused on Dakar. ADEMAS reported that although its product promoters were based in Dakar, every month they did one￾week to ten-day tours in the regions. SMP’s presence is the field was limited to these tours and to some trainings, events, and activities in the regions, but all of these equated to just 25% to 30% of staff time being spent in the field. One central level stakeholder said, ADEMAS n’a-t-elle pas suffisamment de personnel pour bien mener ses activités… 5 personnes pour couvrir 14 régions quand même, c’est faible. English translation: ADEMAS does not have a sufficient number of staff to implement their activities … there are five people to cover 14 regions. That is weak. Both private and public sector participants were asked whether they were familiar with the SMP communications approach in communities, and many said more field activities were warranted. Of the private participants who were familiar with the SMP community communications approach (6 of 13), five thought that it could be improved by giving more training to accompany products and doing more (e.g., health talks and radio spots) to communicate with the people. All the public sector interviewees who were asked (6 of 8) were familiar with SMP community communications. Almost all of them (5 of 6) stated that SMP needed to increase its activities in the regions. Examples given of what to increase were number of health talks per month, number of communication materials to accompany activities, and use of nontraditional channels to reach target populations. ADEMAS’s partners in Dakar, interviewees in the regions, and FGD participants also requested more community behavior change activities. One FGD suggested: Augmenter le temps dédié aux causeries. Le temps ne suffit pas parfois pour bien discuter (quelque fois moins de 30 mn)…Approfondir les causeries, les thèmes évoqués lors des causeries. English translation: Increase the time dedicated to the health talks. The time is not sufficient to discuss the topics in detail. Sometimes the sessions are less than 30 minutes … increase the number of health talks, and the topics that are discussed. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 11 Any BCC activities conducted by the SMP were implemented during the 25%-30% of staff time spent in the field. While more frequent and longer sessions and trainings do not guarantee higher-quality programs, they are indicators of how deeply program staff can go into a particular topic and the diversity of subjects that can be covered within a month. For example, the pharmacists as a group requested more training. One pharmacist said he had received one-day trainings in 2005, 2007, and 2010, which suggests that trainings were not offered annually but only every several years. Some pharmacists also suggested that other staff members in the pharmacy be trained because they have direct contact with consumers. Public sector representatives also wished to see more frequent BCC activities in the field; one midwife recommended more than one or two sessions a month. The earlier focus group statement also indicates that the women would welcome more frequent and longer BCC sessions. Product promotional materials (posters, brochures, calendars) also were reported to be lacking at various POS. Eight of the 13 private sector interviewees reported that quantities of promotional materials needed to be increased. One pharmacist commented: Le principe est bon, mais les outils de communication sur les produits sont presque inexistants. English translation: In principle it is good, but communication materials about the products are almost nonexistent. ADEMAS confirmed that promotional materials were limited and not all POS received them, and the evaluation team observed this to be true: few SMP promotional materials were visible in either private or public establishments. It is recommended that the SMP increase its presence in the field by engaging in more frequent and in-depth BCC activities, so that it can link more closely to regional private and public stakeholders. There are several ways that might be done: establishing regional offices with permanent staff, having staff spend more of their time in the field, or a combination of the two. Private Sector Provision of FP Products and Services The 2006–2011 SMP worked with a variety of private partners, such as the Union Nationale des Commerçants et Industriels du Sénégal (UNACOIS), associations of pharmacists, private clinics, traditional retailers, restaurants, and supermarkets, to increase the number of community distribution points (see Figure 1). Involvement of the private sector was vital in increasing the provision of FP products and services in Senegal. Pharmacies and clinics on the one hand and wholesalers and traditional retailers on the other are also central to promoting health and providing services to the population. Figure 1 demonstrates how ADEMAS worked with a number of partners on its commercial strategy. 12 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Figure 1. SMP Commercial Strategy for Condoms and Aquatabs Valdafrique Wholesalers ADEMAS Pharmacies Supermarkets Traditional retailers, hotels, bars, and other retail POS NGOs and Association Partners Retail POS ADEMAS Outlets Health huts Health cases Community-based organizations (CBO) CONSUMERS USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 13 Private Clinics and Pharmacies The private medical sector is a vital SMP partner; often in a given community there are more private than public providers. It was found that the population appreciated the services provided by the private medical sector and relied on them heavily. One FGD participant said, Il y a plus de discrétion quant à la clinique que dans les structures publiques. English translation: It’s more discreet to go to the clinic or see a private doctor compared to seeing public doctors. For these reasons, it was crucial that they be involved in the SMP. The SMP had a dedicated team of medical promoters whose exclusive mission was to visit organizations in the private medical sector, such as pharmacies and clinics. The SMP mainly involved private sector participants in product launches, trainings, and workshops on specific topics. The objective was to increase both their involvement in the project and their capacity to counsel patients on STI/HIV, FP/RH, and diarrheal disease issues and products and dispel associated myths and rumors. The SMP also sought to increase product distribution and sales through pharmacies and the number of prescriptions written in private clinics for SMP products. Nevertheless, pharmacists and private physicians thought that the SMP did not involve them enough; they stated that the SMP could have done more to involve them in endorsing products and in community activities. Some private physicians had created associations or community organizations connected to their clinics to improve knowledge about medical issues and to exchange information on medical topics. Sometimes the same people were involved in both private sector and SMP activities, and it would have been more effective and efficient to have one high-quality event instead of two. It was also found that the SMP action plan was less flexible due to administrative and financial constraints. Thus, it mostly asked private participants to support and implement SMP activities rather than becoming formal and active partners. It is recommended that the SMP continue to work with the private sector to also improve and strengthen it. The SMP should draw up joint action plans with the private sector and other stakeholders to synergize and harmonize community activities. In establishing closer partnerships with the private sector, the program would be more cost-effective, enabling it to do more using resources that already exist. Retailers, Boutiques, and Traditional Wholesalers Based on the interviews, POS staff were mostly satisfied with their collaboration with the SMP but thought that it could be improved. The majority said they were predominantly involved in increasing sales or in commercial promotions (e.g., buy 2, get 1 free) of socially marketed products and in product launches and other large-scale events. However, most of them said that SMP field staff were mostly concerned with sales and not with enhancing the quality of the program, such as improving their knowledge of certain products, providing marketing materials, and ensuring that products were not past their expiration date. It is recommended that the SMP continue to work with private POS establishments but in closer partnership with them. The SMP should draw up joint action plans with POS stakeholders because they are an important link to the community and cover a large geographical area. Quality measures such as the ones mentioned should be guaranteed. 14 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION The Private Medical Link with the Public Sector Private and public sector participants stated that in Senegal the sectors were not closely linked and that the SMP worked with both to try and strengthen the collaboration. The SMP created opportunities for those in the private sector to discuss their own issues and priorities with the public sector. The goal was to establish a national plan that was more efficient and was harmonized across both sectors. For example, the SMP invited pharmacists and clinic owners to participate in national planning workshops organized by the public sector. The intended outcomes were to draw up a national plan that was accurate and exhaustive and to ensure that private sector issues were incorporated into the national platform. Other opportunities for collaboration were SMP invitations to special dinners to enhance public￾private communication and SPM training sessions involving both sectors. Training tools available to public and private participants aimed at harmonizing the information they provided to clients. However, these initiatives were infrequent because of SMP staff and fund limitations. It is recommended that the SMP work with the MOH to formalize public-private sector interactions and integrate private sector needs and information into the national plan. Because the public-private link is vital for the MOH, it should be responsible for finding resources to support this activity. Increasing Access to FP/RH Products and Services The SMP utilized many strategies to increase access to FP/RH products and services. For instance, it Conducted information and education campaigns to increase understanding of the health benefits of FP methods; Expanded access to more effective FP methods, particularly Depo-Provera, condoms, and Securil products; Increased public and private sector partnerships; and Trained pharmacists and practitioners on such topics as how to counsel clients and provide information about how to socially market products. Overall, the SMP was successful in increasing access to FP/RH products and services in the sense that targets were mainly met and often surpassed. FP educational sessions reached 31,307 people (139% of the target) and FP refresher training sessions reached 1,228 people (113%). Over the five years of the project, 1,246,396 cycles of Securil were distributed (117%), and 49,919 Depo-Provera injections were given (272%). Nearly 19.5 million units of Protec and 8.5 million Fagaru condoms were distributed through nearly 6,800 POS. The program partnered with both the public and private sector to achieve these positive results. These figures represent a total of 328,000 of CYP, about 20% of total CYP in the country, through the promotion of Protec, Securil, and Depo-Provera products; the original target was surpassed by 115%. Despite the fact that the SMP contributed considerably to increasing access to FP products and services in Senegal, there is still room for improvement. In an FGD, members of the National Midwives Association expressed concern about the unmet need among women in rural and suburban areas, which is still very high, and the last DHS showed that fertility rates were still USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 15 high at 6 children per woman aged 15 to 49. Use of modern FP was about 6.5% among the same age group. There is thus a great opportunity for the SMP to reach more individuals whose needs for FP/RH products have not been met by partnering with associations, organizations, public health care staff in rural areas, local NGOs, and formal partners like ChildFund. Some of these organizations have considerable experience; others will need SMP support to scale up their activities. Raising Awareness of New Products Depo-Provera® The official launch of Depo-Provera on July 15, 2009, coincided with the launch of the national FP campaign. According to the ADEMAS Final Report, sales of Depo-Provera during the last two years of the project totaled 49,169 syringes, 272% above the target of 18,100 syringes. SMP’s staff worked to increase sales by providing medical information in visits to private providers, organizing talk sessions, designing and distributing promotional items and leaflets on how to handle side effects, and creating an information sheet for providers to help them dispel any rumors their clients had heard. The most used FP method in Senegal was the injectable, based on preliminary data from DHSV/MICS-2010–2011, but Depo-Provera only made up 4% of the SMP total product share and 2% of its budget. Given the public preference for the product, SMP should give more emphasis to the sales and marketing of Depo-Provera. In 2011 an Institute of Research and Investigations by Survey (IRIS) study of socially marketed products found that the majority of women were aware that injectables (79%) were a modern FP method; of these, 71% were currently using Depo-Provera and 17% had used it in the past. FGD participants in this evaluation had less brand recognition but did mention the injectable as a method of contraception. Since product recognition is one objective for social marketing programs, the new SMP needs to raise the visibility of the Depo-Provera brand. If the previous SMP had had a larger marketing budget for Depo-Provera, sales might have been even higher. Aquatabs® Aquatabs was launched in 2010; this may be why this was the only product in the ADEMAS portfolio that did not reach the 100% target, although it did reach 75%. Based on the ADEMAS Final Report, most of the sales (90%) were not through pharmacies and sales could be increased within the pharmaceutical sector (10%). The Final Report also inventoried SMP promotional activities, such as trainings, awareness and talk sessions, home visits, and radio and TV spots to increase awareness, knowledge, demand for, and use of the product. The 2011 IRIS study found that 18 months after its launch about half (49%) of surveyed households knew of the Aquatabs brand. In the FGDs during the performance evaluation it was reported that Aquatabs was slightly recognized and used by participants. Both studies demonstrated the success of the SMP in promoting the brand. However, the Aquatabs campaign had some weaknesses. Several stakeholders stated that in promoting and distributing the product, SMP had started out strong and then faded. Stock-outs were reported, and the target population was not aware that Aquatabs were available in 16 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION pharmacies. One Department to Protect the Individual and Community (DPIC) representative said, Aquatabs existe dans les pharmacies mais la population n’est pas suffisamment informée sur ce canal de distribution English translation: Aquatabs exist in the pharmacies but the population does not know that was a distribution channel One reason ADEMAS gave for lower-than-expected sales was the limited budget: Pour Aquatabs le budget nous permettait seulement de passer 15 jours sur le terrain alors que pour les autres produits comme Protec, c’était facile car nous avions les ressources pour la publicité et les activités communautaires. English translation: For Aquatabs, the budget did not permit us to go beyond 15 days in the field or put enough money into advertising. For other products, like Protec, it was easier since we had the resources for advertising and community activities. Thus, if the SMP had more funds for promoting the product, perhaps through long-term community activities, sales might have been increased. Given the marketing and distribution budget, Aquatabs targets may also have been too high. Since SMP surpassed so many other original product targets, the problem with Aquatabs might be something other than the ability of ADEMAS to implement a campaign. In the future, budget and targets should be better aligned so that products can be effectively promoted and distributed. Securil® and Protec® The two predominant SMP products were Securil and Protec. Based on the ADEMAS Final Report, the distribution of product share that contributed to the total CYP was 50% for Protec and 25% for Securil, followed by 21% for Fagaru and 4% for Depo-Provera. Sales of Securil in pharmacies reached 1.5 million cycles, 117% of the target. Sales of Protec totaled 19,490,777 units, 83% of the original target. The SMP program was highly successful in raising awareness of Protec. The IRIS study found that 96% of men were aware that condoms are an FP method. Of this group, the majority knew of the Protec brand (88% in spontaneous response and 91% through assisted response) and 75% had used or were using it. One pharmacist said, La plupart de la population connaît parfaitement les condoms Fagaru et Protec, cependant Protec est plus connue même si le produit n’est pas disponible dans le centre. English translation: For the most part the population knows of Fagaru and Protec condoms. Protec is better known even if it is not available in the center. The SMP was less successful in raising awareness for the Securil brand, though most women knew that oral contraception was a FP method. The IRIS study found that a large majority of women were aware of the pill (94%) as a modern method. However, of those women who knew that the pill was a form of contraception, only 16% named Securil spontaneously as a brand and with an assisted response 21% named Securil as a brand. The IRIS study did find that Securil was the most recognized brand, though Lofemenal was also mentioned by 7% of women. Of the women who knew that the pill was a form of contraceptive, 38% were already using USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 17 Securil and 24% had used it in the past. Although not widely recognized generally, in this study Securil was the most recognized brand of pills. Findings from this evaluation were similar in that FGD participants were aware of the pill generically as an FP method but had difficulty identifying the Securil brand name. Four of the six FDGs were reportedly FP users, so for them not to know the brand name suggests an SMP weakness. On the whole, though, the SMP was successful with both products, but in unique ways. Securil surpassed its sales target but few could name it; Protec did not meet its sales target but the majority of men knew its name. It is recommended that the SMP formulate a communications strategy specific to each product. The findings in this section note the differences between products and underscore the importance of having unique strategies for each. The Regulatory Environment for Hormonal FP Products Regulations have significant impact on the effectiveness of a health program. The SMP must work within the current regulatory and legal environment to dismantle barriers, such as the following which emerged during the evaluation as directly affecting the efficiency of ADEMAS: Injection: Because a pharmacist does not have the authority to administer Depo-Provera, the user must spend considerable time and money going to the pharmacy, purchasing the product, returning to a licensed provider, and waiting for the injection to be administered. So many steps tend to discourage use. One doctor stated: Un médicament ou produit de santé doit être facile à utiliser. Son usage ne doit pas être contraignant, car il décourage le patient. C'est un gage de réussite de toute approche de planning familial. English translation: A drug or medical product must be easy to use and not full of constraints to discourage use by patients. That is the best way to guarantee and enhance the use of family planning products. To address this issue, ADEMAS conducted an advocacy campaign to increase the number of medical personnel and private medical staff who could administer Depo-Provera. They also drew up a plan to increase the capacity of pharmacists to properly counsel patients and administer the injectable to ensure that quality of care remained high. If the regulations are changed, demand and use should increase. The hierarchy and organization of the medical sector is heavily regulated in Senegal. For example, guidelines are elaborated by the government and outlined in the PNDSS (Plan National de Développement Sanitaire et Social) and all partners must follow the guidelines. However, individual organizations also have their own administrative procedures, organizational and financial constraints, timelines, and priorities. Any delay or dysfunction in the government or organizations may undermine collaboration and coordination between them. Managing this environment requires sophisticated planning, coordinating, and monitoring skills. Oral contraceptives are classified in the A category, which means that they can only be sold to those who have a medical prescription. The SMP worked to simplify the acquisition and 18 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION sales process in order to increase access by transferring the classification to the C category, where oral contraceptives could be sold without a prescription. The SMP has been waiting for the Ministry of Health to approve this change. Business companies not equipped with internal clinics are not allowed to stock contraceptives (e.g. Depo-Provera), although their employees are potential consumers. However, few Senegalese companies do have internal clinics; therefore, when the SPM visits workplaces for BCC activities to increase acceptance, demand, and use of FP/RH products, those interested in using them must go to a doctor before they can obtain them. If all workplaces were able to stock and manage FP/RH products, the products would be readily available to employees. ADEMAS was working to reduce these access barriers at the time of the evaluation. This regulation also prevented the SMP from establishing public-private partnerships (PPPs) for distribution of FP/RH products. The SMP has worked to create a more conducive commercial environment while also targeting the private sector to increase distribution of and access to its socially marketed products. Pharmacists, clinics, and POS are vital partners in the SMP given the regulations related to these products. In social marketing, it is critical to ensure regular and easy access to products to encourage repeat usage. However, the heavily regulated medical environment did not facilitate easy use of the private sector. First, distribution of FP products is prohibited in easy-to-access establishments such as workplaces not equipped with clinics. Second, by limiting their involvement, it reduced the ability of pharmacists to offer advice on or recommend other products that the pharmacy had available. If pharmacists could give injections, they would be much more involved in patient care and could recommend other FP/RH products as well as counsel the customer. The SMP combined approaches to overcome these barriers to distribution, usage, and sales by lobbying to change regulations and working with the private sector despite relatively few financial resources to increase sales and usage of the products. The SMP also used community activities to reach target populations to increase acceptance of, demand for, and use of FP/RH products. Thus, they worked simultaneously in several areas to increase SMP impact. However, future SMPs need to accomplish more. The new USAID SMP set out specific actions in its 2010–2014 Strategic Plan to improve the regulatory and legal environment for hormonal FP products, which could improve usage and sales. For example, the current legal status of ADEMAS is “association,” so that it cannot undertake trading or commercial activities. To sustain its program and became more efficient, the IA must offer first- and probably second￾generation socially marketed products in order to achieve an accurate and sustained program on its own. Clearly, it must comply with legal requirements but it must also identify products and activities that allow it to make a profit or at least break even. In 2009, at the request of USAID/Senegal, Private Sector Partnerships-One (PSP-One) conducted a rapid assessment of the current and potential markets for FP products and services in the private health sector and made recommendations for strengthening the sector in Senegal. The following points, though mainly based on results of the PSP-One evaluation that used the social marketing sustainability continuum, are still highly applicable: USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 19 Engage new staff, such as the partnership building manager, to create new opportunities and POS. Increase partners in each region. Put in place a strong formal partnership with UNACOIS Improve public relations approaches by increasing the number of socially marketed products. OBJECTIVE 2: WERE PROGRAM ACTIVITIES AND IMPLEMENTATION SOUND? Effective Coordination and Collaboration between the Social Marketing Program Component and the Other USAID Program Components The social marketing activities the SMP implemented for USAID/Senegal Health Program Components were well-coordinated, according to interviews with representatives of ADEMAS, USAID, and ADEMAS partners. ADEMAS used the same finance, administration, and technical staff to manage the FP/RH, STI/HIV/AIDS, and diarrheal disease components, which simplified coordination. ADEMAS viewed the USAID SMP as an integrated whole, discussing all the health topics together in trainings and education sessions, rather than in separate sessions. One interviewee mentioned that during health talks on Aquatabs other topics, such as malaria, tuberculosis, and hygiene, were also discussed. According to USAID and other stakeholders, ADEMAS established solid relationships with such formal partners as Valdafrique, ChildFund, PMC, and IntraHealth to create a strong, streamlined project. For example, Child Fund implemented listening groups in rural areas to stimulate community discussion on health issues presented in the radio serial Ngelawu Nawet, which was produced by another partner, PMC. Child Fund also contributed to the rollout of Aquatabs by leading community activities. In addition to managing the radio serial, PMC disseminated messages on such USAID priority health topics as FP, HIV/AIDS, water and sanitation, and malaria. These examples illustrate how ADEMAS partners worked to help the SMP disseminate the full range of messages, not just one message for one product. Representatives of the private and public sectors were asked about the type and degree of collaboration they had with the USAID SMP. Public sector participants were very positive about their level of collaboration with the project. The majority said that they had collaborated with ADEMAS on product launches (e.g., Aquatabs, Depo-Provera), presentation and participation in health talks, and conducting trainings. However, a common theme in public sector interviews was the need for SMP to collaborate with them more closely to increase promotional and educational activities in the field. Midwives and other health personnel have many opportunities to increase awareness, knowledge, demand, and use of socially marketed products, which would improve public health. The SMP should capitalize on these resources. A member of the Department of Hygiene staff said, ADEMAS doit se servir des associations féminines pour une meilleure distribution des produits au niveau communautaire. English translation: ADEMAS should use women’s associations to improve the distribution of projects at the community level. 20 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION A member of the Association of Midwives added that, Pour Securil, nous pouvons bien aider à booster la consommation, même pour Aquatabs. Pourquoi notre siège ne servirait-il pas de point de distribution par exemple. English translation: For Securil, we could really help increase use; the same for Aquatabs. Why not have us act as distribution points for the products, for example. The midwives and other health personnel in each community, urban and rural, would be excellent resources for ADEMAS. They could also be used as individual POS. The midwives could also offer additional health education sessions in the field as well as provide socially marketed products. The program could develop curricula, with accompanying communication materials, for them to use. The midwives would conduct health talks and women could access the products at the same time, creating a synergy between the program and community health staff. Throughout the evaluation there were numerous examples given of such potential collaborations using existing personnel and structures. ADEMAS could strengthen partnerships with established organizations, associations, and structures to increase acceptance of, demand for, and use of their social-marketed products. Private sector interviewees were less enthusiastic about their collaboration with the USAID SMP. What they primarily talked about was collaboration in terms of the logistics of receiving and selling SMP products, though attending trainings and launch events and participating in health talks were also mentioned. Of the 13 private sector participants, 7 stated that ADEMAS could do more to collaborate with the sector. These are some of their comments: Depuis plusieurs années, la collaboration avec ADEMAS n’a pas été effective, encore moins régulière. English translation: For several years, collaboration with ADEMAS has not been effective; it hasn’t been regular. Pas de protocole commercial établi avec ADEMAS, et pas d’objectifs spécifiques assignés. English translation: There is no commercial protocol established with ADEMAS. There are no specific objectives assigned. Il n’y a pas d’activités d’accompagnement des produits. English translation: There are no activities that accompany the products. Améliorer la communication avec les partenaires et partager les leçons apprises et impliquer toutes les parties prenantes dans la promotion de l’ensemble des produits de Ademas. English translation: Improve upon communications with partners, share lessons learned, and involve all pertinent partners in the promotion of ADEMAS products. Thus participants would like to improve the program by having more regular contact with ADEMAS staff, defined objectives and goals, and a better understanding of the products and the issues surrounding them, and by closer communication so private sector participants have a greater role. It is recommended that ADEMAS increase the staff and regional presence to meet needs such as these. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 21 The Value of the Stand-Alone SMP to the USAID/Senegal Health Program IDI and FGD participants recognized the value the social-marketing component added. They reported that the social-marketing approach was the only one capable of reaching youth with products like condoms that prevent STDs, HIV, and unwanted pregnancies. As a student said during a focus group in Thiaroye (suburb of Dakar), Nous apprécions beaucoup le travail de ADEMAS. C'est la seule organisation qui se soucie des jeunes, qui les informe et leur founit les conseils et produits contraceptifs. English translation: We really appreciate the ADEMAS social marketing component because it is the only one that cares about informing and offering products and services to youth. The stand-alone SMP provided products and services to under-served populations like youth that the national health system was not reaching because of insufficient resources and expertise. Its approach was based on responding to the needs of youth and women, reaching them where public health facilities have limited access. The program placed considerable effort on information and education sessions, behavior change interventions, and campaigns to increase understanding of the health benefits of socially marketed products. The program’s success was built on placement of products in more accessible geographical locations, pricing based on the ability of the target population to pay, and partnerships with the public and private sectors. An increased variety of POS and making health products available through the private sector and various NGO channels increased public access to much-needed products and services and created value-added for the stand-alone SMP. ADEMAS staff reported that more could have been achieved if more funds were available. In fact, they and other partners said that some planned activities were abandoned due to lack of financial resources. Recognizing the value the stand-alone SMP added to the USAID/Senegal Health Program, the evaluation team recommends that the program structure remain. Financial resources are important but not at the center of the success of the SMP. A balance between resource allocation and level of effort of the staff should be considered to sustain the approach. SMP Structure, Scope of Activities, and Effectiveness The SMP organizational structure was based on both donor and legal status requirements as well as on optimizing work in terms of staff capacity and the funding available. The evaluation team found that activities like increasing POS, distribution, and sales volumes were directed toward achieving targets, almost all of which were surpassed. As a pharmacist said in an in-depth interview, ADEMAS was a competent social marketing organization that combined effectiveness and efficiency and “played a critical role in cementing” the collaboration between public and private sectors in the field. The staff mix was mostly appropriate for the goals of the USAID SMP. The diverse professional backgrounds of staff matched the different components of the program. SMP staff was composed of pharmacists and administrators as well as specialists in public relations, promotion, and communications. Based on the interviews and discussions with ADEMAS staff, the workload was substantial but they were content with the job environment and the team spirit atmosphere. 22 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION However, findings from the IDIs and FGDs indicated that there were too few SMP staff to carry out all the activities required for education, communication, distribution, supervision, monitoring, evaluation, research, and reporting. SMP staff spent most of their time in the capital, which caused some of the gaps already noted. The evaluation team recognized that the number of staff was limited by the lack of funds. ADEMAS functions as a project rather than a long-term entity; it is therefore not stable enough to be able to guarantee long-term employment for professional staff. Its sustainability will be a crucial question in terms of maintaining the staff and restructuring to meet the demands of a long-term for-profit organization. Cost-Effectiveness of Various SMP Processes The SMP staff reported that the quality/price ratio of their products was a top priority. In fact, product detailing and placement was based on the users’ ability to afford certain products. However, the SMP team recognized that certain products, such as Depo-Provera, could have been priced more affordably for most women. Advertising costs are very high in Senegal compared to the rest of West Africa. For example, the SMP did not broadcast many TV spots during big events because the cost was not only high but rose annually. Due to the limited budget, the SMP reduced its emphasis on TV and used smaller mass media such as pamphlets, t-shirts, hats, and posters. The SMP used one station, Radio Television Senegal (RTS), as its main channel for marketing products and services because RTS agreed to reduce its cost to further the SMP goal. The SMP also sought to increase its reach cost-effectively by implementing most of its BCC activities through partners (e.g., ChildFund during the launch of Aquatabs, church-based organizations for community activities). Overall, the SMP managed a cost-effective program (see Table 4 and subsequent discussion) while trying to create synergy between the different channels and tools used to change behaviors and encourage target populations to use promoted products. For instance, The SMP had a small but strong promotional team that covered a large geographical area in an effort to be both task- and cost-effective. Instead of having separate team leaders for promoters and medical delegates, both were combined into one staff position. This increased communication and created a more aligned, focused, and efficient program. Cost-effectiveness analysis, defined as a modeling technique that selects among competing wants where resources are limited, helps stakeholders make decisions on how to achieve important health benefits while managing funds carefully. Modeling is a design for economic studies in which the consequences of different interventions are measured against a single outcome, usually in ‘natural’ units (e.g. life-years gained, deaths avoided, heart attacks avoided, cases detected). Alternative interventions are then compared in terms of cost per unit of effectiveness. Measuring cost effectiveness for the SMP is complex since it is linked to the benefits of offering socially marketed products to consumers. For the SMP in Senegal, data for use in a model was limited. There had been no impact evaluation that would allow the evaluators to track parameters quantitatively. However, the analytical process can be simplified as shown: USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 23 Figure 2: Cost Effectiveness Key Parameters Budget management is easier to measure; quality and quantities are more challenging because they are linked to program results. However, quality (e.g., BCC activities implemented effectively, targets reached with evidence-based BCC programs) and Quantity (e.g., number of activities implemented, sales volume per product) are key to determining how accurate the ratio is of budget or costs to achievements. It is evident from the ADEMAS Final Report (Table 9, Achievement of Program Indicators) that the 2006–2011 quantitative targets were met and surpassed. It was also found that the budget evolved to meet the changing programmatic needs as the SMP added more products to its portfolio. However, there was a research gap that must clearly influence any evaluation of the effectiveness of the SMP in Senegal, and that is the lack of impact studies. Any SMP must conduct impact evaluation studies to measure its effectiveness on such indicators as product placement, availability, and whether there was behavior change among the target population. The Financial Picture The SMP successfully managed its budget to meet most of its targets over the course of the five￾year program. Table 3, Column G demonstrates how ADEMAS was able to manage the program according to budget; the burn rates of six of the nine line items were close to 100%, and only three surpassed 100% (Other Direct Costs, Protec, and Securil). Overall, the SMP was eight percentage points above budget, whereas the quantitative target (the amount of in-country private financial resources leveraged for FP/RH in US dollars) was 123%, which meant that ADEMAS brought in more than planned. Thus, the ratio of SMP budget to programmatic target was well-matched. 24 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Table 3. Total Program Expenditures during Program Budget Implementation A B C D E F G Budget Items Total Budget Revised 10/25/2010 Total Disbursement Remaining Budget Projected (RPF) Variation Based on RPF Burn Rate % D=B-C F=B+E-C G=C/B A. Personnel 1,370,516,676 1,370,318,884 197,792 197,792 99.99 B. Other Direct Costs 678,753,598 701,353,749 -22,600,152 -22,600,152 103.33 C. Equipment 112,079,209 107,504,048 4,575,161 4,575,161 95.92 D. Protec 863,330,791 1,123,202,011 -259,871,220 286,957,258 27,086,033 130.1 E. Securil 425,729,278 546,725,663 -120,996,385 128,611,834 7,615,449 128.42 F. Injectable 67,424,206 66,777,013 647,193 647,193 99.04 G. Institutional Support 18,366,004 18,123,154 242,850 242,850 98.68 H. PMC (Radio Serial Drama) 240,747,028 223,934,443 16,812,585 16,812,585 93.02 N. Aquatabs 239,119,066 197,411,450 41,708,392 41,708,392 82.56 Total 4,016,066,632 4,355,350,416 -339,283,783 415,569,092 76,285,308 108.45 Note that the line items for C: Equipment, G: Institutional Support, H: PMC, and N: Aquatabs were all under budget, which compensated for some of the losses in other line items. Other examples that show a balanced budget were these: Burn rates exceeded 100% for D. Protec (130,10%) and E. Securil (128,42%) but were supported by Return to Project Funds (RPF; see Tables 4 and 9). RPF refers to amounts ADEMAS got back from commercial ventures. ADEMAS used the RPF to cover several line￾item shortfalls. In the five years of the program (see Table 4), ADEMAS received CFAF 299,093,172 as RPF, against the budget of CFAF 242,709,600 (see the ADEMAS Final Report, p. 34), for an achievement rate of 123% (see Table 9). The additional RFP funds were used to strengthen the SMP activities, as had been agreed with USAID. This shows that the SMP achieved the expected results. Table 4. Funds Returned to the Project, 2007–2011 Description Jan–Dec 2007 Jan–Dec 2008 Jan–Dec 2009 Jan–Dec 2010 Jan–Sep 2011 Total RPF Received Return to Project Funds (RPF) 58,324,272 42,100,416 67,199,520 65,585,584 65,883,380 299,093,172 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 25 Budget Allocations Table 5 shows the budget allocation to various SMP components; SMP FP products accounted for 34% and Aquatabs and the radio serial each accounted for 6%. The rest (54%) was dedicated to personnel, equipment, and direct costs. Table 5. Cost Allocations Description Achievements Personnel 34% Other direct costs 17% Equipment 3% Protec 21% Securil 11% Injectable 2% Institutional support 0% PMC (radio serial drama) 6% Aquatabs 6% TOTAL 100% As ADEMAS has said, the amount of funds was not sufficient to cover all planned SMP activities and support costs (e.g., research, TV spots, community BCC personnel, equipment). For future SMPs, it is critical to ensure that More of the budget is allocated to product activities, such as promotion, advertising, and BCC activities to increase acceptance of and demand for the products. Funds are dedicated to market research to help design social marketing campaigns and as a sustainability strategy. There is an appropriate amount dedicated to M&E (e.g. baseline, mid-term and final evaluations) and research (formative research, impact studies, marketing studies). The SMP product line is revised as needed and there is continuous strategizing on how existing or new products can create profits to ensure a more sustainable SMP. The evaluation found that the SMP could increase investments in certain areas to improve upon the program. For instance, participants noted: ADEMAS a besoin cependant de renforcer le volet R&D. Cette insuffisance affecte la pertinence du M&E, notamment sur l’évaluation des coûts/efficacité dès lors qu’il n’existe pas de données scientifiquement pertinentes. English translation: ADEMAS needs to strengthen its research and development. It affects the relevance of the monitoring and evaluation data, notably in evaluating the cost and effectiveness of the program. Important scientific data do not exist. 26 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION ADEMAS doit créer une synergie, par un effet de levier entre les produits, les activités et les campagnes publicitaires pour être plus efficace. Ce qui nous conduit à collaborer intensément avec le SNEIPS, le CNLS, IntraHealth, Childfunds etc. English translation: ADEMAS has to create synergy to improve products, activities, and advertising to increase effectiveness. We have to collaborate intensively with SNEIPS, CNLS, IntraHealth, and ChildFund. Improvements could be made on a number of fronts: 1. Additional funds could have been allocated to product activities to increase acceptance, demand, and use. 2. The for-profit product portfolio could have been expanded to create a more sustainable funding base for the SMP. 3. Funds might have been pulled from various line items and put toward research if SMP advocated for such a change with its donors. It is clear that any SMP must be supported by evidence to accurately understand the needs of the target population and construct a strategy to achieve results cost-effectively. 4. The program must also be supported by a strong M&E system that provides accurate measurements on indicators. It is challenging to create a high quality and sustainable social marketing strategy without data or results from impact evaluations. Additional funds might have been allocated to this activity in the past project to better track the impact of the SMP. Product Detailing and Placement within Partnerships The SMP’s distributional activities, as well as the quality of the products, helped to increase sales. Those interviewed for the evaluation reported that product quality was high. In addition, the RPF indicator demonstrated how sales were better than expected (see RPF description and ADEMAS Final Report, p. 34). The diversity of distribution channels also demonstrated that the SMP was effective in its distribution strategy (see Final Report Figure 1, p. 19). The pharmaceutical sector was key to the SMP because the population view pharmacists as credible sources of health advice, making pharmacies important places at which to access social￾marketed products. Pharmacist knowledge of the products and ability to provide advice to the target population supported and influenced positive behaviors. During the five-year project, 60% of the market share for Protec condoms was in pharmacies and 40% elsewhere. Pharmacies effectively linked the products to the community (see Table 6). Table 6. Distribution Channels for PROTEC Condoms Channel Units Sold 2007–2011 Market Share Pharmacies 11,694,466 60% Outlets other than pharmacies 7,796,310 40% Total 19,490,777 100% USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 27 Part of the SMP’s success was in using both channels to increase its efficiency and ability to meet program needs. Tables 6 and 7 show how the SMP had different strategies for unique products. Protec was predominantly sold through pharmacies (60% of distribution points) while Aquatabs were placed mostly in other sales outlets (90% of distribution points). Protec’s original targets were met; Aquatabs could have been improved. ADEMAS’s Final Report does note that Aquatabs could have had more placements within pharmacies. Table 7. AQUATAB Sales by Distribution Channel Channel Tablets Sold Liters of Water Treated Percentage Outlets other than pharmacies 2,937,950 58,759,000 90% Pharmacies 335,200 6,704,000 10% TOTAL 3,273,150 65,463,000 100% The SMP continued to increase the number of POS throughout the life of the project. Table 8 shows that distribution points were mostly nontraditional outlets and grew by 30% from 2007 to 2011. Growth in pharmacies was not as high because the SMP works within the rules of the governmental department, Ordre des Pharmaciens. The SMP worked with as many as were available and was able to achieve its original target. Table 8. POS Numbers, 2007–2011 Type of POS Total POS, Sept. 30, 2007 Total POS, of Sept. 30, 2008 Total POS, Sept. 30, 2009 Total POS, Sept. 30, 2010 Total POS Sept. 30, 2011 Non￾traditional outlets 4,567 4,982 5,024 5,158 5,938 Pharmacies 800 800 800 800 850 Total 5,367 5,782 5,824 5,958 6,788 Quality and Impact Measures Defining SMP quality and impact measures was challenging. There were no baseline, mid-term, or final evaluations of the program to provide a direct causal relationship between the program and such outcomes as increased demand for FP/RH products, increased use of SMP products, or changes in social norms. Trend data throughout the five-year project are also not available. However, the SMP did have several special studies conducted (e.g., IRIS, program monitoring in health centers) that gave some results, such as knowledge and use of branded products and awareness of certain BCC campaigns. The SMP also was able to track product sales and provide numbers of distribution points and knew how much the program contributed to such measures as CYP (20% of total Senegal CYP) over the life of the project. Program results also showed that quantity indicators were almost all met, such as volume of product sales, number of people reached, number of trainings provided, and number of service delivery points that provided FP counseling or services. The USAID- funded SMP was the only program operating in these areas; 28 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION its presence benefitted the Senegalese people. While quantitative impact measures were not available for this evaluation, success of the SMP can be inferred from other data. Although there are some data, it is recommended that in the current SMP, research be given a higher priority, with matching funding. ADEMAS reported that the SMP evaluated had no funds for more complex and detailed research in areas that would have aided the program, such as product lines the population would be willing to invest in, behavior change processes within the population, and trends throughout the life of the project. Quality indicators are also recommended: current indicators all measure quantities; they do not address such issues as behavior change within a group, whether trainees had more knowledge, how current activities can be adjusted to meet current needs, or if FP users were one-time or continual users. Better monitoring of activities could answer these questions. How Major Challenges Were Addressed The main challenges the program encountered were a lack of financial resources and cultural barriers in reproductive health, which slowed the program’s momentum in some activities. On the financial side, the exorbitant cost of advertising in Senegal limited SMP’s use of mass media. For example, it cost CFAF 50 million to place promotional banners during wrestling ceremonies in Senegal compared to CFAF 2 million in Niger, and a 30-second TV spot cost CFAF 400,000 spot in Senegal but only CFAF 25,000 in Niger and the equivalent of CFAF 200,000 in Ghana. Based on interviews with ADEMAS, many planned advertisements were postponed and even cancelled because of the cost. For example, when scented Protec condoms were launched, although 30 spots were planned there was only enough in the budget to pay for 10. Financial challenges also affected the distribution network: not enough resources were available to ensure regular staff visits to pharmacies and other POS. Securil, Depo-provera, and Aquatabs were launched successfully but could have reached more people, especially in rural areas, if more resources had been available. In addition, as some interviewees mentioned, the quantities and variety of promotional materials were not sufficient to reach targeted populations. Limited resources also led the SMP to reduce by half the number of conferences, trainings, and BCC activities. Based on a World Bank survey on a campaign in Senegal for hand-washing with soap published in October 2010, it was found that 60.5% of the behavior change found was attributable to the mass media component (radio and TV); the campaign had allocated 43% of the communication budget to mass media and 57% to interpersonal communication, print materials, and other BCC activities. ADEMAS explained that Aquatabs performed less well than the other products because of the limited funding for the launch and the lack of budget for community activities. The evaluation team encourages the new program to systematically conduct baseline studies not only to have a better understanding of the current marketing environment but also to gauge the success or weakness of current campaigns. Without baselines to track program progress, it is difficult for the SMP or outside evaluations to determine if targets, objectives, and goals were reasonable and were met. It is therefore recommended that the SMP invest in baseline and other research. In dealing with cultural barriers, the program had particular problems in religious areas, such as Touba and Medina Gounass (Casamance), where talking about FP and HIV/AIDS was prohibited. The SMP worked successfully with religious leaders and collaborated with private doctors and pharmacists to sustain its approach. In fact, the SMP implemented successful campaigns involving USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 29 higher authorities (e.g., the Minister of Health, governors, and doctors in both the private and public sectors) and working with large events (e.g., the Magal) to approach hard-to-reach groups like the religious in Touba. Male resistance was a serious challenge. Here the SMP strategy was to go into workplaces, such as plantations and mining, approaching SODEFITEX, ICS Mboro, and Compagnie Sucriere du Senegal in Richard Toll. SMP staff met with men to discuss their involvement, level of support, and acceptance of a positive role in the SMP. Some men became active users of FP/RH products and participated in communication activities during special events and campaigns. OBJECTIVE 3: DID THE PROGRAM ACHIEVE THE COVERAGE EXPECTED? Targets Reached SMP surpassed almost all of its original targets (see Table 9). Table 9. Achievement of Program Indicators N° INDICATORS Targets Results % Achievement Preventing STI-HIV/AIDS; BCC 1 Number of individuals reached through community outreach that promotes HIV/AIDS prevention through other behavior change beyond abstinence and/or being faithful 227,300 233,834 103% 2 Number of individuals trained to promote HIV/AIDS prevention through other behavior change beyond abstinence and/or being faithful 1,894 1,892 99% 3 Number of targeted condom service outlets 6,273 6,788 108% Maternal and Child Health 4 Number of broadcast episodes of radio drama promoting positive maternal and child behaviour 168 168 100% Family Planning and Reproductive Health 5 Couple-year protection (CYP) PROTEC SECURIL DEPO-PROVERA FAGARU Total of CYP (PROTEC + SECURIL + DEPO-PROVERA) 285,144 328,054 162,423 83,093 12,292 70,245 257,808 115% 90% 6 Number of people trained in FP/RH with USAID funds 1,078 1,228 113% 30 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Table 9. Achievement of Program Indicators N° INDICATORS Targets Results % Achievement 7 Number of people who have seen or heard a specific USAID￾supported FP/RH message 22,259 31,307 139% 8 Number of new approaches successfully introduced through USAID-supported programs 1 1 100% 9 Number of USAID-assisted service delivery points providing FP counselling or services (pharmacies) 850 850 100% 10 Amount of in-country private financial resources leveraged for FP/RH in US dollars (returned to project funds) $577,880 $712,126 123% 11 Number of liters of water treated by AQUATABS 87,750,000 65,463,000 75% Source: ADEMAS Final Report. The products that were the most challenging for SMP were Aquatabs and the Fagaru female condom. The only target that SMP did not achieve completely was for Aquatabs, which reached 75% of its original target. Shortages in Aquatabs stocks were noted, especially during major religious events and at the end of the campaign. Any stakeholder who mentioned the female condom noted a lack of demand for them. The KfW representative said that, Il faut déplorer cependant que le condom féminin pose encore des problèmes d’acceptation auprès des femmes English translation: It is unfortunate that the female condom still poses acceptance problems for women. Another pharmacist said, La population ne connaît pas bien le condom féminin. Il doit être soutenu par des actions de communication de proximité, notamment en faire la promotion au niveau des pharmacies : les pharmacies sont crédibles, et les produits qui y sont vendus le sont par ricochet. English translation: The population does not know about the female condom. It must be supported by communication activities, especially at pharmacies. Pharmacists are credible, and products can be sold on their recommendation. The representative of the Senegalese Department of HIV/AIDS reported that a large number of female condoms were purchased for use by female sex workers, since HIV/AIDS is more prevalent in this group; however, there is a lack of demand for and use of female condoms by this as well as other populations. ADEMAS and several pharmacists also reported that sales of female condoms were low. The SMP’s target population for the Fagaru female condom is USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 31 younger women, aged 15–35. The target population should be segmented based on youth risk behaviors or changed to better align with Senegalese government priorities. It is also recommended that the SMP increase its promotion of the female condom during health talks, trainings, and other BCC activities. Demonstrations of the female condom could increase public awareness and knowledge of the product, build skills, and increase acceptance and comfort levels, especially if free samples were given away so women can try it. Target Populations The SMP placed products in geographical areas where products were otherwise difficult to access and priced them based on local ability to pay. However, target populations were simply large groupings according to age and gender, with little other segmentation. The SMP target populations were: Aquatabs—Mothers of children aged 0–6 Securil—Women of reproductive age (15–49) Protec—Men aged 30–59 Fagaru (male)—Younger men, aged 15–35 Fagaru (female)—Younger women, aged 15–35 Radio Program—General population Little mention was made of segmenting based on other characteristics, such as marital status, risk behavior, health characteristics (e.g., previous diarrhea episode in the case of Aquatabs), or population needs, desires, or goals. One of the few examples of segmentation, mentioned in the ADEMAS Final Report, was FP talk sessions for non-users. Women who used or were using FP methods were invited to the sessions as resource people to answer questions and dispel myths about modern contraceptives. During the five-year project, SMP reported that 58% of FP talk session participants were non-users, 32% were users, and 10% were men. ADEMAS should similarly target other communication activities based on population characteristics and work to bring in more men (see the Gender section below). It is recommended that the communication strategy be based on careful segmentation of target populations based on demographic, epidemiological, and social data. The strategy needs to be evolving, not static. Several ADEMAS partners noted the need to revise the communications approach based on the evolving needs of the population. One SMP partner advised: Taylorisation des messages entre les partenaires pour un impact accru et optimum. English translation: Tailor the messages with the partners for greater impact. Producers of the radio serial used feedback from the listening groups to modify their scripts, but that was the only example where outside information was used to revise messages or delivery based on changing needs of the audience. Successful communications and materials are based on the specific needs of the people at whom they are directed. Promoting branded products and programs and specific service delivery points is more likely to achieve the intended outcome than generic messages. When tailored 32 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION messages and materials resonate more with the target population, the communication strategies do more to bring about positive social, cognitive, and behavioral change. Because the SMP often worked with large groupings of people, messages and materials were often general rather than unique. The materials tended to make general statements instead of stating a specific behavioral action to bring about a desired health goal. The ADEMAS Final Report provides rich data that can be used to better segment target populations and lead to a more tailored communications approach. For example, contraceptive use was lower for illiterate and poor women living in rural areas, and for sexually active adolescents and unmarried adult women. It was also stated that women who were part of a couple wanted to use FP methods to space births, but that actual use was low. Since the desire is already there, SMP could focus activities to increase FP use on women, married and unmarried, who are living in partnerships. Thus, the USAID SMP has opportunities it could capitalize on to improve. Epidemiological, behavioral, and social data could be used to design a stronger and more targeted communications strategy. The radio serial and listening groups are a model that can be replicated in the current project. Feedback from listening groups was integrated into program activities to craft new messages and script lines that were more relevant to changing population dynamics. The current project should take advantage of lessons learned and opportunities from the past project. Gender in the Social Marketing Approach IDIs and FGDs reported that both men and women were involved in community communications activities. For example, SMP encouraged men’s involvement in social mobilizations and household visits. Some men led social mobilization activities and community talks. Accceptance of these initiatives created a new, positive image for men. One interviewee said, Il faut cependant noter qu’au cours de la mise en œuvre, ADEMAS a encouragé la participation des hommes. Dans certains secteurs, l’animation des groupes est confiée à un homme ; ce qui ne suffit certes pas mais permet de réduire le niveau de complexe chez les hommes. Il faut passer à l’étape supérieure qui veut que les hommes soient parties prenantes, cibles primaires. English translation: It is necessary to note that in the implementation of the project ADEMAS encouraged men’s participation in the program. In certain activities, men facilitated groups; and it created a way to involve them in a positive way. It is not enough but it was a big step in having men become stakeholders in the project, a primary group. It is vital that men play a key role in the SMP. They must be a primary target population for all SMP products to increase acceptance of, demand for, and use of FP/RH products. The following statements demonstrated the importance of men’s role in the use of FP/RH products: Le condom Féminin fait perdre du temps et crée des risques dans la mesure où les hommes sont impatients et ne peuvent attendre le temps qu’on le place correctement English translation: The female condom wastes time and creates risk since men are impatient and do not want to take the time to put it in place correctly. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 33 J’ai toujours eu des grossesses tellement difficiles que mon mari m’oblige, et m’accompagne, par pitié, vers l’usage des produits PF. Mon expérience est tellement positive que je fais le plaidoyer de l’injectable English translation: I always had difficult pregnancies and my husband supported me by accompanying me to get FP. My experience was very positive and I advocate for getting the injectable. L’homme a pris l’initiative d’appliquer, avec son épouse, la PF, et le couple est devenu un modèle et un vecteur de communication très respecté et écouté. English translation: Men take the initiative, with their spouse, to get FP, and then the couple becomes a model and a channel of communication that is respected and listened to. Thus it appears that men were successfully involved in the SMP. However, quantitatively it was difficult to evaluate the level of men’s participation, since there was little information in the program indicators or the Final Report on how SMP specifically targeted them. The indicators in the Final Report made no distinction by gender or age. Protec condoms did make up 21% of the SMP budget, so it is assumed that to some degree men were uniquely targeted. However, even the “Promotion and Demand Creation Activities” section in the Final Report did not give specific information on how Protec social marketing activities were uniquely targeted to men. The only concrete figure the evaluators had to measure men’s participation was that the ADEMAS Final Report stated that only 10% of FP talk session participants were men. It is recommended that the SMP continue to involve men as active participants and leaders in the program, but that the SMP also monitor participation by gender so it can modify activities accordingly. Specific Cultural Barriers and their Solutions Addressing cultural barriers is crucial to the success of an SMP. During the FGDs, participants provided examples of cultural barriers that inhibited behavior change. For instance, La religion musulmane interdit la PF. English translation: The Muslim religion is against family planning. Les sages-emmes vous demandent de faire le planning familial, alors qu'elles ne le font pas. English translation : Midwives ask you to use FP methods, but they do not use it themselves Le planning familial tue les enfants et crée souvent des complications de santé. English translation : FP kills babies and creates further health complications. The evaluation found that the SMP successfully addressed cultural barriers through Ngelewu Nawet and the associated listening groups. Specific sociocultural issues that were pertinent to the population were identified through formative research and addressed in the program. By modeling healthy and unhealthy behaviors and their consequences, the characters acted as role models for the population. The FGD participants voiced their support for the radio program as follows: 34 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Les groupes d’écoute ont permis d’échanger entre membres, mais aussi de partager l’information avec les voisins, les personnes adultes ou d’âge très avancé pour aborder certains sujets « tabou » comme la sexualité. English translation: Listening groups help members exchange information as well as share with neighbors, adults and the elderly and can tackle certain taboo subjects like sexuality. Si un problème de PF/SR, ou même de bien-être en général ne peut se régler dans un couple et persiste, les membres du groupe se constituent et vont voir le couple pour échanger et trouver des solutions ensemble. Ceci est d’autant plus efficace qu’au niveau du village de Sikatrou, il y a maintenant 10 Couples Modèles qui ont été nommés comme tels car ayant vécu une expérience transformationnelle sur le plan de la PF. English translation: If there is a problem with FP/RH or even with general well-being that can’t be solved within a couple, the members of the group help them find a solution. It is easier at the village level in Sikatrou; there are now 10 model couples who have had transformative experiences with FP. The listening groups also provided useful feedback for the program. It was stated in the Final Report that listening group comments were incorporated into subsequent episodes to meet changing needs. Adaptation of the program based on monitoring data is a best practice that should be continued in the new SMP. The radio and TV spots for FP/RH also helped open up community discussions about sexuality, STIs, and FP/RH issues. A pharmacist stated his support for the Protec and Fagaru TV spots as follows: …le fait de parler de sexualité et de protection sexuelle à la TV a créé un choc positif chez les populations ….Cela montre le professionnalisme et l’audace de ADEMAS qui a introduit dans les mœurs et culture le débat public sur ces aspects English translation: Talking about sexuality and sexual protection on TV has created a positive experience for the population … it shows the professionalism and boldness of ADEMAS in introducing the topic in the customs and public cultural debate. Another activity that demonstrated the SMP’s ability to address cultural barriers was working with religious leaders, which helped surpass certain barriers. Religious leaders who supported FP took part in debates that helped break down cultural and religious barriers. The SMP also put on large-scale events during ceremonies and gave presentations to open discussions about FP/RH topics as a means to change social norms. It is recommended that the SMP continue these activities and supplement them with more frequent BCC activities that are participatory and that facilitate in-depth discussions on cultural issues to support and continue the behavior change process. Nontraditional Channels Among the nontraditional channels the SMP used for communication and advertising were Private sector boutiques, pharmacies, wholesalers, bars, and restaurants The literacy training sector USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 35 SMP worked primarily with pharmacies, boutiques, and wholesalers to increase social marketing communication and advertising. Staff trained pharmacists and personnel in boutiques and wholesalers and had regular contact with them about product supply in their businesses. Staff of ADEMAS’s partner, IntraHealth, said: La formation des agents de pharmacie renforce aussi l’efficacité de ADEMAS sur le plan communication et sensibilisation English translation: The trainings in the pharmacies enhanced the effectiveness of the ADEMAS communication and awareness plan. Most private sector participants in the evaluation stated that their collaboration with ADEMAS was through trainings and product supplies. Many asked for more trainings and for them to be expanded to other staff within their businesses. In both the private and the public sector, participants mentioned posters, calendars, and other promotional materials for products that SMP supplied; these materials were not overwhelming present in these settings. Involvement of literacy trainers in increasing demand for FP products was one innovative nontraditional channel that SMP used for social marketing. SMP trained literacy program instructors in both contraceptive methods and STI/HIV/AIDS prevention. These instructors then conducted about 100 talk sessions over the life of the project in rural communities. The final assessment of the program showed a significant increase of new FP users in areas were talk sessions had been conducted. According to the ADEMAS Final Report, the health districts where these activities were implemented wished that the program could be continued and expanded. OBJECTIVE 4: TO WHAT EXTENT WERE THE PROGRAM APPROACHES SUSTAINABLE? There is no single correct answer on how to build a sustainable SMP because so much depends on the country context, product lines, and population. One cornerstone of a sustainable SMP is creating behavior change so that the population continually purchases the products. The products must also be affordable and accessible so people can easily find and use them. In addition, in creating an organization, whether for- or not-for-profit, investments must be made in new products and there is a risk that the products may not be successful. Therefore, sustainability is a complex issue for any new SMP; it may take several years before it becomes clear whether a product line is worth further investment. The SMP must be able to make investments, take risks, be flexible, and adapt as the environment changes. The approaches used by the 2006–2011 SMP provided many opportunities to build a solid foundation for a sustainable program. Many initiatives were created, such as improving dissemination of public health information, adding ever-more POS, and increasing the use of FP methods. However, the evaluation found both the organizational and management structures of the SMP were donor-driven. Both structures help ensure sustainability, and would have to be modified whenever it was necessary to adapt to a new paradigm. For example, it would be beneficial for the SMP to build staff capacity and create a longer-term program so there is less turnover. Staff who are willing to commit, grow, be flexible, and change based on the new paradigm will help to sustain a SMP. In addition, the SMP was directed toward achieving targets (which were almost all surpassed), but additional distribution channels would be needed to ensure sustainability. The SMP would need to hire more promotional agents and to partner with 36 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION other organizations, associations, and groups to ensure that new POS are established. More emphasis on community-based social marketing to create an enabling environment for social change is also necessary to foster sustainable behaviors. Legal, Regulatory, and Administrative Barriers PSP-One conducted an evaluation of ADEMAS, the IA for the SMP, in order to lead it to a more sustainable program. The assessment recognized many of the organization’s strengths, particularly its ability to successfully manage large social marketing projects throughout Senegal. It also found that the SMP had broad impact through successful partnerships and good management. Yet despite these strengths and the continuing need for social marketing in Senegal, the assessment found that the sustainability of ADEMAS was far from assured. Its status as an association prevented it from taking a more direct role in the import and distribution of health products. Its management systems were designed to comply with donor requirements rather than being suited to managing product lines profitably. SMT funding was entirely dependent on USAID and KfW, and ADEMAS had too few products in its portfolio to achieve economies of scale or attract donor funding outside the RH area. Moreover, the organization does not have effective resource mobilization systems or an entrepreneurial culture that would foster more growth and diversification. The sustainability of the program itself depends on whether it can do the following tasks, which are based on both the PSP-One assessment and the findings from this evaluation: Build staff research and development (R&D) skills, and allocate funds to improve the M&E system. Sufficient and accurate data on the population, products, and market are necessary to reduce costs and raise profits; to achieve these goals, R&D and M&E skills and systems are necessary. Work to build synergy between products, promotional campaigns, and media bursts and across partners and stakeholders [e.g., SNEIPS, the CNLS (National Council for the Fight against AIDS in Senegal), IntraHealth, ChildFund, Midwives Association] in order for activities to become more streamlined and to increase impact and cost-effectiveness. Establish an effective “new business” system that will target private companies, cross-border programs, and other donors as well as invest in new products. Create an organizational and management structure, whether for- or not-for-profit, that supports all the activities necessary to implement a high-functioning SMP. Results of the PSP-One Recommendations The evaluation team reviewed the recommendations from the PSP-One evaluation and investigated how far ADEMAS had gone to act upon those recommendations. PSP-One’s recommendations and the evaluation’s current observations follow: 1. Strengthen the marketing function within SMP: The first step for improving SMP’s marketing efforts is to recruit a qualified senior marketing director who can institute more strategic, evidence-based approaches to marketing SMP products. Through training and mentoring, a skilled marketing director can build the capacity of the supervisor and product director to act as true brand managers. SMP had begun to recruit a marketing director under the supervision of the deputy director, but the process was not completed at the time of the evaluation. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 37 2. Build linkages with rural and community-based sales points: Although ADEMAS had entered into tactical partnerships for distribution with NGOs like Environnement et Développement en Afrique (ENDA) in certain areas, PSP-One recommended that this approach be replicated with many more organizations and community structures. The KfW-financed project opted for a strategy that built on new channels of promotion and distribution through rural community-based associations. This evaluation found that SMP needed to establish and strengthen its linkages with associations, groups, and personnel to improve on its distribution, availability, and communications activities. 3. Consider changing the legal status: Given the restrictions that ADEMAS faces as a nonprofit association, PSP-One recommended a study to determine options for changing its legal status or restructuring its current organization. The report of the legal study was completed in 2009. However, ADEMAS is still a nonprofit association, although it has begun working on fulfilling those recommendations. 4. Pursue product diversification: A key factor in sustainability is to increase the diversity of products in the SMP portfolio. The SMP did launch Depo-Provera, as PSP-One recommended. At the time of this evaluation SMP was exploring the possibility of launching a new product, Uniject. In the last year, ADEMAS also registered all its current brands as its own property. However, much more needs to be accomplished in diversifying products if ADEMAS is to become a sustainable organization. 5. Explore service marketing and social franchising: PSP-One did not identify any immediate opportunities for the SMP in the areas of service marketing or social franchising. However, such opportunities take time to develop; they often emerge from training and supporting providers in the network. One way the SMP could begin to create such opportunities would be to tighten connections with provider associations to generate more support and build opportunities for further collaboration. Currently, ADEMAS has not defined clear actions for this, though the concept is included in its strategic plan. Sustainability and Health Authorities, the Private Sector, and Community Partners It was difficult to ascertain the degree of commitment of the health authorities and private sector and community leaders supporting the program to achieving sustainable SMP systems. ADEMAS needs to develop innovative partnerships with the MOH and other departments; community leaders and stakeholders; local associations and groups; and medical personnel to expand SMP reach and effectiveness. While the past SMP made important connections with the health authorities, the private sector, and community leaders, more effort is needed to ensure sustainability. As noted, private and public representatives have expressed a desire to become more integrated and involved in the SMP, so this recommendation is feasible. Evidence for the Sustainability of Program Approaches Sustainable program planning should take into account national health objectives. While the SMP made some efforts to sustain its program for the long term, much more needs to be done before ADEMAS can be fully self-sufficient. The past project did not respond to a number of questions related to sustainability, among them the following: Does the SMP have a strategy for sustainability (e.g., for-profit products, marketing plan)? The ADEMAS 2010–2014 Strategic Plan is a good start but it does not provide all of the answers needed to ensure a sustainable organization. Is the SMP staff mix sufficient for and able to shift to a for-profit paradigm? 38 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION What contributions or commitments are available at the highest levels of the health sector to ensure sustainability? Even without answers to such questions, there is room for innovation to make the program more sustainable. The SMP has created solid partnerships with the public and private health sectors that can help the new program to establish more formal linkages to support sustainable systems. SMP products are highly respected by the National Health System. The PNA and the SMP are complementary. The SMP has markets and ADEMAS is respected as a strong social marketing organization. Having worked with the public and private sector in providing health services, the SMP has opportunities to entrench its placement within the health system. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 39 IV. CONCLUSIONS The SMP has served its social mission by increasing access in Senegal to FP/RH, STI/HIV/AIDS, and diarrheal disease products and services. Because no other organization was operating in this area, the SMP filled a serious gap in the Senegalese health care system. The SMP met and surpassed all of its original targets: FP educational sessions reached 31,307 people (139% of target); FP refresher training reached 1,228 people (113%); 1,246,396 cycles of SECURIL were distributed (117%), as were 49,919 Depo-Provera injections (272%). SMP products were leaders in their markets, and each was among the lowest-priced and most accessible products available. The SMP was able to add to its product portfolio both an injectable contraceptive (Depo-Provera) and a diarrheal disease prevention tablet (Aquatabs). ADEMAS achieved these results through effective SMP partnerships and good management systems. In elaborating and implementing its program, the SMP involved both private and public sectors and other partners. The number of POS increased by about 27% during the five-year project, from 5,367 to 6,788. Collaboration with the public sector focused on conducting trainings and launching products such as Aquatabs. The private sector attended trainings and promoted and sold SMP products. The SMP also served as a useful link between the private and public sectors; however, based on the IDIs and FDGs, both sectors would like to be even more involved in the program as more formal partners, acting as both implementers of activities and distribution points (e.g., the Midwives Association); working together on drafting work plans and objectives; learning about SMP lessons learned; and increasing their product knowledge. The SMP emphasized distribution and supply of products more than the social or behavior change aspects of social marketing. While it reached the quantitative targets for “people reached” and “trained,” there was some question about the quality and effectiveness of these types of programs. Most SMP activities were one-time rather than numerous and in-depth with the same group of people. The notable exception was the radio serial, Ngelawu Nawet, which was supplemented by listening groups; this should be continued. Public sector representatives were of the opinion that the SMP should have more BCC field activities. Both private and public sector respondents also felt that SMP staff were too focused in Dakar and should have more of a regional presence, which would both increase the number of BCC activities and give the private sector the additional face-time with staff that it sought. The sustainability of the SMP is a major challenge. Though several assessments were conducted during the five-year project, the SMP was slow to act on the recommendations. Sustainability issues included the need for additional, and more profitable, product lines; adaptation of organizational and management systems to match a new for-profit structure; and the need for additional staff. The SMP ought to shift from the current paradigm to a more entrepreneurial culture that would foster growth and diversification so that it can become a long-term presence that continues to meet its social mission. ADEMAS, the IA for the SMP, reported several challenges—limited time to develop the project, limited financial resources, and social and cultural barriers—that affected implementation of the program: 40 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 1. The SMP was not originally intended to be a stand-alone project but was integrated into the larger MCH program being managed by IntraHealth. When it was apparent that the SMP needed a sole IA, USAID gave ADEMAS less than two weeks to draw up a proposal for a five-year $3.5 million project. ADEMAS did so, was granted the project, and became for the first time a sole-recipient of USAID funds. The lack of planning and the change of status meant that ADEMAS did not have sufficient human resources to execute such a large-scale program; nor did it have time to strategize on a system that would be more sustainable. 2. Financial resources were limited. The SMP had $3.5 million for a five-year project that had to have national coverage. Staff thus had to be kept to a minimum, which reduced interactions with partners and the number of activities that could be done. The high cost of advertising narrowed the scope of promotional campaigns and TV/radio spots, and the number of community-based BCC activities also had to be reduced. The minimal ADEMAS budget for research affected its ability to gauge population needs and track sales and use of products. 3. The social context was challenging. Few in Senegal use modern FP/RH products, so there is considerable room for growth, but gender and religious norms are difficult to transcend. The SMP is working to respond to these traditional norms, but that takes time and more still needs to be done. A limitation of this evaluation was that it was conducted over six months after the end of the program and a year after the last field activities. Participants may therefore have forgotten or been less accurate than if the evaluation had been conducted closer to the actual end of the project. ADEMAS provided the evaluation team with the objectives for the five-year project (2006– 2011): 1. Increase sales of Protec and Securil. 2. Launch the injectable contraceptive Depo-Provera. 3. Increase the use of promoted products that contribute to CYP. 4. Promote behavior change with innovative approaches such as education-entertainment radio programs. 5. Increase positive societal behavior change within the USAID MCH strategy. 6. Elaborate the action plan for ADEMAS sustainability. The SMP was successful in fulfilling most of these objectives; however, as with any program, improvements could be made to achieve the long-term health goals of the Senegalese National Health Program. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 41 V. FUTURE PERSPECTIVES AND RECOMMENDATIONS SUSTAINABILITY Organizational Structure Social marketing has long been confused with advertising and social media. SMPs are complex because social marketing is a discipline that develops, promotes, and distributes products; provides services; and attempts to influence and support positive behavior change in society. The USAID SMP provided many services to the Senegalese population, such as disseminating public health information broadly; expanding POS and thus access to products; expanding distribution networks; introducing new products; and implementing BCC activities. The evaluation team recognized that the approaches the SMP used have helped establish a solid foundation for sustainability; however, challenges such as an absence of a sustainability plan, a program that was donor-driven, and too few products in the portfolio are operating against sustainability. Table 10 summarizes the team’s recommendations based on evaluation findings. Table 10. Recommendations Related to Organizational Structure Areas for Improvement What Should be Accomplished Who Is Responsible Expected Results Program approach was successful but sustainability is still a challenge. Conduct studies on potential SM activities (e.g., products that population currently knows about and uses) and the human resource capacity needed to implement the findings USAID Strategic document that outlines new areas of business Staff necessary to address new business ventures are listed and recruitment begins. Research is necessary to support development of the sustainability strategy. Study the feasibility of diversifying the SMP to include new products and services. USAID New products and services, including non-FP products, are identified and explored. A new structure is put in place to support the sustainability plan. Formal partnerships created with both private and public sectors to strengthen ownership by stakeholders. Not enough diversification of products and services to create revenue for the SMP. As a revenue-producing measure, a social marketing training center could be created to build the capacity of health providers and social marketing practitioners in West and Central Africa. USAID & IA Social marketing training center would create additional revenue for the SMP. 42 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Table 10. Recommendations Related to Organizational Structure Areas for Improvement What Should be Accomplished Who Is Responsible Expected Results Improve the SMP M&E system. Mechanism to facilitate frequent feedback from the communities to the program to help modify and improve it. USAID & IA M&E system in place (including learning and sharing platforms with partners) incorporating formative assessments, and baseline, mid-term, and final evaluations. Define both quantity and quality indicators (e.g., behavior change process of group, pre/post tests for trainings). Donor Dependence The IA’s structure reflects its tendency to be donor-driven and respond to the needs of specific projects. ADEMAS has experienced relatively high staff turnover over the life of the project. The insecurity of funding and lack of a long-term program, both of which are attributable to dependency on donor funding, are reasons that staff contracts are shorter-term and that the IA has difficulty in attracting senior staff. Because ADEMAS has been in existence for 15 years, it will be challenging for it to change to a new, sustainable, paradigm. It is recommended that the SMP diversify its products and services, establish more formal partnerships, and commit to changing its way of doing business. Table 11. Recommendations Related to Donor Dependency Areas for Improvement What Should Be Accomplished Who Is Responsible Expected Results Organizational structure of IA was based on donor dependency and legal status requirements The IA organizational structure should focus on a long-term sustainability plan. New professional staff need to be hired and all staff need to be trained to support the change. USAID & IA Staff recruited, trained, and prepared to change to a for-profit paradigm and to put the new business areas into operation Absence of a financial plan that will guide the organization toward sustainability Draw up a financial sustainability strategy, based on the study of organizational structure, that takes into account all possible areas of profit, including commercial, logistics, and production aspects. It must include a strategy that depends less on donors at the end of five years. USAID & IA Financial sustainability strategy implemented Absence of a sustainable action plan with guidance for its implementation Elaborate an action plan. Appoint a professional to be responsible for its implementation. IA Action plan approved and new staff member appointed USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 43 Overcoming Legal Barriers The IA was able to successfully manage the large SMP, yet its sustainability is far from assured. Its legal status as an association prevents it from taking a more direct role in importing and distributing health products. Its management system was designed to comply with donor requirements and is not well suited to manage product lines more profitably. ADEMAS has not created an entrepreneurial culture that would foster growth and diversification. Therefore, it will be difficult for the SMP to achieve economies of scale or attract donor funding beyond the RH sector. Table 12 lists recommendations for overcoming regulatory and legal barriers. Table 12. Recommendations for Overcoming Barriers Areas for Improvement What Should Be Accomplished Who Is Responsible Expected Results Regulatory and Legal status not conducive to a sustainable program Identify a person to analyze study results and select options for changing the IA’s regulatory and legal status. USAID & IA Staff person identified or recruited Data is available on how to change the IA regulatory and legal status based on past assessments, but staff has no time to lead the analysis and decision￾making process Analyze data and submit two options, with clearly articulated positives and negatives, to the Board of Directors for approval. Elaborate and execute a chronogram of activities or milestones to achieve by a certain deadline. USAID & IA Two options are explored in order and one is chosen within six months. Activities are implemented on time to define the final outcomes on the schedule outlined. Lack of clarity about the administrative process Initiate the administrative process to execute the shift of status to the option decided upon. IA Change of legal status to support a sustainable social marketing program. Need for stronger partnerships with associations, groups, structures, and staff Draft a joint action plan with all partners including objectives, activities, and expected results and a specific schedule. USAID & IA Effective partnerships are established Enabling environment is strengthened. Targeting and Advertising Methods A communications strategy is necessary to better target the population and increase awareness of specific health issues and available services; launch and raise awareness of new products; increase the social acceptance of healthy behaviors; increase demand for products and services; and change behaviors. The strategy should identify better-targeted and more effective advertising channels and methods. All product lines should also be explored and detailed in the strategy. For example, the worst-performing product in the SMP portfolio was the Fagaru female condom. If this product were better positioned and marketed, sales and use could be increased. The SMP has a great opportunity to identify and analyze the current needs of the population to raise demand and use of its products and intensify SMP impact (see Table 13). 44 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Table 13. Recommendations Related to Communications Areas for Improvement What Should Be Accomplished Who Is Responsible Expected Results Limited population segmentation, in the communications approach Conduct a formative assessment of the target populations to explore attitudes, beliefs, values, barriers, life goals, media outlets, sources of information, health care practices, and other behaviors related to FP/RH, STI/HIV, and diarrheal disease. USAID & Consultants Formative assessment report with recommendations for, segmentation of target populations, barriers to address, opportunities to capitalize on, recommended media outlets, and behavioral goals for messages. Lack of a written communications strategy Preparation of a strategic communications report USAID & Consultants A strategic communications report, based on data and behavioral theory, guides the SMP in conducting a strong, evidence-based social marketing program focused on bringing about behavior change. Implementation plan listing activities, organizations and staff responsible, staff needed, budget, and timeline for acting on the strategic communications report USAID Implementation plan for five￾year project prepared Improve development of communication tools Effective program tools should be based on the needs of target populations and on best practices. They should be periodically revised as population needs change. IA & consultants Program’s immediate effectiveness and efficiency is increased. Product Placement Placement of products is central to the SMP. The evaluators identified several opportunities missed in the 2006–2011 program that the current program could take advantage of. For example, the SMP should strengthen efforts to reach rural populations and build sustainable linkages by partnering with local associations (e.g., the Midwives Association); public employees (e.g., community-based nurses, community health agents); and private entities (e.g., small markets) to increase distribution and reach (see Table 14). USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 45 Table 14. Recommendation Related to Product Placement Areas for Improvement What Should Be Accomplished Who Is Responsible Expected Results Partnerships could be established with groups, associations, and structures to broaden product placement. Conduct study to determine other channels for placing products USAID & consultants Report identifying other groups, organizations, businesses, NGOs, and associations that can support USAID’s Mission to make products more accessible to the population. Make formal agreements with identified partners to increase product distribution USAID & IA Memorandum of Understanding (MOU) signed with each partner. Develop staffing plan, timeline, and budget adequate to support activities. IA Implementation plan completed and put into action. 46 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 47 ANNEX 1. STATEMENT OF WORK GH Tech Bridge Contract No. AID-OAA-C-12-00004 SCOPE OF WORK (Final: 04/02/12) I. TITLE: SENEGAL SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION II. PERFORMANCE PERIOD: O/A APRIL 9, 2012 – O/A MAY 22, 2012 III. FUNDING SOURCE: THIS ASSIGNMENT WILL BE FUNDED BY USAID/SENEGAL. IV. PURPOSE USAID believes that social marketing is a key resource for promoting public health products and services in Senegal. The purpose of this performance evaluation is to (1) determine the extent to which the expected results in social marketing have been achieved; and (2) provide pertinent information on how the program is being implemented that will inform USAID/ Senegal’s operational decision-making regarding the implementation of the recently awarded social marketing and health communication and promotion program in Senegal. This evaluation will provide a comprehensive look at the Social Marketing Program Component of the USAID/Senegal Health portfolio as implemented by ADEMAS, Agence pour le Développement du Marketing Sociale, from 2003 through the current strategy. V. BACKGROUND ADEMAS, a Senegalese NGO, was created in 1998 and operated as a sub-grantee to the USAID centrally-funded project, Commercial Marketing Strategies. As a part of this agreement, ADEMAS received technical assistance and funding for the social marketing of Protec®. Since September 2003, ADEMAS has been the implementer of USAID/ Senegal’s health social marketing activities under an initial (2003) and follow-on Cooperative Agreement (2007) under the current USAID/Senegal Health Sector Strategy (2006-2011). As a key part of the USAID/ Senegal Health Program, ADEMAS markets male condoms (Protec), injectable (Depo-Provera®) and oral (Securil®) contraceptives, and a point-of-use water purification tablet, Aquatabs®. Although primarily supported by the USAID/Senegal Health Office, ADEMAS has also received support from KFW to socially market female condoms and a condom targeted at youth for HIV prevention and is currently implementing behavior change/ social marketing activities for improved hygiene and sanitation within the USAID/RTI/PEPAM1 program, funded through the USAID/Senegal Economic Growth Office. Opportunities for social marketing in Senegal: Family Planning: Fertility has decreased gradually in Senegal over the past few decades, due to many factors, including delayed marriage, increased urbanization, and improved child survival; 1 Program d’Eau Potable et l’Assainissement du Millenaire. 48 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION however, actual fertility is higher than desired, indicating an unmet need for family planning. Use of modern methods of contraception among married women of reproductive age is currently 12.1% and the unmet need is 31.6%. Rural use of contraception has doubled since 1997 but has stagnated since 2005. Urban use of contraception is approximately triple that of rural use but has also remained relatively stagnant over the past decade (Demographic and Health Survey [DHS] 2010). There are many explanations for low use of modern contraception, including both supply and demand barriers. Despite high awareness among target populations, high unmet need (implied demand), and adequate clinical service ability to provide FP, political-level support is weak. ADEMAS’s activities in social marketing address demand barriers by promoting a safe, effective, socially acceptable method of FP to potential users, as well as supply barriers, by increasing access to FP methods through private sector providers to couples who choose them. At 22% (DHS 2005) the percentage of contraceptives supplied through the private sector in Senegal is low compared to other West African countries; tapping into this network presents a great opportunity to address the high unmet need for FP/RH in Senegal. HIV/AIDS: Senegal has successfully maintained a low prevalence of HIV/AIDS among the general population (0.7% nationally), but pockets of higher prevalence and risk do exist. USAID has a comprehensive HIV/AIDS prevention program which promotes abstinence and fidelity among the general population and condom-use and other risk-reduction strategies among more at-risk populations. ADEMAS’s activities are complementary to these efforts, promoting correct and consistent condom use to all Senegalese who choose to use condoms as an HIV prevention method as well as a primary FP method or a backup method. Diarrhea Prevention: Both the incidence of diarrheal disease and child mortality due to diarrheal￾related diseases are of great concern in Senegal. Preliminary data from the 2012 DHS reveal that the diarrhea rates have remained the same since the 2005 DHS; more than one out of five (21%) children under 5 years old experienced diarrhea in the two weeks preceding the survey. In comparison, other West African countries, such as Benin and Mali, have prevalence rates of 10% and 13%, respectively. Diarrheal-related illnesses are the second leading cause of death for children under 5; they account for about 40,000 preventable deaths each year in Senegal (UNICEF 2002). 2005 DHS results showed that diarrheal prevalence was not only comparable in rural and urban regions of the country (21% and 22%, respectively), but across wealth quintiles as well. These statistics show the need to address the incidence of childhood diarrhea across all populations in Senegal given that cholera is a frequent menace in Senegal, with one outbreak reportedly occurring every year since 2004 (Direction de la Prevention). ADEMAS’s activities have supported the introduction of a point-of-use water purification tablet as a market alternative to address the issues with potable water at the household level. Behavior change: In terms of behavior change, programs in the past have often focused on information, education, and communication (IEC), awareness-raising of specific health issues or services available through targeted messages, and communications with clear health messages presented through theatrical performances, poems, and songs, for example. A different approach, which has proven very effective in other countries, takes the “entertainment” approach much further. The entertainment-education approach presents the targeted health values by integrating them into a primarily entertainment-based production in which positive and negative health values are acted out over an extended period of time by characters that become familiar to the listening audience. In the end, the positive behaviors become more socially acceptable for the general public to act on, and the negative behaviors prove to have negative USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 49 consequences. USAID, through ADEMAS, supports this approach in the form of a radio drama series called Ngalewu Nawet (Winds of Change), which complement the efforts of all implementing partners in the current health strategy (2006–2011). Host-Country Capacity Building: USAID/Senegal’s support for the creation of and subsequent direct funding of ADEMAS since 2003 demonstrates the USG commitment to host-country capacity building. Additionally, USAID, as the organization’s main funding source, provided institutional strengthening technical assistance to ADEMAS with the view to improving the organization’s prospects for sustainability and independence. VI. STATEMENT OF WORK Questions to Investigate The Evaluation Team will review activities that have been implemented to date and identify successes, gaps, and constraints in the implementation of program activities. The team will investigate to what extent the USAID/Senegal Social Marketing Program Component addressed the following key issue and challenge that USAID identified when designing the program. Issue: USAID/Senegal seeks innovative ideas regarding which health commodities and services could be marketed to providers and to the public. Challenge: To expand the range of socially-marketed health products and services in Senegal. Specifically, the Evaluation Team will provide three types of information on the Social Marketing Program Component. First, the team will provide general and specific conclusions on the degree to which the program achieved its main objectives as outlined in the original Cooperative Agreement and in the follow-on agreement and the subsequent amendments. The Team will highlight the main programmatic, technical, managerial, and organizational issues arising during implementation. The team will also review the indicators selected to capture ADEMAS’s performance and program implementation to ensure that these are the most appropriate measures to demonstrate the Social Marketing Program’s progress and will determine which program areas have made significant progress, and which areas are lagging. Second, the Evaluation Team will discuss the efficiency and effectiveness of the processes and methods employed by the Social Marketing Program, e.g., the soundness of the approaches; the beneficiary coverage and response; product coverage and sales; awareness of product brands; perceptions toward promoted health behaviors; and the overall sustainability of the program’s approach. Third, based on this analysis, the team will provide recommendations and identify lessons learned that will inform future program strategy and implementation, particularly USAID/Senegal’s comparative advantage in social marketing and behavior change communications. 7. To what extent were the program’s stated objectives [listed below] met? How effective was the program approach to meeting the stated objectives? – Develop new communication strategies to increase demand for FP/RH products and services; – Increase the involvement of the commercial private sector in the provision of Family Planning products and services; – Expand and consolidate existing product distribution networks (Protec, Securil®); 50 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION – Increase access to FP/RH products and services; – Launch and raise the awareness of new products (Depo-Provera®, Aquatabs®); – Liaise with relevant stakeholders to make the regulatory and legal environment conducive for hormonal family planning products; – Raise the awareness of specific health issues and available services through targeted messages and communications with clear health agendas; – Increase the social acceptance of healthy behaviors in support of USAID’s Health Strategy; – Encourage behavior changes in decision-making based on the social awareness of benefits of healthy behaviors; – Complement the efforts of other USAID/Senegal Health Program Components to reach their behavior change communication (BCC) goals; – Convince, motivate, and recruit new users; – Increase the volume of sales of socially marketed products; – Increase the use of FP products and services; and – Develop and implement a business plan that orients the implanting partner towards sustainability. 8. Were the program activities and implementation sound? – How well have social marketing activities been coordinated with other USAID/Senegal Health Program Components? – How effective has the collaboration been between the Social Marketing Program Component and the other Program Components? – What has been the value-added of a stand-alone Social Marketing Program Component to the overall USAID/Senegal Health Program: – To what extent did the structure of the organization have an effect on the effectiveness/ efficiency of program implementation? – To what extent was the scope of activities appropriate for the skills mix of the implementing partner? – How cost-effective was the implementation of the various key processes of the Social Marketing Program including product detailing and placement advertising community and stakeholder outreach production and broadcasting of the radio drama USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 51 – What major challenges were identified over the period of program implementation and how were they addressed? – How well was gender incorporated into the social marketing approach? – How well were specific cultural barriers and their solutions identified and incorporated into the social marketing approach? – How effective were the established mechanisms for involving key stakeholders? 9. Did the program achieve the coverage expected? – To what extent did the interventions reach the target populations in the expected numbers? – To what extent were the targets realistic? – To what extent were the communication and advertising materials appropriate to the target audience? – To what extent were nontraditional channels used successfully for communication and advertising? 10. To what extent were the program approaches sustainable? – What legal, regulatory, or administrative barriers were encountered that would need to be addressed in a future program? – To what extent have recommendations from the PSP-One assessment for institutional stability been acted upon? – What is the adequate scope of activities against the capacity and skills of ADEMAS? – How were health authorities, the private sector, and/or community leaders engaged in the program to support its sustainability? – What institutional reforms will help keep the momentum? – What evidence does or does not exist that favors the sustainability of the approaches employed by the program? 11. Future Perspectives and Recommendations – After having evaluated the USAID/Senegal Social Marketing Program Component, based on your knowledge of other francophone countries facing similar challenges in social marketing of family planning products, what recommendations can you offer to improve the program design and implementation in the following areas? Overcoming regulatory and legal barriers Organizational structure Advertising channels and methods Population targeting Product placement 52 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Methodology The team conducting this final evaluation shall review all the relevant documents pertaining to the USAID/Senegal Health Program, the USAID/ Senegal Social Marketing Program Component, and the Government of Senegal (GOS) health activities, including those listed in Section D below. In addition to the ADEMAS program headquarters office, the team will identify a sample of target sites to visit in Dakar and in the regions where it will conduct meetings and interviews with representatives from private sector points-of-sale, GOS health care providers and central Ministry of Health stakeholders, local NGOs/CBOs, private pharmacies, and selected end-users, among others. The evaluation team shall propose its own detailed methodology but it is expected that key aspects of the evaluation will include a document review; key informant interviews; and focus group meetings. USAID/Senegal expects that the analysis will consider gender issues like the participation of men during couples-related activities. The assignment work will commence with a two-day Team Planning Meeting. This meeting will allow the team members to meet with the USAID/Senegal staff to be briefed on the assignment. It will also allow USAID/Senegal to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will clarify team members’ roles and responsibilities; review and develop final evaluation questions; review and finalize the assignment timeline and site visit calendar and share with USAID; develop data collection methods, instruments, tools, guidelines and analysis; review and clarify any logistical and administrative procedures for the assignment; establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion; develop a preliminary draft outline of the team’s report; and assign responsibilities for the final report. USAID/Senegal will provide a detailed contact list of key informants and key points of contact to the consultants during the document review period, so planning can begin for appointments and interviews and site visits can be set up for the team’s arrival in-country. USAID/Senegal will also provide a draft schedule for field visits, including duration of stay at various sites to inform the team’s time in-country. Deliverables The Evaluation Team shall provide the USAID/Senegal Contract Officer’s Technical Representative of this contract with the following deliverables: A workplan After review of background materials and initial consultation with USAID/ Senegal and ADEMAS, the Evaluation Team will develop a workplan which will include the proposed data collection methodology, timeline, and tools (including questionnaires) to be used for the evaluation. This plan will be reviewed and approved by USAID/Senegal before field work begins. Debriefing meetings After field work, the Evaluation Team will be responsible for conducting a debriefing meeting with USAID/ Senegal and ADEMAS to present initial key findings and recommendations. Draft evaluation report and final draft evaluation report The Evaluation Team is free to propose an outline for the draft and final draft report; however, it is expected that the report will include at least the following sections: USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 53 Acknowledgement Acronyms list Executive Summary Table of Contents Introduction Discussion – Background – Purpose and Methodology of the Assessment – Findings (re approach, coverage and response, and sustainability) – Conclusions – Lessons Learned Recommendations and strategic options Bibliography Annexes (terms of reference/scope of work; organizations contacted; a discussion of the methodology and data collection tools, etc.) For the draft and final draft evaluation report, it is expected that the report main body (here defined as the Introduction, Discussion, Recommendations) of the final assessment report will not exceed 40 pages. The Evaluation Team Leader shall submit all deliverables to Dr. Baye Mbow, the alternate Agreement Officer’s Technical Representative for the ADEMAS Cooperative Agreement at USAID/Senegal and Antoinette Sullivan. If delivered by e-mail, ambow@usaid.gov and ansullivan@usaid.gov. USAID will provide consolidated comments on the draft evaluation report to the Evaluation Team Leader. The Evaluation Team Leader shall incorporate USAID’s comments and submit the final draft report to USAID in electronic format (Microsoft Word) as well as printed and bound copies (10 copies in English and 20 copies in French). Note: In regard to the final report, GH Tech Bridge contract comes to an end May 22, 2012 and all work must be completed and invoiced by that date. The process for final editing and formatting of approved final report content can take up to 30 days to complete, depending on the length of the report, and the extent of editing required. Additionally, our experience is that translation requires 2-3 additional weeks. Our standard process involves a thorough professional edit, internal review of the editing, professional formatting in the USAID branding-compliant template, and conversion of the document to a 508-compliant PDF for web posting. Due to the time required to complete this process, we cannot guarantee with 100% certainty that any report content approved for editing after mid-April will be completed by the end of the project, although we will do everything we can in order to speed the process along as rapidly as we 54 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION possibly can. We will work with you to ensure that you have the most complete and polished product we can provide before our contract’s end date. There are various options for editing and formatting the final draft report that we can consider, depending on the timing of fieldwork completion. (1) If the fieldwork ends quite close to our project end date, the consultant could provide a working draft of the report to the mission, to be finalized by another mechanism. (2) If the final draft report has been finished and approved by USAID with less than 30 days left in our contract, we can work with you and the editors to determine what can be completed within our timeframe. Information Sources The Evaluation Team shall familiarize itself with USAID/ Senegal and the Social marketing Program Component documentation. USAID/Senegal and ADEMAS will ensure that all relevant documents are available to the Evaluation Team prior to the field work. The documents will include, but not be limited to: USAID Annual Program Statement (2006-2011); ADEMAS Cooperative Agreement including Program Description; ADEMAS follow-on Cooperative Agreement and Program Description; Aquatabs® Feasibility Study; Aquatabs® ADEMAS Program Description; Quantitative and Formative Studies for the Social Marketing of Aquatabs®; PSP-One Scope of Work (technical assistance to ADEMAS; PSP-One Rapid Private Sector Assessment; PSP-One ADEMAS Assessment; Preproduction advertising materials, mock-ups, and product samples; Annual workplans and annual and quarterly reports submitted by ADEMAS; Performance Management Tools prepared by USAID; Activity reports; Field trip reports; Recent sector and/or component assessment/evaluation documents; Plan National de Développement Sanitaire et Social; and Other documents, as required and requested. Evaluation Team Composition It is expected that the Evaluation Team will be comprised of one (1) Evaluation Team Leader and up to two (2) experts with skills defined below. All candidates must be approved by USAID/Senegal. The Evaluation Team will work under the overall direction of the Evaluation USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 55 Team Leader. All team members will contribute to day-to-day problem- solving, technical questions, etc. Desired Qualifications for Key Personnel The Evaluation Team Leader is responsible for clarifying the scope and timeline with USAID, compiling and distributing the background materials to team members, team management and coordination, writing assignments, making transportation and logistics arrangements, field work preparation/scheduling, and briefings/debriefings. Working in conjunction with other team members, s/he will be responsible for data analysis, lessons learned, and recommendations. The Evaluation Team Leader must have the following skills and qualifications: Excellent spoken and written skills in French and English; Postgraduate degree in public health or related field; Extensive (five or more years) experience working with social marketing programs with experience in West Africa highly desirable; Extensive (five or more years) experience in assessment/ evaluation of health activities, particularly social marketing programs; Knowledgeable about USAID policies, objectives and programs; and Be a proven team player. Additionally, other team members must fulfill the following requirements: Bilingual French-English skills; An advanced degree in health sciences, an MPH, or commensurate experience; In-depth knowledge of Social Marketing programs in Africa, West Africa and Senegal experience preferred; Extensive (five or more years) experience working in health in francophone West Africa, particularly Senegal, is highly desirable; Knowledge of organizational/institutional assessments and strengthening. Relationships and Responsibilities Overall Guidance: The USAID/ Senegal Program Office and Health Office will provide overall direction to the Evaluation Team. USAID/Senegal Contact: The co-AOTR for ADEMAS, assisted by the Health Team Leader (or Acting Team Leader) and the USAID/Senegal Program Office will be the official contact for the Evaluation Team. 56 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Responsibilities: USAID/ Senegal Health Office will be responsible for coordinating and facilitating initial assessment-related interviews and meetings in Dakar with ADEMAS and GOS Representatives. The Evaluation Team Leader will be responsible for arranging all subsequent meetings, interviews and field visits. Timeline The tasks in this Scope of Work will be implemented over the period of about six weeks, starting on or about April 9, 2012 through May 22, 2012. The schedule below is illustrative and will be discussed and revised, as required. Illustrative Activities and Estimated Level of Effort Activity Person(s) Responsible Total LOE Period of Performance (illustrative depending on start date) USAID/Senegal sends background documents USAID April 6 Review of documents and initial drafting of evaluation protocol and instruments Evaluation Team 5 April 9-13 Submit draft workplan, methodology, and tools to USAID/ Senegal Team Leader April 13 USAID review of workplan, including data collection methods and tools USAID April 16-17 Travel to Country International Consultant 2 April 13/14 Initial meeting with USAID/Senegal and ADEMAS; Team Planning Meeting (TPM) to finalize workplan, methodology, and tools. Evaluation Team 2 April 16-17 Meeting with USAID to (a) discuss the draft workplan; (b) review and confirm planned dates of submission of deliverables; and (c) brainstorm on key accomplishments, weaknesses, opportunities, and threats. Meeting with ADEMAS to (a) review the information sources and contact list; (b) discuss appointment dates and times; and (c) brainstorm on key accomplishments, weaknesses, opportunities, and threats Evaluation Team 1 April 18 Final workplan submitted to USAID. Meetings with Dakar-based key informants: GOS representatives, other USAID implementing partners, UNICEF, etc. Evaluation Team 2 April 19-20 Field travel and data collection Evaluation Team 8 April 21-27/28 Data analysis and draft summary of key findings Evaluation Team 4 April 30-May 3 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 57 Activity Person(s) Responsible Total LOE Period of Performance (illustrative depending on start date) Travel back International consultant 1 April 27/28 Debriefing for USAID/Senegal and ADEMAS Evaluation Team 1 May 4 Draft report writing Evaluation Team 5 May 7-11 Draft report submitted to Mission GH Tech Bridge/TL May 11 (COB) Mission sends technical feedback/comments to GH Tech Bridge USAID May 15 Draft report revisions Team Leader 2 May 16-17 Revised report submitted to Mission GH Tech Bridge May 18 Total LOE = Local Team Leader (1) 30 Total LOE = International Team Member (1) 33 Total LOE— Local Team Member (1) 28 Travel over weekends may be required during site visits. A six-day work week is approved while in-country. VII. MISSION POINT OF CONTACT Dr. Elhadji Amadou Mbow - Baye Maternal & Child Health / Family Planning ambow@usaid.gov. 338696100-776371266 USAID / Sénégal. Ngor Diarama / Petit Ngor BP: 49 Dakar Sénégal. fax: 338696101 VIII. COST ESTIMATE GH Tech will provide a cost estimate for this activity. 58 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 59 ANNEX 2. LIST DES PERSONNES RENCONTRÉES Structure/ Organisation Statut Participants Titre/ Fonction Contact In-depth Interviews Service National d’hygiène de Dakar Central Mr. Laty Gaye SYLLA 70 102 01 30 SNEIPS Central M.r Seye et Madjiguene Directeur General & Chargée de l'information et événements spéciaux 338275513 DPIC (Direction Protection Ind. & Collective) Central Mr. Gueye et staff Directeur General & Chargée de l'information et événements speciaux ANSFES (Associations Nationale des Sages Femmes d'Etat du Senegal) Central Mme. Marième FALL Présidente Associations Nationale des Sages Femmes du Sénégal (ANSFS) 77 536 48 53 Syndicat des pharmaciens privés Central Dr. Aboubakry Sarr Valdafrique Central Dr. Sakho Directeur d'usine madiagne.sakho @valdafrique. com Division de Lutte contre le Sida (DLSI) Central Dr. Abdou Lahat Mangane Pharmacien Logisticien VIH Tel: 775691741 Childfund Central Mr. Diagne et Staff Skanoute @senegal. childfund.org KFW Central Mme. Marieme Kane IntraHealth Central Dr. El Babacar Gueye a delegué Mma. Awa Mme. Awa, Ex resp Comm et deputy District HLM Dakar Regional Public Dr. Ndiaye MCD 77 324 61 17 District Colobane Dakar Regional Public Dr. Ndiaye MCD 77 634 20 92 Clinique Dr Samba Wade _PA Dakar Regional Private Dr. Wade Manager & owner 77 633 12 15 Clinique Dr Abdoul Khadre Sarr (Dalifort Dakar) Regional Private Dr. Abdoul K Sarr Manager & owner 77 029 33 88 _ dr.kadoo59 @yahoo.fr Pharmacies Coin santé (GY Dakar) Dakar Region Private Dr. Daniel Sarr Manager & owner 33 867 98 40 Amadou BA en face jean de la Fontaine Dakar Region Private Amadou Ba Manager & Propriétéaire 77 211 77 87 60 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Structure/ Organisation Statut Participants Titre/ Fonction Contact (Dakar) Region Medicale de Kaolack KL region Mme. Thiam Sage Femme Région￾Coordinatrice SR 77 659 84 83 Hopital Regionale de Kaolack KL region Mme. Dia Sage Femme hôpital 77 452 34 58 Association des employés de pharmacie/ Pharmacie Mbossé KL region Jean Sagne Coordonnateur National Provisoire Association 77 538 47 02 Demba Ly Auxiliaire de Pharmacie et membre de l'Association Grossiste Secteur privé/ Marché KL region Samba Dione Représentant Dame Diouf/ Manager opérations Commerciales 77 902 44 00 SF hôpital Régional Thies Region Mme. Fall Mounina (Sage Femme) sage femme 77 636 23 64 Camp Gémi Thies Region Mme. Niakhe (Sage Femme) sage femme 77 559 21 40 Clinique conception Thies Region Mme. Ndeye Fama Bâ 33 951 16 94 / 77 308 92 26 Pharmacie Renaissance Thies Region Mr. Niang (Thiés) 77 427 09 99 Focus Group Discussions Agence de Marketing Sociale (ADEMAS) Sujet à Evaluation Mme. Seynabou Mbengue Dr. Cheikh Sarr Structure Evaluée 33 865 01 88 Email: ademas @ademas.sn USAID client Dr. Mbao & team Client Population _ Groupe d'ecoute de Boustan Kaoalck region Mme. Dia Sage femme Sage femme 77 452 34 58 Population _ Groupe d'ecoute de Sikatrou Kaoalck region Moussa Sarr Animateur Case de Santé Sikatrou /rural 77 618 63 64 Keur Assane Ndiaye (CEED) Thies region Selbal Bâ Coordonnateur Animateur 77 403 16 10 Enda Graim Thies region Mr. André Demba Wade Coordonnateur Animateur 33 951 47 69 Tiers enfance (Thiaroye Dakar Dakar Region ONG/Associa tion Laity fall & membres Coordonnateur Animateur 77 563 55 04 / 77 579 93 43 Kakatar (Laïty Fall Dakar) Dakar Region ONG/Associa tion Mme. Diop Sage femme & membres Animatrice 77 567 02 52 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 61 ANNEX 3. BIBLIOGRAPHY Rapport de l’évaluation du système de santé du Sénégal, 2009 USAID Senegal Health Sector Strategy, 2011–2016 Plan National Développent Sanitaire (PNDS), 2009–2018 Document de Politique Economique et Sociale, 2011–2015 ADEMAS Follow-on Program Description ADEMAS Rapport Annuel, 2011 ADEMAS USAID Annual Activity Report, 2007 ADEMAS USAID Annual Activity Report, 2008 ADEMAS USAID Rapport Annuel, FY09 ADEMAS USAID Rapport Annuel, FY10 ADEMAS Final Program Report, 2006–2011 Senegal Private Health Sector Rapid Assessment, 2009 Health Communication RFA, 2011–2016 Plan Strategique, ADEMAS, 2010–2014 PSP-One’s Assessment of ADEMAS Recherche formative pour le développement de feuilletons radiophoniques au Sénégal, Rapport Final, 2008 Recherche formative pour le développement de feuilletons radiophoniques au Sénégal, Revue de la Littérature, 2008 Développement de feuilletons radiophoniques bases sur la méthodologie SABIDO au Sénégal, Enquête de base sur la santé de la reproduction dans les régions de Dakar, Kaolack et Matam Sénégal, 2008 62 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 63 ANNEX 4. CVS 64 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 65 66 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 67 68 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 69 SERIGNE MBAYE SEYE Senior Communications Specialist Nord Foire Villa No 43, Dakar, Senegal, Tel 221-776389544 Email: _seye.mbaye@gmail.com CAREER SUMMARY With more than 20 years of experience in more than 16 countries, especially in Sub Saharan Africa, Mr. Seye has developed a regional reputation for excellence in BCC, education, strategic planning, group dynamics, and strong technical programming and management skills. In recent years, Mr. Seye has worked in water and sanitation, maternal and child health, project management and program sustainability, adding to an impressive array of skills in assisting PVOs and NGOs to prepare and implement their business plans. Specialized in communications, education, qualitative research and marketing related to maternal and child health as well as water and sanitation, he has actively led the management of all phases of development and implementation of BCC activities across all projects, including training, research, and coaching BCC technical experts to assist staff project activities. He participated in the design and implementation of appropriate maternal, child health, and reproductive health programs as well as financial and technical management systems for participating NGOs and their health associations (midwifes and nurses) in USAID projects. As Resident Advisor for the SEATS-Funded USAID project in Senegal, he took the lead in preparation of the multiyear communication plan and was responsible for budgeting process, procurement, logistical management, inventory, and forecasting. Mr. Seye has worked to advance programs for poor populations in urban and rural areas, youth, the private sector, and media. Mr. Seye has established knowledge and information-sharing within West and Central Africa and with strategic audiences and has ensured effective dissemination and outreach of knowledge products. Up to last year, Mr. Seye worked as full-time staff with the Water and Sanitation Program-Africa (WSP) of the World Bank, where he was actively contributing to the successful delivery of the regional communication work, including improving production, dissemination, and effectiveness of knowledge and advocacy tools; supporting country, NGO, and thematic programs; and strengthening client capacity in implementing strategic communications. He was also managing a group of consultants involved in the program. Mr. Seye completed a master’s degree in adult learning at the Applied Economic School in Dakar, Senegal and a master’s degree in communication and culture at the SONY International Institute in Tokyo, Japan He received a special certificate on training methodologies of business owners and workers at the International Labor Organization (ILO) of Torino and earned a certificate in communications design, development, and use of print and audiovisual materials at the International School of Bordeaux, France. Mr. Seye is fluent in Wolof, French, and English and has a good working knowledge of Arabic and Japanese. 70 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Before joining WSP at the World Bank), he worked as a full-time employee with the program for appropriate technology in health (PATH), the Academy for Educational development (AED), and the Medical Care Development (MCD) and was a consultant with several organizations, such as UNICEF, USAID, UNFPA, JHU/PCS, MSH, PRITECH, AFRICARE, ISTI, ACI, ISADE, and UNESCO. With more than ten (10) years spent in USAID health projects, Mr. Seye has a strong knowledge and experience of USAID regulations and procedures. PROFESSIONAL EXPERIENCE December 2004 to May 2011: Water and Sanitation Program (WSP/ Africa Region), Senior Communications Specialist 1. Provide leadership in developing, implementing, and evaluating communication activities related to water hygiene and sanitation in West and Central Africa. 2. Work with African governments, water and sanitation utilities, NGOs, and academic institutions to develop and implement communication strategies strengthening the water sector in Senegal, Rwanda, Benin, Burkina Faso, Niger, and Democratic Republic of Congo. – Provide communications support to thematic/country program activities in development and implementation of their business plans, production of field notes and communications tools, engagement with clients, and product dissemination. – Participate in the development and management of efficient and timely delivery of quality communication tools and products, such as field notes, reports, publications, online versions /formats of new products: CDs, DVDs, videos, electronic newsletter (ACCESS), cartoon calendar, and the website. – Play an active role in establishing knowledge and information within WSP Africa and with strategic audiences. – Lead the development of communications capacity in West and Central Africa September 2001 to October 2004 Children’s Vaccine Program /PATH, Dakar, Senegal, Senior Communication and Training Officer Provide leadership in developing, implementing, and evaluating advocacy, communication, and training activities related to PATH/CVP West Africa Projects and, in collaboration with other GAVI partners, to Global Fund for Children’s Vaccine-awarded countries and to the GAVI Africa Working Group as necessary. Specific responsibilities included: 1. Work with West Africa project team to develop and implement regional and country￾specific program strategies and annual workplans. Develop and take primary responsibility for BCC, advocacy, communication, and training strategies for the CVP West Africa Project. 2. Work closely with the Gates CVP Advocacy, Communication, and Training team to identify and address technical assistance needs for other African countries receiving assistance from GAVI and the Fund. 3. Work closely with the GAVI Africa Working Group and their Advocacy Task Force. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 71 4. Provide leadership needed to ensure effective integration of advocacy and communication activities in overall immunization program planning, implementation, and evaluation in the region. 5. Assess existing communication materials, strategies, and institutional capabilities in the region. 6. Work with country teams as a senior advisor to assess, design, implement, and evaluate national and district-level immunization communication plans. 7. Provide country teams with training and technical assistance in BCC, advocacy, research, materials development, community mobilization, and client-provider interaction. 8. Represent the Gates CVP, the ATF of the GAVI Africa Working Group, and ATF at meetings and conferences as appropriate. 9. Provide senior-level technical assistance and oversight to the Gates-funded Africa Adolescent Reproductive Health Project. September 2000 to September 2001 Senior Communication Advisor, AED at Advance Africa (Washington DC) 1. Provide leadership in developing, implementing, and evaluating advocacy, communication, and training activities related to advance family planning and reproductive health in Africa. 2. Work with African counterparts to develop and implement communication strategies to strengthen family planning programs. 3. Take primary responsibility in the development of regional advocacy, communication, and training strategies for the African regional initiatives. 4. Work closely with other cooperating agencies in communication and training and identify and address technical assistance needs for the African region April 1999 September 2000 Senior Communication Advisor, Family Health Program, Benin 1. Provide technical expertise to the health education and training unit of the Ministry of Health. 2. Take the lead in preparation of the multiyear communication plan. 3. Develop methodologies and materials for outreach training activities and maintain close liaison with the health education unit and other NGOs involved in the program. 4. Prepare and implement research and training communication programs for central, regional and district health providers. 5. Act as Chief of Party in the absence of the Senior Health Manager. 72 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION February 1995 to April 1999 Resident Advisor, SEATS (Service Expansion and Technical Support for Family Planning), Dakar, Senegal office 1. Represent SEATS in Senegal and work closely with US PVOS, international donors and NGO involved in family planning/reproductive health. 2. Assist in identifying NGOs and associations needs and opportunities on family planning activities and provide technical assistance accordingly. 3. Assist in the preparation of regular reports to USAID, other donors, and cooperating agencies. 4. Prepare and develop subprojects and provide technical assistance in program implementation, monitoring, and evaluation. 5. Supervise SEATS program agents involved in the project’s implementation. 6. Identify and supervise consultants involved in subprojects technical assistance. April 1990 to December 1995 Communications Advisor, SEATS Africa, Lomé, Togo 1. Provide technical expertise in reproductive health/FP/IEC to approximately 20 service delivery projects in Madagascar and the West Africa region including Senegal, Togo, Mali, Benin, Côte d’Ivoire, Cameroon, Benin, Guinea, and Burkina Faso. 2. Assist in identifying project opportunities and conducting needs assessments and project planning visits. 3. Design, initiate, monitor, and evaluate IEC project and/or IEC components of projects. 4. Assist project agencies to identify ways to market and improve the quality of their services. 5. Identify IEC technical assistance needs and ensure timely provision of technical assistance to projects. 6. Design and conduct IEC-related training workshops and seminars. 7. Assist in the design, production, and evaluation of IEC materials and use. 8. Document project accomplishments, evaluation results, and lessons learned and assist in the dissemination of this information. December 1988 to March 1990 Communications and Training Specialist: Child Survival Program, Lomé Togo. Provide technical expertise for the health education and training elements of the CS policy and strategy paper; take the lead in preparation of the multiyear training plan; develop methodologies and materials for outreach training activities and maintain close liaison with the health education unit (SNES), CCCD, Regional Social Affairs Directors, and the Peace Corps, as well as with UNICEF in the health education and training areas; serve as a resource person for SNES, to develop IEC activities and training for CS activities such as EPI, ORT, FP, and malaria treatment; prepare training programs for central, regional, and district health providers USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 73 CONSULTANCIES February 2012 Prepare the CDC proposal on building the institutional capacity of local Indigenous organizations implementing comprehensive clinical and laboratory services to sustain and enhance the prevention, care and treatment of HIV/AIDS in the Federal Republic of Nigeria under the President’s Emergency Plan for AIDS Relief (PEPFAR) in Nigeria. Identify the needs for NGO capacity building for effective HIV/AIDS prevention, care and services and come up with an appropriate capacity building program for NGOs. June 2011 to November 2011 Provide TA to ADEMAS HIV/AIDS Youth Project supported by EPOS (German NGO). Prepare and facilitate the national advocacy workshop of the project ending December 2011 and lead the fund raising strategy aimed at sensitizing donors to continue supporting the new phases. 1998/1985, worked with: Johns Hopkins University (PCS) and International Science and Technology Institute (ISTI) Academy for Educational Development (AED) and PRITECH USAID, UNFPA, and UNICEF Carried out several missions in the field of BCC, hygiene, reproductive health, maternal and child health, sexually transmitted diseases, training and communications development, several communications KAP studies and needs assessment, multimedia design and production in DRC, Niger, Chad, Senegal, Mali, Mauritania, and Togo. EDUCATION 1985 International Labor Organization (Torino), and National Institute A/V of Brussels: Major subjects: Audio-visual techniques and social communication. 1982-1984 SONY International Institute, Tokyo, Japan: Master in Communication Culture and Technology Major subjects: Communications and marketing research, production, use and evaluation of video, radio and film programs, interpersonal communication, adult learning, training of trainers, IEC qualitative and quantitative research. 1977-1980 National School of Applied Economy (ENEA), Diploma d'Etat. Master in Pedagogy (adult learning) Major subjects: Economy, pedagogy, psychology, sociology, communication, statistics, and demography. 1975 International School of Bordeaux, France, Major subjects: Communications design, development of print and audiovisual materials. 74 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION COUNTRY EXPERIENCE Senegal, Gambia, Togo, Mali, Benin, Côte D’Ivoire, Cameroon, Ghana, Guinea, Rwanda, Mauritania, Burkina Faso, Chad, Niger, Nigeria, Madagascar, South Africa, Zimbabwe, Kenya, D. R. C, Angola, Morocco, Tunisia, Jordan,. USA, France, Belgium, Switzerland, China, and Thailand LANGUAGES Wolof: Fluent French: Fluent English: Good Japanese: Speaking and understanding Good Arabic: Speaking fair, Reading and writing Good PROFESSIONAL AFFILIATIONS Association africaine des spécialistes en communication Groupe Africain des Consultants en communication de la Banque mondiale. Membre du Club Nation et Développement Volontaire de l’Association Sénégalaise pour le Bien Etre Familial Membre fondateur de l’association “servir et non se servir” Membre fondateur du BEFPA (bureau d’études de formation et de production audio-visuelle) HONORS and AWARDS Médaille ordre du Mérite du Sénégal (June 1987) Path: 10 years of service (April 16, 2000) Five (5) Spots World Bank awards from 2005 to present (in recognition of accomplishments in the qater and sanitation sector) PUBLICATIONS: 1. Training curriculum for community health workers in the Eastern region of Senegal (2003) 2. Training curriculum on communication and counseling for primary health care providers 3. Family Planning in Africa. Who Decides? 4. Strategic planning for communications programs 5. Several field notes on: 6. Development of water and sanitation communication strategy in West and Central Africa 2005–2010 7. Community based distribution 1995–1998 8. Delegation of water management system in 6 West African countries 9. Water and sanitation services state overview in 12 African countries (2008) 10. Participated in the bimonthly production and dissemination of ACCESS (the newsletter of the Water and Sanitation Program of the World Bank). USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 75 TIFFANY LILLIE, PHD, MHS 7020 Yerevan Place Dulles, VA 20189 Cell Phone (Armenia): (374) 93-476-217 lillie.tiffany@yahoo.com EXPERTISE: Behavioral sciences, strategic behavior communications, HIV prevention, monitoring & evaluation, qualitative research, project development, and intermediate French EDUCATION Doctorate of Philosophy (Ph.D.), Johns Hopkins Bloomberg School of Public Health; Health, Behavior, and Society Department, 2006. Masters in Health Science (MHS), Johns Hopkins School of Public Health, Department of International Health, Behavioral and Community Intervention Track, 2002. Bachelor of Arts in English, Ohio University, Athens, Ohio, Cum Laude, 1995. EMPLOYMENT EXPERIENCE Behavioral Sciences and M&E Consultant USAID/Senegal, April—June 2012 Performance Evaluation of ADEMAS, Agence pour le Développement du Marketing Sociale: Developed methodology and designed qualitative questionnaires for the performance evaluation, conducted interviews, analyzed results, and co-wrote final report to (1) determine the extent to which the expected results in social marketing have been achieved; and (2) provide pertinent information on how the program is being implemented to inform USAID/ Senegal’s operational decision-making. John Snow International, AIDSTAR-One Project, May 2011—July 2012 Management and Technical Review of entries for the HIV and AIDS Prevention Knowledge Base, an on-line resource. UNAIDS/Geneva, December 2011—January 2012 HIV/AIDS Technical Writing: Literature review and overview paper on community mobilization and community-based service delivery within six HIV programmatic domains in the context of the UNAIDS Investment Framework [Lancet 377(9782), 2011]. Abt Associates/Armenia, HS-Star Project, May 2011—December 2011 Monitoring & Evaluation: Developed scope of work for a rapid feedback study to assess implementation of the quality of care processes introduced by former USAID projects, which included technical approach, objectives, methodology, sampling, and implementation approach; developed questionnaires; led interviewer training; analyzed in-depth interviews, focus group discussions, and structured questionnaires; and wrote final report. 76 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION World Vision International Middle East/Eastern Europe Regional Office, September 2011 Strategic Behavior Communication (SBC) Training: Developed and led a two-day SBC training in Cyprus. John Snow International, AIDSTAR-One Project, February 2009—September 2010 HIV/AIDS Prevention Technical Writing: Case study on Botswana National HIV/AIDS Prevention Campaign on Multiple and Concurrent Sexual Partnerships (MCP); MCP Technical Brief; Prevention Resource Pages on Partner Reduction, Abstinence, Curriculum Based Education, Transactional and Intergenerational Sex, Peer Education and Outreach, MCP, and Interpersonal Communications. UNAIDS/Geneva, October 2009—January 2010 Drafted Definitions for Prevention Intervention Terms Measure Evaluation, Macro International, Mali, November—December 2009 Monitoring & Evaluation Framework: Led facilitation process and provided technical support to USAID and its Health Implementing Partners to solidify the revised strategic framework as well as begun revising the M&E Framework including indicators, data sources, and collection methods. UNAIDS & Global Fund/Morocco, June—December 2009 Assessment of Sex Worker Peer Education Programs: Designed interview questionnaire; interviewed key project staff who implemented peer education projects to assess quality of current programs; developed and proposed a methodology on how to improve the quality of peer education projects; and led a stakeholder meeting to reach consensus on next steps. University Research Co., LLC, Health Care Improvement Project/Ivory Coast, May—October 2009 Assessment of Peer Education Programs: Led development of three quality assurance assessment tools, including pretesting, for peer education and outreach programs for NGOs, peer educators, and peers; Planned the rapid assessment in three different regions of Côte d’Ivoire, including the identification of specific NGOs; Coached the assessment team on the tools; Analyzed data in completed questionnaires and synthesized into a final report; and Wrote final report with recommendations on how to improve the quality of peer education program in Côte d’Ivoire. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 77 United States Agency for International Development (USAID)/Mozambique, October 2008—June 2009 HIV/AIDS Prevention Technical Writing: Drafted two Indefinite Quantity Contracts (IQC)s on HIV/AIDS prevention projects targeted toward the general population, youth and bridge populations and most-at-risk-populations focusing on multiple and concurrent partnerships, condom use, counseling and testing, and comprehensive prevention services. USAID/Office of HIV/AIDS (OHA), September 2008—March 2010 Prevention and M&E Technical Assistance: Coordinated, organized, and provided technical in￾put to a two-day technical consultation on multiple and concurrent sexual partners; and Provided technical in-put to PEPFAR II sexual prevention output and outcome indicators as part of the TWG subgroup. HIV/AIDS Prevention Advisor (Level III) United States Agency for International Development (USAID), Global Health Fellows Program, Public Health Institute, Washington, DC, August 2007-August 2008 Strategically assessed and provided technical assistance to country programs to ensure portfolios responded to the national epidemic as well as were of high quality, appropriate dose and coverage, and were monitored and evaluated to ensure results. Countries included Haiti, Dominican Republic, Zimbabwe, Caribbean Regional Program, Rwanda, Swaziland, and Lesotho; Participated in interagency working groups on Prevention in the General Population and Youth and Prevention of HIV in Persons Engaged in High-Risk Behaviors to aid countries in keeping up-to-date with the latest scientific findings and best practices to inform country programs; Participated in the development of prevention indicators for PEPFAR II; and Participated in interagency working group on the Evaluation of Prevention Programs. Monitoring, Evaluation, and Reporting Advisor (Level II) United States Agency for International Development, Global Health Fellows Program, Public Health Institute, Washington, DC, June 2006-August 2007 Advised the Caribbean Regional Program, Haiti, and Dominican Republic in their Strategic Information portfolios (i.e., monitoring and evaluation, surveillance, and health information systems), which included providing technical assistance on the design and implementation of national or program-level monitoring and evaluation systems; Lead and co-chaired the Monitoring and Evaluation Technical Working Group & the Monitoring and Evaluation Capacity Building subgroup, which included the development and implementation of training programs, workshops, and other participatory activities that served to broaden knowledge and practice of effective monitoring and evaluation systems; and 78 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION Advised the Prevention of HIV in Persons Engaged in High Risk Behaviors Technical Working Group on strategic information activities, which included the review of behavioral surveillance surveys, and provided recommendations on the improvement of measurement within injecting drug users. Associate Technical Officer, Prevention Family Health International/IMPACT Project, Behavior Change and Communication Department, Arlington, VA, September 2001-February 2006 Analyzed quantitative and qualitative data and co-wrote final report from joint FHI/Population Council baseline study on BCC/ABC; Developed, organized, and co-facilitated two participatory monitoring and evaluation workshops in Kenya for IMPACT BCC Naivasha partners; Reviewed and finalized the analysis of the BCC Formative Assessment conducted in Addis Ababa, Ethiopia; Worked on FHI teams, both BCC and M&E, in writing research study protocols (including ABC study), questionnaires, and final reports; Assisted in the technical areas of BCC as well as monitoring and evaluation, workplace programs, uniform services interventions, and protection of human subjects. Consultant Population Council, Washington, DC, May-June 2005 Analyzed quantitative and qualitative data and co-wrote final report on joint FHI/Population Council baseline study on Abstinence, Being Faithful, and Using Condoms (ABC). Research Assistant Johns Hopkins Bloomberg School of Public Health, Lighthouse Project, Baltimore, Maryland, September 2004-June 2005 Analyzed injecting drug user qualitative data using ATLAS-ti. Wrote paper for publication. Intern Population Services International (PSI), Washington, DC, August-September 2001 Analyzed three knowledge, attitude, and practice surveys from men having sex with men and commercial sex workers on condoms, HIV/AIDS, and social marketing in specific Central American countries and wrote the report to disseminate and use for strategic planning. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 79 Intern TvT Associates/The Synergy Project, Washington, DC, June 2001-September 2001 Researched and wrote a series of country summaries, designated as "rapid scale up" and "intensive focus" under USAID's expanded response to the HIV/AIDS pandemic to be posted on Synergy’s website. Project Coordinator for Business Development University Research Company (URC), International Division, Bethesda, MD, July 1999–August 2000 Administrative Assistant Peace Corps-Office of Planning, Budget, and Finance, Washington, DC, July 1998–1999 Peace Corps Volunteer, Community Health Specialist Peace Corps, Burkina Faso, September 1995–September 1997 LANGUAGE SKILLS: French (working knowledge) COMPUTER SKILLS: Microsoft Word, PowerPoint, ATLAS Peer Reviewed Publications: Lillie, T, Pulerwitz, J, & Curbow, B. (2009). “Kenyan In-School Youths’ Level of Understanding of Abstinence, Being Faithful, and Condom Use Terms: Implications for HIV-Prevention Programs.” Journal of Health Communication, 14(3): 276-92. Smith, K., Lillie, T., & Latkin, C. (2007). “Injection Drug Users’ Strategies to Manage Perceptions of Personal Risk: How do IDUs See HIV as Having Affected Them?” AIDS Education and Prevention, 19(3): 245-57. Lillie, T., Mucheke, S., & Emah, E. (2005, December). Participatory Monitoring and Evaluation: Letting the Community Decide What to Monitor. Poster presented at the 14th International Conference on HIV/AIDS and Sexually Transmitted Infections in Africa (ICASA 2005), Abuja, Nigeria. Presentations Lillie, T. (2008, May) “Deconstructing “Reached”: The Need for Quality, Dosage, and Coverage in Prevention Programs,” Washington, DC. Non-Peer Reviewed Publications: Lillie, Tiffany. 2010. The O Icheke Campaign, Botswana: A National Behavior Change Communication Program to Reduce Multiple and Concurrent Partnerships. Case Study Series. Arlington, VA: USAID, AIDS Support and Technical Assistance Resources, AIDSTAR-One Task Order 1. Pulerwitz, J., Lillie, T., Apicella, A., McCauley, A., Nelson, T., Mwarogo, P., & Ochieng, S. 2006. ABC Messages for HIV Prevention in Kenya: Clarity and Confusion, Barriers and Facilitators. Horizons/Population Council & IMPACT/Family Health International. 80 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 81 ANNEX 5. QUESTIONNAIRE ADEMAS Déroulement Présentation de l’équipe Objet de la rencontre Règles de travail 1. Comment ADEMAS a-t-elle traité les problèmes spécifiques culturelles et de genre dans l'approche de marketing social? ……………………………………………………………………………………………….… ….……………………………………………………………………………………………… ……………………………………………………………………………………………….… …………………………………………………………………………………………….…… 2. Qui étaient les groupes cibles d’ADEMAS? Comment appréciez-vous le matériel de communication et le matériel publicitaire développés pour les différentes populations cibles? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 3. Quelles appréciations faites-vous des canaux de communication modernes utilisés pour appuyer les interventions du programme? Comment ont-ils aidé à atteindre les populations cibles? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 4. Comment ADEMAS a-t-elle impliqué le secteur privé commercial dans la fourniture de produits et services de planification familiale? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 5. Comment ADEMAS a-t-elle élargi et consolidé les réseaux existants de distribution des produits (Protec, Securil ®) pour accroitre l'accès? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 6. Comment la structure de l'organisation a-t-elle influencé l'efficacité de la mise en œuvre du programme? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 82 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 7. Quelles adéquations voyez-vous entre le volume et l’organisation des activités mises en œuvre par ADEMEAS et le dosage des compétences du personnel? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 8. Comment appréciez-vous la règle de couts/efficacité des différents processus clés du Programme de marketing social, y compris: i. La nature des produits et leur placement ii. Les aspects publicitaires iii. Les approches Communautaires iv. La Production et la diffusion de la série radiophonique 9. Quelle est la stratégie utilisée par ADEMAS pour impliquer les parties prenantes clés dans le programme? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 10. 10. Selon vous est-ce que le programme a atteint les objectifs de couverture qui étaient fixés ? Si oui /non, Quelle est votre appréciation ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 11. Dans quelle mesure les recommandations de l'évaluation PSP-One pour la stabilité institutionnelle ont été suivies d'effet? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 12. Quelle appréciation faites-vous de la pérennité du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 83 12. B. Quels sont les obstacles juridiques, réglementaires ou administratifs rencontrées dans le cadre de ce projet (2006-2011) qui doivent être pris en compte dans le programme actuel ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 13. Pouvez-vous partager avec nous trois choses qui pourraient renforcer le succès du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 14. Pouvez-vous partager avec nous trois améliorations ou défis qui pourraient menacer la poursuite du programme ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 15. Avez-vous autre chose à partager avec nous ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… Merci pour votre disponibilité 84 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 85 ANNEX 6. QUESTIONNAIRE IDI USAID Déroulement Présentation de l’équipe Objet de la rencontre Règles de travail Personnes rencontrées : 1. De façon générale, pensez vous que ADEMAS a bien atteint ses objectifs dans le cadre du partenariat avec les autres composantes du programme de santé financé par l'USAID/Sénégal ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 2. De façon spécifique, comment appréciez-vous l’atteinte des objectifs assignés à ADEMAS (Objectifs tels que): Convaincre, motiver et recruter de nouveaux utilisateurs L'augmentation du volume des ventes de produits du marketing social; L'augmentation de l'utilisation des produits et services; Développer et mettre en œuvre un plan d'affaires qui oriente le partenaire d'exécution ……….……………………………………………………………………………………..… …...……..……..…….…………………………………………………………………………. ………………………………………………………………………………………….……… ………………………………….……………………………..… 3. Comment appréciez-vous les efforts de coordination/collaboration initiés par ADEMAS dans le cadre du partenariat avec les autres composantes du programme de santé financé par l'USAID/Sénégal ? Si oui si non, pouvez vous donner des exemples ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 4. Comment appréciez-vous la part d’ADEMAS dans la fourniture nationale de produits et services du Ministère de la Santé? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 86 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 5. Que pensez-vous de la stratégie utilisée par ADEMAS pour impliquer le secteur privé commercial et les parties prenantes clés dans le programme? Que suggérez-vous pour une amélioration ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 6. Comment appréciez-vous l’accessibilité des produits et services offerts par ADEMAS aux bénéficiaires EN CE QUI CONCERNE: i. La nature des produits et leur accessibilité géographique et financière ii. Les aspects publicitaires iii. Les approches Communautaires iv. La Production et la diffusion de la série radiophonique ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 7. Quelle appréciation faites-vous de la pérennité du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 8. Pouvez-vous partager avec nous trois domaines qui pourraient être renforcés pour assurer le succès du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 9. Pouvez-vous partager avec nous trois domaines ou défis á améliorer qui pourraient menacer le programme ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 87 10. Avez-vous d’autres recommandations pour améliorer l’intervention d’ADEMAS? ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… Merci pour votre disponibilité. 88 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 89 ANNEX 7. QUESTIONNAIRE IDI MSP ET PARTENAIRES ADEMAS Déroulement Présentation de l’équipe Objet de la rencontre Règles de travail Personnes rencontrées : 1. Quels sont les domaines de collaboration sur lesquels vous travaillez avec ADEMAS ? Que pensez vous de cette collaboration? ………………………………………………………………………………………………… ….……………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 2. Avez-vous contribué dans le choix des groupes ciblés par ADEMAS? Si oui comment avez￾vous été associé ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 3. Quel a été votre apport dans la conception et l’utilisation du matériel de communication pour les différentes populations cibles? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 4. Que pensez-vous de la stratégie utilisée par ADEMAS pour impliquer le secteur privé commercial et les parties prenantes clés dans le programme? Que suggérez-vous pour une amélioration ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 5. Comment appréciez-vous l’efficacité et le professionnalisme d’ADEMAS dans la mise en œuvre du programme? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 6. Comment appréciez-vous l’accessibilité des produits et services offerts par ADEMAS aux bénéficiaires EN CE QUI CONCERNE: 90 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION i. La nature des produits et leur accessibilité géographique et financière ii. Les aspects publicitaires iii. Les approches Communautaires iv. La Production et la diffusion de la série radiophonique ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ……………………………………………………………………………………… 7. Selon vous est-ce que le programme a atteint les objectifs de couverture qui étaient fixés ? Si oui /non, Quelle est votre appréciation ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 8. Quelle appréciation faites-vous de la pérennité du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 9. Pouvez-vous partager avec nous trois domaines qui pourraient être renforcés pour assurer le succès du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 10. Pouvez-vous partager avec nous trois domaines ou défis á améliorer qui pourraient menacer le programme ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 11. Avez-vous d’autres recommandations pour améliorer l’intervention d’ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… Merci pour votre disponibilité. USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 91 ANNEX 8.QUESTIONNAIRE IDI PARTENAIRES PRIVES ADEMAS Déroulement Présentation de l’équipe Objet de la rencontre Règles de travail Personnes rencontrées : 1. Quels sont les domaines de collaboration sur lesquels vous travaillez avec ADEMAS ? Que pensez-vous de cette collaboration? ….……………………………………………………………………………………………… …….…………………………………………………………………………………………… ………………………………………………………………………………………………… ……………….....……………………………………………………………………………… 2. Avez-vous contribué dans le choix des groupes ciblés par ADEMAS? Si oui comment avez￾vous été associé ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 3. Quel a été votre apport dans la conception et l’utilisation du matériel de communication pour les différentes populations cibles? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 4. Que pensez-vous de la stratégie utilisée par ADEMAS pour impliquer le secteur privé commercial et les parties prenantes clés dans le programme? Que suggérez-vous pour une amélioration ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 5. Comment appréciez-vous l’efficacité et le professionnalisme d’ADEMAS dans la mise en œuvre du programme? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 92 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 6. Comment appréciez-vous l’accessibilité des produits et services offerts par ADEMAS aux bénéficiaires EN CE QUI CONCERNE: i. La chaine logistique et l’accessibilité géographique et financière des produits et services ii. Les aspects publicitaires iii. Les approches Communautaires iv. La Production et la diffusion de la série radiophonique ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 7. Selon vous est-ce que le programme a atteint les objectifs de couverture qui étaient fixés ? Si oui /non, Quelle est votre appréciation ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 8. Quelle appréciation faites-vous de la pérennité du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 9. Pouvez-vous partager avec nous trois domaines qui pourraient être renforcés pour assurer le succès du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 10. Pouvez-vous partager avec nous trois domaines ou défis á améliorer qui pourraient menacer le programme ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 93 11. Avez-vous d’autres recommandations pour améliorer l’intervention d’ADEMAS? ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… ……………………………………………………………………………………………………… Merci pour votre disponibilité. 94 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 95 ANNEX 9. POPULATION QUESTIONNAIRE FDG Déroulement Présentation de l’équipe Objet de la rencontre Règles de travail Liste de Participants: 1. Que savez-vous du programme de marketing social de ADEMAS? ..………………………………………………………………………………………………… ….………………………………………………………………………………………………. ..………………………………………………………………………………………………… ..………………………………………………………………………………………………… 2. Quels sont les produits de planification familiale que vous connaissez? Avez-vous entendu parler de Protec, le Depo-Provera, Securil, ou des produits Fagarou? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 3. Connaissez-vous quelqu'un qui utilise des produits de planification familiale ou les produits Aquatabs? Si oui, quels produits utilisent- ils (Protec, le Depo-Provera, Securil, Fagarou)? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 4. Quelle est votre opinion sur les produits PF / SR offerts par Ademas (Protec, le Depo￾Provera, Securil, Fagarou)? (EX. qualité bonne, La nature des produits et Les aspects publicitaires,) ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 5. Où pouvez-vous trouver ces produits? Ces produits sont-ils accessibles? Géographique (faciles à trouver) et financière (abordables) ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 96 USAID/SENEGAL: SOCIAL MARKETING PROGRAM PERFORMANCE EVALUATION 6. Avez-vous écouté le programme de radio Ngelawu Nawet? Si oui, combien de fois avez￾vous l'écouter? Que retenez-vous à ce sujet (messages clés)? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 7. Avez-vous parlé à vos amis ou en famille sur les messages qui Ngelawu Nawet discute? Si oui, quels sont les sujets parlez-vous? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 8. Avez-vous participé à des groupes d'écoute? Si oui, combien de fois avez-vous participé? Que pensez-vous des groupes d'écoute? De quel sujet spécifique avez-vous discuté? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 9. Votre ecoute de l’émission Ngelawu Nawet / ou votre participation dans les groupes d'écoute, vous a-t-elle incité à réfléchir ou à agir différemment en ce qui concerne votre santé et l'accès à certains services de PF? Si oui, comment? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 10. Pouvez-vous partager avec nous trois domaines qui pourraient être renforcés pour assurer le succès du programme ADEMAS ? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 11. Pouvez-vous partager avec nous trois domaines ou défis á améliorer qui pourraient menacer le programme ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… 12. Avez-vous une autre recommandation ? Aimeriez-vous ajouter autre chose sur le programme et les interventions de’ ADEMAS? ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… ………………………………………………………………………………………………… Merci pour votre disponibilité. For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com