September 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently for DevTech by Dr. Carol S. Shepherd, Mr. Iain Mclellan and Mr. Willie Kachaka. SSDI-COMMUNICATIONS ACTIVITY PERFORMANCE EVALUATION ACKNOWLEDGMENTS The SSDI-Communication evaluation team would like to thank the individuals who generously took time to provide candid information and perceptions on the communication and community mobilization activities implemented under the Support for Service Delivery Integration (SSDI) Communication and Services projects. The team wants to thank the more than 400 beneficiaries who answered survey questions, graciously invited the team into their homes to visually inspect improvements, and participated in focus group discussions in the 10 village research sites, often with little advance notice. Throughout this evaluation, the team benefited from the support and assistance of Ministry of Health directors, managers and health care providers of the Essential Health Package in Malawi. The team would like to acknowledge Chimwemwe Chitsulo, USAID/Malawi’s Monitoring, Evaluation and Learning Specialist and Contracting Officer’s Representative, for his meticulous guidance throughout this evaluation process. Last, the team is grateful to JHU-CCP, Jhpiego, Save the Children, Plan International, Care, Galaxy Media, Mercantile, ADECOTS, the Red Cross, the Center for Alternatives for Victimized Women, Orphans and Children, and the Salima AIDS Support Organization for making their staff and project information available to the team. These professionals are working tirelessly to improve access to quality health care services to the Malawian people. A special thank you also goes to Ward Jacobs, health management information systems consultant, Health Section, UNICEF Malawi. SSDI-Communications Activity Performance Evaluation September 9, 2016 AID-612-TO-16-00003 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. SSDI Communications Activity Performance Evaluation i CONTENTS Acronyms............................................................................................................................................................................. ii Executive Summary .......................................................................................................................................................... iii I. Evaluation Purpose and Questions ............................................................................................................................. 1 II. Project Background....................................................................................................................................................... 2 III. Evaluation Methods and Limitations ........................................................................................................................ 3 IV. Findings, Conclusions and Recommendations ...................................................................................................... 6 Question 1: To what extent did the SBCC packages, including the Moyo ndi Mpamba platform, effectively promote increased awareness and practice of preventive behaviors in target communities? 6 Question 2: To what extent did SSDI-Communication improve the capacity of the HES and local institutions to develop, coordinate, implement and provide oversight to health sector integrated SBCC? ............................................................................................................................................................................ 10 Question 3: To what extent did SSDI-Communication’s fellowship program improve strategic planning and implementation of SBCC, and media reporting of health issues? ............................................................ 13 Question 4: What effect did the joint implementation of community mobilization by Services and Communication have on the effectiveness of this intervention? ..................................................................... 15 Question 5: What are the most significant accomplishments, best practices and lessons learned from SSDI-Communication? Explicitly identify and document the facilitating and inhibiting factors to positive performance for each of the above questions. .................................................................................................... 20 Annex I: Evaluation Statement of Work .................................................................................................................... 23 Annex II: Data Collection Instruments ....................................................................................................................... 26 Annex III: Key Informants Interviewed ....................................................................................................................... 53 Annex IV: GPS Site MAPS .............................................................................................................................................. 57 Annex V: Community Action Cycle ............................................................................................................................ 62 Annex VI: Disclosure of any Conflicts of Interest ................................................................................................... 63 SSDI Communications Activity Performance Evaluation ii ACRONYMS BCC Behavior change communication CAC Community Action Cycle CAG Community Action Group CDCS Country Development Cooperation Strategy CM Community mobilization DHMT District Health Management Team EHP Essential Health Package FGD Focus group discussion GVH Group Village Head HES Health Education Section HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome IPC Interpersonal communication JHU-CCP Johns Hopkins University Center for Communication Programs KII Key informant interview M&E Monitoring and evaluation MDHS Malawi Demographic and Health Survey MNCH Maternal, neonatal and child health MoH Ministry of Health NGO Non-governmental organization SBCC Social and behavior change communication SSDI Support for Service Delivery Integration SSDI–Communication Support for Service Delivery Integration–Communication SSDI–Services Support for Service Delivery Integration–Services SSDI–Systems Support for Service Delivery Integration–Systems USAID United States Agency for International Development SSDI Communications Activity Performance Evaluation iii EXECUTIVE SUMMARY EVALUATION PURPOSE AND QUESTIONS Purpose: The purpose of this evaluation was to determine the effectiveness of the SSDI-Communications multilevel approach to promote normative behavior change and health-seeking practices. Evaluation questions: 1. To what extent did the social and behavior change communications (SBCC) packages, including the Moyo ndi Mpamba (Life is precious) platform, effectively promote increased awareness and practices of preventive behaviors in target communities? 2. To what extent did SSDI-Communication improve the capacity of the Health Education Section (HES) and local institutions to develop, coordinate, implement and provide oversight for health sector integrated SBCC? 3. To what extent did SSDI-Communication’s fellowship program improve strategic planning and implementation of SBCC and media reporting of health issues? 4. What effect did the joint implementation of community mobilization by SSDI-Services and SSDI￾Communication have on the effectiveness of this intervention? 5. What are the most significant accomplishments, best practices and lessons learned from the SSDI￾Communication activity? Explicitly identify and document the facilitating and inhibiting factors to positive performance for each of the above questions. PROJECT BACKGROUND USAID’s health investment: In 2011, at the inception of the SSDI project, Malawi suffered from poor health indicators, inefficient and understaffed service delivery systems, gender inequalities and widespread use of harmful traditional practices. USAID/Malawi sought to address these issues through the interrelated SSDI projects: (1) Services, (2) Communication and (3) Systems. EVALUATION DESIGN, METHODS AND LIMITATIONS Study design: This performance evaluation assesses changes in behaviors and practices that are related to SSDI-Communication activities. The team visited five purposively selected sites, one in each zone, to evaluate the hypothesis, “Greater improvements to behavior change and practices indicators occurred in the SSDI-Communication areas.” The team identified general trends to triangulate and validate existing data. Data collected included: media content study: 304; focus group discussions (FGDs): 30; key informant interviews (KIIs): 97; Likert surveys: 264; observation checklist: 55 households. Limitations: The sampling strategy captures areas targeted by SSDI-Communications interventions. Results are not representative of the larger population. As KIIs and FGDs represent a significant portion of the data, potential biases related to personal opinions and recollection cannot be ruled out. SSDI Communications Activity Performance Evaluation iv FINDINGS AND CONCLUSIONS QI: SBCC Packages Findings1 Radio and posters were the most effective channels for SBCC messaging: In the media content study, beneficiaries cited radio as the primary source of information on Essential Health Package (EHP) issues. Among Likert survey participants, 61.7 percent were familiar with Moyo ndi Mpamba radio programs and advertisements. In the 15 project districts in SSDI-Communication’s endline survey, 59.5 percent had heard of Moyo ndi Mpamba radio and 45.2 percent had listened. Both men and women reached: Exposure to Moyo ndi Mpamba messages and materials was roughly the same for men and women (media content study). Women’s access to Moyo ndi Mpamba radio programing was less than that of men (20 percent fewer had heard of Moyo ndi Mpamba radio and 24 percent fewer listened). Moyo ndi Mpamba brand well recognized: In the endline survey, 83.6 percent had heard of Moyo ndi Mpamba. The platform was appreciated and the concept understood by 82.1 percent (media content study). Women’s knowledge increased, and 98.6 percent said that health practices improved as a result of Moyo ndi Mpamba (media content study). Effectiveness of media channels: The network of 18 radio stations provides good value for a wide listenership, though a third of beneficiaries were not reached. Posters were attractive and seen by 61.65 percent in the endline survey. In addition, 22.7 percent had ever heard of the Family Health Booklets, and 6.25 percent had read the booklet. Flip charts have great potential but were produced too late. Open days had impact, but billboards, road shows and music festivals were not effective. Conclusions  The Moyo ndi Mpamba platform has high value and denotes quality, and its use should be continued.  The evidence-based integrated SBCC package proved to be effective and reduced duplication.  Radio set the stage for effective interpersonal communication (IPC) by community outreach workers.  Posters, Family Health Booklets and flip charts have the greatest value and reach; billboards have the least value. Recommendations  Continue use of the Moyo ndi Mpamba brand into the foreseeable future by future projects of the Ministry of Health (MoH) and its partners. There is room to integrate other EHP issues.  Continue investments in radio, particularly spot ads, magazine and community radio.  Study improving women’s radio listenership through low-cost, wind-up radios. 1 NB: The SSDI-Communications Endline Survey covered all 15 project districts. The data collected by the team included only the districts visited during the evaluation (Salima, Machinga, Karonga, Lilongwe and Chikwawa) and covered approximately 25 percent of the Group Village Heads (GVH) in the four traditional authorities in each district where community mobilization activities were implemented. Hence, data may differ. SSDI Communications Activity Performance Evaluation v  Increase production of flip charts and provide training on their use in IPC.  Continue investments in print materials, including posters and Family Health Booklets, and reduce billboards, music festivals and road shows. Q2: Capacity Building Findings The SBCC leadership role of HES slow in developing: The HES feels more like an observer than an actor. Challenges such as staffing, low budget and vertical programs handicapped its evolution. The HES continues to need leadership, coordination and advocacy skills for the future. Training in community mobilization worked well: The Community Action Cycle (CAC) manual and tools are user-friendly. However, problems with management of the community mobilization activities affected the training and support of community groups (lack of follow-up training, allowances, and insufficient supervision and transport). Conclusions  The widespread introduction of evidence-based strategic SBCC planning bodes well for its use well into future.  No specific terms of reference were developed for HES capacity development; project and HES collaboration was ad hoc in nature. It was more intense at the beginning of the project. Recommendations • Specific HES capacity-building outcomes with clear objectives and indicators are needed for the HES to develop the skills required to take the lead in overseeing and managing SBCC. These should be set collaboratively between the project and the MoH. • Continue with Leadership in Strategic Communication Planning workshops. • Expand training for community radio stations. Q3: Fellowships Findings Fellowships and internships were successful, but their scope was limited. Chancellor College, the key partner for the fellowships, was pleased with the collaboration on SBCC. Journalists’ support was diverse and resulted in increased and more accurate media coverage on health. Conclusions  Internships and fellowships have the potential to develop tomorrow’s SBCC expertise.  Chancellor College is an enthusiastic partner for enhancing SBCC expertise.  Costs of increasing SBCC expertise and media coverage are relatively low. Recommendations  Revamp the fellowship and internship program to increase the number of participants.  Study the outcome of SSDI-Communication’s journalist training.  Continue to train journalists covering health. SSDI Communications Activity Performance Evaluation vi Q4: Community Mobilization Findings Community mobilization (CM) start-up was delayed by movement from SSDI-Communication to SSDI￾Services. By the end of 2015, 306 Community Action Groups (CAGs) had completed their first CAC. Among Likert survey respondents, 86 percent liked the CAC and felt it was easy to use. Most feel that the CAGs are sustainable and are an integral part of the health system. Community mobilizers agree that there are more pit latrines and handwashing stations, and more families now advocate for their own health and timely use of EHP, than five years ago, according to the Likert study. However, SSDI￾Communication’s endline survey found that only 8.9 percent of men and 6.1 percent of women in the 15 project districts had received a home visit, an indicator of the small scope of CM. Weak management diminished CM effectiveness, and supervision fell through the cracks. Conclusions  CM was successful, despite management issues and funding delays that limited its impact. It was also conducted on a relatively small scale in 13 of the 15 districts. CM interventions covered approximately 25 percent of Group Village Heads (GVH) in the targeted traditional authorities.  The participatory and inclusive CAC approach promoted ownership and sustainability.  Mobilization made communities self-reliant. Recommendations  Simplify and articulate the CM management structure.  Use the same CM structure for all organizations under the project.  Strategize and budget to scale up CM at the beginning. Q5: Best practices and lessons learned Conclusions: Facilitating factors  Embodiment of evidence-based planning throughout the project, including all media products, EHP and CAC process  Adoption of GVH bylaws  Integrating EHP messages and services  Effective community mobilization methodology  Good collaboration between government, donor, non-governmental organizations (NGOs), the private sector and mass media  Building capacity from the ground up to empower communities through CAGs and Community Management Teams  Adequate financial resources to make impact with SBCC Conclusions: Inhibiting factors  SBCC inhibited by limited IPC at the community level  Limited access to radio by up to one quarter of women  Cumbersome and inefficient CM management structures  Delays and inefficiencies resulting from the decision to move CM to SSDI-Services  Implementation inhibited by funding cutbacks and delays in funding to subgrantee level  Investments insufficient to reach beneficiaries on the scale needed to bring large-scale change SSDI Communications Activity Performance Evaluation vii  Collaboration with government limited by its funding constraints Recommendations: Best practices  Use of mass media, mid-media and IPC via CM informed and inspired beneficiaries.  CM generated behavior change, including male involvement in family planning, antenatal care and maternal, neonatal and child health.  Integration of vertical programs into one cohesive intervention strengthened response.  Coordination between Communication, CM and Services is needed to maximize efficiencies and program results.  GVH bylaws generated huge behavior change at the community level but need monitoring.  Investments in integrated health communication reaped dividends and merit continuation. SSDI Communications Activity Performance Evaluation 1 I. EVALUATION PURPOSE AND QUESTIONS PURPOSE The purpose of this evaluation was to determine the effectiveness of the Support for Service Delivery Integration (SSDI)-Communication’s multilevel approach to promote normative behavior change and health-seeking practices. The focus was to determine the extent to which SSDI-Communication achieved its primary objectives, as identified in Annex I. The evaluation also focused on the extent to which SSDI-Communication was able to build the capacity of the Health Education Section (HES), the degree to which its campaigns resonated with individuals and communities, and an appraisal of the community mobilization implementation model. The target audiences for this evaluation include USAID/Malawi, the Malawi Ministry of Health (MoH), other stakeholders, USAID/Washington, and the implementing partners: Johns Hopkins University Center for Communication Programs (JHU-CCP), Jhpiego and Abt Associates. Findings and recommendations from this evaluation will inform USAID/Malawi’s new SBCC and health activities. The results of this evaluation will contribute to the lessons learned on integrated health programming under the Global Health Initiative and USAID/Malawi’s Country Development Cooperation Strategy (CDCS). EVALUATION QUESTIONS The following questions formed the basis for the performance evaluation, as outlined in the scope of work (Annex I): 1. To what extent did the social and behavior change communications (SBCC) packages, including the Moyo ndi Mpamba platform, effectively promote increased awareness and practices of preventive behaviors in target communities? 2. To what extent did SSDI-Communication improve the capacity of the HES and local institutions to develop, coordinate, implement and provide oversight for health sector integrated SBCC? 3. To what extent did SSDI-Communication’s fellowship program improve strategic planning and implementation of SBCC and media reporting of health issues? 4. What effect did the joint implementation of community mobilization by SSDI-Services and SSDI￾Communication have on the effectiveness of this intervention? 5. What are the most significant accomplishments, best practices and lessons learned from the SSDI￾Communication activity? Explicitly identify and document the facilitating and inhibiting factors to positive performance for each of the above questions. SSDI Communications Activity Performance Evaluation 2 II. PROJECT BACKGROUND JUSTIFICATION OF USAID’S HEALTH INVESTMENT In 2011, at the inception of the SSDI project, Malawi suffered from poor health indicators, inefficient and understaffed service delivery systems, gender inequalities and widespread use of harmful traditional practices. According to the 2010 Malawi Demographic and Health Survey (MDHS), the maternal mortality ratio was 675 deaths per 100,000 live births. Early childbearing, high unmet need for modern contraceptive methods and high fertility rates increased the risks of death and disability to both mother and child. One quarter of all children under 5 were underweight. Ten percent of young adults were HIV-positive, and approximately 1.1 million Malawians were living with HIV. Malaria was endemic in 95 percent of the country, and less than 40 percent of Malawians owned mosquito nets. Together, these statistics justified USAID’s health investment in Malawi. USAID’S OVERARCHING STRATEGIC GOAL AND 3-C APPROACH USAID/Malawi’s CDCS 2013–2018 has as its overarching strategic goal to “Improve the quality of life for Malawians.” USAID/Malawi’s CDCS hypothesis states, “If assistance is integrated, then development results will be enhanced, more sustainable, and lead to achievement of the CDCS goal.” The CDCS promotes integration through the concentration of program and financial resources by what USAID calls a “3-C Approach”: co-locating interventions, coordinating better and collaborating to foster linkages. See Annex I for further detail. USAID/MALAWI FLAGSHIP HEALTH ACTIVITY One way that USAID/Malawi seeks to achieve the CDCS goal is through the SSDI project. This activity consists of three interrelated sector activities: (1) SSDI-Services, (2) SSDI-Communication and (3) SSDI￾Systems. In close collaboration with the MoH, SSDI interventions support the increased availability and quality of Essential Health Package (EHP) services, reinforce health promotion and disease prevention among households and strengthen elements of the health system to sustain effective EHP delivery. STRIDES IN IMPROVING HEALTH IN MALAWI While causality cannot be inferred, the preliminary results of the 2015/2016 MDHS offer evidence that USAID’s and other partners’ investments in health are improving the health status of Malawians. Malawi has made significant strides toward improving the health of its people, as shown by the improvement in Table 1, and the ensuing health indicators. Overall, 40 percent of households have access to an insecticide-treated mosquito net, and 82 percent of women and 68 percent of men had ever been tested and had received the results of their last HIV test. Half of households use an improved and not shared toilet/latrine facility, whereas 31 percent use facilities that would be considered improved if they were not shared by two or more households. The majority of households (87 percent) obtain drinking water from an improved source, compared to 80 percent in 2010. A closer look at SSDI-Communication will show how it has contributed to these results. Table 1: Health Indicators Indicators MDHS 2010 MDHS 2015 Total fertility rate 5.7 112 Early childbearing 26 29 Contraceptive prevalence rate 46 59 Modern method use 42 58 Unmet need for family planning 26 19 Infant mortality: Deaths/1,000 live births 66 42 Under-5 mortality: Deaths/1,000 live births 112 64 Proportion of live births in health facility 73% 92% SSDI Communications Activity Performance Evaluation 3 III. EVALUATION METHODS AND LIMITATIONS The evaluation was conducted between May 24, 2016 and Sept. 9, 2016 and covered the period from SSDI project inception in September 2011 through the end of July 2016. The evaluation team included three consultants: Carol Shepherd, team leader; Iain McLellan, SBCC specialist, and Willie Kachaka, research analyst. In addition to the core team, six research assistants were hired to assist with data collection and transcription. The evaluation methodology and data collection tools (Annex II) were developed as part of the inception report. The research assistants were trained and the tools translated into local languages during the first week of the in-country portion of the evaluation. Pretesting occurred in Salima City prior to start of data collection. STUDY DESIGN This performance evaluation assesses changes in behaviors and practices that are related to SSDI￾Communication activities. Specifically, it provides a conceptual and contextual understanding of campaign messages and how they are understood by recipients, and it assesses the effectiveness of activities such as the Moyo ndi Mpamba platform, use of multimedia channels, effectiveness of different materials and interpersonal communication (IPC) and the effect of community mobilization at the Group Village Head (GVH) level. It also recognizes that multiple activities have been implemented by similar programs; therefore, evaluating the influence of SSDI-Communication in the project districts would assess strength and intensity of implementation rather than complete attribution. The evaluators, therefore, will not infer causality. The evaluation team visited five SSDI districts (Annex IV), one in each zone, to evaluate the behavior change hypothesis: “Greater improvements to behavior change and practices indicators over a five-year period occurred in the SSDI-Communication service areas.” The team identified general trends with which to triangulate and validate existing data. The evaluation team used a mixed methodology comprising both qualitative and quantitative data collection. The team conducted a review of the project literature; key informant interviews (KIIs) with stakeholders at the national, zonal, district and health facility (Annex III) levels; focus group discussions (FGDs) with community health structures and men and women in urban and peri-urban areas where the community mobilization interventions took place; and a media content study with beneficiaries to evaluate the impact of the communication campaigns. A short survey using a Likert scale was also administered to KII and FGD participants. These approaches were complemented by direct household observations. SSDI Communications Activity Performance Evaluation 4 Table 2: Evaluation Methodologies by Question Question Media Content FGDs KIIs HH Obs Likert Survey Secondary Data 1: Did SBCC packages effectively promote increased awareness and practice of preventive behaviors in target communities? 304 30 FGDs (10 with 80 women; 10 with 75 men; 10 with 81 CAG members 97 55 167 beneficiaries Desk review SSDI￾Communication endline survey data 2: Did SSDI￾Communication improve SBCC capacity of the HES and local institutions to develop, coordinate, implement and provide oversight? 0 0 97 0 188 national, district and community SSDI￾Communication partners Desk review Policies and strategies 3: Did SSDI￾Communication fellowships improve strategic planning and implementation of SBCC and media reporting of health issues? 0 0 9 0 9 fellows and interns Desk review 4: What effect did joint implementation of community mobilization by SSDI-Services and SSDI-Communication have on the effectiveness of this intervention? 304 30 FGDs (10 with 80 women; 10 with 75 men; 10 with 81 CAG members 97 55 264 Desk review SSDI￾Communication and SSDI-Services endline survey 5: What are the most significant accomplishments, best practices and lessons learned from the SSDI￾Communication activity? 305 30 FGDs (10 with 80 women; 10 with 75 men; 10 with 81 CAG members 97 55 264 Desk review Policies and strategies Total 305 30 97 55 264 See Annex IV for maps of evaluation sites. SAMPLING METHODOLOGIES The evaluation was conducted in five purposively selected sites, one in each health zone—Salima (Central East), Karonga (North), Lilongwe (Central West), Machina, (South East), Chikwawa (South West). In each district, the team selected a sampling frame of census enumeration areas or villages within 15 km buffers around two randomly selected SSDI-supported health facilities—one peri-urban and one rural. SSDI Communications Activity Performance Evaluation 5 LIMITATIONS OF EVALUATION METHODS This sampling strategy is designed to capture only areas targeted by SSDI-Communication interventions, in line with the evaluation design. Therefore, all evaluation results presented in this report are statistically representative only of districts benefitting from SSDI interventions and cannot generate findings that represent the larger population of Malawi. As KIIs and FGDs represent a significant portion of the data collected, potential biases related to personal opinions and recollection cannot be ruled out. The evaluation team attempted to mitigate these risks by using multiple interviewers and facilitators and by triangulating results across multiple data collection methodologies. DATA QUALITY Data quality was overall quite good. However, the indicator database maintained by SSDI-Services is somewhat problematic in some areas. Behavior change indicators collected by Community Action Groups (CAGs) at the household level are not complete and have not been as thoroughly vetted as they might have been. The presentation of these indicators in this report is to illustrate the magnitude of potential changes in behavior, as documented by the CAGs. USAID will embark on a data validation exercise of all community-level data and present the comprehensive and validated findings through a separate exercise. DATA ANALYSIS Quantitative data The media content survey and the Likert surveys from key informants were cleaned and analyzed using STATA 14.0 and Excel software, using indicator-specific computations. Frequencies, means and chi￾squared tests for overall significance were conducted for all key indicators. The descriptive data were disaggregated by sex, life stage and location (rural/peri-urban) of the respondents to identify disparities in practices and behaviors. Secondary data sources from the community mobilization indicators database were also analyzed using the same approach, but using comparative analysis between baseline and endline data to assess changes in behavior as a result of the communication campaign. Qualitative data KIIs were manually transcribed, and FGDs were digitally recorded and transcribed from local languages into English. The research analyst then coded each of the FGD transcripts. A preliminary thematic analysis was conducted on the transcribed notes, based on the primary research questions and a review of a subset of transcripts. This approach allowed for a certain level of inductive analysis, which is crucial in working with qualitative data, while simultaneously maintaining structures that would allow the analysis to be tailored to specific evaluation questions. The transcripts also included direct quotes by topic that can be used for illustrating trends. Thematic analysis involved organizing data into categories by identifying recurring themes in the data and creating labels under different categories. SSDI Communications Activity Performance Evaluation 6 IV. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS QUESTION 1: TO WHAT EXTENT DID THE SBCC PACKAGES, INCLUDING THE MOYO NDI MPAMBA PLATFORM, EFFECTIVELY PROMOTE INCREASED AWARENESS AND PRACTICE OF PREVENTIVE BEHAVIORS IN TARGET COMMUNITIES? Findings SBCC central creative concept Strong evidence-based strategic SBCC instilled: Despite a delay in completing the baseline study, SSDI-Communication has been successful in changing the mindset at all levels to make evidence-based strategic planning the basis for developing all health communication. Through close collaboration and capacity building with government, non-government and private-sector partners, the focus is now decidedly on: (1) understanding target populations and persuading them to adopt positive behaviors; and (2) extensive pretesting of strategies and materials. The SSDI-Communication SBCC strategy was developed collaboratively with partners, is based on a solid theoretical approach and has well-defined target audiences based on a Lifestyles Strategic Approach. Branding was enormous success: The Moyo ndi Mpamba (Life is precious) platform is widely recognized: in the endline survey, 83.6 percent of roughly equal numbers of men and women had heard of it, and 95 percent of beneficiaries recognized it in the media content study. Its concept of taking responsibility for one’s own health was understood by 82.1 percent in the same study. In the Likert survey, 99.3 percent agreed with the statement that they “are aware of the Moyo ndi Mpamba campaign and found the advice to be useful.” In a Machinga FGD, when asked what Moyo ndi Mpamba means, a man said, “It means we must take care of our lives,” and, “As individuals it is our responsibility to look after our lives.” A woman in a Salima FGD said, “To do well in life you need to have a healthy life.” Multimedia SBCC package Multimedia SBCC packages inspired behavior changes: Mass media, mid-media and IPC of the six EHP messages informed and inspired beneficiaries to adopt preventive and health-seeking behaviors. Radio programs and spot ads were considered the “best channels” according to the media content study done in the traditional authorities with community mobilization. Outreach conducted by CAGs and community health volunteers ranked a close second. Though only 7.5 percent were exposed to household visits in the traditional authorities, beneficiaries listed outreach by CAGs and community health volunteers as their second most preferred channel for Moyo ndi Mpamba communication after radio (2016 SSDI-Communication Endline Survey). In the Likert study, 95 percent of beneficiaries agreed that communities are more aware of “how to improve their health” today than they were five years ago. Evidence of positive behavior changes: In the media content study, 94 percent of respondents were familiar with the six EHP issue messages, and 92.6 percent said they took action on them. Evidence of substantial behavior change included latrine construction and handwashing, malaria diagnosis and treatment, use of long-lasting insecticide-treated bed nets, HIV testing and more women delivering in health facilities, according to qualitative and quantitative research with beneficiaries. In the Likert survey, water and sanitation (36.2 percent), family planning (25.4 percent), long-lasting insecticide-treated bed nets (15.2 percent) and HIV testing (15.2 percent) were areas with the “biggest improvements.” Impact on male-female relationships: When asked about the impact of Moyo ndi Mpamba on male￾female relationships in the media content study, men and women both cited more discussions by SSDI Communications Activity Performance Evaluation 7 couples about family planning as the biggest change; men attending antenatal and maternal, neonatal and child health (MNCH) clinics with their partners and increased awareness of the harm of gender-based violence were second and third, respectively. Also in that study, 96.9 percent of beneficiaries said women’s knowledge had increased and health practices had improved as a result of Moyo ndi Mpamba, and only 6.9 percent of respondents said they were not aware of any impact on male-female relations. A woman in a FGD in Salina commented, “Now we are discussing with our husbands on issues affecting our lives and we make decisions as family.” Radio principal tool Network of radio station partners created: Mass media tools employed by SSDI-Communication included an interactive, magazine-style radio program focused on real-life stories from ordinary people recorded in communities, a drama and spot ads. In the endline survey, 59.6 percent of respondents had ever heard of Moyo ndi Mpamba radio. A network of 18 national, regional and district radio stations broadcast the programs and spots. This included Zodiac and the two MBC stations with the highest listenership nationally, seven other national stations and eight regional and district community stations. Some radio stations were given technical assistance, recording equipment and laptops for editing audio in exchange for broadcasting project programming. Twenty-two spot ads were produced, and 11,500 were aired. In terms of programs, 104 episodes of “Cheni Cheni Nchiti” were produced. Though there were plans to produced twice as many episodes of the Moyo ndi Mpamba radio drama series/magazine, production was cut short and only 89 were produced. Radio primary source of information on Moyo ndi Mpamba: Radio spot ads and programs were the primary source for information on Moyo ndi Mpamba, according to the media study. In the endline survey, 45.15 percent said they had ever listened to its programs. This was reflected also in the FGDs with beneficiaries. Radio was mentioned as the first source of information on Moyo ndi Mpamba in 16 of 20 sessions. In the Likert study, 71.7 percent of beneficiaries “agreed very much” or “agreed” with the statement: “In the radio drama and magazine you find stories about people and topics that are true to life.” The end-of-project indicator target of 20 radio spots produced was surpassed by two. Total radio spots aired were 12,150, a third of them on malaria; 1,500 each on nutrition in pregnancy and complementary feeding; a generic one on Moyo ndi Mpamba; and 1,050 each on water, sanitation and hygiene, MNCH and family planning. Radio cost $4,643,143, including radio program production and airing ($3,476,177) and radio spots production and airing ($1,635,816). Important gaps in access to radio remain: In the media study, 31.1 percent of respondents said they had never heard Moyo ndi Mpamba on radio, and 25 percent in FGDs said they rarely or never listened to radio. “We don’t really sit down and listen, we just hear bits here and there when we are passing by the radio because we are always busy,” a woman said in a Lilongwe FGD. A woman in Chikwawa commented, “Most of us don’t have radios or don’t have time to listen.” Twenty-four percent more men than women had ever listened to Moyo ndi Mpamba radio programs. Print materials’ impact Print materials’ varying degrees of success: Insufficient numbers printed, erratic distribution and relatively high levels of illiteracy handicapped use. In the observation survey, half of the 55 households visited had Moyo ndi Mpamba materials. The most common print materials included the Family Health Booklet and leaflets or pamphlets on malaria, oral rehydration solution/zinc, and long-acting contraceptives, according to the media study. Material development and printing cost $3,665,022. Posters attractive and widely seen: According to the 2016 endline survey, 61.65 percent had seen the Moyo ndi Mpamba posters. The posters included color photos of ordinary people, often smiling, and often featured both men and women together. There were reports of posters being so attractive they were stolen to decorate homes, especially the ones featuring celebrities such as popular musicians. SSDI Communications Activity Performance Evaluation 8 Family Health Booklets appreciated: These colorful booklets covering the EHP messages were seen by 22.95 percent in the endline survey. Half of FGD beneficiaries had seen them and a quarter said they had one in their home. There were reports of children reading the contents to their parents and community mobilizers using them in outreach in the absence of flip charts or other materials specifically designed to support IPC. In the media content study, one in six had seen the booklet. Support materials for outreach work lacking: House-to-house visits by CAG members and health surveillance assistants were considered the second biggest source of information, after radio, according to the FGDs. These two groups were the preferred human sources of information by 56.1 percent of respondents in the media content study, followed by facility-based health providers (25.4 percent) and village heads (6.6 percent). All those conducting outreach were critical of the shortage of support materials that could be used in outreach. The Moyo ndi Mpamba flip chart was appreciated by those who had seen it, but few had been given a copy and only 6.4 percent of beneficiaries had seen it, according to the media study. The end-of-project indicator target of 6,000 community health volunteer flip charts was surpassed: 8,340 were produced. All but 420 of them were produced in the fifth year, so it was not available when the bulk of the community mobilization was done. Billboards’ low visibility: Large billboards were installed on heavily travelled routes and major intersections. Eighty-eight percent of the billboards featured messages on malaria and family planning. Forty percent had seen the billboards in the endline survey, but only 29.3 percent had in the media content study. Another mid-media, posters, had much higher visibility. The 44 billboards cost the project $1,374,059, or $31,228 per billboard. Events Events popular but low visibility: Events organized by SSDI-Communications included: Open Days, which featured district health officials giving health talks, drama groups, and provision of services such as voluntary counseling and testing; Family Road Shows with musicians, DJs and free t-shirts held at market towns in districts; and regional music festivals. They were enjoyed by those who had seen them, but only a small percentage of beneficiaries had seen any of the events. Open Days were mentioned in only three of the 20 beneficiary FGDs, and two mentioned dramas. In the media content study, only 2.6 percent of respondents had seen a road show, and 7.5 percent in the endline survey said they had been exposed to one or more Moyo ndi Mpamba activities. Table 3: Major Communication Initiatives Major Initiatives Funding Training/Capacity building $2,753,425 Fellowships/Internships (22 interns and fellows: 4 health communication fellows, 6 University of Malawi interns, 12 journalist fellows) $397,195 Media training $231,576 Material development and printing $3,665,022 Production and airing of radio spots $1,166,966 Airing radio programs $1,635,817 Billboards $1,374,060 Radio program production $1,840,361 Dramas $924,733 Community mobilization $3,389,467 TOTAL $17,378,622 SSDI Communications Activity Performance Evaluation 9 Conclusions  The Moyo ndi Mpamba platform has high value because it has come to denote good quality and attractive communication products. The health topics were integrated successfully into the campaign. All activities were branded as Moyo ndi Mpamba, which brought coherence to the media products and consistency to the recommended behaviors, as well as the notion of engaging beneficiaries in improving their own health.  The vast majority of people interviewed in the media content study and FGDs and who filled out the Likert surveys were familiar with the campaign and stated that their knowledge and behavior had changed as a result of hearing messages and participating in project activities. Other data and observations supported these findings. The training and learning-by-doing activities within the public and private sectors resulted in individuals and organizations developing knowledge and commitment to sustain SBCC interventions. It also resulted increased reporting of accurate information on health issues.  The design of the mass media and community campaigns with their strong evidence-based strategies has proven to be successful in promoting preventive and health-seeking behavior related to the six EHP issues.  SSDI-Communication’s SBCC efforts demonstrate that integrated health communication provides good value by grouping intervention messages and reducing the parallel and duplicated messaging of vertical programs.  Limiting communication to the six EHP focal areas without including each of the 15 interventions in the initial campaigns was prudent and contributed to the success, but there is room to expand to new focal areas, new support materials and training, as well as coordination with SSDI-Services.  SSDI-Communications has established a reliable network of collaborators with 18 radio stations, including national stations with high listenership and popular community radio stations. Use of radio allowed for SBCC messages to reach large numbers and set the stage for IPC and community mobilization.  Production capacity, particularly that of community radio stations, was increased, which resulted in higher production values and more interactive programming and community involvement.  Radio listenership is limited among one quarter to one third of women due to lack of access to radios.  Posters, Family Health Booklets and flip charts have the greatest value among print materials in terms of reach and utility. Billboards have the least value in terms of visibility and cost.  Initial plans to broadcast live or record and rebroadcast events were not implemented, but they have great potential for expanding the reach of Open Days and road shows.  The evaluation found a difference in radio listenership in project areas with community radio stations and those without, because communities appreciate community radio, which is more likely to be broadcast in their languages and reflect local issues and perspectives. The new project should explore the potential to support development of new community stations and build the capacity of all community stations. Recommendations  Continue use of the Moyo ndi Mpamba brand into the foreseeable future. The MoH has already adopted it for use in new materials. The platform lends itself to catching attention and adding credibility to other EHP focal areas and emerging health issues. SSDI Communications Activity Performance Evaluation 10  Continue investments in radio, particularly spot ads, interactive magazines and the network of community radio stations, through capacity building and additional production equipment to enable more direct interaction with communities.  Study the possibility of improving radio listenership among women by distributing low-cost, wind-up radios to selected beneficiaries through CAGs, as well as encouraging use of the radio chip found in mobile phones but often not turned on.  Increase production of Moyo ndi Mpamba flip charts to support interactive IPC by CAG members, HSAs and facility-based health care providers, along with training in their use.  Continue investments in print materials, including posters and materials like the Family Health Booklet, and reduce investments in billboards, music festivals and road shows.  Amplify impact of events such as Open Days and road shows by broadcasting them live or recording and playing them on the radio. QUESTION 2: TO WHAT EXTENT DID SSDI-COMMUNICATION IMPROVE THE CAPACITY OF THE HES AND LOCAL INSTITUTIONS TO DEVELOP, COORDINATE, IMPLEMENT AND PROVIDE OVERSIGHT TO HEALTH SECTOR INTEGRATED SBCC? Findings Impact on SBCC planning Evidence-based strategic planning entrenched among partners: Through capacity building and training in SBCC strategic planning, mentoring and learning by doing, SSDI-Communications has been successful in developing an environment where evidence-based strategic planning has become the norm. Evidence-based strategies were “used every time,” and partners worked closely with the SSDI￾Communications research unit. In the Likert survey, 60 percent of MoH respondents agreed very much with the statement that SSDI-Communications capacity building had “enabled me to increase skills for planning and implementing SBCC”; 40 percent neither agreed nor disagreed. Leadership in Strategic Health Communication workshop appreciated: A wide range of project partners, from HES national and district staff to private sector partners, participated in the two￾week workshop. All liked the balance between the theoretical and the practical. In KIIs with a dozen workshop participants, many said they are using the skills they developed, such as evidence-based planning and understanding target audiences. “We have applied what we learnt,” an HES health educator said. “Some things we still apply to this day like making materials visually appealing and action oriented.” In the Likert survey, 80 percent of MoH respondents agreed with the statements regarding increasing their ability to “promote EHP messages” and “communicate messages and inspire health-seeking behavior.” The end-of-project indicator target of 146 “Malawians trained through state-of-the-art SBCC training programs” was not met: Only 87 were trained. Capacity improvements Technical assistance for policy development useful: The HES was able to finalize the National Health Promotion Policy and develop a health communication strategy, as well as a specific SBCC strategy for malaria, with the assistance of a seconded expert from SSDI-Communications. The end-of￾project indicator of “three program SBCC strategies refined and integrated” was met. The HES deputy director said in a KII that he was satisfied with the help from the consultant and with printing and launches. “We have already started to use the strategy at the national level,” he said. The end-of-project indicator of five SBCC advocacy sessions held with policymakers was surpassed by three. SSDI￾Communications further assisted with the adoption of the health promotion policy through high-level advocacy. However, implementation has not been formalized for any of the three documents. SSDI Communications Activity Performance Evaluation 11 SSDI-Communication was responsive to HES demands for material assistance: The HES requested that the project assist it in establishing an electronic archive of materials and creating a HES web site with links to the materials. It also hoped to upgrade its audio production studio. This was done with varying degrees of success because of inadequacies with the server and the MoH’s difficulty in hiring staff to operate the studio. “SSDI-Communications had an ear to listen and the intelligence to see things worth spending money on. Its legacy is left with us,” the acting HES director said in a KII. Capacity building among private sector partners brought dividends: Project partners Mercantile, Galaxy and ADECOTS reported that SSDI-Communication’s capacity-building interventions improved staff skills. “Our staff capacity is quite a bit different now as the project demanded the best from us,” commented a Mercantile staffer in a KII. “Everything is driven strategically now. We are now prepared with capacity, skills and knowledge to provide resources in health SBCC for the country into the future.” In the Likert survey, private media, including the three partners, was positive about the legacy of the project; 66.7 percent agreed or agreed strongly with the statement: “I feel confident the legacy of SSDI-Communications is entrenched and will continue to reap dividends long into the future.” HES engagement challenges HES learning by doing and project mentoring was more active at the start: The HES was actively involved in the development of the SSDI-Communications SBCC strategy and the creation of the Moyo ndi Mpamba platform and attended working sessions with project partner Mercantile to develop and pretest the initial materials. However, according to the HES staff interviewed, this “working together” diminished over the last two years. The two HES health educators dedicated to work with SSDI-Communications estimate that only 15 percent of their work time was spent collaborating with the project. “It is easier to give feedback if involved in the process from the beginning rather than just being asked to comment without knowing the background,” one HES health educator said in a KII. MoH management challenges handicapped HES involvement: High levels of HES staff turnover, unfilled positions, limited and overstretched staff, disappointments with allowances, and poor collaboration between the vertical programs and the HES all contributed to HES difficulties in fulfilling its leadership role in SBCC. The HES did take the lead in inviting stakeholders to meetings for strategy and materials review, though there were more of them in the first years of the project than more recently, requiring fewer inputs from the HES. No specific definition of HES relationship with SSDI-Communication: Though there were yearly planning meetings with the HES when SSDI-Communications prepared its work plan, there were no formal agreements on tasks to be performed. The original project design placed a great deal of importance on developing HES capabilities in leadership, coordination and advocacy for SBCC. But the learning-by-doing and mentoring approach has met with less success than envisioned. Some at the HES are even critical of SSDI-Communications for depending on private sector partners to do its communication work with limited inputs from the HES. A more concrete strategy is needed to further improve HES capacity to lead and coordinate health SBCC at all levels. HES leadership, coordination and advocacy skills needed: The capacity of the HES to manage material development and production and oversee, monitor and evaluate SBCC is important for future sustainability of SBCC initiatives. Setting critical functions for the HES, with specific capacity-building outcomes and indicators for the skills developed, is essential. When asked in the Likert survey to name challenges, MoH respondents cited lack of involvement and coordination between SSDI￾Communications and ministry staff. The high level of staff turnover at the MoH and transportation were also raised. User-friendly community mobilization training District-level capacity building in community mobilization successful: The CM methodology SSDI Communications Activity Performance Evaluation 12 was found to be user-friendly and appropriate by the trainer of trainers, subgrantee staff, district-level ministry officials, CAG members and others at the community level who were trained. The training tools, including materials such as picture cards and CAC manuals, were well understood and used. The CAC involves communities in identifying health challenges, their root causes and solutions (Annex V). The picture cards illustrate the challenges and solutions. “It was a good practical workshop and it enabled us to mobilize communities,” one district health staffer said in a KII. One trainer found the CAC manual to be of good quality and “kept it handy” for reference when training others. “We learnt how to go through stages like exploring health issues with the community and setting priorities,” a Kayuni CAG member said in a FGD. The only complaint about the training was that some CAG trainees found it too rushed, considering the detailed contents. Training duration varied from two to five days. Five-day training was often split into two or three sessions to allow for practical experience in between classes. Community mobilization management problems affected training: Budget cuts for CM resulted in no refresher training and no training for new staff due to high turnover. Other training issues involved complaints about the density of the training by some CAG members, extension workers avoiding training over allowances, lack of transport, lack of budgets for CM at district-level training, and insufficient CM training or orientation for health center staff. District-level health education officers complained that they were trained in CM but not given opportunities to apply their skills. “I was trained in community mobilization but was only involved in one CAG training,” reported an interviewed district health promotion officer in Salima. However, subcontractors and subgrantees frequently found district health promotion officers to be busy and unavailable when they were requested to support training. They were also tasked with providing supervision, which was often delayed due to lack of transport. Conclusions  SSDI-Communications will have an impact on the quality of evidence-based strategic planning of SBCC well into the future.  SSDI-Communications training resulted in widespread use of evidence-based strategic planning.  The lack of specific terms of reference for collaboration between SSDI-Communications and the HES has made collaboration more ad hoc, unofficial and more intense at the project’s beginning.  Support for SBCC policy documents had impact, but more senior-level advocacy is needed to help resolve MoH structural problems such as staffing, ensure that vertical programs benefit from HEC expertise, and provide resources for SBCC development.  The HES is now better equipped and trained to carry out its communication mandate within the MoH to develop, implement and supervise SBCC interventions. However, ongoing funding constraints within the Government of Malawi will likely handicap the HES from leveraging these investments to fully complete its mission.  The CAC training model and materials were user friendly, and the cascade training method worked well. The training brought CAGs up to speed on the EHP. All this was successful, despite a dearth of follow-up training and supervision due to management inadequacies.  Training formed the basis for empowering communities, which allowed them take control of their environment and lives.  Not training/orienting all health center staff in community mobilization was a missed opportunity.  Community mobilization training was not incorporated into the national planning and budgeting cycle. However, the HSA curriculum has already incorporated community mobilization and the CAC. SSDI Communications Activity Performance Evaluation 13 Recommendations  Continue with Leadership in Strategic Health Communication workshops to reinforce evidence￾based SBCC strategic planning skills.  Support the vulgarization of the SBCC policy documents to ensure they are understood and applied.  SSDI-Communications should continue high-level ministry advocacy to solve structural challenges regarding staffing, allowances and insufficient collaboration between vertical programs and the HES.  The well-orchestrated use of mass media (radio in particular), mid-media and IPC through community mobilization has informed and inspired beneficiaries to adopt positive changes in behavior. The integration of vertical programs into one cohesive intervention also greatly strengthened the response. These strategies merit continued USAID/Malawi support.  USAID/Malawi should continue and expand support for public and private media, including community radio, especially in the area of encouraging dialog with communities on their development of health-seeking behaviors.  Future investments should maximize the opportunity for sustainability of SBCC by ensuring HES direct involvement in the development and supervision of SBCC interventions, including supporting establishment of the unit’s research, monitoring and evaluation function to sustainably develop evidence-based message development.  Create tripartite memoranda of understanding between the MoH/HES, SSDI-Communications and its private sector partners that clearly delineate a supervision and oversight role for the HES.  Conduct community mobilization training or orientation for all health center staff, and standardize training allowances such as “full board” from the start to eliminate misunderstandings.  Integrate community mobilization and broader SBCC activities into district and traditional authority-level planning and budgeting. This will help enhance ownership and sustainability.  Investigate the possibility of including the CAC in the community health nursing and environmental health curricula. QUESTION 3: TO WHAT EXTENT DID SSDI-COMMUNICATION’S FELLOWSHIP PROGRAM IMPROVE STRATEGIC PLANNING AND IMPLEMENTATION OF SBCC, AND MEDIA REPORTING OF HEALTH ISSUES? Findings Fellowships and internships Fellowships and internships successful but scope limited: Fellows and interns were grateful for the enhancement of their professional skills in SBCC. A total of 22 interns and fellows were engaged. Fellowships and internships improved the SBCC skills of participants, but due to the limited number involved, their impact on improving strategic planning and implementation of SBCC was limited. Fellows well trained and integrated into SSDI-Communications: Despite being limited in scope, high-quality fellows were identified and well mentored by SSDI-Communication staff. Most now work in SBCC. All four of the fellows in the Likert survey agreed with the statement, “SSDI-Communications capacity building enabled me to increase my skills for planning and implementing SBCC,” with three agreeing very much. Comments from interviewed fellows included: “Exposed to many things.” “Acquired leadership skills.” “Worked with team leaders who were always willing to help with on the job training.” SSDI Communications Activity Performance Evaluation 14 Student strikes limited duration of University of Malawi internships: Six undergraduates successfully completed an internship program, but the time they actually spent working was cut short due to student strikes. Two were placed in SSDI-Communications, one in Story Workshop and one in Galaxy Media. There were 200 applications for the positions in the first year. The private sector media partner Mercantile has indicated that it would be available to accommodate interns in the future. University of Malawi Chancellor College pleased with collaboration: Both students and staff increased their SBCC knowledge through lectures, participation in Leadership in Strategic Communication workshops and preliminary preparations to develop a SBCC curriculum. “The SBCC training materials are very good,” the collaboration point person at Chancellor College said in a KII. “If they had gone outside for all of the training it would have cost a lot.” Audiovisual recordings were made of the lectures. An upgrade of audio production equipment was also included. In the end, only five of the 16 planned guest lectures were conducted; these were attended by 515 students. Media coverage of health Diverse journalist support: Support to journalists to increase the amount and the quality of reporting on health took a variety of forms. The project supported 12 journalist fellows. Capacity assessments of media outlets identified their training needs. Based on the assessments, the project designed training for print and electronic journalists on health reporting. Media forums looked at ways of improving health reporting beyond training individual journalists. Memoranda of understanding were signed between media houses and the MoH. The end-of-project indicator target of 70 journalists trained in effective health reporting was surpassed: 97 were trained. The target of 12 media cafés conducted was not met; only 9 forums were conducted for 248 journalists. In the Likert survey, “increased participation of media in health campaigns” was the second most cited achievement, mentioned by 28.6 percent of MoH respondents. Support to media reaped benefits in terms of increased coverage: Anecdotal evidence suggests that media reporting on health issues increased after capacity building. “Prior to the training there was almost zero coverage on health and it tended to be very confrontational,” observed the director of training partner Galaxy in a KII. The emerging community radio stations were particularly grateful for the training and the gift of professional equipment for recording and editing. “The media houses have really helped to bring this debate that people are responsible for their own health to the forefront. People are responsible for their own health,” said the director of Galaxy. Support to journalists was reduced at the request of USAID. The issue was media freedom of expression vs. monitoring of reporting to ensure accuracy. Conclusions  Internships and fellowships have potential to develop tomorrow’s SBCC expertise.  Chancellor College is an enthusiastic partner for enhancing SBCC expertise.  Coverage of health in the mass media increased from almost none to regular coverage.  Content quality of the health coverage increased and misinformation decreased, especially when the same trained reporters covered health continually.  Community radio stations upgraded equipment, improved reporting skills and became enthusiastic partners for creating and rebroadcasting health programing.  Costs to increase SBCC expertise and media coverage are relatively low. Recommendations  Study the outcome of the SSDI-Communications journalist training to find out exactly how much media coverage of health increased, how many journalists cover health exclusively and SSDI Communications Activity Performance Evaluation 15 how much misinformation persists.  Considering the turnover of journalists, there is a need to continue training on covering health.  Emerging health issues beyond the six EHP areas will need to be introduced to journalists to encourage accurate coverage of them.  Continue support to community radio stations to further increase their skills in accurately covering health and to allow them to give voice to communities through field visits and other interactive elements.  The fellowship and internship program should be reviewed to look into ways to increase the number of participants, including the involvement of additional university partners and placing interns and fellows with additional project partners. QUESTION 4: WHAT EFFECT DID THE JOINT IMPLEMENTATION OF COMMUNITY MOBILIZATION BY SERVICES AND COMMUNICATION HAVE ON THE EFFECTIVENESS OF THIS INTERVENTION? USAID awarded funding to SSDI-Communications for CM and SSDI-Services for community-based services. Both were provided by volunteers and implemented via subgrantees. In year two, USAID moved management, supervision, funding and implementation of subgrantees from both projects to SSDI-Services, leaving SSDI-Communications to provide technical support and training to CM activities. The move, along with the subsequent budget cut, deprived the activity of the core technical expertise of JHU-CCP and its partner Save the Children. It also prevented JHU-CCP from fully measuring the impact of its SBCC activities and made attribution of CM efforts difficult. The change resulted in a cumbersome management structure and delayed implementation of CM activities for two years. In spite of these challenges, the activity was very successful, though on a much more limited scale than originally planned. Findings Community mobilization model Community mobilization model easy to use and well-liked: The movement of CM to SSDI￾Services necessitated a stop and restart of implementation that resulted in an almost two-year delay from the original start. The projects jointly chose and trained new subgrantees and created 557 new CAGs at the GVH level.2 It was not until 2014 that most CAGs began implementing the CAC. Once started, the process moved forward on a steady basis. Fifty-five percent of CAGs (306) reached the “evaluate together phase” of the first CAC by September 2015 and started work on the second round.3 The rest were awaiting funds to begin the same phase. According to the Likert survey, 86 percent of respondents like the CAC model and feel it is easy to use and effective. To date, SSDI-Communication has conducted 157 technical support visits with very limited staff. In spite of its success, CM was implemented in less than 25 percent of project-covered GVHs due to delays in start-up and funding. CAGs are sustainable: The participatory and inclusive CAC approach has promoted ownership and sustainability. More than half of KII and Likert survey respondents believe that the CAGs are sustainable and will continue functioning outside of the project. They are now considered an integral part of the system for ensuring good health in participant communities. The collaboration between the community and the CAGs inherent in this structure has led to better project results and community health. The 2 Anglican Diocese of Southern Malawi was unable to fulfill its contract and dropped out of the project, leaving Nsanje and Phalombe without support. 3 SSDI–Services FY 15 Annual Report. p. 102. SSDI Communications Activity Performance Evaluation 16 structure has also led to an organic scaling up in nearby communities. Chiefs developed GVH bylaws: With support from the projects, municipal leaders developed GVH￾level bylaws to encourage communities to adopt heathy behaviors in the six EHP areas. The bylaws were accompanied by fines to ensure adherence. Examples of rules and potential fines are shown in Table 4. The fines vary by village and are sometimes returned if the family complies with the bylaw at a later date. The disposition of the fines also varies: The fine can stay with the village chief or the GVH, go to the local health center to be used for equipment or other clinic needs, or to a fund held at the GVH to be used for improvements in the villages, such as under-5 shelters. Some chiefs do offer notes excusing behaviors due to extenuating circumstances, although it is not clear how often this is done. Table 4: GVH Bylaws and Fines Bylaw Fine Families must dig pit latrine and build handwashing facilities 5,000 MWK; 1 or 2 goats; not allowed to attend funerals; not allowed to be buried in community cemetery Women must deliver at a health facility 5,000 MWK; 1 or 2 goats; must pay for infant immunizations and post-delivery care for herself and infant at clinic Families must use mosquito nets at night 2,500 MWK Health center policy–Women who attend antenatal care with their husband or partner jump to the head of the queue Have to wait in line. Excuse note sometimes available. Community mobilization made communities self-reliant: In SSDI-Communication’s 2013 formative research, when asked who was responsible for their health, respondents answered government and NGOs. In FGDs and KIIs conducted as part of this evaluation, respondents now believe that they are responsible for their own health and are taking action to remain healthy. The Likert survey found that 95 percent of beneficiaries agreed or agreed strongly that the community is more aware of how to overcome health problems today, while 98 percent of NGOs and community mobilizers agree or agree strongly that families now advocate for their own health and the timely use of EHP services. Household data show positive behavior changes: All evaluation data suggest that behavior change indicators at the household level improved dramatically. This is consistent with the data collected from the subgrantees from households in the GVHs with CM intervention in the five districts, shown in Table 5. The data were collected directly from the subgrantees, so their quality has not been verified. The percent of those with healthy behaviors at the household level changed from the baseline in April 2014 to March 2016 in intervention GVHs. For example, the percentage of households with pit latrines increased, ranging from 46 to 110 percent (some fell and were rebuilt during the month) during the two-year period, a percentage increase of 35 percent to 75 percent. The percentage of women delivering in a health facility increased, ranging from 21 to 89 percent, and those getting tested for HIV increased, ranging from 69 to 80 percent. One female FGD participant in Mkwepere T/A Nyambi Machinga stated, “It becomes much easier when we go with our husbands more especially when there are people in line we just go straight in front–we are role models. This usually happens when we go to the under-5–we go in first.” A health worker in Iponga health center, Karonga, reported, “We have noticed an increased number of women these days coming for ANC [antenatal care] and coming to deliver at the facility such that our labor ward is always full. The good thing is that even those that deliver at home due to delay in coming here for observations, would always come to the hospital soon after delivery for check-up and postnatal care.” SSDI Communications Activity Performance Evaluation 17 Table 5: Health Behavior Improvements at the Household Level in Five Evaluation Districts **** Data/Indicators Karonga Machinga Lilongwe Salima** Chikwawa*** % increase of households with pit latrines from April 2013 to March 2016 35% ↑ 51% ↑ 73% ↑ 75% ↑ N/A Households with pit latrines 65% 80% 46% 110% N/A % increase of households with handwashing facility at pit latrine from April 2013 to March 2016 80% ↑ 72% ↑ 47% ↑ 90% ↑ N/A Households with handwashing at pit latrine 33% 57% 25% 110% N/A % increase of households with new access to safe drinking water from April 2013 to March 2016 50% ↑ 55% ↑ 76% ↑ 63% ↑ 31% ↑ Households with access to safe drinking water 58% 74% 35% 85% N/A % increase in households with members who use long-lasting insecticide-treated bed nets from April 2013 to March 2016 49% ↑ 99% ↑ 91% ↑ 64% ↑ 29% ↑ Households with members who use long-lasting insecticide-treated bed nets 53% 86% 26% 100% N/A Women who delivered at health facility 21% ↑ 89% ↑ 49% ↑ 71% ↑ 39% ↑ # tested for HIV and received results 69% ↑ 72% ↑ N/A 80% ↑ N/A ** Some toilets fell and were rebuilt. ***Data available April 2014 to March 2015 only and data quality is questionable. ****Data were collected directly from subgrantees so has not undergone verification. Management and supervision Weak management affected CM implementation: The management structure that was developed after CM moved under SSDI-Services was complex and confusing, resulting in inefficiencies. Each district had slightly different structures, depending on the subcontractor and subgrantee. The project design called for SSDI-Services to support CM at the district level, primarily through the district community coordinator position, and also to support subgrantees on a day-to-day basis, monitoring work plans and supervising the work on the ground. However, in most districts visited, this did not occur consistently, often leaving the subgrantees to manage on their own. The subgrantees did receive supervision and training on financial vouchers from SSDI-Services. Irregular finding: Funding delays and inadequate training were stated as the second and third biggest challenges during program implementation. The processes for requesting funds for district activities, reconciling funds against expenditures, communications and data collection varied from district to SSDI Communications Activity Performance Evaluation 18 district, with weaknesses at both the subcontractor and subgrantee levels. Inconsistent supervision: District-level Community Mobilization Teams were supposed to supervise and monitor the CAGs. Lack of transportation meant that they could not make supervisory visits unless they were paid allowances and provided transport from district hospitals. The district health promotion subcommittees, also tasked with providing CM supervision, were very busy and had the same transportation issues. According to subgrantees interviewed, budgets were insufficient to pay for transportation and allowances for these groups on a regular basis. SSDI-Communications provided technical support to each CM-implementing district every quarter. The roles of its zonal and national technical teams were designed to complement and not replace the SSDI￾Services district team. This planned complementarity was not implemented consistently across subgrantees. The subgrantees mostly saw SSDI-Communication staff, whom they viewed as technical support, and not SSDI-Services staff, whom they viewed as supervisory staff. Inconsistent data collection: When USAID moved CM from SSDI-Communications, the results framework for SSDI-Services was not modified to incorporate the change. This meant that neither project was formally accountable for CM, although both projects reported on it. The CAGs collected household indicators on a monthly basis and reported them to the subgrantees, who in turn reported to the subcontractor’s monitoring and evaluation (M&E) officer at either the district or zonal level. The M&E officer was expected to enter the data. However, data collection and entry were often inconsistent, and data quality validation was rarely done. The forms and processes used varied between districts. Better supervision and management would have detected this problem early in the process and corrected it. Jhpiego is currently trying to rectify this problem. Transportation Transportation was the biggest challenge: The major challenge, according to district-level key informants, was transportation to conduct supervision and outreach. In the Likert survey, 38 percent of District Health Management Team (DHMT) members, 32 percent of health care providers, 55 percent of NGOs and 35 percent of community mobilizers cited transportation as the major challenge. Traditional authorities are large and the terrain is rugged. CM facilitators had to use bicycles to deliver print materials to CAGs for further distribution. CAGs faced similar challenges, except they were often on foot, resulting in the inefficient use of scarce resources. Conclusions  The goal of SSDI-Communications was to contribute to reducing the total fertility rate, risk of HIV/AIDS and maternal, infant and under-5 mortality rates. Data collected in the evaluation districts and project indicator data show that this goal was met.  CM was extremely successful, despite major management difficulties and limits to its funding and scale. There were several reasons for this success: - Mass media, mid-media and IPC inspired and informed the CAGs and communities about the six EHP areas. - The participatory and inclusive CAC approach promoted local ownership and put people in charge of their own lives, which created sustainability. - The bylaws provided powerful incentives to communities to change behavior. - Integration of the vertical programs led to more focused interventions.  While the bylaws were very successful in changing behaviors, there is no information available on potential negative impacts, especially on poor and vulnerable women. SSDI Communications Activity Performance Evaluation 19  In baseline FGDs, when asked who was responsible for their health, individuals stated that government was responsible. Five years later, the Moya ndi Mpamba campaign and interventions have succeeded in bringing about change. Beneficiaries in FGDs and the media study stated that they are responsible for their health and will continue to pursue health-seeking behaviors, which make them heathier. This is the essence of sustainability and development.  USAID’s decision to move CM under SSDI-Services delayed its start and reduced the influence of SSDI-Communication’s SBCC and CM expertise. While SSDI-Communications provided technical support, budget cuts prevented it from providing the level of support required at start￾up.  Management of CM implementation was weak, causing confusion and inefficiencies in most districts visited by the evaluation team. The subcontractors and subgrantees were structured differently from each other, which made standardized management difficult. The multiple layers resulted in communication lapses and delays in funding that left district subgrantees with no funds for 2-4 months at a time several times per year.  Community Management Teams were unable to procure transport and allowances to conduct site visits to the CAGs, resulting in limited to no external supervision.  Due to transportation problems, many CAG members were required to walk between the GVH villages, which, due to the sheer size and often difficult terrain, limited their effectiveness. Recommendations  CM should be implemented under a structure that takes advantage of core SBCC competencies. CM management, supervision, technical support, capacity building and data collection should be resident within the parent project and extend to the zonal and district levels. This could be located either within the Communications or the Services project, but not split between both. The other project should support implementation. To ensure that this takes place, CM should be incorporated into the results framework of both projects, appropriately to each project’s activities and with no overlap between the projects. This can help monitor demand creation and needed supply.  USAID/Malawi should invest more heavily in CM to complement the SBCC because both have been very successful. This should include developing and costing a strategy to rapidly scale up CM.  The new project should standardize and simplify project management structures, clearly articulate roles and responsibilities and ensure they are shared across the prime and all subs. Subgrantees should be located under the prime to eliminate inefficient management layers; they should have M&E officers and be responsible for supervising the CAGs. Conducting annual inter￾CAG meetings and visits will help to rapidly scale up the model.  To relieve transportation limitations, the new project should provide district staff, including area supervisors and facilitators, with motorcycles or electric bicycles with solar chargers, if appropriate and not currently available. Solar chargers would eliminate fuel costs and allow staff to cover the large distances required in most districts. Provide all CAGs with push bicycles, in addition to gum boots, umbrellas, tablets and pens. In addition, all CAGs and community volunteers should be given t-shirts, wrappers, hats, etc. that identify them as official community workers.  If the Community Management Team is to be used to supervise the CAGs, its composition needs to be formalized, linkages reinforced and coordination among all participating line ministries strengthened. SSDI Communications Activity Performance Evaluation 20  The new project should support the MoH to redefine the roles and responsibilities of all existing community groups at village and GVH levels for greater efficiency and effectiveness in community participation. QUESTION 5: WHAT ARE THE MOST SIGNIFICANT ACCOMPLISHMENTS, BEST PRACTICES AND LESSONS LEARNED FROM SSDI-COMMUNICATIONS? EXPLICITLY IDENTIFY AND DOCUMENT THE FACILITATING AND INHIBITING FACTORS TO POSITIVE PERFORMANCE FOR EACH OF THE ABOVE QUESTIONS. Accomplishments  The combination of mass media and IPC between beneficiaries and CAGs using an integrated platform proved to be very effective, as evidenced by the response to the intervention and the substantial shifts in behaviors.  SSDI-Communications has embodied evidence-based planning throughout the project, including all media products, EHPs and the CAG process.  SSDI-Services and SSDI-Communication succeeded in creating a CM structure using CAGs that is both successful and sustainable.  A significant and low-cost accomplishment with a discernable impact was the development and enforcement of health-related bylaws at the GVH level.  SSDI-Communications and SSDI-Services should be commended for the positive results of communication and CM efforts. However, challenges related to funding cuts, project design, movement of the bulk of CM from SSDI-Communications to SSDI-Services, pressure from USAID to obtain results, and weak management due, in part, to project structure resulted in inefficiencies and much confusion about roles; these were exacerbated by severe funding constraints within the Government of Malawi. These challenges likely prevented SSDI￾Communications and SSDI-Services from achieving the scale of success originally envisioned. Facilitating factors  Embodiment of evidence-based planning throughout the project, including all media products, EHP and CAC process  Integrating health communication messages and services  Effective CM methodology  Good collaboration between government, donor, NGOs, private sector and mass media  Capacity building from the ground up to empower communities through CAGs and Community Management Teams  Adequate financial resources to make impact  Adoption of GVH bylaws Inhibiting factors  SBCC inhibited by limited IPC at the community level  Collaboration with the MoH limited by its funding and structural constraints  Limited access to radio by up to one quarter of women  Cumbersome and inefficient CM management structures  Delays and inefficiencies resulting from the decision to move CM implementation to SSDI￾Services SSDI Communications Activity Performance Evaluation 21  Implementation inhibited by funding cutbacks and funding delays to the subgrantee level  Investments insufficient to reach beneficiaries on the scale needed to bring large-scale change  Collaboration with government limited by its funding constraints Best practices and lessons learned The best practices identified below are the result of lessons learned throughout project implementation and have given rise to substantial behavior change across all six EHPs.  Adoption of community bylaws and enforcement recommendations resulted in significant improvements in latrine construction, handwashing, malaria diagnosis and treatment, long-lasting insecticide-treated bed net use, male involvement in antenatal care, family planning, MNCH and delivery in health facilities, making it a best practice. However, as stated above, the government should be vigilant that these practices do not have an unfair impact on women. For example, health centers have adopted the practice of allowing a pregnant woman to go to the head of the queue if her husband or partner attends antenatal clinic with her. While this has increased male participation, it could unfairly target women whose husbands cannot attend due to work. Some health centers accept a note from the chief that excuses the husband to prevent unfair treatment.  The vertical programs were integrated into one cohesive intervention presented on a single platform, Moyo ndi Mpamba. This was then used in mass media, mid-media and IPC via community mobilization, which both inspired and informed beneficiaries to change behaviors. Combining these three project components was a best practice that garnered substantial benefits.  Coordination between Communication, CM and Services is needed to maximize efficiencies and program results. SSDI Communications Activity Performance Evaluation 22 ANNEXES I. Evaluation Statement of Work II. Data Collection Tools III. Key Informant Interviews IV. GPS Maps V. Community Action Cycle VI. Disclosure of any conflicts of interest by evaluation team members SSDI Communications Activity Performance Evaluation 23 ANNEX I: EVALUATION STATEMENT OF WORK Support for the Service Delivery Integration (SSDI) Activity Evaluation (Services, Systems, & Communications) SCOPE OF WORK AND BACKGROUND INFORMATION USAID/Malawi’s Country Development Cooperation Strategy (CDCS), which covers the period 2013 - 2018, has as its overarching strategic goal to: Improve the Quality of Life for Malawians. This is supported through three development objectives (DOs): DO 1: Social Development Improved; DO 2: Sustainable Livelihoods Increased; and DO 3: Citizen Rights and Responsibilities Exercised. USAID’s CDCS hypothesizes: If assistance is integrated, then development results will be enhanced, more sustainable, and lead to achievement of the CDCS goal: Malawian’s Quality of Life Improved. The CDCS will promote integration through the concentration of program and financial resources by what the Mission calls a “3-C Approach”: • Co-locating interventions to the extent that it is sensible; • Coordinating better within USAID and with other Development Partners (DPs), and • Collaborating to foster linkages among implementing partners and the DPs to improve results and sustainability. There are 28 districts in Malawi. All three DOs will implement activities in the three CDCS focus districts (Lilongwe, Balaka and Machinga); two DOs will implement activities in ten districts; and one DO will implement in seven districts. The three DOs will also provide limited nation-wide assistance in the remaining seven districts. One way that USAID/Malawi seeks to achieve the CDCS goal is through increased availability and improved quality of essential social services. The Mission is engaged in a range of health system strengthening activities to expand facility- and community-level service delivery and increase the number of people receiving high impact, high quality services. Support for Service Delivery Integration (SSDI) is USAID/Malawi’s flagship health activity. This activity consists of three inter-related sector activities, namely 1) SSDI-Services, 2) SSDI-Communications, and 3) SSDI-Systems. In close collaboration with the Ministry of Health (MoH), SSDI interventions support the increased availability and quality of the Essential Health Package (EHP) services; reinforce health promotion and disease prevention among households; and strengthen elements of the health system to sustain effective EHP delivery. SSDI’s development hypothesis postulates that: Programming health interventions through an integrated platform, consisting of activities in health policy and systems strengthening, support for integrated health service delivery, and social and behavior change communication, will result in significant expansion of coverage, quality and utilization of priority EHP services at community clinics, health centers and district hospitals. SSDI’s interventions align with the CDCS’s DO 1: Social Development Improved; and with crosscutting Sub-Intermediate Results (SIR) 1: Capacity of Institutions Improved), and SIR 2: Positive Behaviors Adopted). SSDI interventions also contribute directly to the Malawi Health Sector Strategic Plan (HSSP) 2011-2016. SSDI-Services’ strategies and interventions are designed to complement and support the social and behavioral change communications and health system strengthening interventions of Sectors II and III of the SSDI overall activity (SSDI-Communications and SSDI-Systems). SSDI Communications Activity Performance Evaluation 24 SSD-I Services Support for Service Delivery (SSDI-Services) provides an integrated service delivery program to improve the health and well-being of Malawians by improving the quality of priority Essential Health Package (EHP) services at the community- and referral- (health centers and District hospitals) levels. SSDI￾Services, implemented by JHPIEGO, is a five year USAID-funded project that runs from November 2011-November 2016. Its primary objectives include: • Increase access and utilization of EHP services for women and children and engage men in health care; • Improve quality of health services at community and facility level in target districts; • Improve health-seeking behavior by individuals, families and communities; • Strengthen health care delivery system via the development, testing, and scaling up of innovative and sustainable community-based service delivery approaches; and • Develop coherent and mutually supportive activities between the Government of Malawi (GoM), the three SSDI elements, the Private Sector and Social Marketing (PSSM) partners, and other national stakeholders to ensure integration and leveraging of program inputs to scale up service delivery. SSD-I Communication SSDI-Communications is a five-year (September 2011- September 2016) USAID/Malawi social and behavior change communication (SBCC) activity. SSDI-Communications promotes normative and individual behavior change in several priority health areas, including HIV and AIDS, maternal and child health, malaria, nutrition, water and sanitation and family planning. The activity addresses barriers to behavior change at the structural, service delivery, societal, and personal levels. SSDI-Communication primary objectives include: 1. Strengthening national and targeted district level planning and coordination on EHP priorities; 2. Developing and producing evidence-based SBCC packages under multi-level media campaigns; 3. Building capacity of key national institutional partners and targeted district SSDI partners for effective SBCC strategic planning and delivery; and 4. Identifying best practices for SBCC implementation through formative research and testing innovative approaches. SSD-I Systems SSDI-Systems is a five-year USAID-funded project awarded to Abt. Associates Inc. running from September 2011 to September 2016, whose mission is to assist the Ministry of Health (MoH) to improve policies, management and leadership, and fiscal responsibility to advance Malawi’s health system and the sustainable impact of the Essential Health Package (EPH). SSDI-Systems provides appropriate, relevant, and coordinated interventions at the national, zonal, district, and local levels. This sector’s primary objectives are to: • Provide the MoH with expert technical assistance in policy development; • Clarify and strengthen management functions at all levels for quality assurance; SSDI Communications Activity Performance Evaluation 25 • Improve the current health management information system (HMIS) to ensure that key staff can carry out rigorous and routine high-quality data collection to support evidence-based decision making; • Adapt proven tools and metrics to bolster monitoring and evaluation (M&E), financial management, and Human Resources for Health (HRH); and • Execute gender-sensitive programming that takes into account the exponential benefits resulting from activities that advance women’s and girls’ equity and health status. PURPOSE OF THE PERFORMANCE EVALUATION The Contractor must carry out up to a minimum of two end-of-activity performance evaluations out of the following three. The specific purposes of each evaluation are as listed below. Purpose of Performance Evaluation of SSD-I Services: To determine the effect of SSDI-Services’ interventions on improved service delivery and quality of care at supported community clinics and health facilities; expanded coverage of quality EHP services; and increased uptake of quality integrated EHP. The main objectives are to measure and determine the extent to which SSDI-Services interventions had on quality of and access to care; provide in-depth insights into the facilitating and limiting factors of increased service utilization at each level of service delivery; and document progress made towards building MoH capacity to deliver quality EHP services. The evaluation outputs must provide evidence based recommendations on key actions required of USAID/Malawi and MoH to improve their activity planning. In addition, the evaluation outputs must provide recommendations to USAID/Malawi and the MoH to inform future implementation of integrated health service delivery-focused programming. Recommendations must go beyond general high level recommendations, and be based on a review of what worked well under SSDI-Services, and must articulate specific key approaches for the future. Purpose of Performance Evaluation of SSD-I Communications: To determine the effectiveness of the SSDI-Communications’ multilevel approach to promote normative behavior change and health seeking practices. Findings and recommendations from this evaluation will inform the implementation of USAID/Malawi’s new SBCC and integrated health activities. The main objectives are to determine the extent SSDI- Communications achieved its four primary objectives (see C1 above), with specific focus on extent to which SSDI-Communications was able to reposition the Health Education Section (HES); the degree to which SSDI-Communications’ campaigns resonated with individuals and communities; and an appraisal of the community mobilization implementation model. The evaluation outputs must provide recommendations to USAID/Malawi and the MoH to inform future implementation of SBCC programming in Malawi. Purpose of Performance Evaluation of SSD-I Systems: To determine the effectiveness of the SSDI-Systems approach to support Malawi-led and Malawi-owned efforts to achieve sustainable health results in line with current health priorities. The main objectives of this end-of-activity performance evaluation are to assess SSDI-Systems methodologies and approaches to capacity strengthening, and institutionalization of key MoH functions, including supportive supervision; management structures, responding to stakeholder expressed needs, and mentorship) in relation to the activity’s achievements. The evaluation outputs must provide recommendations to USAID/Malawi and MoH to inform future implementation of health system strength. SSDI Communications Activity Performance Evaluation 26 ANNEX II: DATA COLLECTION INSTRUMENTS Question 1: To what extent did the Social and Behavior Change Communications (SBCC) packages, including the Moyo ndi Mpamba platform, effectively promote increased awareness and practices of preventative behaviors in target communities? Tool 1.1 KII/FGD on increased awareness and practice of preventable behaviors prevention by beneficiaries Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing an assessment for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this assessment. Please answer « yes » if you agree to participate. 1. What does the slogan Moyo ndi Mpamba mean to you? 2. What is the idea behind the slogan? 3. Where have you heard the slogan being used? 4. What health behaviors is Moyo ndi Mpamba recommending? 5. What do you know about the Moyo ndi Mpamba radio drama from two years ago? 6. What health problems were discussed by the characters on the radio drama? 7. What health problems were discussed by the characters of the radio drama that had an effect on your own or your family’s health? 8. What do you know about the Moyo ndi Mpamba magazine radio program that is being broadcast now? 9. What do you think about the real stories about real people told on radio program? 10. What songs or music is associated with Moyo ndi Mpamba? 11. What healthy activities were recommended by Moyo ndi Mpamba regarding each of the following: Malaria? Maternal, Neonatal and Child Health? Family Planning? HIV/AIDS? Nutrition? Water, hygiene and sanitation? 12. What is your experience getting information on Moyo ndi Mpamba topics from health care providers? 13. What is your experience getting information on Moyo ndi Mpamba topics from community health volunteers? 14. What sort of Moyo ndi Mpamba printed support materials did the health providers use, if any (flip charts, picture books, booklets? 15. What sort of Moyo ndi Mpamba printed support materials did community health volunteers use, if any (flip charts, picture books, booklets? SSDI Communications Activity Performance Evaluation 27 16. What other Moyo ndi Mpamba information from other sources did you get in your community, if any? 17. What channel did you find to be the most useful for communicating Moyo ndi Mpaba messages? 18. What influence has Moyo ndi Mpamba communication had on relations between husbands and wives, if any? 19. What is your experience of husbands and wives going to antenatal care together? 20. What is your experience of husbands and wives discussing Family Planning together? 21. What is your experience of husbands and wives going to Family Planning services together? 22. What can be done to reduced children under 5 from dying? 23. Over the last five years, how has your attitude about taking your health and that of your family into your own hands? 24. What groups of people should be given priority for sleeping under a mosquito net? 25. Why get medical help within 24 hours when developing a fever? 26. What has been your experience in building toilets, latrines or rubbish pits near your household for human waste within the last 5 years? 27. What has been your experience with increased handwashing in your household within the last 5 years? 28. What have you done to improve handwashing in your household (buy soap, set up handwashing station, getting plastic container with spout)? 29. What are your priorities for good family health? 30. What has been done in the community to ensure good hygiene and sanitation in the community? 31. What modern contraceptives have you used, if any? 32. What specific health-seeking actions, if any, have you and your family taken based on the Moyo ndi Mpamba recommendations? 33. What project materials have you seen (posters, billboards, Flip Chart, Family Health Booklet)? 34. What project events, if any, did you attend including (road shows, dramas, open days, concerts)? 35. What is your experience with visits to your household by Community Action Group member? 36. What is your experience with Health Surveillance Assistants being the source of information of the 6 Essential Health Package messages? SSDI Communications Activity Performance Evaluation 28 Tool 1.2 Likert on increased awareness and practice of preventable behaviors prevention with beneficiaries male__ female__ age___ location_________________________ Introduction: We are doing an assessment of health projects. We would appreciate if you can help us with the assessment. Please be advised that you are not in any way obliged to participate in this survey. Please answer « yes » if you agree to participate. PART A: Please indicate which of the following health services you or your family have ever received: =1 Malaria =2 Maternal, neonatal and child health =3 Family planning =4 HIV/AIDS =5 Nutrition =6 Water, hygiene and sanitation =7 Infant vaccinations =8 Bed nets for malaria =9 Malaria treatment =10 Birth outside the home with medical assistance =11 Prenatal care =12 Postnatal care =13 Water and sanitation: soap/protection of water and food/latrines =14 Nutrition education =15 Vitamin A PART B: We will read you several statements. We would like you to tell us how much you agree or disagree with the statements on a scale of one to five. Five meaning that you agree very much and one meaning that you don’t agree at all. (5) agree very much (4) agree (3) neither agree nor disagree (2) disagree (1) disagree very much 1. I am aware of the Moyo ndi Mpamba campaign and have found the advice given to be useful. 1 2 3 4 5 2. I am familiar with Moyo ndi Mpamba radio dramas and magazine program and find the stories about people and topics to be true to life. 1 2 3 4 5 3. The people providing health services have improved over the last five years in their ability to communicate to me about health issues. 1 2 3 4 5 4. The community health volunteers have contributed to improving my health and that of my community. 1 2 3 4 5 SSDI Communications Activity Performance Evaluation 29 5. The community is more aware about how to improve family health now than five years ago. 1 2 3 4 5 6. I have been able to get all the information I need on improving the quality of my family’s health. 1 2 3 4 5 7. I am very satisfied with the way information about health is communicated to me now. 1 2 3 4 5 8. I am much more aware of what families can do to overcome health problems today than I was 5 years ago. 1 2 3 4 5 9. I have enough mosquito nets in my household for everyone and they are used regularly. 1 2 3 4 5 10. My partner and I have no problems getting modern contraceptives which we use. 1 2 3 4 5 11. I have installed toilets, latrines, rubbish pits or handwashing facilities in my household within the last 5 years. 1 2 3 4 5 PART B: Please answer in your own words the following questions: 11. What has been the biggest improvement you have made in your health or the health of your family due to Moyo ndi Mpamba communications? _____________________________________________________________________________ _____________________________________________________________________________ ____________________________________________ 12. What might be done to improve health communication to you and your family? _____________________________________________________________________________ _____________________________________________________________________________ ____________________________________________ SSDI Communications Activity Performance Evaluation 30 Tool 1.3 Site observation checklist Directly identify or observe evidence of targeted health practices =1 urban =2 rural District____________________Village/Town__________________________ __Evidence of use of bed nets (for children, for adults, for pregnant women)=1 __Presence of soap=2 __Handwashing container=3 __Appropriateness storage of water (any of the following: covered, away from animals, spouts)=4 __Availability of sanitation facilities (latrines, toilets, rubbish pit)=5 __Health card=6 __Project materials (leaflets, brochures or booklets)=7 SSDI Communications Activity Performance Evaluation 31 Tool 1.4 Media Content Analysis Survey Section A: Identification Zone: North =1 CE =2 CW =3 SE =4 SW =5 District: Karonga=11 Machinga=41 Salima =21 Chikhwawa=51 Lilongwe=31 TA:_______________________________ Village:_________________ Location of interview: Rural =1 Urban =2 Sex: (circle one) Male =1 Female =2 Age of respondent:_______ Category of respondent (circle one) Young couple (about to get married/and married) =1 Parent of under-5 children =2 Parent of older children adolescents (12 to 19) =3 INTRODUCTION Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing an assessment for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this assessment. Please answer « yes » if you agree to participate. Section B: Moyo ndi Mpamba Campaign B1) What is the idea behind the slogan « Moyo ndi Mpamba, Usamalireni? [choose one] Have no idea, never heard of it before =1 People should vote in elections =2 Take action to improve your health =3 Be friendly to your neighbors =4 Other, specify____________________ =5 B2) Where have you heard or seen the slogan « Moyo ndi Mpamba » used in the past? [mark all mentioned] SSDI Communications Activity Performance Evaluation 32 Never heard or seen it being used =1 Radio spot advertisements =2 Radio drama program =3 Seen it on posters and billboards =4 Seen it on brochures and leaflets =5 Heard it in music and roadshows =6 Other, specify____________________ =7 B3) What health behaviors is Moyo ndi Mpamba recommending? [Mark all mentioned] Keep children in school for better health =1 Sleep under long lasting insecticide-treated bed nets every night =2 Reduce population growth by getting married later =3 Couples going to antenatal clinic =4 Couples going for HIV testing and counseling together =5 Know danger signs in pregnancy =6 Use family planning methods =7 Eating all six food groups =8 Prevent mother-to-child transmission =9 Get immediate care for a fever =10 Don’t know any of recommendations =11 Other, specify____________________ =12 B4) What specific health changes have you and your family made as a result of Moyo ndi Mpamba recommendations? [Mark all mentioned] Have planted more profitable crops =1 Husband and wife attended antenatal care =2 Use condoms in casual relationships =3 Get diagnostic test and treat malaria rapidly =4 Built pit latrines in household =5 Built handwashing stations in household =6 Other, specify____________________ =7 B5) What are your priorities for good family health? [Mark all mentioned] Reduce chances of HIV and AIDS infection =1 Access contraceptives at family planning services =2 Improve communication between couple on health =3 Eat better food for better family nutrition =4 Access clean water =5 Maintain sanitary conditions =6 Ensure everyone sleeps under bed net every night =7 Other, specify____________________ =8 SSDI Communications Activity Performance Evaluation 33 B6) What are the most serious health-related problems you are facing in order of importance? [Mark 1-3] __Lack of safe drinking water =1 __Poverty =2 __Lack of food =3 __Inadequate health services =4 __Untreated illness =5 __Limited access to medications =6 __Other, specify___________________ =7 B7) What Moyo ndi Mpamba radio spot advertisements have you heard? [Mark all mentioned] Have never heard any Moyo ndi Mpamba radio spots =1 Promotion of long lasting insecticide-treated bed nets =2 Good nutrition in pregnancy =3 Moyo ndi Mpamba generally =4 Cholera prevention and treatment =5 Complementary feeding of infants =6 Other, specify___________________ =7 B8) What do you think of the Moyo ndi Mpamba radio drama that ended two years ago? [Circle all mentioned] Never heard of the radio drama =1 Program is bit boring and not interesting for me =2 Find the characters resemble people I know =3 Issues discussed are helpful to all in my family =4 This drama has inspired me to make health changes =5 Other, specify___________________ =7 B9) Which of the following topics did you hear discussed in the radio drama? [Circle all mentioned] Never heard radio drama =1 Malaria prevention with bed nets =2 Importance of wearing clean clothes =3 Family planning methods =4 Good nutrition during pregnancy =5 Benefits of neonatal care =6 Benefits of safe delivery =7 Introduction of weaning foods =8 Good breastfeeding practices =9 HIV =10 Other, specify___________________ =11 B10) What do you think of the Moyo ndi Mpamba magazine radio program on the radio now? SSDI Communications Activity Performance Evaluation 34 Never heard of the radio magazine program =1 Program is bit boring and not interesting for me =2 Find the stories are about resemble people like me =3 Issues discussed are helpful to all in my family =4 This radio magazine has inspired me to make health changes =5 Other, specify___________________ =6 B11) Which of the following topics did you hear discussed in the radio drama? [Circle all mentioned] Never heard radio drama =1 Malaria prevention with bed nets =2 Importance of wearing clean clothes =3 Family planning methods =4 Good nutrition during pregnancy =5 Benefits of neonatal care =6 Benefits of safe delivery =7 Introduction of weaning foods =8 Good breastfeeding practices =9 HIV =10 Other, specify___________________ =11 B12) What channels did you hear the radio spots, the radio magazine and radio drama broadcast on? [Circle all mentioned] ZBS =1 MBC1 =2 MBC2 =3 Galaxy =4 Community radio =5 Other, specify___________________ =6 B13) Which of the following Moyo ndi Mpamba materials have you seen or received? [Circle all mentioned] Family health booklet =1 Leaflets on long acting reversible contraception =2 Malaria comic book =3 Brochures =4 Marriage counsellor guide =5 Community health flip chart =6 Malaria flyers =7 ORS and zinc leaflets =8 Other Specify___________________ =9 B14) Which of the following Moyo ndi Mpamba posters have you seen? SSDI Communications Activity Performance Evaluation 35 [Circle all mentioned] Not seen any Moyo ndi Mpamba posters =1 Long-lasting insecticide-treated bed nets =2 Maternal nutrition and child health =3 Family planning =4 Antenatal care attendance =5 Handwashing =6 Contraceptive implants =7 Complimentary feeding =8 Nutrition in pregnancy =9 Promoting Moyo ndi Mpamba =10 Other, specify___________________ =11 B15) Which of the following Moyo ndi Mpamba promotions have you seen? [Circle all mentioned] Billboards on long-lasting insecticide treated bed nets =1 Billboards on long-acting family planning methods =2 Songs or music that is associated with Moyo ndi Mpamba =3 Songs from Music4Life Music Album =4 Music4Life festivals =5 Moyo ndi Mpamba roadshows =6 Local drama groups =7 Moyo ndi Mpamba promotional T-Shirts =8 Wrappers =9 Other, specify___________________ =10 B16) What human sources of information on Moyo ndi Mpamba have you been exposed to? [Circle all mentioned] Health care provider =1 Community health worker =2 Community Action Group =3 Village head =4 Religious leader =5 Journalist =6 Other, specify___________________ =7 B17) What are the best channels that have been used by Moyo ndi Mpamba to communicate with you? [Circle all mentioned] Radio spot ads =1 Radio programs =2 Posters =3 Roadshows =4 Leaflets and flyers =5 SSDI Communications Activity Performance Evaluation 36 Family Health Booklet =6 Interpersonal communication by health worker =7 Music4Life songs =8 Billboards =9 Other, specify___________________ =10 B18) What is the overall purpose of Moyo ndi Mpamba? [Circle all mentioned] Get people to advocate for their own health =1 Increase the practice of healthy behaviors =2 Increase the number of people who vote =3 Benefit from a responsive health care system =4 Get government to spend more on education =5 Other, specify___________________ =6 B19) What has been the impact of Moyo ndi Mpamba on male and female relations? [Circle all mentioned] Not aware of any impact on male and female relations =1 Family planning discussed more now by married couples =2 Men attend antenatal clinics more often now with their wives =3 Women have better paying jobs than they used to =4 More awareness of harm of gender-based violence by men =5 Other, specify___________________ =6 B20) What changes have been made as a result of Moyo ndi Mpamba to improve the lives of women? [Circle all mentioned] More women deliver in health clinics now =1 Women have more control of decisions regarding family health =2 Women discuss with men more about family planning =3 Men go with women to antenatal care services =4 Social norms changed to allow women more influence =5 Other, specify___________________ =6 B21) To what degree has your knowledge on health increased in the last five years ? (circle one) Not increased at all =1 Increased =2 Increased a great deal =3 No opinion =4 B22) To what degree have you improved your health practices in the last five years ? (circle one) Not improved at all =1 Improved =2 Improved a great deal =3 No opinion =4 SSDI Communications Activity Performance Evaluation 37 Section C: Services C1) Family planning messages C1 Which family planning messages have you heard? Which ones have you taken action on? C1a Family planning methods are safe. Heard=1 Not heard=2 C1b If you have side effects, what action would you take? Go to clinic/took action=1 Didn’t take action=2 C2a Choose temporary or permanent family planning methods according to your needs. Heard=1 Not heard=2 C2b What action did you take? Took action=1 Didn’t take action=2 C3a Pregnancy after the age of 35 puts a woman at greater risk. Heard=1 Not heard=2 C3b What action did you take? Took action=1 Didn’t take action=2 C4a Getting pregnant when you are too young (younger than 18) puts your health and that of the baby at risk. Heard=1 Not heard=2 C4b What action did you take? Took action=1 Didn’t take action=2 C5a Discuss with your partner on how best to plan your family. Heard=1 Not heard=2 C5b What action did you take? Took action=1 Didn’t take action=2 C6a You have a right to decide when to get pregnant. Heard=1 Not heard=2 C6b What action did you take? Took action=1 Didn’t take action=2 C7a Make healthy choices as a couple. Talk about family planning. Heard=1 Not heard=2 C7b What action did you take? Took action=1 Didn’t take action=2 C8a Real men talk to their spouses about family planning. Heard=1 Not heard=2 C8b What action did you take? Took action=1 Didn’t take action=2 Section D: Malaria messages D What of the following malaria messages have you heard? Which ones have you taken action on? SSDI Communications Activity Performance Evaluation 38 D1a Anyone can get malaria. Sleep under a long￾lasting insecticide-treated net every night, all year round to protect yourself. Heard=1 Not heard=2 D1b What action would you take? Go to clinic/took action=1 Didn’t take action=2 D2a Everyone in the family should sleep under the LLIN. Heard=1 Not heard=2 D2b What action did you take? Took action=1 Didn’t take action=2 D3a Long-lasting insecticide-treated bed nets are safe for everyone in the family. Heard=1 Not heard=2 D3b What action did you take? Took action=1 Didn’t take action=2 D4a Visit the nearest health center as soon as you notice fever in any family member (within 24 hours). Heard=1 Not heard=2 D4b What action did you take? Took action=1 Didn’t take action=2 D5a Make sure to take all your malaria drugs as prescribed at the health center. Heard=1 Not heard=2 D5b What action did you take? Took action=1 Didn’t take action=2 D6a Keep surroundings clean and dry to prevent mosquitoes from breeding. Heard=1 Not heard=2 D6b What action did you take? Took action=1 Didn’t take action=2 Section E: Maternal, newborn and child health messages E What of the following MNCH messages have you heard? Which ones have you taken action on? E1a It is important to plan carefully for childbirth and have a birth plan that includes when to go to the clinic, how to get there and the resources that will be needed during that period. Heard=1 Not heard=2 E1b What action would you take? Go to clinic/took action=1 Didn’t take action=2 E2a Having a birth preparedness plan will help ensure a healthy outcome for your pregnancy and delivery. Heard=1 Not heard=2 E2b What action did you take? Took action=1 Didn’t take action=2 E3a All couples should learn about, and know the E3b What action did you take? SSDI Communications Activity Performance Evaluation 39 danger signs of pregnancy, delivery and after delivery. Heard=1 Not heard=2 Took action=1 Didn’t take action=2 E4a Pregnant women should attend antenatal care at least four times before delivery and once within the first three months of pregnancy. Heard=1 Not heard=2 E4b What action did you take? Took action=1 Didn’t take action=2 E5a Pregnant women should deliver at the health center for safe and skilled delivery. Heard=1 Not heard=2 E5b What action did you take? Took action=1 Didn’t take action=2 E6a All children should be fully immunized by the age of 1 year. Heard=1 Not heard=2 E6b What action did you take? Took action=1 Didn’t take action=2 E7a Parents should take their under-5 children to the health facility as soon as they observe any danger sign. Heard=1 Not heard=2 E7b What action did you take? Took action=1 Didn’t take action=2 SSDI Communications Activity Performance Evaluation 40 Section E: Nutrition messages E Which of the following nutrition messages have you heard? Which ones have you taken action on? E1a Eat foods from the six food groups: plant, proteins, animal proteins, carbohydrates, fats and oils, minerals and vitamins, to stay strong and healthy. Heard=1 Not heard=2 E1b What action did you take? Took action=1 Didn’t take action=2 E2a Pregnant and breastfeeding women need more food. Pregnant women should eat one extra meal per day, while breastfeeding women should eat two extra meals per day. Heard=1 Not heard=2 E2b What action did you take? Took action=1 Didn’t take action=2 E3a Breastfeed your baby exclusively for the first six months. Heard=1 Not heard=2 E3b What action did you take? Took action=1 Didn’t take action=2 Section F: Water, sanitation and hygiene messages F Which of the following water, sanitation and hygiene messages have you heard? Which ones have you taken action on? F1a Leftover food should be covered properly and reheated before eating to avoid disease. Heard=1 Not heard=2 F1b What action did you take? Took action=1 Didn’t take action=2 F2a Wash your hands with clean water and soap/ash before preparing or eating meals, after visiting the toilet and after changing baby nappies. Heard=1 Not heard=2 F2b What action did you take? Took action=1 Didn’t take action=2 F3a Drink only clean and safe water. Make water safe by boiling, or by treating with chlorine or other treatment agents. Heard=1 Not heard=2 F3b What action did you take? Took action=1 Didn’t take action=2 F4a Avoid open defecation to prevent disease. Heard=1 Not heard=2 F3a What action did you take? Took action=1 Didn’t take action=2 SSDI Communications Activity Performance Evaluation 41 Section G: HIV AIDS messages G Which of the following HIV AIDS messages have you heard Which ones have you taken action on? G1a If you do not know your status, you cannot get treatment. Get tested for HIV, get treated. Heard=1 Not heard=2 G1b What action did you take? Took action=1 Didn’t take action=2 G2a STIs predispose you to HIV infection. Seek early treatment for STIs. Heard=1 Not heard=2 G2b What action did you take? Took action=1 Didn’t take action=2 G3a Unprotected sex puts you at risk of HIV infection. Use condoms each time, every time. Heard=1 Not heard=2 G3b What action did you take? Took action=1 Didn’t take action=2 G4a Partners who stay faithful to each other greatly reduce their risk of HIV infection. Heard=1 Not heard=2 G4a What action did you take? Took action=1 Didn’t take action=2 G5a HIV-positive mothers can give birth to HIV￾negative babies. Find out how at the health center. Heard=1 Not heard=2 G5b What action did you take? Took action=1 Didn’t take action=2 G6a Talk to your partner about HIV and your options to protect each other. Take the test together. Heard=1 Not heard=2 G6b What action did you take? Took action=1 Didn’t take action=2 SSDI Communications Activity Performance Evaluation 42 Question 2: To what extent did Communications improve the capacity of the Health Education Section (HES) and local institutions to develop, coordinate, implement, and provide oversight for health sector integrated SBCC? Tool 2.1 KII/FGD for National SBCC planners Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing a study for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this survey. Please answer « yes » if you agree to participate. (circle one) =1 National partners (HES, UoM, Chancellor College), =2 Media sector (Story Workshop, Galaxy media, Phingo and Nyimbo) =3 Private media (Mercantile International Advertising Agency) =4 SSDI staff (JHU, Save the Children) CAPACITY BUILDING 1. In what way did SSDI-Communication contribute to the building of technical capacity building to develop, coordinate, implement and oversee SBCC programming? 2. What specific trainings did you participate in and how would you assess them? 3. What specific SBCC methodologies, tools and techniques were introduced by SSDI-Communication that you found to be particularly useful? 4. What specific skills were improved to allow you to coordinate, implement and oversee SBCC efforts? 5. How would you consider the balance between practical and theoretical capacity building in terms of preparing you for developing and implementing SBCC strategies? 6. What were the most effective means for capacity building that were used (training, guides, mentoring, learning by doing, etc.)? 7. What was the impact on increased SBCC capacity of the Leadership in Strategic Health Communications workshops, the fellowships for “rising SBCC stars” and the University of Malawi lectures? 8. What has been the impact of SSDI-C on the capacity of the Health Education Service nationally and at the district level? TOOLS AND STATEGIES 9. How would you assess the utility of the SSDI-Communication tools like the message guides and campaign strategy guides? 10. What role did SSDI-Communication play in the development of national communication policy and strategies for health, malaria and the Essential Health Package? 11. How would you assess the utility of the Institution Capacity Assessment that evaluates communication capacity? SSDI Communications Activity Performance Evaluation 43 EVIDENCE-BASED STRATEGIES 12. How was the influence of the capacity building on the ability of SSDI and its partners to conduct evidence-based strategic SBCC planning? 13. How do you assess the impact of use of evidence-based strategic planning of SBBC planning and implementation? 14. To what degree was flexibility and adaptability built into SSDI-C that allowed it to respond to emerging issues? MESSAGES AND MATERIALS 15. In what ways did the SSDI-Communication capacity building influence the way key Essential Health Packages were promoted? 16. What has been the impact of branding SBCC outputs under Moyo ndi Mpamba? 17. How do you assess the use of mass media under Moyo ndi Mpamba? 18. How do you assess the use of facility and community support materials under Moyo ndi Mpamba? COMMUNICATION AND SERVICES 19. To what degree was SSDI successful in coordinating Communication, Services and Community Mobilization? 20. What were the challenges in coordinating Communication, Services and Community Mobilization? 21. What were the specific enabling and inhibiting factors in developing capacity among the SSDI￾Services partners to conduct SBCC and promote services and health-seeking behavior? GENDER 22. How was gender dealt with in capacity building, planning and implementation? 23. What could be done to better deal with gender issues in SBCC? INNOVATION 24. What was your experience, if any, with the introduction of SBCC innovations such as: -new research methodologies -approaches to maximize integration -partnerships to involve the private sector and other non-governmental actors -uses of traditional media and digital media -resource catalogs ACCOMPLISHMENTS, BEST PRACTICES AND LESSONS LEARNED (5.1) 25. What were the most significant accomplishments of SSDI-C? 26. What do you consider the most important best practices that were used by SSDI-Communication that you would recommend are used in the future? 27. What were the most significant challenges encountered in achieving the SSDI-Communication goals? 28. What were the specific enabling factors in developing and implementing SBCC strategies? 29. What more could be done to improve the quality of SSDI-C with beneficiaries? SSDI Communications Activity Performance Evaluation 44 30. What changes would you recommend in the way communication and community mobilization activities were designed, implemented and managed? 31. What lessons were learned that will be useful in planning future similar interventions? SSDI Communications Activity Performance Evaluation 45 Tool 2.2 KII/FGD for District and Community-level SBCC Implementers Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing a study for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this survey. Please answer « yes » if you agree to participate. (circle one) =1 Health care providers =2 Subgrantee staff =3 Community mobilisers =4 Community Action Group =5 Community Mobilization Sub grantees =6 District officers (DHPR, DHMT, DHPC) 1. In what way did SSDI-Communication contribute to the building of technical capacity building to develop, coordinate, implement and oversee SBCC programming? 2. What specific trainings did you participate in as part of the SSDI-Communication and how would you assess them? 3. In what ways did the SSDI-Communication capacity building influence the way you promoted Key Essential Health Packages (EHP) messages? 4. What difference did the evidence-based SBCC Strategic Planning make to the effectiveness of the SBCC and Community Mobilization at your level? 5. What role did SSDI-Communication play in performance monitoring and and evaluation as well as knowledge management tracking of SBCC and Community Mobilization initiatives? 6. How did SSDI-Communication enable District and Community partners to effectively focus on the very diverse and wide range of EHP messages? 7. What synergies were there between the National, District and Community levels for coordinated SBCC and Community Mobilization planning and implementation? 8. What was the role of SSDI-Communication in the development of those synergies and the coordination? 9. How has community mobilization specifically been used to promote use of priority EHP services? 10. How has community mobilization specifically been used to promote use of priority EHP behavior change? 11. What were the challenges in decentralized health action planning and a bottom up planning approach involving village heads and traditional authorizes? 12. How would you assess the Community Mobilization models and tools that were used including the Community Action Cycle process? 13. What was the impact at your level of the SSDI-Communication SBCC packages? 14. What was the impact at your level of the SSDI-Communication multimedia campaigns? 15. What was the impact of the Family Health Booklet, the Marriage Counseling Guide and the community health flip chart? SSDI Communications Activity Performance Evaluation 46 16. To what degree has knowledge and awareness of the benefits of appropriate health practices like handwashing and complementary feeding increased? CAGs 1. Describe training in the Community Action Cycle including who conducted it, its duration? 2. What was the utility of the « Picture Cards » in the Community Action Cycle process? 3. What kind of supervision did you recieve after the training, how often and by whom? 4. What support materials did you receive from the project (flip chart, Family Health Booklet, picture cards) and how were they used? 5. Describe any events that you might have attended related to the project (road shows, dramas, open days, concerts)? Facility Health Staff 1. Describe any training you received by the project in community mobilization, the Essential Health Packages and Interpersonal communication, if any? 2. What role did you play, if any, in coordinating with Community Action Groups or supervising their activities? 3. What project communication materials did you receive, if any, and how were they used (Posters, Flip Chart, Family Health Booklets? 4. What were efforts were made, if any, to match the delivery of project-related services with the promotion of services? Health Surveillance Assistants 1. Describe your experience with training in project Community Mobilization? 2. What was your experience, if any, using the picture cards when conducting the Community Action Cycle process? 3. What was your experience being supervised in your Community Mobilization work? 4. Describe your relationship with the Community Action Groups in terms of supporting them? 5. How well was the delivery of project services coordinated with the promotion of those services? District Health Promotion Officers 1. What was your experience with training in Communication Strategic Planning? 2. What was your experience with training in Community Action Cycle? 3. What was your role, if any, training or supervision of Community Action Groups and community mobilization? 4. How well were the 6 Essential Health Package messages communicated? 5. What were the most effective communication channels used (radio, posters, Flip Charts, Family Health Booklets)? 6. What events did you participate in (road shows, dramas, open days, concerts)? 7. What project materials, if any, were produced at the District level? SSDI Communications Activity Performance Evaluation 47 Tool 2.3 Likert Survey on Capacity Building for National, District and Community SSDI￾Communication Partners Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing a study for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this survey. Please answer « yes » if you agree to participate. (circle one) Organizational affiliation (National) National partners (HPRS, UoM) =1 Media sector (Story Workshop, Galaxy media, Phingo and Nyimbo) =2 Private media (Mercantile International Advertising Agency) =3 SSDI staff (JHU, Save the Children) =4 Organizational affiliation (District and Community) Health care providers =5 Subgrantee staff =6 Community mobilisers =7 NGOs (community mobilization sub grantees) =8 District players (DHCC, DHMT, DEC, DHPD) =9 PART A: We are presenting you here with several statements. We would like you to tell us how much you agree or disagree with the statements on a scale of one to five. Five meaning that you agree very much and one meaning that you don’t agree at all. (5) agree very much (4) agree (3) neither agree nor disagree (2) disagree (1) disagree very much 1. The SSDI-Communication capacity building enabled me to increase my skills for planning and implementing SBCC 1 2 3 4 5 2. The SSDI-Communication capacity building enabled me to better promote the key Essential Health Packages (EHP) messages 1 2 3 4 5 3. Despite the diversity and large number of EHP intervention areas, we were able to communicate the messages well and inspire health seeking behaviors 1 2 3 4 5 4. Dynamic synergies were created between the National, District and Community levels for coordinated SBCC planning and implementation 1 2 3 4 5 5. The Community Mobilization models and tools, including the Community Action Cycle process, were used effectively 1 2 3 4 5 SSDI Communications Activity Performance Evaluation 48 6. I am very satisfied with the way information about health is communicated to me 1 2 3 4 5 7. I feel confident the legacy of SSDI-Communication is entrenched and will continue to reap dividends long into the future 1 2 3 4 5 8. There are more toilets, latrines and handwashing facilities and fewer births at home as a result of Community Mobilization efforts over the last 5 years 1 2 3 4 5 9. There is now broad stakeholder commitment and a true national movement behind EHP priorities as compared to 5 years ago 1 2 3 4 5 10. Families now advocate for own health and the timely use of EHP following the SSDI-Communication interventions compared to 5 years ago 1 2 3 4 5 PART B: Please answer in your own words the following questions: 11. What were the biggest constraints and challenges faced when designing, implementing and managing the communication activities? _____________________________________________________________________________ _____________________________________________________________________________ ____________________________________________ 12. What were the biggest achievements and the best practices of SSDI-Communication? _____________________________________________________________________________ _____________________________________________________________________________ ____________________________________________ SSDI Communications Activity Performance Evaluation 49 Question 3: To what extent did Communications’ Fellowship program improve strategic planning and implementation of SBCC, and media reporting of health issues? 3.1 KIIs for SSDI-Communication fellows 1. If you participated in the Media Café or other media trainings, explain in what ways that you benefited from that experience? 2. What were your experiences with the Health Communication Fellowship Program, if any? 3. What did you learn and skills did you develop during your fellowship? 4. How would you assess how well the fellowship program was organized? 5. What did you work on following after completing your fellowship? 6. What were the most important skills you gained from the fellowship that were used in your work? 7. What are the other fellowships participants you know working on now? 8. What were the benefits of being selected for student internships for undergraduate students in communications at University of Malawi? 9. What special skills are needed for covering health issues? 10. How did the fellowship or internship impact the way you cover health issues? 11. What are the differences between journalists who have participated in the media cafés, the media training, and the fellowship program and those who have not? SSDI Communications Activity Performance Evaluation 50 Question 4: What effect did the joint implementation of community mobilization by Services and Communications have on the effectiveness of this intervention? 4.1 KII/FGD with SSDI Services managers and medical staff Backgound KIIs on effectiveness of planning and coordination will be held with MoH managers at the central, zonal, and district levels as well as SSDI staff. Interviews will explore constraints and challenges related to how activities were designed, implemented, and managed to achieve the stated SBCC objectives. Hello! My name is________________________, I am an interviewer for DEVTECH, which is doing an assessment for the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All answers will remain confidential. It should take less than an hour. Please be advised that you are not in any way obliged to participate in this assessment. Please answer « yes » if you agree to participate. Facility Level KIIs: 10 frontline healthcare providers (Nurse, Medical Assistants) at health facilities to determine if the capacity building activities helped improve their deliveries of project communication messages. Collect qualitative information on the use use of communication materials in inter-personal communications. FGDs at community level: 5 FGDs to collect qualitative information at community level including: Village health committee mumbers, Health Advisory Committee members, Village Development Committee members (VDCs); Community Volunteers, Health Surveillance Assistants; these are the structures that provide outreach at community level. 1. What was your role related to Moyo ndi Mpamba planning and implementation? 2. What training do you get to improve your communication and community mobilization skills? 3. What things did you learn from the training that you were able to apply? 4. What other outside help did you get besides training? 5. What things did you learn from the training that you were able to apply? 6. What were the biggest constraints and challenges faced when designing, implementing and managing the Moyo ndi Mpamba communication and community mobilization activities? 7. What were the biggest achievements of the Moyo ndi Mpamba communication and community mobilization activities? 8. What were the biggest achievements of the Moyo ndi Mpamba community mobilization? 9. What changes would you recommend in the way communication and Community Mobilization activities were designed, implemented and managed? 10. What was the most significant accomplishment of the Moyo ndi Mpamba and why was it? 11. How were SSDI-Services and SSDI-Communication coordinated? 12. What was the net result of the SSDI-Services and SSDI-Communication joint implementation? 13. What were the challenges in the joint implementation? 14. What role did SSDI-Services play in building SBCC (Social and Behavior Change Communications) capacity at the facility level? SSDI Communications Activity Performance Evaluation 51 15. What was your role, if any, in SBCC with services clients? 16. What training, if any, did you get in SBCC from SSDI-communication? 17. How has your work in SBCC changed, if at all, since having contact with SSDI-communication? 18. What is the idea behind Moyo ndi Mpamba? 19. What are the most important Moyo ndi Mpamba messages you have communicated with your clients? 20. What were the most significant positive behaviors that were adopted by your clients after being exposed to Moyo ndi Mpamba messages. 21. What support materials have you used, if any, that have the Moyo ndi Mpamba logo on them? 22. How were the Social and Behavior Change Communication « packages » used by your service to promote services? 23. How were the Social and Behavior Change Communication « packages » used by your community to promote behavior change? 24. What use, if any, did you personally make of the SBCC packages? 25. What more could be done to improve the quality of your communications with your clients or community? 26. How has demand for and use of priority of Essential Health Packages been increased? 27. What has been done specifically to in terms of community mobilization and communication to increase demand for EHP services? 28. To what degree has voluntary family planning been established as a community norm in your community? 29. What are examples of increases in knowledge and awareness of the benefits of appropriate health practices in your community? 30. What caused the knowledge and awareness increases primarily? 31. What could be done to improve coordination between those conducting Communication, Community Mobilization and Service Delivery? SSDI Communications Activity Performance Evaluation 52 Question 5: What are the most significant accomplishments, best practices, and lessons learned from the SSDI-Communication activity? Tool 5.1 KIIs accomplishments, best practices and lessons learned 1. What were the most significant accomplishments of SSDI–C? 2. What lessons were learned that will be useful in planning future similar interventions? 3. What things would do recommend be done differently in the future? 4. To what degree was flexibility and adaptability built into SSDI –Communication that allowed it to respond to emerging issues? 5. What more could be done to improve the quality of SSDI - Communication communications with beneficiaries? 6. What were the biggest constraints and challenges faced when designing, implementing and managing the communication activities? 7. What were the biggest achievements of the communication and activities? 8. What changes would you recommend in the way communication and community mobilization activities were designed, implemented and managed? 9. What was the most significant accomplishment of the SSDI-Communication and why was it? 10. What do you consider the most important best practices that were used by SSDI-Communication that you would recommend are used in the future? 11. What were the most significant enabling factors in achieving the SSDI-Communication goals? 12. What were the most significant challenges encountered in achieving the SSDI-Communication goals? 13. What were the specific enabling and inhibiting factors in developing and implementing SBCC packages? 14. What were the specific enabling and inhibiting factors in developing SBCC capacity among the SSDI￾Communication partners? 15. What were the specific enabling and inhibiting factors in developing capacity among the SSDI￾Services partners to conduct SBCC and promote services and health-seeking behavior? 16. What were the challenges in coordinating Communication, Community Mobilization and Services? SSDI Communications Activity Performance Evaluation 53 ANNEX III: KEY INFORMANTS INTERVIEWED Name Designation Location District Dziko Chatata Research, M&E Specialist, SSDI-C, JHU SSDI-C Office, Lilongwe Lilongwe Alinafe Kasiya Chief of Party, SSDI-C, JHU-CCP SSDI-C Office, Lilongwe Lilongwe Triza Kakhobwe Hara Deputy Chief of Party, SSDI–C, JHU SSDI-C Office, Lilongwe Lilongwe Chimwemwe Chitsulo Monitoring, Evaluation and Learning Specialist USAID, Lilongwe Lilongwe Edson Dembo Malaria Program Specialist, Agreement Officer’s Representative SSDI Communications, USAID USAID, Lilongwe Lilongwe Angela Chitisme SBCC Technical Assistant, SSDI-C, JHU SSDI-C Office, Lilongwe Lilongwe Linda Mwambakulu National Coordinator Red Cross and Program Manager SSDI-C Red Cross, National HQ Lilongwe Chancy Maulka Social and Behavior Change Communication Advisor, SSDI-C, JHU SSDI-C Office, Lilongwe Lilongwe Anna Chinombo Community Mobilization Advisor, SSDI-C, Save the Children SSDI-C Office, Lilongwe Lilongwe Thokozadi Bema Zone Manager South East Zone-Zomba, Save the Children, Acting National Director SSDI SAVE the Children National Office, Lilongwe Lilongwe/ Zomba Mavuto Mizwa Community Mobilization and Capacity Building Manager, SSDI–C, Save the Children SSDI-C Office, Lilongwe Lilongwe Patricia Chirombo Zonal Community Mobilization Coordinator, SSDI–C, Save the Children SSDI-C Office, Lilongwe Lilongwe Wezi mMula Mawwlana Medical Officer DHMT DHMT, Lilongwe Lilongwe Thel Ramino Medical Officer DHMT–Acting District Medical Officer DHMT, Lilongwe Lilongwe Jalume Shaba Health Promotion Officer DHMT, Lilongwe Lilongwe Trinitus Mvaio Senior Health Promotion Officer DHMT, Lilongwe Lilongwe Silvester Kachale District Coordinator–Red Cross Red Cross District Office Lilongwe Abigail Groadigios Senior Nursing Officer–DHMT DHMT, Lilongwe Lilongwe Khanji Nyambo District Community Coordinator–SSDI Services, Jhpiego SSDI-S District Office Lilongwe Nitta Aufi M&E Officer Community Mobilization– SSDI–S, Jhpiego SSDI-S District Office Lilongwe Jimmy Nkhoma Nurse Midwife Technician/Family Planning Coordinator Nsalu health center Lilongwe Wongani Batoni Ass. Environmental Health officer Nsalu health center Lilongwe John Galanje Senior-HSA Nsalu health center Lilongwe Jenkins Banda Environmental Health Officer Mitundu health center Lilongwe Emmanuel Njerewa Clinical Officer Mitundu health center Lilongwe Dr. Fanny Kachale Director Reproductive Health Services National Office, Lilongwe Lilongwe SSDI Communications Activity Performance Evaluation 54 Dan Wendo Chief of Party, SSDI-Services Jhpiego SSDI Jhpiego Office Lilongwe David Amoruso Program Management Officer, SSDI Services SSDI Jhpiego Office Lilongwe Patricia Chirombo Zonal community Coordinator, SSDI Services SSDI Communication Office Lilongwe Watson Kaunde Managing Director Mercantile International Lilongwe Limbani Silungwe Projects Coordinator Mercantile International Lilongwe Marvin Mbwana Creative Manager Mercantile International Lilongwe Mercy Simbi Executive Director ADECOTS Lilongwe Livinia Kaunda Former SSDI-C fellow Counterpart International Lilongwe Tamanda Masambuka Former SSDI-C fellow Documentation Officer UNICEF/Malawi Lilongwe Lusayo Banda Former SSDI-C fellow MA in international communication at University of China Lilongwe Lolade Jhpiego Office Lilongwe Kabango Malewezi Database Manager Jhpiego Office Lilongwe Phino Mchenge Central Monitoring & Evaluation Division (CMED), Acting Head Ministry of Health Lilongwe Eluby Maganga UNICEF Lilongwe Maganizo Monawe Ministry of Health, Central Monitoring & Evaluation Division Ministry of Health Lilongwe Vitima Ndoui Program Officer, Materials and Package Development SSDI-C Lilongwe Violet Orchardson Nutrition Advisor USAID Lilongwe Precious Phiri Deputy Director Preventive Health Ministry of Health Lilongwe Mathrew Ramirez AMP Health Management Partner Aspen Institute Lilongwe Ward Jacobs Health Management Information Systems Consultant UNICEF Malawi Lilongwe Arthur Champiti Acting District Health Officer, Ministry of Health Salima District Hospital Salima Angella Nyongani Sukwata Health Education Officer, Ministry of Health, Salima District Hospital Salima Paul Duncan Executive Director Salima AIDS Support Organization (SASO) Salima Chiyemekezo Chabvu Programs Manager Salima AIDS Support Organization (SASO) Salima Takondwa Dzoole Mwale Community Mobilization Coordinator Salima AIDS Support Organization (SASO) Salima Claire Nyiribgo Nurse/Midwife Khombedza health center Salima Mphatso Masowa Nurse Midwife Technician Khombedza health center Salima Prisca Chiphwanya Medical Assistant Khombedza health center Salima SSDI Communications Activity Performance Evaluation 55 Edson Khomba Health Surveillance Assistant Maganga health center Salima Mataka Mkubwi Nurse Midwife Technician Maganga health center Salima Jambo Kamwana Senior Health surveillance Assistant Mbonechera health center Machinga Andy Mwafongo Medical Assistant Mbonechera health center Machinga Kondwani Chikakuda Medical Assistant Namandanje health center Machinga Worried Gausi Chief Clinical Officer Liwonde District Hospital Machinga Rotina Mlombwa Administrator Liwonde District Hospital Machinga Clifton Ngozo Senior Health Promotion Officer Liwonde District Hospital Machinga Wongari Nyirenda Health Promotion Officer Liwonde District Hospital Machinga Zacharia Kachimanga Area Supervisor for TAs Liwonde and MLomba, Machinga District Red Cross Machinga District Red Cross Machinga Nomawthu Chide District Coordinator, Machinga District Red Cross Machinga District Red Cross Machinga Lorent Milliasi HSA Mbonechera health center Machinga Frank Matiya HSA Mbonechera health center Machinga Grace Maida Nurse Midwife Mkwepere health center Machinga Chimwemwe Chavinda Project Director for SSDI Communications Chancellor College-UoM Zomba Chikosa Ngwira Zonal Manager–SSDI–S, Plan International Plan International Zonal Office Mzuzu Ella Billy Nyirenda District Community Mobilisation Coordinator Red Cross, Karonga Karonga Harold Mtambo District Health Education Officer-retired Karonga District Hospital Karonga Cynthia Team Leader Plan International District Office Karonga Trzzer Luesha Area Supervisor Red Cross, Karonga Karonga Lucy Mfune Area Supervisor Red Cross, Karonga Karonga Mavuto Kaonga HRO Karonga District Hospital Karonga David Tukula District Environmental Health Officer Karonga District Hospital Karonga Maloni Nyirenda District Nursing Officer Karonga District Hospital Karonga David Sibale Acting District Health Officer Karonga District Hospital Karonga Joseph Mwachande Administrator Karonga District Hospital Karonga Rodgers Mkuziwaduka Medical Assistant Lupembe health center Karonga Alethea Ngosi Nurse Midwife Technician Lupembe health center Karonga Jane Sichali HDA Counsellor/Diagnosis Asst. Lupembe health center Karonga Blessings Chirwa Medical Assistant Iponga health center Karonga SHSA Iponga health center Karonga Hendrix Mwenelupembe Zonal Coordinator, Save the Children, SSDI-S Blantyre Blantyre Joyce Phekani Executive Director-Center for Center for Alternatives Blantyre SSDI Communications Activity Performance Evaluation 56 Alternatives for Victimized Women, Orphans and Children for Victimized Women, Orphans and Children HQ Maxwell Kaliati Program Manager-Center for Alternatives for Victimized Women, Orphans and Children Center for Alternatives for Victimized Women, Orphans and Children HQ Blantyre Tamiwa Chikopa Finance Administrator-Center for Alternatives for Victimized Women, Orphans and Children Center for Alternatives for Victimized Women, Orphans and Children HQ Blantyre Benson Nikhoma Somba Director-Galaxy Media Galaxy Media Office Blantyre Virginia Faeh Enim Family Planning Coordinator Chikwawa District Hospital Chikwawa Chisomo Kalitsiro Facilitator-Center for Alternatives for Victimized Women, Orphans and Children Chikwawa District Office Chikwawa Enoch District Coordinator-Center for Alternatives for Victimized Women, Orphans and Children Now lives in Blantyre Chikwawa Vennie District Team Leader, Save the Children, SSDI-S Save the Children District Office Chikwawa Francis Mwale Medical Assistant Makhuwira health center Chikwawa Alex Mlilima Nurse Midwife Technician Makhuwira health center Chikwawa Willy Limited Senior Health Surveillance Assistant Ngabu health center Chikwawa Grace Kachala Senior Nurse Midwife Ngabu health center Chikwawa SSDI Communications Activity Performance Evaluation 57 ANNEX IV: GPS SITE MAPS SSDI Communications Activity Performance Evaluation 58 SSDI Communications Activity Performance Evaluation 59 SSDI Communications Activity Performance Evaluation 60 SSDI Communications Activity Performance Evaluation 61 SSDI Communications Activity Performance Evaluation 62 ANNEX V: COMMUNITY ACTION CYCLE Organize Community for Action Explore Health Issues & Set Priorities Plan Together Evaluate Together Act Together Prepare to Mobilize Prepare to Scale-up SSDI Communications Activity Performance Evaluation 63 ANNEX VI: DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Carol S. Shepherd Title Consultant Organization Self-Employed Evaluation Position? X Team Leader Team member Evaluation Award Number (contract or other instrument) AID 612-TO-16-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. X Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. Previous work experience with the Jhpiego Chief of Party SSDI Communications Activity Performance Evaluation 64 I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date August 7, 2016 Name Willie Kachaka Title Mr. Organization DevTech Evaluation Position? Research Analyst Evaluation Award Number (contract or other instrument) AID 612-TO-16-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. SSDI Communications Activity Performance Evaluation 65 I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date August 7, 2016 Name Iain McLellan Title SBCC Specialist Organization DevTech Evaluation Position? Team Leader X Team member Evaluation Award Number (contract or other instrument) AID 612-TO-16-00003 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) SSDI-Communication I have real or potential conflicts of interest to disclose. Yes No X If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 15. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 17. Current or previous work experience with an organization that 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. SSDI Communications Activity Performance Evaluation 66 may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 28 Aug 16 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523