USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION COMMUNICATION AND MALARIA INITIATIVE IN TANZANIA JUNE 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Dr. Willard D. Shaw, Dr. Carol Baume, and Dr. Alex Mwita through the GH Tech Bridge Project. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION COMMUNICATION AND MALARIA INITIATIVE IN TANZANIA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-01-053) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00004. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION i CONTENTS ACRONYM LIST ............................................................................................................................... iii EXECUTIVE SUMMARY ................................................................................................................... v I. INTRODUCTION ....................................................................................................................... 1 Overview of the Project ............................................................................................................. 1 Performance Evaluation Process .............................................................................................. 2 II. PROJECT IMPLEMENTATION ................................................................................................ 5 Achievement of Project Objectives ......................................................................................... 5 Key Achievements .....................................................................................................................13 Gender Approach ......................................................................................................................14 Communication Approaches ...................................................................................................15 Capacity-Building Impact ..........................................................................................................19 Use of Funds ................................................................................................................................21 Staffing Structure ........................................................................................................................22 III. PROJECT IMPACT ....................................................................................................................25 IV. RECOMMENDATIONS AND OBSERVATIONS ..............................................................31 General Recommendations and Observations ....................................................................31 Recommendations for Future Tanzanian Projects .............................................................35 Implications for PMI Worldwide ............................................................................................36 APPENDIXES APPENDIX A. SCOPE OF WORK ..............................................................................................37 APPENDIX B. PERSONS CONTACTED ...................................................................................47 APPENDIX C. REFERENCES .........................................................................................................51 FIGURES Figure 1: Exposure to Malaria Messages by Various Channels ...............................................29 Figure 2. Percentage of Net-owning Households in which all Individuals Slept Under a Net ......................................................................................................................29 Figure 3. Percentage of Female Respondents Reporting They Sought Treatment for a Child’s Fever within One Day of Onset...........................................................29 Figure 4. Percentage of Households in which All Children Under 5 Slept Under a Net the Previous Night, by Exposure [Lindi and Mtwara 2009 n=450] ..............30 ii USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION TABLES Table 1. Project Progress against Objectives ............................................................................... 5 Table 2. Community-level Activities .............................................................................................14 Table 3. All-Cause Under-Five Mortality DHS and THMIS 1992-2010 ................................25 Table 4. Key Malaria Behavioral Indicators DHS and THMIS 2004–2010 ...........................26 Table 5. Ifakara UCC Surveys 2010 & 2011 ...............................................................................26 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION iii ACRONYM LIST ACT Artemisinin Combination Therapy ANC Antenatal care BCC Behavior change communication CA Cooperative agreement CBO Community-based organization CCA Community change agent CM Case management COMMIT Communication and Malaria Initiative in Tanzania DED District executive director DHS Demographic and Health Survey DMFP District malaria focal person DOT Directly observed treatment FH Fund holder FP Family planning GFATM Global Fund to Fight AIDS, Tuberculosis, and Malaria GH Tech Global Health Technical Assistance Project IMCI Integrated management of childhood illnesses IRS Indoor residual spraying ITN Insecticide-treated mosquito net IPC Interpersonal communication IPT Intermittent preventative treatment IPT1, IPT2 Intermittent preventative treatment dose 1, dose 2 IPTp Intermittent preventative treatment during pregnancy Jhpiego Originally known as the Johns Hopkins Program for International Education in Gynecology and Obstetrics JHU Johns Hopkins University JHU-CCP Johns Hopkins University Center for Communication Programs LLIN Long-lasting insecticide-treated mosquito net M&E Monitoring and evaluation MFP Malaria focal person MIP Malaria in pregnancy MoHSW Ministry of Health and Social Welfare MVU Mobile van unit iv USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION NGO Non-governmental organization NMCP National Malaria Control Program PMI President’s Malaria Initiative PSI Population Services International RCC Rolling Continuation Channel (Global Fund) RDT Rapid diagnostic test RFH Regional fund holder RMO Regional Malaria Office RTI Research Triangle Institute SOW Scope of work SP Sulfadoxine-Pyrimethamine TCCP Tanzania Capacity Communication Project TDHS Tanzania Demographic and Health Survey THMIS Tanzania HIV/AIDS and Malaria Indicator Survey TRaC Tracking Results Continuously Tsh. Tanzania shilling UCC Universal Coverage Campaign USAID United States Agency for International Development VEO Village executive officer WEO Ward executive officer USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION v EXECUTIVE SUMMARY A performance evaluation of the Communication and Malaria Initiative in Tanzania (COMMIT) was carried out in Tanzania from April 23 through May 21, 2012, by a GH Tech Bridge Project consultant team consisting of three development professionals with extensive malaria, BCC, development, and Africa experience: Dr. Carol Baume, Dr. Alex Mwita, and Dr. Willard Shaw. The Communication and Malaria Initiative in Tanzania (COMMIT) is a five-year, $15-million President’s Malaria Initiative (PMI)-funded behavior change communication (BCC) project that was awarded on October 23, 2007, to the Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (JHU-CCP). Project partners include Population Services International (PSI), Jhpiego, Research Triangle Institute (RTI), and a large number of Tanzania non-governmental organization (NGOs). Its goals are to contribute to the reduction of malaria morbidity and mortality through 1) fostering correct use of insecticide-treated mosquito nets (ITNs) and associated behaviors, 2) encouraging correct treatment-seeking and case￾management behaviors, 3) emphasizing the importance of intermittent preventative treatment during pregnancy (IPTp) for all pregnant women, and 4) supporting acceptance of indoor residual spraying (IRS) in target areas. COMMIT works nationally via mass media and collaboration with other projects, while its rural communication initiative with NGOs and community change agents (CCAs) is active in only 11 regions in mainland Tanzania. A project implemented by PSI under a Global Fund Rolling Continuation Channel grant (RCC) runs the same rural communication initiative in the seven other rural regions. Although the focus is on rural districts, urban districts also benefit from mass media interventions and some of the other communication materials. COMMIT is a solid, well-run project that has met, and often exceeded, the requirements of the cooperative agreement and its annual workplans. An exception is its work with the private and commercial sector, which was limited because of developments beyond COMMIT’s control. COMMIT has developed and employed a wide range of communication materials and created a rural network of community-based organizations (CBOs) and local change agents that is highly valued by the communities it serves and presents a model for community action in Tanzania that should be strengthened and expanded. Despite being in the forefront of malaria research and ITN social marketing since the late 1990s, Tanzania had never before had a communication program that implemented an integrated malaria approach using multiple and complementary channels at the individual and system level. The profile of malaria has been raised considerably in communities and the health system. Millions of Tanzanians, particularly in rural areas, are now more knowledgeable about malaria and the key interventions for malaria prevention and treatment. COMMIT has been very active in creating and disseminating a wide range of approaches and materials on various aspects of malaria prevention and treatment (long-lasting insecticide-treated mosquito nets (LLINs), intermittent preventative treatment (IPT), case management, IRS and creatively using multiple channels to promote optimal behaviors. COMMIT worked through the National Malaria Control Program’s (NMCP) BCC working group on the conceptualization, refinement, testing, revision, and production of all its communication materials in accordance with the Communication Strategy for Malaria Control Interventions 2008––2013, which it helped develop. COMMIT also helped other groups with their materials and strategies, including vi USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION development of all the media and print for the Red Cross Hang Up Campaign and the Zanzibar malaria program. COMMIT utilized: Mass Media: Resources included radio spots broadcast on a national and/or regional level; a malaria program for school children called PataPata which is inserted into children’s programming Saturday mornings with some CCAs forming listening groups of children; and a distance education program for CCAs that is now under development. Television and radio spots were developed based on interviews with well-known Tanzanians, including a previous president, talking about their personal experience with malaria. Chumo, an award-nominated, entertainment-education film was developed with malaria in pregnancy (MIP) messages. Print Materials: Throughout the life of the project, print materials were produced and distributed in large quantities, including a wide variety of brochures, pamphlets, posters, stickers, scratch cards, and job aids (e.g., flip charts, pregnancy wheel). At start-up, COMMIT immediately reproduced existing materials to support the ongoing program for treatment of nets using Ngao insecticide and improved Ngao (Ngao ya Muda Mrefu) then moved on to the promotion of LLINs and to developing new materials to support its rural communication initiative. Interpersonal Communication (IPC) Training for Health Workers: A Jhpiego-led activity sought to improve IPC skills among health facility staff through training in IPC and supervision and the provision of job aids (e.g., flip charts). This initiative provided training to 886 health workers and supervisors who were then asked to pass on the results of their training to their colleagues. In addition, 200 nurse tutors from 63 midwifery schools were trained on IPC. Rural Communication Initiative: This was perhaps the largest and most significant of COMMIT’s communication strategies. A network of NGOs and volunteers at the regional, district, and ward levels was created with the task of raising community knowledge and action through at least four events per month and one-on-one interactions. This network consists of the following: An NGO in each region with good management capabilities serving as regional fund holder (RFH) to channel project funds from PSI to CBOs, advise on programming, and monitor and report on CBO performance and progress. CBOs at the district level to supervise, support, and work with the CCAs at the ward level based on a quarterly plan of activities developed with the CCAs and local officials and submitted to the RFH for approval and funding. CCAs selected at the community level by ward executive officers (WEOs), village executive officers (VEOs), local council members, and the district malaria focal person (MFP) using a process defined by COMMIT. Volunteers were generally between the ages of 20 and 35, lived and worked in the ward, were respected members of the community, and were willing to devote time to organizing at least four events per month in collaboration with the CBO and local leaders and providing counseling to families. They received a token honorarium of 10,000 Tanzanian shillings (Tsh.) per month, a bicycle to visit other villages in the ward, and a solar-powered radio to use with listening groups they formed. The four events organized by the CCAs each month generally followed an agenda determined by COMMIT that included group talks, village-level talks, school programs, home visits, and talks at health facilities. Bigger events such as sporting events and cultural shows were carried out by USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION vii the CBOs with the help of the CCAs. This network was supported by PSI regional offices that used its fleet of mobile van units (MVUs) to visit villages according to a negotiated schedule. Since the inception of the project, more than 27,000 events (including MVUs, road shows, cultural shows, school campaigns, etc.) have been produced, reaching an estimated 12,691,200 people. The current COMMIT structure consists of 11 RFHs, 69 CBOs, and 1,200 CCAs. In meetings with RFHs, CBOs, CCAs, and local administrative and public health officials at the regional, district, and village level in Coastal, Mtwara, and Mwanza regions, the evaluation team was impressed with the level of collaboration, spirit, and commitment shown by all involved. Everyone involved in this network seems to have emerged the better for it. CCAs were now seen as local malaria experts. The CBOs and RFHs acquired expertise and experience in malaria, improved their management and financial systems, and increased their potential as partners for other donors. The regional and district MFPs greatly appreciated the role of CCAs in reaching into the communities. Government and civil society were collaborating in a productive and mutually appreciative way. The creation of this network has provided the malaria program (and other health programs) with an excellent infrastructure for the promotion of health activities at the community level. PSI has adopted this structure for the seven regions in which it is operating with Global Fund support, thus achieving harmonization of strategy across the 18 regions. COMMIT has had other significant capacity-building impact. It has worked closely with the NMCP to finalize the National Communication Strategy for Behavior Change Communication 2007–2014 and been a valuable member of the NMCP’s BCC working group. It has provided training to key staff members of NMCP. All district MFPs in the 11 regions participated in the training of trainers program in preparation to roll out COMMIT activities in their districts. In summary, COMMIT has done a very good job of producing and disseminating a wide variety of communication materials and creating a rural communication network that has reached many people. It has been an excellent partner with the NMCP and is highly valued by all groups contacted by the evaluation team. The general consensus is that COMMIT has done a very good job and made a significant contribution to the fight against malaria. At the village level, CCAs, VEOs, and WEOs are adamant that villagers are better equipped to avoid malaria because of the efforts of the project and strongly feel that their activities should continue until malaria is eliminated. One of the tasks of the evaluation team was to try to identify the impact of BCC efforts on changing key malaria behaviors; however, the way that USAID included the monitoring and evaluation (M&E) function in the Request for Application does not yield the precise answers that it is now seeking at the end of the project. Furthermore, the original Performance Monitoring Plan (PMP) submitted as part of the proposal was amended at the request of USAID in Year 1 to eliminate data collection on changes in behavior and increase reliance on the use of data from already planned surveys like the Tanzania HIV/AIDS and Malaria Indicator Survey (THMIS) and Demographic and Health Survey (DHS). Those surveys, however, were not geared to assess COMMIT’s impact. COMMIT’s M&E component was primarily set up to track process indicators, such as numbers of materials produced and distributed, events held, and people reached and trained, and intermediate indicators, such as exposure, knowledge, attitudes, self￾efficacy, and norms. In looking at some of the M&E done by COMMIT and others, the team saw weaknesses in the questions or implementation process that limited their usefulness. There was no single person in charge of M&E; it was handled by staff in Dar es Salaam and Baltimore, who viii USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION were splitting time among other projects, and by partners PSI and Jhpiego. It did not produce consistent, quality data that the evaluation team could use to make confident statements about the link between project activities and behavioral outcomes. In the best of cases, precisely delineating the impact of BCC is a difficult process. Based on three decades of implementation in multiple countries, it is generally accepted that BCC is an essential, but not sufficient, component of any public health initiative. It is not enough to provide commodities and services; one also has to educate the public about their use and importance and build a continuing demand for them. National data clearly show that there has been a reduction in malaria morbidity and mortality, that use of LLINs has greatly increased, and that there has been progress in getting artemisinin combination therapy (ACT) to febrile children promptly but minor progress in increasing IPTp rates. Given the paucity of focused data, however, the evaluation team cannot make a definitive judgment on the extent to which COMMIT’s BCC activities contributed to the positive changes achieved. Common sense, consistent testimony from people on the ground, and some data from M&E strongly suggest that the millions of print materials, thousands of radio spots and community-level events, and hundreds of CCAs have had a significant impact on the malaria behaviors of many Tanzanians. In conclusion, the evaluation team feels that COMMIT was a strong, energetic, and well managed project that appears to have played an important role in raising awareness and knowledge of the desired malaria behaviors among a significant portion of the rural population, but the team cannot define its level of impact or provide a precise assessment of which communication strategies and materials worked the best (a tricky issue at best in that materials and channels are meant to be complementary and mutually reinforcing). It appears that the PMI investment served its purpose and, in the process, developed a model for rural communication through NGOs that could be replicated other places; but future projects should be designed with an assessment component that obtains all the answers to the questions PMI wants answered. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 1 I. INTRODUCTION OVERVIEW OF THE PROJECT The Communication and Malaria Initiative in Tanzania (COMMIT) is a five-year, $15-million, PMI-funded malaria behavior change communication (BCC) program awarded on October 23, 2007, to the Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (JHU-CCP). Partners on the project include Population Services International (PSI), Jhpiego, and a large number of Tanzanian non-governmental organizations (NGOs). Initially Research Triangle Institute (RTI) was a COMMIT partner, however, it received only BCC support and no COMMIT funds in the early years and became a separate initiative when it was awarded a United States Agency for International Development (USAID) indoor residual spraying (IRS) contract in 2010. The COMMIT project contributes to USAID’s strategic objective (SO) 11, “Health Status of Tanzanian Families Improved.” Primarily, the COMMIT project responds to intermediate result (IR) 11.1, “Communities empowered to practice key behaviors and use services for target health problems” (specifically malaria prevention and control). It will achieve this objective by directly promoting specific behaviors—insecticide￾treated mosquito net (ITN) use and proper treatment-seeking behaviors (intermittent preventative treatment (IPT) and prompt, correct treatment when ill). The National Malaria Control Program (NMCP) Communication Strategy and the President’s Malaria Initiative (PMI) set ambitious goals—reducing malaria morbidity and mortality by 50% by 2010 (NMCP) and increasing coverage of key interventions to 85% by 2010 (PMI). These goals rest on the assumption that household and individual behaviors related to the acquisition and use of malaria control commodities can be changed, and changed rapidly. The COMMIT project unfolded in stages as activities were started up in different regions against a backdrop of other donor activities, such as the mass distribution of long-lasting insecticide￾treated mosquito nets (LLINs). In most regions, activities have only been implemented over the last two years. Major Events FY08 FY09 FY10 FY11 FY12 COMMIT officially starts X Mtwara and Lindi regions are activated XXXX XXXX XXXX XXXX Ruvuma and Rukwas are activated XX XXXX XXXX XXXX Morogoro, Mara, Shinyanga, Mara, Mwanza, Kagera, and Coast are activated XXXX XXXX XXXX Mass distribution of LLINs U5CC UCC Major surveys: DHS‘05 THMIS ‘07/08 DHS THMIS 2 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION PERFORMANCE EVALUATION PROCESS This performance evaluation was carried out in Tanzania from April 23 through May 21, 2012, by a GH Tech Bridge Project consultant team consisting of three development professionals with extensive malaria, BCC, development, and Africa experience: Dr. Carol Baume, Dr. Alex Mwita, and Dr. Willard Shaw. The evaluation process consisted of multiple stages: Briefing Stage of reading background materials on the project (e.g., annual workplans, quarterly reports, cooperative agreement (CA), etc.). Goal Clarification Stage in Dar es Salaam when the team met with USAID staff to review the purposes, objectives, and key questions listed in the team scope of work (SOW) and ensure that all had the same understanding of the evaluation’s goals. Planning Stage of soliciting input into creating a list of the people to be interviewed; the program interventions and materials to be reviewed; and the field trips that might be made to incorporate feedback from implementers at all levels. A draft outline of the proposed evaluation report was provided to the Mission to once again ensure the team was addressing the Mission's needs and that there was agreement on the evaluation content. Information-Gathering Stage which consisted of the following: Interviews in Dar es Salaam with members of the COMMIT project (JHU-CCP, Jhpiego, and PSI), USAID, NMCP, Centers for Disease Control and Prevention, Mennonite Economic Development Associates Red Cross, and others that had worked with the project. A list of people contacted is in Appendix B. Topic guides were drawn up for each type of implementer and stakeholder to make sure that key topics were addressed in each interview. A review of all the major communication products developed by COMMIT and the process used to create them; the training materials and approaches used; and the structure of the project and flow of decision-making, funding, and quality control. Field interviews in three geographically spread-out regions with heavy malaria (Coastal, Mtwara, Mwanza). Notification and permission letters were sent to the regions by NMCP a week prior to field travel. To make the best use of limited time and gather feedback from a sufficient number of implementers, the team asked that each district within the region send a representative from the participating community-based organization (CBO) (usually the head) to the regional capital for a meeting. The first day of the field visit was in the regional capital, beginning with courtesy visits to the Regional Malaria Office (RMO) official and district executive director (DED) followed by a half-day group meeting with the malaria focal persons (MFPs) from the Ministry of Health and Social Welfare (MoHSW), the regional fund holder (RFH), the CBO representatives from each district in the region, and 1–2 PSI staff members. There were 10–15 people at each of those meetings. On the following day, the team traveled to a ward to meet with the “grassroots” implementers—community change agents (CCAs), village executive officers (VEOs), and ward executive officers (WEOs)—and to visit a clinic to inquire about IPC training and materials. A visit to the village of Bupoo in the Coastal region was also made for a group interview of the VEO, CCA, health worker, teacher, and 10 members of the community who had formed a drama group. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 3 Report Writing Stage in Dar es Salaam interspersed with interviews with other organizations and follow-up discussions with COMMIT staff to clarify issues, facts, and perceptions. A draft report was submitted to USAID followed by an oral presentation and discussion at the Mission. Report Finalization Stage was in Washington after getting feedback from USAID Tanzania and PMI and filling in any information gaps by Dr. Mwita in Tanzania. A presentation to PMI was made in Washington. 4 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 5 II. PROJECT IMPLEMENTATION ACHIEVEMENT OF PROJECT OBJECTIVES Overall, the COMMIT Project has done an excellent job of meeting or exceeding almost all of the objectives in the cooperative agreement (CA). The table below summarizes what COMMIT accomplished under each objective in the CA as stated in its annual implementation plans and quarterly reports. Table 1. Project Progress against Objectives Objective 1: To engage communities and individuals with common sets of messages and information through a variety of mutually reinforcing sources, including mass media, rural community outreach, and community-initiated approaches. 1 Summary of Activities Accomplished: 1.1 Strengthening of the BCC cell within NMCP One-month training in Boston for the head of BCC cell on Leadership and Strategic Health Communication. One-month in-country training for three NMCP staff on Leadership and Strategic Health Communication (A visiting lecturer came to Tanzania from Boston.). 1.2 Completion of the National Malaria Communication Strategy This is a strategic guide on key malaria messages. 1.3 BCC working group with members from NMCP, COMMIT, PSI, and other NGOs Working group met regularly to craft and harmonize malaria messages and pretest them before dissemination via print and electronic media. Outputs: Four NMCP staff with leadership and strategic health communication knowledge and skills. A wide range of communication materials was produced, including: – Brochures on how to hang and repair a net, artemisinin combination therapy (ACT) dosing, IPTp regimen – Pregnancy wheel – Malaria in pregnancy movie (Chumo) – IPC flip charts – CCA flipcharts – Scratch cards Remarks: More than 17 print materials were developed. 6 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Table 1. Project Progress against Objectives Objective 2: To provide powerful community-based BCC strategies to change rural household malaria behaviors through community dialogue and community-initiated action, extended and supported by external rural engagement activities. 2 Summary of Activities Accomplished: 2.1 Community dialogue and actions on malaria through the work of the CCAs who engaged communities with malaria messages through one-on-one meetings, family talks, and group talks. 2.2 With support from CBOs, the CCAs arranged events (e.g., football matches, dramas, cultural shows, etc.) that attracted scores of people, even from nearby villages. 2.3 The Rural Engagement Teams operated nationally, beyond the 11 COMMIT regions. PSI’s mobile vans and road shows, contracted to private commercial companies, delivered malaria messages via film and interpersonal communication. Outputs: A network of RFHs, CBOs, and CCAs in the 11 regions consisting of: – 11 RFHs, – 65 District CBOs – 1,200 CCAs The number of people reached with information, education and communication BCC activities for insecticide-treated nets (ITNs), indoor residual spraying (IRS), intermittent preventative treatment during pregnancy (IPTp), (ACTs), and rapid diagnostic tests (RDTs): – FY 2008—1,360949 – FY 2009—3,190,581 – FY 2010—3,670,404 – FY 2011 Target—3,000,000; Reached—3,871,065; Achievement— 129% – FY 2012 Target—1,800,000; Reached—943,000 to date; Achievement—52% Remarks: The dropout rate for CCAs has been very low, and many of the original volunteers continue to work for COMMIT. To date, only two RFH changes were made to improve management. Lindi and Mtwara were too large to be covered by one RFH, so a second RFH was recruited. In Ruvuma, the logistics of having an RFH based in a district almost 100km away from the regional center proved too challenging. All changes were made smoothly. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 7 Table 1. Project Progress against Objectives Objective 3: To sub-contract with community and faith-based organizations to support CCAs and scale up community-initiated activities within districts. 3 Summary of Activities Accomplished: 3.1 COMMIT selected and trained RFHs and CBOs. The selection was on merit, based on a detailed application process. Competitive bidding in all 11 regions ensured transparency in the selection process. The RFHs provided oversight to district-based CBOs, channeled funds from COMMIT via PSI to CBOs, and collected district implementation reports from CBOs for onward transmission to COMMIT. Regular meetings were held between the RFHs and CBOs to discuss implementation issues. CBOs coordinated and supported activities of ward-based CCAs and held monthly meetings with them. CBOs conducted CCA refresher training with assistance and oversight of PSI and district malaria focal persons (DMFP). Outputs: Refer to Objective 2 outputs. Remarks: The evaluation team met RFHs from Coast, Mtwara, and Mwanza Regions. They appeared to be capable, credible, and committed organizations. All of them appreciated that their association with COMMIT had improved their managerial and financial capabilities and built their malaria expertise. Some noted that they had attracted funding from other donor organizations for work beyond malaria because of their COMMIT experience. Objective 4: To identify, train, and equip community members to act as change agents in stimulating the community dialogue and community-initiated action process to change rural household malaria behaviors. 4 Summary of Activities Accomplished: 4.1 The community in consultation with the WEOs identified CCAs. These are trusted men and women, residing within the community and working as peasants, tradesmen, etc. The majority of them were village health workers or had previous exposure in health-related projects. Each CCA was designated to work in four to eight villages constituting a ward. 4.2 The CCAs received a four-day training on malaria organized and conducted by the district CBOs and were assigned to be messengers for malaria behavioral change in their wards. 4.3 The CCAs were provided with flip charts, solar-powered radios, and reference guides as basic tools for their work and bicycles for commuting from village to village. 8 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Table 1. Project Progress against Objectives 4.4 Their responsibilities involved conducting a minimum of four malaria events per month. They used opportunities like family talks, village meetings, cultural events, and school and health facility visits to reach groups of people to deliver malaria prevention and treatment messages. 4.5 With funding and planning support from the CBOs, CCAs organized sports events (football) and cultural shows, which attracted the youth and other villagers and used those opportunities to deliver malaria messages. 4.6 They visited schools to give lectures on malaria to students. Each Saturday they carried solar-powered radios to schools where students who enrolled in malaria clubs came to listen to a radio malaria program (PataPata) followed by discussions. 4.7 They counseled parents and families on malaria prevention and treatment. In some instances, they succeeded in convincing community members that convulsions (degedege), which is a common manifestation of cerebral malaria in children, is not due to supernatural events and that health facilities provided superior care for convulsions compared to traditional healers, an achievement which saved lives. 4.8 Eventually, the community accepted them as true malaria change agents and regarded them as “malaria experts.” In Magu district in Mwanza, the ward leadership was so impressed with their work that they decided to include the CCAs in the Ward Development Council meetings and other activities. 4.9 CCAs did all this for a modest monthly stipend of 10,000 Tsh. (US$6.40). 4.10 The CBOs provided support and oversight to the work of CCAs, organized refresher courses, and helped organize and pay for events (sports, cultural shows, etc.). They also collected reports and forwarded them to the RFH. Outputs: In the 11 regions where COMMIT was working, the CCAs received: Solar-powered radios CCA flip charts Bicycles Numerous print materials on malaria interventions Remarks: The evaluation team held half-day meetings with RFHs, CBOs, and MFPs in the regional centers and half-day meetings with CCAs, VEOs, and WEOs in the district centers of Mkuranga district (Coast region), Mtwara district (Mtwara), and Magu district (Mwanza). The discussion centered around the work done to date, their opinions of the communication materials and channels, challenges and rewards of their jobs, how to improve the program, problems with promoting the key malaria behaviors, and other issues pertaining to their work. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 9 Table 1. Project Progress against Objectives All CBOs and MFPs in the three regions visited gave high marks to CCAs and their work. Without exception, they said the use of CCAs was the most effective way of reaching the rural community with malaria messages. In some cases, MVUs or radio was cited second. The CCAs, themselves, remarked that their effectiveness was due to the trust they enjoyed from community members. Overall, participants in all districts felt that the combination of different communication methodologies was important for effective message delivery. All participants agreed that there were noticeable changes in the behavior of the community in embracing malaria interventions. They said that many community members had a better understanding of malaria, and many sought early treatment for fever, used ITNs, and used sulfadoxine-pyrimethamine (SP) for IPTp when pregnant. Acceptance of IRS, which is implemented by RTI in the Lake regions (Kagera, Mwanza, and Mara), was high; although, CCAs said it had been a hard sell because of misunderstandings, including that IRS caused impotence among males. The CCAs were emphatic that a majority of their community members understood malaria prevention and treatment messages, and there were fewer malaria cases and deaths in their communities compared to before the COMMIT project. All CCAs argued that it was important for this project to continue because malaria is yet to be eradicated. In addition, halting current efforts would make people lapse into their old habits. All CCAs said their main challenge was transportation. Reaching all villages in their wards via bicycle, particularly during the rainy season, was difficult. Some felt the stipend should be raised as it was not much help for them and their families, and they sometimes had to spend it for transportation. The CBOs spoke of inadequate funds and said they could not supervise effectively because of it. They also wished to have transportation to help them with district supervision activities. Some wished to have computers to help them with report writing, communication, etc. The CBOs noted that COMMIT had not provided them with a single piece of equipment. Objective 5: To expand coverage of key messages via a variety of local entertainment events, complementing and supporting change agents and community-initiated action. 5 Summary of Activities Accomplished: 5.1 Local entertainment events such as sports and cultural gatherings were leveraged by CCAs for malaria communication. Outputs: FY 2010 as an example: – CBO/Mid-Media events planned and performed Cultural shows 3000 1459 10 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Table 1. Project Progress against Objectives Sports events 1000 1372 School events 4000 6671 Mobile cinema 600 1318 – CCA events performed: Individual IPC- 4630 Group IPC 7268 Population reached with mobile cinema, sports, cultural shows, group talks, school events, and public meetings: - 1,962,804 Males and 1,245,903 Females = Total 3,208,707 Remarks: In the three districts visited by the evaluation team, CCAs confirmed using local events for message delivery. In fact, this was their main mode for reaching hundreds of people at once compared to family or group visits. Fluctuations occurred in the number of certain categories of events planned and implemented due to changing circumstances in the area and unforeseen circumstances, including, weather, funding flow, etc. Objective 6: To develop a campaign with NMCP that builds on current promotions and demand creation to: Support and reinforce community-based BCC activities Raise awareness of malaria among general public, local government officials, and civil-society organizations Echo themes identified in the harmonized message guide 6 Summary of Activities Accomplished: 6.1 In February 2010, Malaria No More, in collaboration with NMCP, COMMIT, PSI, and other partners, launched a national campaign against malaria. The campaign, dubbed “Zinduka, Malaria Haikubaliki” (Wake Up, Malaria Is Not Acceptable), was inaugurated in Dar es Salaam by President Kikwete and broadcast live by leading radio and TV channels. Popular music artists entertained and were the center of attention, drawing a huge crowd. Speeches by the president and the U.S. ambassador and a special song from the artists highlighted the problem of malaria and emphasized what people should do to fight and defeat malaria. This single event popularized the Malaria Haikubaliki slogan so much that it became the talk of the country for months. The roll out of the Zinduka Malaria Haikubaliki campaign has reached 11 other regions (Mara, Tanga, Shinyanga, Lindi, Mtwara, Mwanza, Dodoma, Kagera, Tanga, Morogoro, and Ruvuma). The main follow-on activity after the launches has been the establishment of Zinduka clubs in schools. These campaigns have further raised public awareness on malaria and reinforced COMMIT’S community-based BCC activities. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 11 Table 1. Project Progress against Objectives 6.2 COMMIT implemented mass-media campaigns, including the following: Mid Media: Road shows and mobile video cinema in the rural areas. Saturday Children’s Radio Program: Aims at building a malaria prevention culture by inspiring children to actively participate in the fight against malaria by talking to their parents and sleeping under an LLIN. Two Minutes of Wisdom: Words of wisdom on malaria from prominent political leaders like former President Mwinyi, business leaders like Abraham Mengi, religious leaders like Polycarp Cardinal Pengo, and prominent artists. The key message is that people should take malaria seriously and should use preventive and treatment interventions. Net Norm: The campaign followed the universal distribution of LLINs and aims at building a net-use norm. The “goodnight” stickers produced by COMMIT and distributed by the Tanzania Red Cross Society during the campaign communicate the fact that a household with a sticker is a caring household with everyone sleeping under a net every night. Malaria-in-pregnancy movie Chumo communicates the dangers of malaria in pregnancy. It is shown through a mobile video van in the rural areas and is also commercially available for sale on DVD. Integrated Safe Motherhood Campaigns: Initially developed to address SP uptake issues, it has attracted other partners and combines themes including early antenatal care (ANC) attendance, malaria (net use and IPTp), prevention of mother-to-child transmission (PMTCT), birth planning, and other health pregnancy messages. Radio Distance Learning Campaign: The program was designed to address the prohibitive cost of refresher training for CCAs. COMMIT teamed with the USAID￾funded Tanzania Capacity Communication Project (TCCP) to develop a comprehensive half-hour radio distance learning program that covers malaria, HIV, family planning, etc. Volunteers learn about communication and mobilization skills and how to apply those skills to a health topic. Malaria and the Media: COMMIT has worked very closely with the media. It has held three media forums, helped create the Journalists against Malaria Network, and, for the Media Council of Tanzania, supported the malaria category for Print and Radio awards that cover 20 different categories. Outputs: Refer to achievements table. Remarks: The malaria campaigns have helped raise the profile of malaria among the highest￾ranking government officials and the general public. 12 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Table 1. Project Progress against Objectives Objective 7: To implement a sustainable plan for improving health provider interpersonal communication (IPC) skills to effectively transmit messages on prevention and case management (CM) of malaria in a manner that ensures client comprehension and adherence. 7 Summary of Activities Accomplished: 7.1 COMMIT had a component for strengthening the malaria interpersonal communication (IPC) skills of health workers. Implemented by Jhpiego, this component promoted IPC skills relevant to counseling in clinics. Every health facility in the project areas had a health worker trained in malaria communication. Flip charts were issued to health facilities as job aids to help make clients more knowledgeable about malaria and more likely to adhere to advice from health providers. The trainees were urged to train colleagues when they went back to their health facilities. For sustainability, this component (funded through Jhpiego/Mothers and Infants, Safe, Healthy and Alive also included training of tutors from nursing, medical, and para-medical training colleges who will impart communication skills to future health care providers. Outputs: Year 1: Baseline client exit interviews to assess quality of client/provider interactions. Year 2: Development and translation of communication materials into Kiswahili. Year 3: Other outputs: Distribution of malaria-in-pregnancy brochures, reminder cards, pregnancy wheels, supportive supervision of 32 health facilities in nine regions. Remarks: The evaluation team interviewed two health providers in Mtwara and Magu districts. Both of them demonstrated the use of the flip chart. In the study carried out to assess the impact of the IPC training, the exit interviews with clinic clients did not differentiate between women counseled by trained health workers vs. untrained health workers. Therefore, this was a lost opportunity to gather good information. Objective 8: To increase collaboration and cost-sharing activities between the private sector and the project in order to increase overall coverage with key messages and positive behavior reinforcement through: Increased coordination of stakeholder malaria activities Increased allocation of district health funds and attention to malaria activities Increased focus on malaria by high-level national officials Increased involvement of the business community in malaria activities USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 13 Table 1. Project Progress against Objectives 8 Summary of Activities Accomplished: 8.1 COMMIT has used different approaches in sensitizing, networking, and partnership building for malaria control, including the following: Active participation in World Malaria activities in 2008, 2009, 2010, and 2011 Participation in International Trade Fair and Farmer’s Day in 2008, 2009, 2010, and 2011 Advocacy meeting with district medical officers in 2008 Training of Journalists in 2009, 2010, and 2011 Dissemination of Malaria Communication Strategy in 2010 The 50th Independence Anniversary 8.2 At the district level, the district health authorities lauded the contributions made by CCAs in helping communities become more knowledgeable on malaria and increasing the use of ITNs, IRS, IPTp, and ACTs provided by the government and other stakeholders. 8.3 In recognition of the CCA’s contributions, Magu district will be allocating funds in their annual plans for CCA activities. 8.4 Coca-Cola and a few other companies have, at different times, contributed funds for ITNs. These were distributed by PSI to disadvantaged groups in Dar es Salaam. 8.5 Support from Lottery Canada ensured that all secondary schools were provided with LLINs. Remarks: The commercial component was the weakest of all. Because of the government’s decision to exclusively purchase polyethylene nets, three of the four Tanzanian polyester net manufacturers ceased producing nets, including one company that had installed a capacity to produce polyester LLINs. A number of people noted that LLINs were not available for sale. A comparison of the annual workplans and quarterly reports shows that COMMIT had successfully implemented all of its proposed activities. The first year was used for preparatory work and setting up the program in consultation with NMCP and other implementing partners, strengthening of the BCC working group, selection of the regions where the project would commence, and travelling to districts to introduce the project to the district and regional leadership. Recruitment and selection of RFHs and district CBOs for the first seven regions was completed. From the second year onward, the project rolled out to cover 11 regions and 69 districts. KEY ACHIEVEMENTS 1. COMMIT’s extensive delivery of malaria messages by deploying multiple channels is unprecedented in the history of malaria control in Tanzania. There is no doubt that a majority of Tanzanians are more knowledgeable about malaria and the interventions for malaria prevention and treatment. 14 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 2. COMMIT has created a well-functioning network of local NGOs and CBOs working together on malaria in collaboration with health officials. Before COMMIT, hardly any local NGO or CBO was working on malaria, the leading cause of infant and childhood morbidity and mortality in Tanzania. The NGOs and CBOs have now developed competence in malaria programming and have strengthened their organizational capacity and skills that enable them to work at a more professional level and, in some cases, attract funding from other donors. 3. COMMIT’s work has been an eye-opener to the district and ward authorities in terms of the power CCAs wield as catalysts for behavioral change. Some districts are now planning to budget for their work and to use CCAs to reach where they are unable to reach. 4. MFPs state that there has been a decline in malaria cases and deaths and that COMMIT, and the CCAs in particular, played an important role in that decline through their work in sensitizing communities to adopt malaria interventions. 5. A wide variety of malaria communication materials have been produced and disseminated. 6. The visibility of malaria as a critical health problem has been established from the top leadership to the village level. There is strong political support involving the head of state, who is also the outgoing African Leaders Malaria Alliance (ALMA) chairperson. 7. Thousands of community-level activities have been planned and implemented that have carried malaria messages to millions of people as shown in the table below. These numbers are cumulative for all regions, where implementation time varies from three to four years. Table 2. Community-level Activities Type of Event Avg. pop. size per event Number of Events Total Number of events Estimated Number Yr1 Yr2 Yr3 Yr4 Yr5 reached MVU shows 1,200 300 600 600 420 360 2,280 2,736,000 Road Shows 1,500 200 400 400 n/a n/a 1,000 1,500,000 School Campaign 280 1,800 2,880 2,880 2520 1,000 11,080 3,427,200 Sports 1,800 300 480 480 420 360 2,040 3,672,000 Cultural shows 200 1,080 1,440 1,440 1,080 500 5,540 1,152,000 Meetings 100 300 480 480 420 4,000 5,680 204,000 Total 5,080 3,890 6,280 6,280 4,440 6,220 27,620 12,691,200 GENDER APPROACH COMMIT addressed gender issues on a number of levels: 1) diversity of core staff to allow for a variety of perspectives, 2) attention to gender issues among target audiences and through formative research, 3) development of materials with messages for both genders, 4) the participation of all genders in project activities as participants and as audiences, and 5) a multichannel approach that aimed at reaching gender-specific audiences with key messages. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 15 The COMMIT core staff is diverse with 13 males and 11 females serving as full or part-time staff. Female staff members played the leadership role in the work with clinics, where females form the largest part of the work force. The interpersonal training reached 564 health workers, including 424 females and 141 males, with a focus on how to counsel mothers and bring fathers into the process and into the antenatal care clinic. The IPC flip chart featured men as partners in facilitating health pregnancy outcomes and highlighted the need for the husband to come with his wife to the clinic to better understand how to prevent malaria during pregnancy. Illustrations in other materials depict both genders. At the community level, malaria activities were largely carried out by volunteer CCAs who were selected by local leaders for the role of providing malaria information to an average of six to eight villages in a ward. Of the 1,200 CCAs recruited at the village level, the male/female split was approximately 66/34. Both female and male CCAs were in agreement that their genders did not hinder their ability to provide information to counsel men, women, or children. In some project areas, there was a matriarchal bias that allowed women to play a lead role in home and financial decision-makings. In other areas, while women were able to make many of the health decisions for themselves and their families, they needed to discuss health actions (e.g., going to a clinic) with their husbands and to obtain money for transportation. In those cultures, the CCAs spent time educating husbands to be supportive of the health needs of their pregnant wives and young children. In fact, CCAs saw their roles as educating the whole family, and during home visits they tried to reach husbands, wives, and children together with their malaria information and counseling. Radio messages also engaged men with the theme of providing moral and financial support for their wives to prevent malaria. Although COMMIT did not have a written gender strategy, gender was always integrated into its thinking since the key interventions largely focused on women. Women running households had to deal daily with the hanging and care of nets. COMMIT print materials depicted the husband taking the lead in hanging up the net but most often showed the pregnant woman and young child under the net. In the case of IPT, families were targeted with messages about the importance of making an early visit to the clinic to receive SP and returning for a second dose. The goal was to make both husbands and wives cognizant of the need for a pregnant woman to get to a clinic early. The same process was involved in the promotion of early treatment of malaria. Both mothers and fathers were targeted with messages to get children with a high fever to a clinic as soon as possible. In the case of IRS and, to some extent, LLINs, the project faced a belief in some parts of the country that the spraying of houses or sleeping under a treated net could lead to male impotency. A radio spot was created of two men discussing the issue and going to a doctor to get his educated viewpoint. At the community level, CCAs sometimes found the resistance to IRS spraying to be an obstacle, and they focused on persuading males that IRS would have absolutely no effect on their potency. COMMUNICATION APPROACHES COMMIT utilized a systematic, multichannel approach based on its Pathways Model for behavior change. It created a wide range of communication materials to reach various audiences with a variety of malaria messages and calls to action. Each material seems to have followed a systematic process of developing a creative brief stating its purpose and intended audience, draft preparation and review (with NMCP’s BCC working group), pretesting, revision, production, 16 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION distribution, training of users when needed, and periodic monitoring. The communication interventions developed and implemented by COMMIT followed multiple tracks: Mass Media: Radio spots broadcast on a national and/or regional level; a malaria program for school children called PataPata, which is inserted into children’s programming Saturday mornings with some CCAs forming listening groups of children; and a distance education program for CCAs that is now under development. Two-minute television and radio spots were developed based on interviews with well-known Tanzanians, including a previous president, talking about their personal experience with malaria. Chumo, an entertainment film, was developed by COMMIT to be launched with a malaria-in-pregnancy (MIP) campaign; however, this MIP campaign turned into an integrated Safe Motherhood Campaign that has been collaboratively developed with the TCCP. The movie was commercially launched, broadcast on television, and made available for sale on DVD by a local distributor, with reported sales of more than 25,000 DVDs and VCDs. The movie has been nominated for awards in the Zanzibar International Film Festival (where it won best director and best actress) as well as in the African Academy Awards, the Pan African International Film Festival, and the Durban International Film Festival. Print Materials: A wide variety of brochures, pamphlets, posters, stickers, scratch cards, and job aids (e.g., flip charts, pregnancy wheel) were produced and distributed in large quantities over the life of the project. At start-up, COMMIT immediately reproduced existing materials to support the existing program for treatment of nets using Ngao insecticide and improved Ngao, then moved on to the promotion of LLINs) and the development of new materials to support its rural communication initiative. All materials were submitted to USAID for branding approval. Interpersonal Communication Training for Health Workers: A Jhpiego-led activity sought to improve IPC skills and supervision among health facility staff through training in IPC and supervision and the provision of job aids (e.g., flip charts). This initiative provided training to 886 health workers on basic IPC skills and 559 on IPC supervision skills. Health providers were asked to pass on the results of their training to their colleagues. Rural Communication Initiative: This was, perhaps, the largest and most significant of COMMIT’s communication strategies. A network of NGOs and volunteers at the regional, district, and ward levels was created with the task of raising community knowledge and action through at least four events per month and one-on-one interaction for creating positive behavior change. COMMIT partner PSI was tasked with creating a network that consisted of: An NGO in each region with good management capabilities serving as RFH to channel project funds from PSI to CBOs and to supervise CBO performance. CBOs at the district level that were to supervise, support, and work with CCAs at the ward level based on a quarterly plan of activities developed with the CCAs and local officials and submitted to the RFH for approval and funding. CCAs who were selected at the community level by WEOs, VEOs, local council members, and district MFPs using a defined process set out by COMMIT. These were volunteers generally between the ages of 20 and 35 who lived and worked in the ward, were respected members of the community, and were willing to devote time to organizing at least four events per month in collaboration with the CBO and local leaders and providing counseling to families. These volunteers received token honoraria of Tsh. 10,000 per month (a modest government salary would be 150,000/month), a bicycle to visit other villages in the ward USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 17 (generally four to eight villages per ward), and a solar-powered radio to use with listening groups they formed. Communication materials were provided to the CCAs via the RFH￾CBO network. Monthly meetings of CCAs were organized by the CBOs to share successes and problems and make future plans. This structure was supported by the PSI office in the region that also operated a fleet of MVUs that visited villages according to a schedule developed in collaboration with regional and district officials and the RFH, CBOs, and CCAs. Advance announcements by a PA system were used to attract an audience. The MVUs showed malaria-related, public health films in the evening in villages, thus, attracting large crowds for this novel educational entertainment. Question-and￾answer sessions were held following the playing of the film or video. The four events organized per month generally followed an agenda determined by COMMIT that included group talks, village-level talks, school programs, home visits, and talks at health facilities. Bigger events were carried out by the CBOs with the help of the CCAs. These included cultural shows such as dance, theatre and sports events such as soccer matches that would attract teams and fans from multiple villages. At halftime, CCAs would deliver malaria messages and print materials. With the launch of the PataPata radio program for children, CCAs and other community members such as teachers organized listening groups that gathered to hear the broadcast and then discuss its content. Since the inception of the project, over 27,000 events (including MVUs, road shows, cultural shows, school campaigns, etc.) were carried out, reaching 12,691,200 people. The current COMMIT structure consists of 11 RFHs, 69 CBOs, and 1,200 CCAs. In meetings with RFHs, CBOs, CCAs, and local administrative and public health officials at the regional, district, and village level in Coast, Mtwara, and Mwanza regions, the evaluation team was impressed with the level of collaboration, spirit, and commitment shown by all involved. It appeared that very good relationships had been established, and all were working together as a team. All expressed great enthusiasm for the project and the work they have done to fight malaria. Each region reported occasional funding delays, and all CCAs cited the large geographical areas they had to cover (e.g., six to eight villages, some 8 kilometers away) as a challenge; however, all thought that the system was working well and that their relationships were strong and transparent. Everyone involved in this network seems to have emerged better for it. CCAs are now seen as local malaria experts with a heightened visibility and level of respect in their communities. The CBOs and RFHs, many of whom had been exclusively involved in HIV-AIDS work, had extended their expertise and experience into malaria, improved their management and financial systems, and increased their profiles as potential partners for other types of programs. Some had been involved in other malaria efforts (e.g., the Hang Up campaign), while others had secured additional external funding from other donors for nonmalaria initiatives largely based on their reputation as a COMMIT partner and an organization that had learned to manage according to USAID regulations. The regional and district MFPs greatly appreciated the role of CCAs in reaching into the communities. Government and civil society were collaborating in a productive and mutually appreciative way. The creation of this network has provided the malaria program (and health programs in general) with an excellent infrastructure for the promotion of health activities at the community level. Across the board, everyone was pleased with the CCA system—particularly that the CCAs were recruited locally unlike other health workers, VEOs, WEOs, etc., who are civil servants and likely from outside the local community. PSI has adopted this structure for the seven 18 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION regions in which it is operating with Global Fund support, thus achieving harmonization of strategy across the 18 regions. Impact of Communication Approaches It is clear that COMMIT had wide reach. COMMIT’s BCC Impact Study found that 69% of people reported hearing a malaria message on the radio, about 25% saw something on malaria on TV, about 25% had listened to PataPata in the prior six months, 19% had seen a road show, and 17% had attended a video event. CCAs also reached people: 22% reported attending a CCA event and 19% had received a home visit. The evaluation team has been asked by some people to identify which material or channel or activity has been the most effective. To a certain degree, this is a moot question, as effective BCC programs use multiple channels and materials to create synergy and an environment of reinforced messages. No single material or channel is a silver bullet, and different audience segments may be affected differently by different channels at different times. Nonetheless, in the regional meetings with RFHs, CBOs, and MFPs, the evaluation team asked what they thought was most effective, and then if they had to cut one piece of the communication program, what it would be. People always wanted to name several things as “most effective,” and CCAs along with mobile vans and radio were most frequently mentioned. In most cases, print materials were cited as the one channel they would reduce if forced to limit channels. Some CCAs liked the print materials because it gave them something to hand out, and they felt that people read them; however, many more thought that people did not pay attention to them or like reading them and said they often found some on the ground after events. This, however, is based on discussions with a very limited number of CCAs and CBOs, though it does match up with findings in other countries, where posters, billboards, and brochures do not register a great impact. They add to an environment of information flow but are not often cited by people as a key source of information. The tremendous amount of materials produced and disseminated by COMMIT did create a rich information base about malaria, but it is not clear if all of these materials were necessary and if they all made a contribution to behavior change. The evaluation team’s assumption is that some were more valuable than others in stimulating or facilitating action. The evaluation team discerned a lack of strategic decision-making as to precisely which material carrying which messages was needed at which intervention points, or how to allocate different messages among channels. To some extent, the approach seemed to follow the Pathways Model that was in the workplans for Years 3–5, to utilize as many channels as possible. There could have been more in-depth analysis of key behaviors at the outset to identify the specific barriers (conceptual, structural, and product-specific) and to strategize how to overcome them. For behaviors requiring interaction with a health product, it is important to see how the health product is used in the house or clinic, etc.; much can be gained by observation (e.g., of nets in homes, IPT administration in clinics, etc.) that does not come out of group discussion. For example, the problem of getting pregnant women to take at least two doses of SP has been one of the most intransigent, and it is not because women do not know they should take SP or are afraid of taking it. Several people thought that intermittent preventative treatment dose 2 (IPT2) might be underreported because of poor record-keeping. Health workers cannot find the documentation for a woman’s first visit, so they simply record her as new, and it shows up as a first visit; therefore, there might be some over-reporting of intermittent preventative treatment USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 19 dose 1 (IPT1) and underreporting of IPT2. Undoubtedly, very limited supplies of SP, mentioned in every region, are a major barrier that must be addressed quickly and completely. Because the current policy is directly observed treatment (DOT), other barriers such as a lack of drinking water or a cup might come into play. Another barrier is the long walk to the health facility— which sometimes means walking in extreme heat or rain while pregnant, caring for other young ones, or leaving behind duties in the home and field. If the whole behavioral context were understood, then a dialogue could take place with the health system and others involved, addressing how to get SP closer to home while still monitoring doses. Possibly, CCAs or other village health workers could provide second doses and record them on standard forms that would go to the health center. The same in-depth understanding is necessary for net use problems. In each region, multiple people mentioned that the nets were too small for their beds or that the single-size nets were easily torn while trying to stretch them over the bed frame. A quick check of four houses around one district meeting hall found one house with a net with a number of holes about the size of a nickel that the mother said were there when she opened the package, while another house had a six-month-old Olyset net with a three-foot tear made when trying to stretch the net around the bed frame. In the regional and district interviews, a number of people reported that villagers were concerned about the large mesh size (Olyset has 56 holes per inch versus 132+ for other leading LLINs) allowing mosquitoes to get into the net. Also, seeing mosquitoes resting on the net made people feel that the net was not effective. The extent to which tears, net size, shape, mesh size, and other factors prevent proper net use is not known but warrants systematic investigation. COMMIT did messages on net care and repair in response to these net problems and worked with CCAs to discuss options such as sewing the smaller nets together. CAPACITY-BUILDING IMPACT The CA mainly focused on the NMCP as the object of capacity building. The project, therefore, carried out a number of activities designed to increase the ability of NMCP staff to design and implement effective BCC. An initial step was to send the head of the BCC working group to the United States to take the Johns Hopkins University (JHU) course, Leadership in Strategic Health Communication. Given the satisfaction with this course, COMMIT then worked with other JHU staff to improve the course’s focus on malaria. The two-week course Leadership in Strategic Malaria Communication was run twice in Tanzania for DMFPs . The NMCP’s head of the BCC cell cofacilitated this course. All DMFPs received a CD with reference materials and a toolkit for planning, implementing, and evaluating district BCC programs. COMMIT also linked with TCCP, also run by JHU, to form a network of course alumni, who then held an alumni meeting to share their work experiences. COMMIT later teamed up with the Centre for Enhancement of Effective Malaria Interventions to conduct refresher training for district and regional MFPs with input from NMCP. The MFPs are provided with an action-planning module to use in their regions that is based on the NMCP’s Medium-Term Strategic Plan 2008–2013 and its Communication Strategy for Malaria Control Interventions 2008–2013. Other capacity-building activities with NMCP were carried out through collaborative efforts and on-the-job training. The COMMIT project director worked closely with NMCP staff on the refinement and finalization of their National Communication Strategy for Behavior Change Communication. COMMIT staff were active members of the BCC working group and involved 20 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION that unit, as required, in the review and elaboration of ideas for and drafts of communication materials that enhanced their hands-on experience with the conceptualization and creation of BCC materials. COMMIT’s approach was focused on developing skills in BCC planning and materials development but did not develop a systematic capacity-building plan that incorporated organizational development approaches aimed at strengthening the entire organization. This type of organizational development approach did not arise in any of the workplans that were approved by the Mission, so the focus on BCC skills must have been satisfactory. COMMIT’s mandate was to work in mainland Tanzania, but six members of the Zanzibar Malaria Control Program participated in Leadership in Strategic Health Communication courses. COMMIT helped the Zanzibar team develop the communication BCC plans for the BCC unit and was asked to develop the promotion and media for the Universal Coverage Campaign (UCC) there that distributed more than 700,000 nets in March 2012. Capacity was also built in IPC for health providers. Through the subcontract with Jhpiego, COMMIT conducted 19 IPC trainings that reached 565 health providers. These included 12 IPC supportive supervision trainings for antenatal care in-charges and district reproductive and child health coordinators (304 females and 11 males) from COMMIT districts and Jhpiego sentinel sites. Another seven trainings were run for district malaria integrated management of childhood illnesses (IMCI) focal persons and district nursing officers (120 females and 130 males). Following training, field support supervision was conducted in 32 health facilities in nine regions to observe the interaction of participants with clients, provide on-the-job support, and obtain feedback from providers on the use of the job aids provided to them. More supportive supervision is planned for April–July 2012 to reach 51 CCA-covered districts in the 11 COMMIT regions and 33 Jhpiego sentinel sites. The most impressive and innovative capacity-building output of the project, however, was the creation of the network of FHs, CBOs, and CCAs that carried out the major part of the rural communication initiative. The creation of this network was largely implemented by PSI, who carried out the following key steps of the process as set forth in the proposal submitted by JHU: 1. Recruiting RFHs who would channel funds from JHU/PSI to CBOs, 2. Working with fund holders to recruit one CBO per district, and 3. Working with district and ward officials to select local residents as CCAs. COMMIT used a systematic, multistep, well-documented process of identifying and selecting RFHs by surveying the NGOs/CBOs working in each region with a particular focus on those with strong managerial and financial capabilities. A number of organizations were then asked to submit proposals for becoming the COMMIT fund holder for a given region. Interested organizations submitted proposals, which were reviewed by regional health officials who then selected the best organization. COMMIT then completed a CA with the organization and provided training in financial management and technical oversight. COMMIT worked with the fund holder to implement a similar approach to identifying one CBO per district that would oversee and work with CCAs to implement malaria activities every month. The CCAs were selected by the CBO in collaboration with local officials, including the VEOs, WEOs, and the district councils. In meetings with fund holders, CBOs, CCAs, WEOs, VEOs, MFPs, and district health officers at the regional, district, and village level in Coastal, Mtwara, and Mwanza regions, the evaluation USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 21 team was impressed with the level of collaboration, spirit, and commitment shown by all involved. It appeared that very good relationships had been established between the fund holders, CBOs, and CCAs and that they were working together as a team. All expressed great enthusiasm for the project and the work they have done to fight malaria. To some extent in each regional meeting, problems were reported on occasional funding delays, and all CCAs cited the large geographical areas they had to cover (e.g., four to eight villages, some 8 kilometers away) as a challenge; however, all thought that the system was working well and that their relationships were strong and transparent. In discussing the changes that had occurred to the CBOs and fund folders during the course of COMMIT, many of them noted the following: 1. The financial and management training provided by COMMIT had improved their ability to operate and implement programs. 2. They had gained experience in malaria work, whereas previously many of them were only focused on HIV/AIDS. 3. The COMMIT process had led them to develop stronger relationships with local officials (e.g., WEOs, VEOs, MFPs, clinics, health officials). 4. Participation in COMMIT and the broadening of their technical capacity provided them with an opportunity to broaden their work into other areas and activities (e.g., hang up campaign, other health programs). 5. The network of CCAs, who were recruited from the community, gave them stronger ties and heightened visibility in these communities. 6. Fund holders, especially, mentioned that their participation in a USAID-funded activity provided them with a level of legitimacy that made them attractive to other donors who were seeking local partners with reliable financial and management systems. The creation of this network of fund folders, CBOs, and CCAs has provided the malaria program (and health programs in general) with an excellent infrastructure for the promotion of health activities at the community level. Across the board, everyone was pleased with the CCA system—particularly that the CCAs were recruited locally unlike VEOs, WEOs, and other health workers, who are civil servants and often from outside the local community. USE OF FUNDS Initially, a five-year, $15-million project, COMMIT, by the end of its lifespan, will have received less than $14 million. As a result of a shortfall in PMI funding, some activities for Year 5 will be curtailed. This funding was spread among project operations and the design and implementation of activities in four malaria intervention areas: LLINs, IPTp, CM, and IRS, which was the lowest priority for COMMIT since the IRS project run by RTI did its own promotion. While it is difficult to identify the amount of funds used for each intervention, given that the project had an integrated approach and many cost elements benefited multiple interventions simultaneously, the team worked with COMMIT staff to get a general idea of the distribution of resources among the four intervention areas and the communication channels. It is important to understand that this “$15-million project” did not focus all its resources on a single technical area; thus, its impact must be understood in light of the level of resources applied to each component. In reviewing the expenditures to date by COMMIT, the staff have made the following calculations of the funds spent on BCC activities in each intervention area. As expected, the 22 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION largest amount went for encouraging the use of LLINs. Thirty million LLINs were distributed free during COMMIT (no LLINs were purchased by COMMIT), and the promotion of their acceptance and nightly use was a main focus of the project. The IRS component, which was a tangential one linked to RTI’s implementation of a major IRS project, received the lowest total level of support from the project. As noted earlier, RTI was an original project partner but became separate when it was awarded an IRS contract in 2010. Estimated Percentage of COMMIT Funds Used for BCC in Each Malaria Intervention Area LLINs: 38.2% IPT: 31.2% CM: 25.9% IRS: 4.7% In addition, the evaluation team asked the COMMIT staff to attempt to assess the relative investments made in various types of communication interventions (e.g., radio, print, IPC at clinics, mobile vans, CBO-CCA structure). The investment pattern that could be teased from the numbers was as follows, not including staff time. Subgranting: $2,607,770 (mainly for PSI & Rural Communication Initiative) Mass Media: $1,616,000** IPC: $1,356,500 Print Materials: $1,008,000 Mid Media: $552,846 (mobile vans) Advocacy: $190,000 TOTAL $6,323,116 ** (COMMIT negotiated 50% off radio rates and also received $250,000 of free media time) Although these numbers are inexact, they do show the general trend of investment by the project in various interventions and channels. Without solid impact data, it is not possible to say which investments provided the greatest returns. STAFFING STRUCTURE COMMIT feels that the structure it proposed and eventually implemented proved to be effective for project needs. The original project director left for personal reasons after the first year, but her replacement was managed smoothly. The use of two major subcontractors, PSI and Jhpiego, essentially decentralized the project with PSI running the rural communication component, while Jhpiego developed the capacity of health facilities in IPC. JHU, as the prime, did not have a presence in the regions, which were basically managed by PSI in terms of the rural communication component. With the presence of other JHU activities in Tanzania, COMMIT was able to share staff time with other projects (e.g., its M&E person split time between COMMIT and other JHU projects), while M&E staff in Baltimore also provided some support. The evaluation team thought it would have been better if COMMIT had an onsite USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 23 person focusing on M&E, forming ongoing links with other organizations, gathering relevant data, and ensuring the consistency of the questions used in COMMIT’s data-gathering efforts. COMMIT staff believe that their arrangement worked well. Collaboration COMMIT has done an excellent job of working with a wide variety of groups involved in malaria and safe motherhood. It has been very open to collaboration with a host of organizations in sharing ideas and resources and providing technical assistance. Following are some examples: 1. Working with PSI to share materials and strategies across the COMMIT project and the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM)-funded RCC project that was doing similar malaria work in the seven rural regions not covered by COMMIT. PSI adopted the rural communication strategy developed by COMMIT and shared equipment, such as the mobile vans, when possible. COMMIT purchased three MVUs for PSI, while PSI used its fleet of MVUs to support COMMIT activities nationwide giving COMMIT project a national coverage. 2. COMMIT’s Year 4 workplan called for the creation of a MIP initiative, and the project worked with a number of groups on its planning and then linked with the new TCCP project that focused on safe motherhood. 3. COMMIT developed the communication activities for the Hang Up Campaign implemented by the Red Cross. 4. COMMIT also provided technical assistance to the Zanzibar Malaria Control Program. Six Zanzibar staff attended the Leadership in Strategic Health Communication courses, and COMMIT helped develop the BCC plans for the BCC unit and promote the distribution of 700,000 nets in March 2012. 24 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 25 III. PROJECT IMPACT There have been questions raised by some within PMI as to whether money spent on BCC is justified, given that money is needed for critical commodities, such as LLINs, ACTs, RDTs, and spraying supplies for IRS. Although it is clear that BCC has made significant contributions in other health areas, some ask what difference it makes for malaria, and if so, what kinds of communication activities are most effective. There is consensus among all stakeholders that there has been a tremendous amount of malaria￾related behavior change in recent years. In order to seek documentation of that change and assess COMMIT’s role in it, the team reviewed all M&E reports by COMMIT as well as all large￾scale external studies conducted in the time period just before and during project implementation. External (non-COMMIT) Studies: The major data sources were the Tanzania Demographic and Health Surveys (TDHS) from 2005 and 2010, the HIV-AIDS and Malaria Survey (THMIS) from 2007–08, and the more recent Ifakara studies of net indicators following the UCC. (The Tanzania National Voucher Program conducted national yearly surveys beginning in 2005, but those were discontinued after 2008 and, therefore, do not correspond to COMMIT’s implementation period.) These studies document considerable behavior change and show that childhood mortality has declined as interventions have been scaled up since 2000. Table 3. All-Cause Under-Five Mortality DHS and THMIS 1992-2010 Year Cause 1992 DHS 1996 DHS 1999 DHS 2004–2005 DHS 2007–2008 THMIS 2010 DHS Number of deaths per 1,000 live births 143 138 148 112 92 81 Data that roughly correspond to “pre-COMMIT” and “mid-COMMIT,” i.e., between 2005 and 2010, show very large increases in the percentage of people sleeping under a treated net, even though the data were collected prior to the UCC campaigns. The more recent 2010 and 2011 Ifakara surveys conducted after the UCC and Hang Up campaigns in the Coastal, Southern, and Lake zones show continued increases in ITN-related indicators. (Those zones were predominantly COMMIT regions.) The CM indicators show a small increase in percentage of childhood fever cases taken to the health center promptly and a large increase in those cases that were given an ACT the same or following day that the fever appeared. There were modest increases in IPT indicators. The following two tables show the specific data for net, CM, and IPT indicators. 26 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Table 4. Key Malaria Behavioral Indicators DHS and THMIS 2004–2010 Data Source: Indicator 2004–2005 DHS 2007–2008 THMIS 2010 DHS Households owning an ITN or LLIN 23% 39% 64% Household members who slept under an ITN last night 20% -- 45% Under-fives who slept under an ITN last night 16% 26% 64% Pregnant women who slept under an ITN last night 20% 27% 57% Pregnant women receiving 1 dose of IPT 53% 59% 63% Pregnant women receiving 2+ doses of IPT 22% 31% 27% Under-fives with fever in past 2 weeks who were taken to a formal health provider 61% 57% 65% Under-fives with fever in past 2 weeks who received ACT the same or next day -- 14% 26% Table 5. Ifakara UCC Surveys 2010 & 2011 (carried out immediately following UCC and Hang-Up Campaigns) Zone and Year: Indicator Lake 2010 2011 Southern 2010 2011 Coastal 2010 2011 Households with a treated net 82% 95% 61% 96% 62% Kisarawe 78% Rufiji 98% Household members who slept under a treated net last night 57% 70% 38% 81% -- 61% Under-fives who slept under a treated net last night 71% 79% 56% 84% 72% Kisarawe 79% Rufiji 73% Kisarawe 73% Rufiji Women 15–49 who slept under a treated net last night 62% 76% 47% 82% -- 65% Nets owned that were used the prior night -- 65% -- 66% -- 49% Although these data document change, they do not tell us COMMIT’s role in that change. At first the evaluation team planned to compare data from the 11 COMMIT regions with the seven regions that RCC funded. In reality, however, COMMIT was a national program that worked with the RCC-funded regions, the NMCP, and other NGOs such as Red Cross to implement activities nationally. PSI was the COMMIT partner that implemented the subgranting and managed that component in COMMIT regions as well as in those under the RCC grant. The CCAs covered almost all of the country because of RCC and COMMIT, with COMMIT USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 27 providing materials such as the CCA flip charts, training manuals, guides, and print materials to CCAs nationwide. COMMIT lead the national media campaign on malaria, including radio and TV spots, and developed the radio show PataPata, the movie Chumo, and the national journalist initiative. The term “COMMIT regions” delineates the funding source rather than the program activity. Therefore, there were no differences of substance that would make for a meaningful comparison of the 11 COMMIT regions with others. At the same time, because COMMIT was the spearhead, if not the implementer, for malaria communication during this time period, any behavior change resulting from communication activities can be attributed to COMMIT working with others nationally. COMMIT Monitoring and Evaluation: The team reviewed COMMIT’s own M&E work to look for quantified evidence of impact there. COMMIT is fulfilling its M&E mandate, but currently there is little such evidence available. One reason is constraints that USAID put on the project’s evaluation work; another is some shortcomings in COMMIT’s tracking mechanisms, and a third is the timing. Analysis from the evaluation component has not been completed, and data from the 2012 THMIS is not yet available. From the outset (in the RFP), USAID discouraged primary data collection, envisioning use of the DHS and THMIS and other data sources to measure program impact. The RFP noted: Applicants should expect to establish a baseline of individual and community behaviors that are the target of this request for applications. Baselines should draw on indicators recommended by WHO–RBM and NMCP. Most of these indicators will already have been collected with specialized studies and surveys so the successful applicant should not have a need to collect them again…It is not expected that the implementation of the BCC strategy will require significant additional research or studies. Rather, the successful applicant should draw on research and studies that already exist. Applicants should attempt to keep Research activities to a minimum, and design their applications and budget accordingly. The COMMIT proposal that now forms the SOW for the CA states: A rigorous monitoring and evaluation plan will inform, monitor and evaluate the program’s activities. To maximize savings in cost and time, this evaluation will draw upon existing sources of data and, whenever possible, develop linkages to the data collection activities conducted by other programs in Tanzania… The outcome evaluation will assess the project’s effects at multiple levels, including effects on individuals’ knowledge and attitudes regarding malaria and its prevention and treatment, effects on community-level norms related to malaria and gender roles... PSI’s annual TRaC surveys will supplement this, providing interim measures of the trends in the target behaviors and assessing a broader range of the psychosocial characteristics, such as perceptions of self-efficacy and social norms that may mediate the effects of program exposure on the intended behaviors. COMMIT submitted a PMP during the first year of the project. Upon review by USAID, COMMIT was asked to remove measures of behavior (e.g., percent of people under a net, percent of under-fives under a net, percent of women, percent taking child to clinic within a day of onset of fever) because national surveys, including DHS and THMIS, were already measuring the key behaviors. The indicators that remained in COMMIT’s PMP were general measures of exposure and “behavioral determinants” such as awareness, knowledge, and self-efficacy. As the 28 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION national surveys show, there has been good progress in most malaria-related behaviors. However, attribution of effect requires measurement of exposure to specific elements of the program and behavioral determinants, as well as behavior, so that associations among them can be explored in analysis. The external national surveys are not set up to do this. Use of Existing National Surveys: COMMIT was able to get the THMIS to agree to insert questions on exposure to communication in the 2012 survey (data being collected now) so that analyses can be performed to try to assess the impact of communication exposure on behavioral outcomes. Comparison of the 2007 and 2012 THMIS may shed some light on COMMIT’s impact, as those survey dates coincide well with COMMIT’s time span. There will be limitations to that analysis. Only a limited number of questions could be inserted, and some of those questions are not optimal. Moreover, studies designed for one purpose almost always have shortcomings when one tries to “piggyback” and use them for another. Some of the outcomes of interest to COMMIT, such as fixing holes in the net or having the husband accompany the pregnant woman to ANC, are not included in these surveys. In addition, COMMIT’s target sample is mothers/guardians of a child under five, but the THMIS sampled all adults aged 15–49 living in the sampled household. Some of those households will not have a child under five, and some will have two adults with a child under five. Therefore, it may be difficult to extract the appropriate sample for COMMIT’s analysis. Nonetheless, a plan should be drawn up now to address such issues and lay out analyses that will provide as much information as possible about the impact of malaria BCC in Tanzania so that work should start as soon as the data set is available. Interim Tracking: COMMIT used PSI’s annual Tracking Results Continuously (TRaC) surveys, “community surveys,” and questions inserted in the Steadman/Synovate Omnibus Survey to provide some interim information about progress. These instruments are not consistent in geographical areas covered or questions asked. Some of the results are contradictory and counterintuitive. For example, according to data collected via the Omnibus survey, the percentage of households owning at least one ITN dropped from 85 to 72% between April and September 2010. The community surveys conducted in Lindi and Mtwara are well constructed but were carried out only in 2009 and used data from other surveys in 2008 to ascertain progress during that year in those sites. The PSI TRaC surveys are carried out nationally on an annual basis and have the potential to provide solid tracking data, but they do not use indicators that correspond well with the specific malaria behaviors or their determinants, and they do not cover the range of behaviors related to treatment, IPT, and ITNs. Therefore, these surveys are not reliable as documentation of change during the program, although they do provide some feedback for program planning. Ideally, there would have been one sound tracking mechanism used throughout the program. Outcome Evaluation: In order to learn more about how well the project worked and to tease out potential differences in effect by channels and exposure level, COMMIT collaborated with NetWorks to carry out an outcome evaluation in 2011–2012. The study was conducted in three regions—Lindi, Rukwa, Mwanza—with a sample of 400 households in each region, for a total of 1,200 households. Data analysis has not been completed, but preliminary results show good exposure to three main channels: CCA, mass media, and community activity. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 29 Figure 1: Exposure to Malaria Messages by Various Channels However, the data show little direct impact of exposure on sleeping under a net and on prompt treatment. Figure 2. Percentage of Net-owning Households in which all Individuals Slept Under a Net Figure 3. Percentage of Female Respondents Reporting They Sought Treatment for a Child’s Fever within One Day of Onset 30 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION These results appear to be inconsistent with the universal and emphatic claims of COMMIT impact from those at all levels in the field. There are inherent limitations to claims of causality in field research, where input variables cannot be manipulated and control groups cannot be found (and in this case, where a comparable baseline was lacking), so perhaps some of those limitations are affecting results here. Furthermore, communication operates by indirect as well as direct effect. Especially when a behavior starts becoming the norm, those not directly exposed can still be influenced by what they see and hear around them; direct exposure is not needed in order to be affected by the campaign. These are only a few preliminary results, and further refined analyses may show a more positive effect. The earlier COMMIT community study in Lindi and Mtwara (2009) did find a large difference between those who had seen a road show or video and those who had not. It is not clear why the current survey did not find similar results. Figure 4. Percentage of Households in which All Children Under 5 Slept Under a Net the Previous Night, by Exposure [Lindi and Mtwara 2009 n=450] Monitoring, Evaluation, and Research Staffing: The original staffing plan included a full-time M&E staff person. However, partially because USAID discouraged research, M&E was carried out by staff shared by other CCP programs in Dar es Salaam and a researcher at Johns Hopkins on an as-needed basis; there was no single person responsible for this function. COMMIT feels that this arrangement worked well, but the result was a somewhat scattershot approach, and some of the M&E suffered because of it, principally in consistency of monitoring data and quality of the measures in some studies. Especially since COMMIT was the first PMI-funded effort focused on communication for multiple malaria interventions, it would have been ideal to have a skilled researcher on staff to plan and oversee integrated data collection, maintain quality control, make sure results were used to their fullest and fed into future efforts, and keep abreast of related research in Tanzania. Exposed vs. Unexposed: p<0.001, controlling for age, education, number of household possessions, currently works for money, malaria knowledge scale, receipt of voucher, and use of media. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 31 IV. RECOMMENDATIONS AND OBSERVATIONS Based on its review of COMMIT documentation, research data, interviews with project participants from the national to the village level, and general observations, the evaluation team agreed upon the following recommendations and observations that could help to improve current and future BCC and malaria activities. They range from long-term recommendations, such as looking for ways to continue the rural communication network, to several small, but valuable, steps like providing CCAs and other project participants with certificates at the end of the project. USAID/Tanzania asked the team to prioritize these recommendations or to select the most important four or five. Given the fact that the team sees value in all of these recommendations, it decided to use a star system to indicate a level of priority: *** 3 stars for a high priority ** 2 stars for a useful/doable recommendation * 1 star for a point that deserves serious consideration GENERAL RECOMMENDATIONS AND OBSERVATIONS Rural Communication Initiative *** a. The FH-CBO-CCA system seems to be functioning very well in the regions visited with high commitment and morale and a team spirit among FHs, CBOs, CCAs, and local officials and, therefore, presents the possibility of being a continuing force for malaria prevention and treatment and other health issues. Ensuring its sustainability and expansion and helping the NGOs/CBOs to plan focused activities based on data from the district level in coordination with district government and health stakeholders should be a priority. This system strengthens civil society and is a positive force for development. It has upgraded the skills and capacities of thousands of people and has provided a model for cooperation and problem-solving. *** b. Consideration might be given to increasing the CCA stipend from Tsh. 10,000, given inflation factors and the fact that some CCAs have to dip into their own pockets to cover expenses such as local travel. It is a careful balance between maintaining the current spirit of volunteerism and contributing to the community vs. raising compensation to the point where that volunteer spirit withers and people are motivated to seek the CCA positions more for the monetary reward. A modest increase (e.g., to Tsh. 20,000 or 30,000) would not be very costly but would recognize their valuable contributions, provide some morale boost, and account for inflation. *** c. Emphasis should be placed on facilitating and building district-level capacity to develop, plan, and implement integrated malaria activities. This should be linked to the RCI and the planning should be driven by data. Districts need to take ownership of the malaria activities so that they are focused on the needs of those districts. The infrastructure exists with the regional and district MFPs as well as the district health officers. Integrated plans can be developed that could be funded through the NGOs with supervision and agreement from the district levels. 32 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION ***d. Given the pattern of dispersed settlement and the distance between villages, many CCAs cite transportation difficulties as a major hurdle for them and suggest having motorcycles. This would be costly, and it is unlikely that CCAs would be able to maintain the motorcycles for long. Another option is to increase the number of CCAs to several per ward or one per village, thus adding coverage and reducing distances to be traveled. ***e. A lesson learned early on is that the VEOs and WEOs and other local officials had to be briefed on and convinced of the idea of CCA and CBO participation in malaria so that they were supportive and knowledgeable about the role of each and the modest resources available. They, and all stakeholders, should be included in the initial project announcement and planning. ** f. A VEO suggested that all CCAs, VEOs, and WEOs, should receive a certificate from COMMIT at the end of the project. This is a very good idea that should be implemented. ** g. Consideration might also be given to a suggestion by a Mwanza CCA that the CCAs be issued IDs, given that they cannot wear their malaria t-shirts all the time and an ID would provide credibility (and probably a little additional motivation/distinction). *h. The CBOs noted that they have received no equipment during COMMIT. Given the emphasis on increasing their capacity and requiring regular reports and tracking systems, it would be reasonable, in a future project, to assess the feasibility and wisdom of providing every CBO with a laptop, software, printer, a basic pack of supplies, and some orientation to its use, including templates for tracking forms and reports. *i. Many CCAs are requesting more training. It is not clear if there is a real need for refresher training and new information, or if this reflects a desire to both upgrade their morale and status as well as their knowledge. Some are very bright and would welcome an opportunity to learn more. The new distance education program via radio should be monitored to learn its effects on CCAs as well as on other health providers and community members. *j. Periodic checks should be made of the equipment issued to CCAs (bicycle and radio) to ensure they are working and still under the control of the CCA. LLINs, ACTs, and SP *** a. The Quality Control process for the purchases of LLINs should be reviewed, particularly the use of the bursting strength test. People in all three regions visited complained of nets “tearing easily” or arriving with 1–2” holes in them. This should not be happening to 150- denier high-density polyethylene. BCC promotion of the use of nets cannot be successful if the consumers believe that the nets are of poor quality. The LLIN is the most visible symbol of malaria prevention activities. *** b. The “one size fits all” net distribution policy should be re-examined given the large number of complaints that the single-size nets are “too small” for the beds or sleeping places. It does no good to provide free nets that people find unsuitable and are unlikely to use, or tear when they try to stretch them around larger beds. The cost and added complexity of making at least two sizes available should be balanced against the likelihood that nets of the proper size will be more likely to be used. The Tanzania Voucher Program USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 33 distributes a larger net than that handed out in the free distributions. The power of BCC is greatly limited if the product it is promoting is seen as flawed. *** c. In every region visited by the team, the problem of severely limited supplies of SP and ACT were hindering success. BCC cannot solve this. If the BCC convinces a woman to go to the clinic to get SP and it is not available, the BCC and the health system lose credibility. ** d. With four domestic net manufacturers, Tanzania once had the greatest net production capacity of any African nation. It still does in terms of millions of nets produced each year, but now it has only one active manufacturer instead of four. LLINs are not readily available in the market anymore, so people report difficulty in purchasing nets. Given the fact that the funds available for free nets will soon begin to dwindle, and that questions are being raised about the quality and variety of the LLINs currently available, it would be prudent to examine the status of the commercial net market in Tanzania to determine what can be done to increase the variety of LLINs available and to assess the possibility for recreating a commercial market for LLINs that can serve the long-term needs of the Tanzanian people. When BCC builds a demand for LLINs, people should have the option of buying one when they want it rather than waiting a year or two for another free net. BCC Materials *** a. The integrated approach of promoting multiple malaria behaviors (use of LLINs, ACTs, IPTp, early treatment-seeking, IRS) has presented no problem other than spreading available funds among different messages. Given the overlap of the target audiences and the behaviors, it makes sense to have an integrated approach. *** b. Continued funding for mass media and mid-media interventions should be considered if no other malaria stakeholder will take over COMMIT’s role. The focus should be to better understand the new reality with reduced malaria cases due to UCC and other commodities being readily available. Messaging needs to be developed to keep people under nets and using other commodities. Attention should be given to specific messaging to geographic areas based on data from the regions as well as new surveillance that will be happening in the coming years. ** c. A review of the current print materials should be made to determine which ones should continue to be used in terms of their usefulness and the value of the messages they are carrying, given that knowledge of malaria has greatly increased. Some of the materials appear to be very wordy, carry messages that are no longer salient, and may be costly to produce and distribute. COMMIT should also investigate whether there is a Swahili “readability scale” that can assess the grade level of the language they are using. The print materials seem to be the least valued of the communication materials, according to discussions with CCAs, CBOs, etc. Some people think they are useful or at least give them something to give out; even more think people pay little attention to them and sometimes throw them away after a sporting event. The cost of production and distribution of existing print materials should be reviewed and a determination made as to which ones should be continued. *d. The flipchart for clinic use for IPC should be reviewed to see if it has too much information for the health worker to pass on and if all the pictures are useful in conveying 34 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION the right information. The flipchart could also be made more accurate by depicting a rectangular net in use rather than a conical one, which is not distributed. * e. The brochure for hanging and repairing an LLIN should start with a recommendation that users hang up the LLIN 24 hours before using it in order to disperse any loose insecticide and to reduce any insecticide smell. This is often a step listed on the LLIN bag. *f. Project documents (e.g., quarterly reports, workplans, annual reports) show a consistent lack of editing (e.g., two quarterly reports note that the COP has “relocated twice to Tanzania”) and proofreading (e.g., one report spells COMMIT in three different ways). Project deliverables should reflect the level of accuracy and attention to detail that the project applies to its main work. Data Collection, Analysis, and Distribution *** a. Draw up a plan now for analysis of THMIS data that will provide as much information as possible about the impact of BCC in Tanzania. Analysis should start as soon as the data set is available. *** b. Have a skilled researcher on staff to plan and oversee implementation of data collection, maintain quality control, make sure results are used to their fullest and fed into future efforts, and keep abreast of related research in Tanzania. *** c. USAID must decide what impact data it wants and support its collection. Donors cannot ask for evidence of effect but disallow the type of study that would provide answers to questions about effect. ** d. Two-way data flow should increase among the various levels: village (CCAs), district, region, and national. It is not certain that what the CCAs are learning about key behaviors at the grassroots is being systematically collected and shared at all levels, and it does not appear that data is flowing downward, particularly information on progress made against malaria and data on the reduction in incidence and date rates. Specific feedback on progress helps planning and can motivate people to achieve more. (Anecdote: A drama troupe member said she didn’t know that rates of IPT2 were very low and added that the group would now incorporate the fact into their work.) ** e. It is unclear whether there is enough quality information flowing into the project on the impact of its messages, materials, and activities. A review should be made of the project’s information needs, analyzing the existing sources of data and feedback for efficacy and timeliness and determining if they are sufficient for the project to function optimally in fine tuning its communication messages and channels to current needs. ** f. Include the indicator “percentage of nets used (slept under) last night” in all major studies. The denominator is number of nets owned, and the numerator is number of nets used. This indicator is important for tracking non-use of nets owned. Non-use above 15% or so implies that efforts need to be improved to get people to use the net(s) they have or that there is surplus, or waste, in net distribution. Only the Ifakara studies carried out following UCC calculated this indicator. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 35 Mass Media *** a. The PataPata program for school children was mentioned in every meeting the team had, with most people feeling that it was important to get malaria messages to the next generation. There seemed to be a feeling that there will always be a significant share of today’s adults who will not adopt the needed malaria behaviors; therefore, it is essential that the youth be educated. Several people suggested that CCAs and others (e.g., local teachers) running listening groups be provided with some “rewards” for the children (e.g., pencils, exercise books) that could be handed out to one or two children per meeting. Future Directions *** a. Data show that people know the symptoms of malaria, that mosquitoes cause malaria, that ITNs help prevent malaria, and other basic facts. BCC does not have to spend time on these things and should move beyond them. Efforts should now focus on very specific barriers or misperceptions. For example, it was mentioned several times that people see mosquitoes resting on their nets, so they doubt the efficacy of the net. They get bitten by mosquitoes when outside the net. They see that some people still get malaria even though they use a net. They note that some children die even when taken to a hospital. *** b. As the project moves into its final year, it should take stock of what it has learned and focus its resources on the critical issues that affect its achievements, perhaps doing fewer things and concentrating on the most effective activities or materials or issues, such as the low rates for IPT 1 and 2, provided the supply issues are solved soon. RECOMMENDATIONS FOR FUTURE TANZANIAN PROJECTS There still remains a need for USAID to invest in malaria prevention and treatment in Tanzania and to help the NMCP and communities promote the behavior change that is needed to reduce malaria to a very low level. Awareness levels on key malaria messages are now very high. People understand the dangers of malaria and are familiar with the steps needed to prevent or treat it; however, action needs to be focused on converting that increased awareness into actual behavior change. A more strategic phase of behavior change work needs to occur that looks at each desired behavior, particularly those that have been resistant to change, to better understand the barriers to achieving high levels of compliance with malaria recommendations. Why aren’t women coming back for IPT2? Why aren’t more pregnant women sleeping under an LLIN every night? An up-to-date examination of current behaviors and the conditions that are facilitating or hindering behavior change needs to be undertaken. This should be done in collaboration with a broad range of MOH staff in a multidisciplinary context so that it focuses on all barriers and possible strategies and solutions, not just on communications. Is the unreliable supply of SP the major reason for the slow growth of IPT consumption, or are there other factors that are causing late first visits and no second visit? To what extent is record￾keeping an explanation for the poor numbers? Should the DOT policy of SP administration be changed to allow for home consumption of SP? Communication may not be the real answer to any of these problems or to solving the lagging growth of desired behaviors. A fresh look at the issues, the elements of the desired behaviors, and the circumstances that aid and inhibit the adoption of these behaviors is needed as the basis for the formulation of new policies, systems, activities, etc. that can stimulate high levels of the behaviors the MOHSW seeks to instill. 36 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION IMPLICATIONS FOR PMI WORLDWIDE The intense focus on understanding the barriers to achieving high levels of adoption of the main malaria actions is something that could also be supported by PMI. The success of the Rural Communication Initiative and its utilization of NGOs, CBOs, and CCAs should be of interest to PMI given USAID’s goal of increasing the involvement of civil society organizations in development. Any similar effort in another country should start with an assessment of the existing civil society and volunteer networks that might be tapped. The Tanzanian model will be applicable to some countries, but other models will be more viable in other situations. Still, the ability of a credible spokesperson to deliver key messages locally is an approach that has been successful in many countries. PMI would also benefit from looking toward the future supply of the commodities that underlie its malaria prevention and treatment strategies. With the inevitable decline in the amount of donor funding for commodities over the next decade, how can adequate supplies be maintained, even though the total need will decline with the decline in malaria cases? Now that tens of millions of LLINs have been distributed and millions of families have seen the benefits of using LLINs, will those families who can afford a net be willing to buy one if free supplies are constrained? What is the game plan beyond free nets for all and free medicine for all? We cannot wait until the money runs out before we define a solution. To what extent will local production of nets, ACTs, and SPs be an answer in countries with a domestic production capacity? If imports are to be relied upon, how can you have a vibrant market with multiple quality products in a variety of price ranges? As PMI moves into its seventh year of operations, it may wish to take stock of what has happened in behavior change to date and to assess future directions. Knowledge and awareness has greatly increased in PMI countries, and morbidity and mortality rates for malaria have dropped. There is a need to focus resources on those key areas that can make a large difference. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 37 APPENDIX A. SCOPE OF WORK Global Health Technical Assistance Bridge Project (GH Tech) Contract No. AID-OAA-C-12-00004 (Amend #1: 05-10-2012) I. TITLE Performance Evaluation of the Communication and Malaria Initiative (COMMIT) Project Contract: Global Health Technical Assistance Bridge Project (GH Tech) II. PERFORMANCE PERIOD (SCOPE START AND END DATES) April 12–June 22, 2012 III. FUNDING SOURCE Tanzania Mission IV. PURPOSE The purpose of this evaluation is to: 1. Determine the extent to which the COMMIT project’s objectives and key tasks have been achieved, 2. Document lessons learned to inform key management decisions that can be applied to other future programming of President’s Malaria initiative (PMI) in other countries, and 3. Assess the various communication approaches used by COMMIT, to determine what worked best to affect both provider and consumer behaviors. V. BACKGROUND Cooperative Agency: The Johns Hopkins University Agreement Number: CA- 621-A-00-08-00005-00 Effective dates: October 23, 2007 to October 22, 2012 Ceiling Value for CA: $15,000,000 Obligations to Date: $13,950,000 On October 23, 2007, the USAID Mission in Tanzania awarded a competitive five-year Cooperative Agreement to the Johns Hopkins University for the Communication and Malaria Initiative (COMMIT) Project ending on October 22, 2012, with a ceiling of $15,000,000. The project is operating in 11 regions and is implemented by the Johns Hopkins’ Bloomberg School of Public Health Center for Communication Programs (CCP) with PSI, JHPIEGO, and 11 Tanzanian non-governmental organizations (NGO) and a community-based organization (CBO) in each district of operation in the 11 regions. RTI International was also a subcontractor during the first two years of the project. The project is implemented in partnership with the National Malaria Control Program (NMCP) and it is PMI’s key mechanism to promote the behaviors necessary to achieve malaria prevention and control and create the enabling environment that support these behaviors. 38 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION The COMMIT project operates at the national, community, and household level to foster social norms and self-efficacy around malaria prevention and treatment. The project has a substantial rural communication component known as Rural Communication Initiative that is implemented through 65 community-based organizations (CBO)—one CBO per district—that supervise 1,200 Community Change Agents (CCAs). These CCAs are responsible for mobilizing communities around malaria prevention, treatment, and control. Through the Rural Communication Initiative the project proposes to reach targeted communities directly with malaria messages from activities such as home visits, group talks, and school-based activities, to community initiated activities in which communities develop and implement action plans for malaria prevention. Village-level activities such as road shows and mobile video units engage communities through participatory talks and videos that use entertainment to promote social norms and personal reflection around malaria prevention and control. The project’s mass media component supports the community-level mobilization by emphasizing prevention and treatment messages that focus on increasing self-efficacy around net use, malaria in pregnancy, and case-management using different channels and formats. In addition to radio campaigns focused on specific thematic area, one campaign, “Two Minutes of Wisdom” (radio and TV), showcased well-known Tanzanians discussing their experiences with malaria. More recently a 10-minute children’s weekly radio program, PataPata (Get It), was developed to empower children to become change agents in malaria prevention within their own homes and communities. COMMIT has also used an entertaining approach for the prevention of malaria in pregnancy by producing a mass-marketed feature film that has been distributed commercially as well as through the Rural Communication Initiative’s mobile video units. With partners and the NMCP, COMMIT has implemented the umbrella campaign Malaria Haikubaliki (Malaria is Unacceptable) in which all partners used the logo and slogan for their activities. Together, these activities support the community-level mobilization activities. The project’s health-facility component is aimed at strengthening health providers’ skills around counseling on under-five case management and for ANC/malaria in pregnancy. This has included adding malaria interpersonal communication counseling (IPC) skills in national focused antenatal care trainings, including IPC in pre-service training, as well as specific training with health centers in the COMMIT focus districts for both case management and ANC/malaria in pregnancy. To ensure implementation and continued practice of the skills, a district level supervision component is included in the package. To date, $13,950,000 has been obligated to the project. VI. DETAILED PROJECT DESCRIPTION Please see a full program description (Annex 2) of this SOW. VII. OBJECTIVES OF EXTERNAL EVALUATION This external evaluation shall: Determine the overall achievements in relation to the expected results of the COMMIT project since the beginning of the program; Determine progress made in the capacity building and institutional strengthening of the Ministry of Health and Social Welfare (MOHSW)/NMCP; USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 39 Identify the lessons learned from the COMMIT project since the beginning of the program taking into account the perspectives of the stakeholders, the donors and the beneficiaries; and Propose key activities for the follow-on project. VIII. EVALUATION METHODOLOGY The evaluation team shall use facilitative methods and activities that will enhance collaboration and dialogue among counterparts, particularly MOHSW and NMCP. The evaluation team shall work under the supervision and guidance of the Agreement Officer Representative (AOR) for the COMMIT project. The AOR will organize all internal PMI meetings including linking the evaluation team with the Health Office Team Leader and other team members. This performance evaluation will address six key questions: What have been the COMMIT Project’s key achievements so far? How successfully was the project implemented? To what extend have the COMMIT objectives been achieved? How is the project perceived and valued by malaria stakeholders in Tanzania? In what ways has COMMIT integrated gender considerations into its activities? Assess the various communication approaches used by COMMIT to determine what worked best to affect both provider and consumer behaviors? Are there specific lessons from the project that can be applied to other PMI countries? The evaluation team shall develop an evaluation design and data collection methods, using a mixed methods approach to gather both quantitative and qualitative information that is based on sound social science methods and tools used in a manner to minimize potential biases. The proposed evaluation design data collection methods, tools to be used, and work plan will be submitted to USAID/Tanzania and discussed during the team-planning meeting in Tanzania. The final evaluation design and workplan shall be presented to the AOR and relevant PMI and health team members for comments during the last day of the Team Planning Meeting (TPM) in Tanzania. The COMMIT AOR will arrange for an initial introductory meeting with appropriate MOHSW and NMCP staff at the outset of the process. Where necessary, the AOR may participate in meetings with the Government of Tanzania (GOT) representatives and partners. A general list of relevant stakeholders and key partners will be provided to the evaluation team by the AOR at the time of arrival, but the evaluation team will be responsible for expanding this list as appropriate and arranging the meetings and appointments so as to develop a comprehensive understanding of the program and services offered through the COMMIT cooperative agreement. The final methodology, together with evaluation tools and work plan, will be developed as a product of the Team Planning Meeting (TPM) and shared and approved by the Mission, NMCP, and NIMR prior to application. 40 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Document Review Prior to conducting field work, the evaluation team will review existing literature and data, including NMCP strategies, M&E plan (Performance Management and Malaria Medium-term Strategic Plan), BCC strategy, quarterly and annual reports of the cooperative agreements and modifications, and other reports and documents reflecting COMMIT’s work in Tanzania. Over time, the project has documented the evolution of the project and implementation of various malaria communication activities. Annex 1 is a detailed list of reports, studies, publications and other documents for review. All team members will review these documents in preparation for the initial team planning meeting. Baseline and midterm evaluation reports. Team Planning Meeting A two-day team planning meeting will be held in Tanzania before the evaluation begins. This meeting will allow PMI to present the team with the purpose, expectations, and agenda of the assignment. In addition, the team will: – clarify team members’ roles and responsibilities, – establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion, – review and develop final evaluation questions, – review and finalize the assignment timeline and share with PMI, – develop data collection methods, instruments, tools and guidelines, – review and clarify any logistical and administrative procedures for the assignment, – develop a preliminary draft outline of the team’s report, and – assign drafting responsibilities for the final report. Internal PMI/Tanzania Meetings These will include, at a minimum: Initial organizational/introductory meeting at which the evaluation team will present an outline and explanation of the design of the evaluation; Mid-term evaluation review with the AOR and Health Team leader to outline progress and implementation problems; and Final evaluation debrief/summary of the data, draft recommendations, and report. Field Visits/Key Informant Interviews Key informant interviews will be conducted as required. The evaluation team will conduct interviews with donor organizations, selected NGOs, and other key respondents identified during the planning meeting. Gender representation must be taken into consideration. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 41 The evaluation team shall arrange to visit selected sites supported through COMMIT in consultation with the AOR and Johns Hopkins University in-country leadership. The selected sites should include a representative sample of activity sites and should represent variety along the following dimensions: geographical location; technical focus of activities; subpartner types; and level of COMMIT’s support. The evaluation team will be accompanied by a member of staff from PMI/Tanzania and/or NMCP. The site visits will involve interviews with district health management teams, commodity outlet sites and clients, and targeted beneficiary groups and observation of interpersonal communication and community mobilization activities coordinated by COMMIT. The purpose of these site visits is to gain a better understanding of the technical competence of COMMIT staff and subpartners, the attitudes and knowledge of key beneficiaries and purchasers of commodities, the constraints encountered in the various categories of activity implementation, and key target audiences’ perceptions of their needs in order to provide quality services. The evaluation team should outline key meetings to coordinate post field visits in order to share findings and get final inputs before preparing the report. Wrap Up and Debriefing At the conclusion of the field visits and key informant interviews, there will be a debrief meeting with PMI/Tanzania, NMCP, and key malaria stakeholders in addition to the Tanzania PMI Country Support Team at the Ronald Reagan Building in Washington, DC. The purpose of the meeting will be to share findings and get final inputs before preparing the draft assessment report. IX. TEAM COMPOSITION, SKILLS, AND LEVEL OF EFFORT Team Leader/Senior Evaluation Specialist: Should have a postgraduate degree in health or an applicable social sciences field. S/he should have at least 15 years of senior level experience working in behavior change communication in a developing country. S/he should have extensive experience in conducting qualitative evaluations/assessments. Excellent oral and written skills are required. The Team Leader should also have experience in leading evaluation teams and preparing high-quality documents. The Team Leader will take specific responsibility for assessing and analyzing COMMIT’s progress towards targets, factors for such performance, and benefits/impact of the strategies and compare these with other possible options. S/he will provide leadership for the team, finalize the evaluation design, coordinate activities, arrange periodic meetings, consolidate individual input from team members, and coordinate the process of assembling the final findings and recommendations into a high-quality document. S/he will write the final report. S/he will also lead the preparation and presentation of the key evaluation findings and recommendations to the PMI/Tanzania team, NMCP, and other major partners. Capacity-Building Technical Advisor: Should have a postgraduate degree in organizational development or health systems. S/he should have at least five years of experience with institutional capacity building and organizational development in developing countries. S/he should be knowledgeable in program assessment and evaluation methodologies in human resource and work plan development and training in information monitoring systems. S/he 42 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION should have extensive experience, and demonstrate state-of-the-art knowledge, in conducting programmatic evaluations/assessments. Malaria Technical Advisor: Should have a postgraduate degree in public health or related subject. S/he should have at least 10 years of experience with malaria control and behavior change communication program design and implementation in developing countries and demonstrated knowledge of state-of–the-art strategies. S/he should have extensive experience in conducting program evaluations/assessments and a thorough knowledge of evaluation methodologies and organizational and institutional capacity building. One of the evaluation’s team members must be a local Tanzanian with malaria and/or BCC experience. Timeline and LOE: PMI/Tanzania anticipates that the period of performance of this assessment will be approximately 35 days. This would include preparation days, in-country work in Dar es Salaam and the regions, and report writing and finalization. The in-country assessment will take place o/a April 12–May 21, 2012. The following is a sample timeline: Week Activity Mwita Days Baume Days Shaw Days April 18–21 Preparation Work—3 days 3 3 3 Travel to Dar es Salaam 0 1 1 April 22–28 Mission meeting; information gathering 6 6 6 April 29–May 5 Mkuranga-Bupoo visits; information gathering 6 6 6 May 6–12 Visits to Mtwara & Mwanza regions 6 5* 6 May 13–19 Writing of final report in Dar es Salaam 6 4** 6 May 20–26 Presentation to USAID/debrief 1 0 1 Travel to USA 0 1 1 Follow up in Dar on any issues from debrief 1 0 0 Waiting for feedback from USAID/PMI 0 0 0 May 27–June 1 Revise final draft based on feedback; review & approval of final draft by team; presentation to PMI/Washington; final editing/formatting for submission 2 3 1 3 1 1 Grand Total 31 30 35 *Baume does not work Saturday, May 12; No charge for per diem. **Baume returns to USA Thursday evening, May 17, after completing her sections of the report. X. LOGISTICS A six-day workweek is authorized when team is working in-country; local holidays are not authorized. The evaluation team will be responsible for all off-shore and in-country logistical support. This includes arranging and scheduling meetings (with exception to previously mentioned meetings with GOT and initial introductory meetings), international and in-country USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 43 travel (including vehicle rentals), hotel bookings, working/office space, computers, printing, and photocopying. A local administrative assistant/secretary may be hired to arrange field visits, local travel, hotel, and appointments with stakeholders. In addition, the evaluation team leader is responsible for draft and final report development, as well as other eligible expenses associated with the completion of the assignment. XI. EXPECTED DELIVERABLES The contractor deliverables shall include: 1. A proposed timetable for evaluation period. 2. A written methodology plan and tools (evaluation design/operational work plan) during the pre-evaluation meeting. 3. A proposed outline of the report on the findings and recommendations of evaluation with possible issues for discussion during the mid-evaluation meeting (within two weeks of arrival in-country). 4. A completed draft of evaluation report to PMI/Tanzania for presentation during the debrief meeting that will be held approximately three days before departure (see report format provided in section XII. “Reporting Requirements” below). After the debrief meeting, the evaluation team should incorporate oral comments received from PMI and stakeholders. 5. Draft report should be completed prior to the team leader’s departure from Tanzania. The written report should clearly describe findings, conclusions, and recommendations (using the report format provided in “Reporting Requirements” below). PMI/Tanzania team will provide comment on the draft report within five working days of submission. 6. A final report that incorporates the team responses to Mission comments and suggestions. The draft final report should be completed within 5 days after PMI provides its feedback on the draft report, incorporating the comments received from the review of the draft and sent to the Mission. The report shall not exceed 30 pages, excluding references and annexes. After the final but unedited draft report has been reviewed by PMI, GH Tech will have the documents edited and formatted and will provide the final report to PMI/Tanzania for distribution (2 hard copies and a CD ROM). It will take approximately 30 days for GH Tech to edit/format and print the final document. 7. List of all reviewed/cited sources in final report. XII. REPORTING REQUIREMENTS The findings from the evaluation will be presented in a draft report at a full briefing with PMI/Tanzania and possibly at a follow-up meeting with key stakeholders. The format for the evaluation report is as follows: Executive Summary: concisely state the most salient findings and recommendations (2 pp); Table of Contents (1 pp); Introduction: purpose, audience, and synopsis of task (1 pp); Background: brief overview of malaria and behavior change communication in Tanzania, PMI’s strategies and priorities, brief description of the COMMIT Project, purpose of the evaluation (2–3 pp); 44 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Methodology: describe evaluation methods, including constraints and gaps (1 pp); Findings/Conclusions/Recommendations: for each objective area (17–20 pp); Issues: provide a list of key technical and/or administrative issues identified (1–2 pp); Future Directions (2–3 pp); References (including bibliographical documentation, meetings, interviews and focus group discussions); Annexes: evaluation methods, schedules, interview lists and tables should be succinct, pertinent and readable; list of documents consulted, and SOW. The report’s contents: The evaluation report should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why. Evaluation reports shall address all evaluation questions included in the scope of work. The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the technical officer. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. Evaluation findings will assess outcomes and impact on males and females. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparable groups, etc.). Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings should be specific, concise, and supported by strong quantitative or qualitative evidence. Sources of information need to be properly identified and listed in an annex. Recommendations need to be supported by a specific set of findings. Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. XIII. MISSION AND/OR WASHINGTON CONTACT PEOPLE/PERSON Naomi A. Kaspar Project Management Specialist Health and Population Office USAID Tanzania USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 45 Plot 686 Old Bagamoyo Road, Msasani P.O. Box 9130, Dar es Salaam, Tanzania Email: nkaspar@usaid.gov Tel: +255 22 2294490 Ext. 4309 Cell: +255 755 766600 XIV. REFERENCES Background documents for this assignment will include, but not be limited to, the following: ANNEX 1: Selected List of Background Materials Request for Applications (RFA) Number USAID-TANZANIA-07-005-RFA: Behavior Change and Communication in the Prevention and Case Management of Malaria in Mainland Tanzania COMMIT’s revised program description submitted to USAID/Tanzania on August 1, 2007 COMMIT’s Annual Reports COMMIT’s Annual Work Plans PMI/Tanzania and Health Team strategy documents Project products (designed materials such as fliers, posters, publications, etc.) ANNEX 2: Program Description (Separate File) XV. COST ESTIMATE GH Tech will provide a cost estimate for this activity 46 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 47 APPENDIX B. PERSONS CONTACTED TANZANIA: DAR ES SALAAM Johns Hopkins University CCP Organization Robert Ainslie, Chief of Party Waziri Nyoni, BCC Program Manager Fauziyat Abood, Media Advocacy Officer Pamela Kweka, BCC Program Officer Furaha Kabuye, Sustainability Coordinator Population Services International Romanus Mtunge, Executive Director David Dadi, MCH Director Edwin Mtei, Rural Communication-IPC Jhpiego Maryjane Lacoste, Country Director Rebecca Mdee, IPC Coordinator Ukende Shalla, PSE Nurse-Midwifery Advisor Victor Mponzi, Monitoring and Evaluation United States Agency for International Development Naomi Kaspar, Project Management Specialist Dr. Jessica M. Kafuko, PMI Resident Advisor Ludovicka Tarimo, Gender Specialist Angela Mwaikambo, Monitoring and Evaluation Specialist Moses Busiga, Monitoring and Evaluation Specialist Centers for Disease Control and Prevention Dr. Peter D. McElroy, PMI Resident Advisor Ministry of Health: National Malaria Control Program Dr. Ally J. Mohamed, Program Manager Leah Ndekuka, Head BCC Cell MEDA, National Voucher Program Faith Patrick, Country Director Tanzania Red Cross Society Jane Lweikiza, Program Manager 48 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION TANAM (Tanzania National Malaria Movement) Beatrice Minja, Executive Director TANZANIA: COAST REGION Mkuranga District Meeting (April 30, 2012) Ms. Elinet Nsemwa, CVM/AIDS Partnership Africa (Regional Fund Holder), Coast Region Dr. Philemon Karugilo, Ag. District Medical Officer, Mkuranga District Mr. Aaron Wilson, District Malaria and IMCI Focal Person Mr. Joseph Mganga, SHIMAJEMA CBO Mr. Mambo Kabwere, SHIMAJEMA CBO Mr. Dando Issa, Community Change Agent (CCA) Bupoo Village, Mkuranga District (April 30, 2012) Mr. Mohamed Mwago, Village Executive Officer Ag. Ward Executive Officer Ms. Mwamosi Ponza, CCA, Bupu Ward Mr. Severine Sambala, Community Member Mr. Athumani Lukubwaga, Head of Chemi Chemi Cultural Troupe Ten members of Chemi Chemi Cultural Troupe TANZANIA: MTWARA REGION Dr. Saifudun Kabuma, Regional Medical Officer, Mtwara Mtwara Regional Meeting Dadi Mohamed, Masasi District Malaria Focal Person Hamis Nakuraha, Neda CBO Lucy Millanzi, Nanumbu District Malaria Focal Person Msosa Gabriel, Shikum CBO Abasi Ibrahim, Nanyumbu CBO Fadina Saida, Twaha CBO Dr. Ally Namiyundu, Tandahimba District Malaria Focal Person Devotha Kapembe, Newala District Malaria Focal Person William Kunamula, Mtwara Mukemba CBO Cyprian Lungu, PSI Regional Manager, Mtwara Erick Mwinuka, PSI Regional Manager, Mtwara Allan Mkopoka, Masha Regional Manager Carol Sevin, Masha Regional Manager Nanyamba Village: Mtwara District Mr. Geoffrey Masumbuko, Ward Executive Officer, Nanyamba Ward USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 49 Mr. Yusuf Abdulrehmani, Village Executive Officer, Nanyamba Village Mr. Hamisi Luwono, CCA Mr. Mustafa Chono, CCA Mr. Saidi Namonde, CCA Mr. Hamisi Libuburu, CCA TANZANIA: MWANZA REGION Dr. Valentino Bangi, Mwanza Regional Medical Officer Mwanza Regional Meeting Mr. James Lumala, CBO, Ukerewe District Mr. Elias Seleli, CBO, Sengerema District Mr. Gideon Lyakugwile, CBO, Missungwi District Ms. Jane Benedict, Regional Fund Holder, Mwanza Mr. Thomas Gikaro, PSI, Mwanza Ms. Naike Nathaniel, M&E, Regional Fund Holder Mr. Noel Thomson, Journalist, TANDABUI Health Radio Mr. Edgar Kasasse, Regional Malaria and IMCI Focal Person Mr. Julius Mwengera, CBO, Magu District Mr. Mussa Mugaiwa, CBO, Geita Mr. Salala Kilumani, CBO, Kwimba Mr. Winston Nongwe, Malaria and IMCI Focal Person, Ukerewe Mr. Abdul Mgonja, Malaria and IMCI Focal Person, Sengerema Mr. Wilie Luhangija, Malaria and IMCI Focal Person, Geita Dismas Dotto, Malaria and IMCI Focal Person, Missungwi Mr. Joseph Mandago, Malaria and IMCI Focal Person, Magu Ms. Joyce Kasimbazi, Malaria and IMCI Focal Person, Kwimba Sukuma Ward: Magu District Mr. George Msoga, VEO, Lumeleji Village Mr. Masalu Mazoka, CCA, Sukuma Ward Ms. Happyness Mashenene, CCA, Magu Township Mr. Kulwa Pauline, CCA, Nkungululu Ward Mr. Bujiku Kasubi, Chairman, Sukuma Village Ms. Lina Amaya, WEO, Sukuma Ward Mr. Dotto Isack Habi, Dispensary in Charge, Lumeleji Mr. Mandago, District Health Officer 50 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 51 APPENDIX C. REFERENCES Ainslie, Robert. “COMMIT Community Survey Preliminary Results Overview, 2012.” Bonner, Kimberly, A. Mwita, P. McElroy, S. Omari, A. Mzava, C. Lengeler, N. Kaspar, R. Nathan, J. Ngegba, R. Mtung'e, and Nick Brown. “Design, implementation and evaluation of a national campaign to distribute nine million free LLINs to children under five years of age in Tanzania.” Accessed at www.malariajournal.com/content/10/1/73, March 2011. Boulay, Marc. “BCC Impact Study, Preliminary Results, Presentation, 2012.” Boulay, Marc. “Evaluating PMI BCC Programming in Tanzania—The COMMIT Project, Presentation, 2010. COMMIT. “BCC Impact Study” (slides showing preliminary results). COMMIT. “Behavior Change and Communication in the Prevention and Case Management of Malaria in Mainland Tanzania, 2007–2012.”COMMIT Project Annual Implementation Plans: Years 1, 2, 3, 4 & 5. COMMIT “Behavior Change and Communication in the Prevention and Case Management of Malaria in Mainland Tanzania, 2007–2011.” COMMIT Project, Annual Reports: Years 2, 3, 4 (no report for Year 1). COMMIT. “Behavior Change and Communication in the Prevention and Case Management of Malaria in Mainland Tanzania 2007–2011.” COMMIT Project Quarterly Reports Years 1, 2, 3, 4, and Q1 and Q2 of Year 5. COMMIT. “CCA Qualitative Assessment.” COMMIT. “Children’s Radio Program PataPata.” COMMIT. Chumo, Media for Development International, 2011. COMMIT. “Chumo Movie Pre-Testing Report.” COMMIT, 2011. COMMIT. “Combined Results for the Omnibus 2010 Results.” COMMIT, 2010. COMMIT. “COMMIT Overview Document.” 2012. COMMIT. “COMMIT Success Stories.” 2011. COMMIT. “Community Survey, 2009.” COMMIT. “Compendium of COMMIT Print Materials.” (binder of materials) 2012. COMMIT. “Formative Research, 2009.” COMMIT. “Omnibus Data, 2010.” COMMIT. PataPata Children’s Radio Program Pre Test Results, 2011. COMMIT. PataPata Children’s Radio Program Posttest Report, 2011. COMMIT. “Performance Monitoring Plan, 2009.” COMMIT. “Project Monitoring Plan.” 52 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION COMMIT. “TRaC 2010 (PSI National Survey), Tables.” The Communication and Malaria Initiative in Tanzania (COMMIT): Program Description. JHU Proposal to USAID. 2007. “Communication Impact: Rural Communication Activities Increase Net Use in Tanzania.” Report No. 25, Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs, April 2010. “Communication Strategy for Malaria Control Interventions 2008-2013.” Malaria Control Series No. 21. Ministry of Health and Social Welfare, National Malaria Control Programme, United Republic of Tanzania. “Focus on Mainland Tanzania.” Progress and Impact Series Country Reports, No. 3. World Health Organization, January 2012. “Fund Manager’s Quarterly Report Model.” Population Services International, 2010. “How to Ensure Regional Fund Holders Assist You in Malaria BCC Activities: Guide for Zonal Managers,” 2008. Ifakara Health Institute. “Down But Not Out. The Impact of Malaria Control in Tanzania.” Spotlight, Issue 2, May 2009. Ifakara Health Institute. “Health Facility Committees: Are They Working?” Spotlight, Issue 7, August 2011. Ifakara Health Institute. “Tanzania DHS 2010: What Has Changed?” Spotlight, Issue 9, October 2011. “Impact of Malaria Interventions in Tanzania, 2007–08.”, National Malaria Control Programme, 2009. Jhpiego. Training of Interpersonal Communication Supervision Skills for Supervisors in Kigoma Region. Jhpiego, July 2011. Jhpiego. Training of Interpersonal Communication Supervision Skills for Supervisors in Shinyanga Region. Jhpiego, July 2011. Jhpiego. Provider’s Interpersonal Skills in Communication on Malaria: Clients’ Perspectives from Tanzania. A report on providers / client interpersonal communication from ANC and Outpatient Department clients. October 2010. Jhpiego. Training of Health Care Providers on Interpersonal Communication Skills in Kilimanjaro, Arusha and Manyara Regions. Jhpiego, April 2010. Jhpiego. Training of Health Care Providers on Interpersonal Communication Skills in Mtwara Region. Jhpiego, March 2010. Jhpiego. Training of Health Care Providers on Interpersonal Communication Skills in Mwanza, Shinyanga, Mara and Kagera. Jhpiego, December 2009. Jhpiego. Training of Nurse Tutors on Interpersonal Communication Skills for Prevention of Malaria in Pregnancy. Jhpiego, May 2011. USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION 53 “Malaria Medium-Term Strategy 2008–2013.” National Malaria Control Programme, July 2009. “Malaria Sub-Granting Communication Flow.” Population Services International, 2008. Marchant, T., J. Bruce, R. Nathan, M. Mponda, Y. Sedekia, and K. Hanson. Monitoring and Evaluation of the Tanzanian National Net Strategy Report on 2008 NATNETS Household, Facility Services, and Facility Users Surveys. London: Ifakara Health Institute, London School of Hygiene and Tropical Medicine, February 2009. Medium-Term Malaria Strategic Plan 2008–2013. United Republic of Tanzania, Ministry of Health and Social Welfare, National Malaria Control Programme. February 2008. Mlwangwa, Susan. “Community Change Agents Qualitative Assessment Topline Report,” 2011. “Mtwara Supportive Supervision Report.” Population Services International, March 2010. National Malaria Control Programme. Summary of Five Household Surveys to Monitor Population‐level Coverage and Impact of Malaria Interventions in Tanzania, 2007–08. Dar es Salaam, Tanzania: National Malaria Control Programme, 2009. Population Services International. “Request for Expression of Interest: Fund holding for Ruvuma.” March 2009. Population Services International. “Request for Proposal: COMMIT Project Kagera Region Fund Holding Services.” 2008. Population Services International. Sub Recipient Pre-Award Assessment Tool. 2010. Population Services International. “TRaC Survey Main Malaria Indicators Presentation.” 2010. President’s Malaria Initiative: Behavior Change Communication Priority Focus for FY2011/12. United States Agency for International Development/President’s Malaria Initiative January 2011. President’s Malaria Initiative. Malaria in Tanzania, 2008. Available online at http://www.fightingmalaria.gov/countries/profiles/tanzania.html. Roll Back Malaria Partnership. Guidelines for Core Population‐Based Indicators. Calverton, MD: January 2009. Rose, Nathan and Yovitha Sedekia. Monitoring and Evaluation of the Tanzanian National Net Strategy, Universal Coverage Campaign. Household Survey Report—Coastal Zone. London: Ifakara Health Institute, London School of Hygiene and Tropical Medicine, December 2011. Rose, Nathan and Yovitha Sedekia. Monitoring and Evaluation of the Tanzanian National Net Strategy, Universal Coverage Campaign. Household Survey Report—Lake and Southern Zones. London: Ifakara Health Institute, London School of Hygiene and Tropical Medicine, December 2011. Skinner, Joanna and M. Boulay. “The important thing is education on prevention: A qualitative study on malaria prevention and treatment in Tanzania.” February 2010. 54 USAID/TANZANIA: COMMIT PROJECT PERFORMANCE EVALUATION Tanzania Demographic and Health Survey 2004–05. Dar es Salaam Tanzania: National Bureau of Statistics Tanzania, and ORC Macro, 2005. Tanzania Demographic and Health Survey 2005. Dar es Salaam, Tanzania: National Bureau of Statistics (NBS) and ICF Macro: NBS and ICF Macro, 2006. Tanzania Demographic and Health Survey 2009—10. Dar es Salaam, Tanzania: National Bureau of Statistics Tanzania, and ORC Macro, 2010. Tanzania Demographic and Health Survey 2010. Dar es Salaam, Tanzania: National Bureau of Statistics (NBS) and ICF Macro, 2011. Tanzania HIV/AIDS and Malaria Indicator Survey 2007-08. Dar es Salaam, Tanzania: Tanzania Commission for Aids (TACAIDS), Zanzibar AIDS Commission, National Bureau of Statistics (NBS), Office of the Chief Government Statistician, and Macro International Inc., 2008. Trainer’s/Supervisor’s Reference Guide on Interpersonal Communication Supervision Skills: Prevention and Management of Malaria in Pregnancy and in Children under Five Years of Age. Ministry of Health and Social Welfare/Tanzania. Second Draft. April 11, 2012. Training on Interpersonal Communication for Antenatal Care Health Providers, Nurse Tutor’s and Providers for Malaria Case management of Children under Five Years. COMMIT. Unpublished notes. United States Agency for International Development. “Communication and Malaria Initiative in Tanzania (COMMIT) Cooperative Agreement.” Washington, DC: USAID, October 2007. United States Agency for International Development. “Request for Proposal: Behavior Change and Communication in the Prevention and Case Management of Malaria in Mainland Tanzania.” World Health Organization. 2008. The World Malaria Report, 2008. Available online at http://malaria.who.int/wmr2008/malaria2008.pdf. POWERPOINT PRESENTATIONS “CCA Qualitative Evaluation Research Top Line Findings.” JHU Center for Communication Programs. 2011. “COMMIT Overview of Activities.” COMMIT. 2012. “Omnibus Presentation: Steadman Results: Malaria.” COMMIT. June 2011. PataPata: Children’s Radio Program, JHU Center for Communication Programs. 2011. “Trac 2010 Key Findings: Internal Dissemination,” Population Services International, August, 2010. Training on Interpersonal Communication Supervision Skills. Jhpiego. March 2012. For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com